Shaders 103 – Lighting

Hello!

By now you should know what shaders are, and how they work.  You should also know how to integrate them into your code.  Since I have spent a lot of time putting lighting and what not into our game, I have become a bit of an expert with it.  So today I am going to go over how to do some fragment based lighting.

Changes from OpenGL and Movement Matrices

While I didn’t do the lighting in our game last semester, you can’t take old OpenGL code with lighting and a whole bunch of glTranslate and glRotate calls and expect it to work.

The first thing we are going to have to do is build a whole bunch of matrix functions that build a perspective, look at, rotation, translation, multiplication, invert and transform matrices.  When you download the CG API some of the sample code does have these functions build in, but they expect you to know what they do and how they work.

Here is how we will now be rendering objects instead of using the ‘gl’ draw calls.

/*** Render brass solid sphere ***/

setBrassMaterial();

/* modelView = rotateMatrix * translateMatrix */
makeRotateMatrix(70, 1, 1, 1, rotateMatrix);
makeTranslateMatrix(2, 0, 0, translateMatrix);
multMatrix(modelMatrix, translateMatrix, rotateMatrix);

/* invModelMatrix = inverse(modelMatrix) */
invertMatrix(invModelMatrix, modelMatrix);

/* Transform world-space eye and light positions to sphere's object-space. */
transform(objSpaceEyePosition, invModelMatrix, eyePosition);
cgSetParameter3fv(myCgFragmentParam_eyePosition, objSpaceEyePosition);
transform(objSpaceLightPosition, invModelMatrix, lightPosition);
cgSetParameter3fv(myCgFragmentParam_lightPosition, objSpaceLightPosition);

/* modelViewMatrix = viewMatrix * modelMatrix */
multMatrix(modelViewMatrix, viewMatrix, modelMatrix);

/* modelViewProj = projectionMatrix * modelViewMatrix */
multMatrix(modelViewProjMatrix, myProjectionMatrix, modelViewMatrix);

/* Set matrix parameter with row-major matrix. */
cgSetMatrixParameterfr(myCgVertexParam_modelViewProj, modelViewProjMatrix);
cgUpdateProgramParameters(myCgVertexProgram);
cgUpdateProgramParameters(myCgFragmentProgram);
glutSolidSphere(2.0, 40, 40);

Now this may seem like a lot, but it is necessary for working with shaders.

The beginning where we call the setBrassMaterial() function is where we set the objects parameters.   We will get to that a bit later.  For now think of it as your glColor call.

The first part where we create the matrix using a simple rotation and translation matrix is fairly simple.  You would just pass on those parameters as if you were doing a normal glRotate or glTranslate call.  You can replace these with variables so you can move these.  For now this object is stationary so we do not need it to move

However the next part is where you  multiply them to get your modelMatrix and invert it to get your final matrix.  This is so we can calculate lighting with respect to the sphere object.  We then update our eye and light Cg parameters that we will see later.

The last bit of code creates the modelView matrix and actually draws the sphere.

Using Materials

The book uses this method of creating functions that set the emissive, ambient, diffuse, specular and shininess values.  Like this:

static void setBrassMaterial(void)
{

const float brassEmissive[3] = {0.0, 0.0, 0.0},
brassAmbient[3] = {0.33, 0.22, 0.03},
brassDiffuse[3] = {0.78, 0.57, 0.11},
brassSpecular[3] = {0.99, 0.91, 0.81},
brassShininess = 27.8;

cgSetParameter3fv(myCgFragmentParam_Ke, brassEmissive);
checkForCgError("setting Ke parameter");
cgSetParameter3fv(myCgFragmentParam_Ka, brassAmbient);
checkForCgError("setting Ka parameter");
cgSetParameter3fv(myCgFragmentParam_Kd, brassDiffuse);
checkForCgError("setting Kd parameter");
cgSetParameter3fv(myCgFragmentParam_Ks, brassSpecular);
checkForCgError("setting Ks parameter");
cgSetParameter1f(myCgFragmentParam_shininess, brassShininess);
checkForCgError("setting shininess parameter");

}

So this function just sets the colour of each of the light parameters that we want.  Using this we can make several material functions for different objects and control them independently in whatever way we want.  You can make a character, enemy and level material.  Right before you load your character, you can make their lighting bright so that they stand out.  For enemies, you can give them a bit of a red highlight to show the player that they pose a threat.

What to Initialise

Now we are in our initCg() function let us break it down into a vertex and fragment area.

Vertex Initialisation

myCgVertexProfile = cgGLGetLatestProfile(CG_GL_VERTEX);
cgGLSetOptimalOptions(myCgVertexProfile);
checkForCgError("selecting vertex profile");

myCgVertexProgram =
cgCreateProgramFromFile(
myCgContext,              /* Cg runtime context */
CG_SOURCE,                /* Program in human-readable form */
myVertexProgramFileName,  /* Name of file containing program */
myCgVertexProfile,        /* Profile: OpenGL ARB vertex program */
myVertexProgramName,      /* Entry function name */
NULL);                    /* No extra compiler options */
checkForCgError("creating vertex program from file");
cgGLLoadProgram(myCgVertexProgram);
checkForCgError("loading vertex program");

#define GET_VERTEX_PARAM(name) \
myCgVertexParam_##name = \
cgGetNamedParameter(myCgVertexProgram, #name); \
checkForCgError("could not get " #name " parameter");

GET_VERTEX_PARAM(modelViewProj);

This is a fairly simple vertex initialisation.  The main point is to see that we are passing the modelViewProj matrix.  If you go back up to our draw code you can see where we update myCgVertexParam_modelViewProj parameter.

Vertex Shader Code

void v_fragmentLighting(
float4 position : POSITION,
float3 normal   : NORMAL,

out float4 oPosition : POSITION,
out float3 objectPos : TEXCOORD0,
out float3 oNormal   : TEXCOORD1,

uniform float4x4 modelViewProj)
{
oPosition = mul(modelViewProj, position);
objectPos = position.xyz;
oNormal = normal;
}

You can still see that this vertex shader is still simple.  We take our model view matrix and multiply that by our position and output both our position and our object position.

Fragment Initialisation

#define GET_FRAGMENT_PARAM(name) \
myCgFragmentParam_##name = \
cgGetNamedParameter(myCgFragmentProgram, #name); \
checkForCgError("could not get " #name " parameter");

GET_FRAGMENT_PARAM(globalAmbient);
GET_FRAGMENT_PARAM(lightColor);
GET_FRAGMENT_PARAM(lightPosition);
GET_FRAGMENT_PARAM(eyePosition);
GET_FRAGMENT_PARAM(Ke);
GET_FRAGMENT_PARAM(Ka);
GET_FRAGMENT_PARAM(Kd);
GET_FRAGMENT_PARAM(Ks);
GET_FRAGMENT_PARAM(shininess);

/* Set light source color parameters once. */
cgSetParameter3fv(myCgFragmentParam_globalAmbient, myGlobalAmbient);
cgSetParameter3fv(myCgFragmentParam_lightColor, myLightColor);

This not the full code for the initialisation.  This smidgen of code contains the new parameters that we will be passing into our fragment shader to compute our lighting.

Fragment Shader Code

void basicLight(
float4 position : TEXCOORD0,
float3 normal   : TEXCOORD1,

out float4 color : COLOR,

uniform float3 globalAmbient,
uniform float3 lightColor,
uniform float3 lightPosition,
uniform float3 eyePosition,
uniform float3 Ke,
uniform float3 Ka,
uniform float3 Kd,
uniform float3 Ks,
uniform float shininess)
{
float3 P = position.xyz;
float3 N = normalize(normal);

// Compute emissive term
float3 emissive = Ke;

// Compute ambient term
float3 ambient = Ka * globalAmbient;

// Compute the diffuse term
float3 L = normalize(lightPosition - P);
float diffuseLight = max(dot(L, N), 0);
float3 diffuse = Kd * lightColor * diffuseLight;

// Compute the specular term
float3 V = normalize(eyePosition - P);
float3 H = normalize(L + V);
float specularLight = pow(max(dot(H, N), 0), shininess);
if (diffuseLight <= 0) specularLight = 0;
float3 specular = Ks * lightColor * specularLight;

color.xyz = emissive + ambient + diffuse + specular;
color.w = 1;
}

This code takes in our parameters that we pass in our C++ code to compute emissive, ambient, diffuse and specular lighting.  Emissive and ambient are fairly easy to compute, however diffuse and specular require some more work.

Emissive Light

Emissive is the light that is emitted or given off by a surface.  This can be used to stimulate glowing
Equation: emissive = Ke
Ke is the materials emissive color

Ambient Light

Ambient or ambience is light that has bounced around from different objects.  This can be used to make your environments better.  You can have a grey ambient for smoggy cities or a nice bright yellow ambient for forests and nature environments.
Equation: ambient = Ka * globalAmbient
Ka is the material’s ambient reflectance
globalAmbient is the color of the incoming ambient light

Diffuse Light 1

Diffuse light is reflected off a surface equally in all directions.  Even if an object has small nooks and crannies, the light will bounce of its rough texture
Equation: diffuse = Kd * lightColor * max(N dot L, 0)
Kd is the material’s diffuse color
lightColor is the color of the incoming diffuse light
N is the normalised surface normal
L is the normalised vector toward the light source
P is the point being shaded

Diffuse Lighting 2
Specular Light 1

Specular lighting is light scattered from a surface around the mirror direction.  It is only seen on very shiny and metallic materials.  Unlike the above types of light, Specular depends on where the viewer is looking at for it to work.  It also takes into account how shiny a surface is.
Equation:  specular = Ks * lightColor * facing * (max(N dot H, 0))^shininess
Kd is the materials specular color
lightColor is the color of the incoming specular light
N is the normalized surface normal
V is the normalized vector toward the viewpoint
L is the normalized vector  toward the light source
H is the normalized vector that is halfway between V and L
P is the point being shaded
facing is 1 is N dot L is greater then 0 and 0 otherwise

Specular Light 2

Then you add all the lights together and that is lighting in a nutshell.

Fragment Lighting

Thank your for reading,
– Moose

Shaders 101 – Intro

Hello!

Overview

I have read up to chapter 5 in the CG textbook (almost halfway done) and I thought it would be good to do a general summary of what I have learned so far.  Granted I might be misinformed or have fragmented knowledge about some aspects, but I hope I will be corrected in the comments.

What are Shaders and How Do They Work?

First of we are talking about the programming language Cg created by NVIDIA.  The point of shaders and the Cg language is to help you communicate via code with your graphics card to control the shape, appearance and motion of objects drawn.  Essentially it allows you to control the graphics pipeline, like a boss.  Cg programs control how vertices and fragments (potential pixels) are processed.  This means that our Cg programs are executed inside of our graphics cards.

Shaders are a powerful rendering tool for developers because they allows us to utilise our graphics cards.  Since our CPU’s are more suited toward general purpose operating system and application needs, it is better to use the GPU that is tailor built for graphics rendering.  GPU’s are built to effectively process and rasterize millions of vertices and billions of fragments per second.  The great thing about CG is that it gives you the advantages of a high level language (readability and ease of use) while giving you the performance of a low level assembly code.  Cg does not provide pointers and memory allocation tools.  However it supports vectors and matrices and many other math operations that make graphics calculations easier.

Cg is not meant to be used as a full fledged programming language.  We still need to build our 3D applications in C++ (or any language) then use our shader language (CG, HLSL, GLSL, RenderMan etc.) to optimise our graphics using the GPU.

The Graphics Pipeline

Graphics Pipeline: From the CG Textbook

In order to understand how shaders work, we have to have a general understanding on how the graphics pipeline (stages operating in parallel) work.  First your 3D application sends several geometric primitives (polygons, lines and points) to your GPU.  Each vertex has a position in 3D space along with its colour, texture coordinate and a normal vector.

Vertex Transformation

This is the first processing stage of the pipeline.  First several mathematical operations are performed on each vertex.  These operations can be:

  • Transformations (vertex space to screen space) for the rasterizer
  • Generating texture coordinates for texturing and lighting to determine its colour

Primitive Assembly and Rasterization

Once the vertices are processed, they are sent to this stage of the pipeline.  First the primitive assembly step assembles each vertex into geometric primitives.  This will result in a sequence of triangles, lines or points.

Geometric Primitives

After assembling the primitives will need to be clipped.  Since we are limited to a screen we cannot view the entire screen.  So according to our view frustum we clip and discard polygons (culling).  Once our screen is clipped our next step is to rasterize.  This is the process of determining what pixels are covered by a geometric primitive.

Rasterisation

The last important item in this process is for the user to understand the difference between pixels and fragments.  A pixel represents a location on the frame buffer that has colour, depth and other information.  A fragment is the data needed to generate and update those pixels.

Fragment Texturing and Coluring

Now that we have all our fragments the next set of operations determine its final colour.  This stage performs texturing and other math operations that influence the final colour of each fragment.

Raster Operations

This is the last stage of the graphics pipeline.  It is also one of the more complex stages.  Once the completed fragments come out of the previous stage the graphics API perform several operations on the incoming data.  Some of them are:

  • Pixel ownership test
  • Scissor test
  • Alpha test
  • Stencil test
  • Depth test
  • Blending
  • Dithering
  • Logic operations
This is pretty much the above process in a nutshell.
Pipeline In a Nutshell

Programmable Graphics Pipeline

So what was the point of talking about the pipeline?  Now that we know how the fixed pipeline works and how normal graphics API’s send information to be processed we can see where are shaders are executed and what they do.

Programmable Graphics Pipeline

The two important parts of this diagram are the programmable vertex processor that runs our Cg vertex programs and the programmable fragment processor that runs our Cg fragment programs.  The biggest difference between each one is the fact the the fragment processor allows for texturing.

Summary

Now that we know how the graphics pipeline works we can create programs to manipulate the pipeline however we want.  Next week we shall take a look at how to make programs for Cg and how they work in your 3D application.

Thank you for reading,
-Moose

Game Opinion: Japanese Culture and Western Game Design also RPG’s

Hello!

The next blog posts will be in preparation for a game idea I have had sitting in a dark corner for a while waiting to be let out to roam free and changed.  In order to do that, I have to go over two topics.  One is the East vs West culture differences pertaining to games which I will be blogging about today.  The other topic I wanted to talk about is how I think RPG’s should be cultivated.

Overview

Why do Japanese games do so horrible in the western market?  Why are so few eastern titles seen on the top selling charts in the Japense market? Can we make games for both audiences or do we have to make independent games for each region?

Rant

Animation

Lets start out by looking at the differences between Japanese anime and North American Cartoons.  Famous Japanese anime like Bleach, Naruto, One Piece and Fairy Tail all have huge epic stories that have been going on since 1999 and are still published weekly to this date.  Also, Japanese anime all have very deep stories and characters that can be enjoyed by many different age groups.

  • Bleach – Protagonist is a very strong and serious teenage boy.  The story is about how this boy turns into a “death god” that is able to walk among and interact with the afterlife.  The protagonist is only able to advance through the many antagonist’s by developing his hidden power only to protect his friends.  The most notable aspects of this anime is the depth at which they choose to portray the afterlife.
  • Naruto – Protagonist is a comical young boy with a very troubled past having persevered through lots of hate and animosity from others because of a demon monster that lives inside him.  This anime takes place in a large fictional land inhabited by ninjas.  The main plot that drives this anime is the protagonist trying to reclaim his best friend from the dark side or the evil side.  This anime has many great underlying themes.  The most notable is the cycle of hatred and revenge.  Since the protagonist has survived through a decade of hate he is constantly trying to remove hate and revenge from the world.  The anime tries to define hate in many ways and it also shows how the cycle starts and begins several times over the course of the anime.
  • One Piece – This is truly one of my favourite shows of all time.  It is an epic tale of a very comedic and amiable protagonist that can create a bond greater then friendship with almost everyone on the show.  The setting is a giant world where pirates roam free to pillage the unexplored seas of this fantasy realm.  The main plot is that a great Pirate King left a great treasure in the most dangerous sea and whoever get it will be able to become the next pirate king.  The reason why I love this anime is the cast of the protagonists allies.  His pirate crew is filled with the most interesting characters that eventually you feel as a viewer immersed that you are laughing as if you are on that crew with them.  Most notably they use a word to describe the crews relationship with one another that is “nakama“.  Many people translate this Japanese word to friend, however there is no real word in the English language that can properly state the meaning of this word.  A Nakama is someone who is greater the a friend but not blood related, it is someone who is essentially your family and the bonds you share cannot be broken.  This concept of brotherhood is really the best part about the anime, it shows the most perfect group of friends that I have seen on screen.
  • Fairy Tail – This show is essentially a magical version of One Piece.  Whereas rather then a pirate crew, we have a guild of magicians

While this montage does not to justice to the many hilarious moments on One Piece, it gives you a good idea of the show and Anime.  There is a lot of slapstick type comedy in many anime and One Piece uses it a lot in the delivery of their jokes. (The guy with the straw hat is the protagonist Luffy).

The main point I wanted state was that their shows have a lot of depth to them in comparison to the North American cartoons.  Look at Family Guy, Futurama/Simpsons, Archer then the more childish shows like Spongebob, Fairly Odd Parents and so on.  These shows are primarily episodic that reference pop culture and current events.  The mature shows are more humorous rather then going for deep stories.  The childish shows have short 15 minute skits that rely on cheap morals driving the plot to teach kids basic morality and ethical lessons.  Japanese anime don’t have these “morals” driving their plot, instead they show deep and emotional scenes through their characters.

People may think that Japanese anime are all the same, however the thing that defines each show are their characters.  Luffy from One Piece is a funny, approachable and likeable leader yet when shit goes down he turns into a complete badass that forces you to sit on the edge of your seat to see him fight.  Naruto from Naruto is a rebel with a harsh past and a long history of bullying because of a demon that lives inside him.  However his strength is the ability to look past that and still fight and ‘love’ everyone equally and fight for the sake of the village that ridiculed him as a child.  Ichigo from Bleach is more a serious character that is always fighting to save his buddies.

Overall, the point is that Japanese media is more focused on creating a long and epic story built on the backbone of its characters whereas American media is more populated by shows giving the viewer an emotional response through episodic shows.  This is also because of the way these shows are produced.  Japanese shows will have one writer and a team of artists working on a project for years at a time but American shows change writers almost every episode.

Games

Looking at some of the more famous Japanese games like Final Fantasy, Dragon Quest, Monster Hunter, Persona 3, Tales of Series, Dark/Demon Souls, etc.  The majority of these games have very large and expansive worlds that follow a fairly linear plot structure but for the most part have very difficult gameplay mechanics and/or very deep and satisfying stories. These games are heavily focused on narrative and want their players to follow this linear plot they have created.  If you look at almost every major Japanese RPG developer (Squaresoft, Enix, Koei, Falcom) they all started out making visual novels for their RPG genre.  All their games after that use those visual novels as a template for creating their future games. In essence they take a dynamic like an action RPG, turn based RPG and take a long story like a visual novel and create a game.

Looking at western RPGs like Mass Effect, Dragon Age, Skyrim, Fallout, etc.  These games also have very large and expansive worlds however their stories are not the driving force of the game.  The most important aspect of WRPGs is the player experience.  They try to build a player centric RPG where the player is free to either play the story, kill NPC’s altering plot lines or explore the world.

The guys over at Extra Credits bring up a valid point about the whole JRPGs vs Western RPGs.  As an industry/community we end up lumping both JRPGs and WRPGs in the same genre but the reality is that they are two completely different genres.  It is like calling both apples and oranges, bananas.  When in reality they are both fruits they have their own sub classification as either an apple or an orange.  That was most likely the worst analogy ever.

Personally I was fairly surprised that I am a fan of Japanese animation however I prefer the WRPGs.  I like the freedom and ability to do whatever I want in a game, JRPGs are fairly restrictive.

Summary

What was the point of the past 1300 words? Why do Japanese games do so horrible in the western market?  Why are so few eastern titles seen on the top selling charts in the Japense market? Can we make games for both audiences or do we have to make independent games for each region?

First of the rant was a large intro that will allow me to answer these next few questions.

Take a look at VgChartz software and hardware sales for North America and Japan for a second. For now you should notice a bit of a trend.  Japan’s consumers like handheld consoles, mainly the 3DS and the North American consumers prefer the home console.  So unless Mass Effect 3 comes out with an amazing PS Vita or 3DS port, their eastern audience is fairly limited.  Aside from the business aspect of why WRPGs don’t sell in Japan, I think that Japanese consumers are looking for an indepth RPG experience on a handheld device.  Most of the great WRPG’s are console based and there has not been a huge effort to make a similar game experience on a handheld device.  This is primarily because the horsepower on the consoles is greater then that of most of the handheld devices.  As I stated above, WRPG’s are more about creating a great innovative gameplay experience that require more processing power.

At the top of the 2011 Japanese charts we see Mario Kart 7 (3DS), Super Mario Land (3DS), Monster Hunter 3G (3DS), Monster Hunter Freedom 3 (PSP), Final Fantasy Type (PSP).  The first eastern game on that chart is Just Dance (Wii) in 17th place. Skyrim has sold about 200,000 units in Japan and Mass Effect 2 has sold about 90,000.  I think this goes back to the idea of making handheld games geared to the Japanese market.  However, there are hundreds of JRPGs already attempting this strategy and this is evident by the number of 3DS  and PS Vita games released.

Closing Remarks

Overall, the idea of WRPGs vs JRPGs should not be a head to head competition of which one is better.  Each one is a genre on their own.  It is like comparing a FPS with a puzzle game.  I do not think it is possible to make a RPG game that can satisfy both regions equally. One may argue by bringing up Final Fantasy, Pokemon and Super Mario.  However those games have insane teams of marketing and game developers with million dollar budgets and a strong foundation in those markets after developing for many years.

Thank you for reading,
-Moose