Cube Map Lighting
Charles Bloom
March 27, 2004

Cube Map Lighting is a technique similar to Spherical Harmonics, but it
works on GeForce 1 class hardware, and has some nice other properties.
To render, all you need is a rasterizer that can sample from an
environment map. We use cube maps, but any kind of map could be used.

The basic idea is that in the rasterizer you will simply be taking a
per-pixel Normal direction, and sampling the map in that direction. The
lighting is thus quite trivial :

L = DiffuseMap[N]

You can also do specular or reflected light if you can compute a
reflection vector. (Note that GeForce3 class hardware has custom
register combiner modes to do exactly this - look up a cube map with a
Normal converted to a reflection vector). If you can compute the
reflection vector R(N), then you can make the specular light :

S = SpecularMap[ R(N) ]

Now, you can also store a "gloss" factor in the alpha channel of the
normal, and combine them :

Out = L + N.a * S

This would require two passes on GeForce3-class hardware (Dx8), but only
one on Radeon-class hardware (Dx9). Note also that the Normal here may
come from a normal map texture, or it could come just from vertex
normals. Also note that we don't need to re-normalize the normal at all,
because the cubemap lookup *implictily* normalizes for us. Thus, we can
use just vertex normals and interpolate them per-pixel and lookup the
cubemap and get nice lighting. Also note that there is *zero* positional
dependence in the per-pixel work. All the positional dependence is
already encoded in the cubemap. If you wanted you could do some
positional falloff per-vertex and pass that down, but I choose to not do
even that. (that would be important if your objects were very large or
your triangles were very large, neither of which are a good idea in
general). Note that technically you should use different maps for the
diffuse and specular lookup. The diffuse map should be convolved with
the (N*L) diffuse BRDF, while the specular map should be basically a
straight picture of the incoming light.

So, all we have to do is compute the maps. You can do this with the CPU
or the GPU. I like to use very small maps and do it on the CPU. You can
use a diffuse map as small as 4x4 per face and still get very good
lighting. Note that that's 4*4*6 = 96 samples, which is still a lot more
than the 9 or 16 samples that you would use for Spherical Harmonics.
Also note that the GPU cost is unrelated to the size of the map, but if
you are doing CPU sampling, the map needs to be small. I've successfully
used 8x8 maps as well, which is 8*8*6 = 384 samples.

To compute the DiffuseMap, you need to compute all the light that a
normal in the direction N would see. I'm going to describe two ways you
might want to do this, though there are really any number of ways. The
first way is by taking a picture of the world, either pre-computed or
dynamic. The second is algorithmic by sampling point lights and such
around you. So, in the first method, you have a picture of the incoming
light at your object; this should be taked from the center of the
object, and it's not positionally correct for all the points of the
object, but that's usually not a big deal if your environment is
reasonably low-frequency. You can take this IncomingLight picture either
by taking a real picture of a physical space with a camera (see
Debevec's Light Probes, for example), or dynamically be rendering to
textures and generating a cubemap. This IncomingLight picture is exactly
what you need for the SpecularMap - you can just use it as is (it's just
an environment map). If you sample IncomingLight[N] it literally tells
you how much light is coming in to the solid angle for the texel of the
cubemap in direction "N". To generate DiffuseMap, you need to convolve
this. You compute :

DiffuseMap[N] = Integral{L} (N*L) * IncomingLight[L] dL

and you have to make sure you get all your solid angle terms right and
your normalization and all that. The solid angle issue is a little funny
if you're using cube maps, because the solid angle of each texel is
different - the ones near the corner of the cube are smaller (imagine
mapping the cubemap onto a sphere). Amusingly enough, the fastest way to
compute this convolution is using spherical harmonics, but I won't go
into that here. This convolution can also be computed using the GPU if
you want to generate very large DiffuseMaps or if you somehow have a
system with a fast GPU and a slow CPU. I won't get into that either.

The other method is to algorithmically use lights placed around the
world. Note that the GPU per-pixel cost is unrelated to the number of
lights. To do this, you can first compute all the position dependence of
the light for the whole object, since you're going to use the same
position for all the cubemap samples. The only thing that differs from
one texel to the next is the normal. So, you can compute all your
falloff in whatever you want, which gives you an overall light
intensity, then you just compute for each texel :

DiffuseMap[N] = intensity * [ (N*L) ]
SpecularMap[R] = intensity * (R*L)^P

So, this math can be very fast on your 96 or 384 samples.

If your objects and lights are static, you can pre-compute your
DiffuseMap & SpecularMap and associate them with that object and never
have to compute them again. Another option would be to have some sampled
IncomingLight maps stored around the world, then you could pick the
nearest few and interpolate them as your object moves around. For fully
dynamic objects and lights, you just use the algorithmic method and
compute them as you need them. Note that in this case you will filling
out a lot of cubemaps from the CPU, so you need to manage your dynamic
textures well. With bad drivers on the PC this can be catastrophic for
performance; I do it on the XBox, so there's no performance hit at all.
In a racing game, or something where you have just a few important
characters, the render-to-texture cubemap generation would be good.

If your object is rigid, you can easily compute your Maps in object
space, so that the pixel-shader operation is literally just Out =
Map[N], assuming the normal N is also in object space. If your normal is
in tangent space or something else, you will have to apply a matrix
transform in the lookup, which geForce3-class hardware easily does : Out
= Map[ Matrix * N ].

Now, one nice thing about this cubemap lighting is that the only thing
you need per-pixel is a normal. That makes the amount of per-object data
needed quite small. Also, it means you can easily use it on skinned and
animating objects, since it's quite easy to transform the normals by the
skin matrix. Your per-pixel normals are usually stored in "rest pose
space" then, and you just use a Matrix that transforms from that space
to the space of the Map (usually world space). This is often the same
matrix you used to do the vertex transformation for skinning, but you're
just using the 3x3 rotation part (technically you should use the
inverse-transpose, and you don't need to worry about normalization).

Note that the positional falloff is only being done per-object. That
means if you put very simple object right next to each other and try to
make them abutt seamlessly, it won't work, the seam will be visible. The
solution to this is just art - don't try to stick flat surfaces
together. Always have a bevel or something at object boundaries, which
is enough to visually hide the change in lighting.

Another funny point is the cubemap sampling is sort of weird and broken.
The problem has to do with bilinear filtering and how the texels are
assigned. Normally in a texture, if you sample the middle of a texel,
you will get the color of that texel. As your sample point moves away
from the center you pick up the neighboring colors. If you sample at a
corner, you get an even 4-way pixel fo the neighboring texels. Now with
cube maps, you have 6 separate textures, each of which is samples and
filtered independently. The lookup routines for the cubemap choose of
the of the 6 faces and then samples it just like a regular texture. The
problem with this is at the edges. At the edges of the cubemap, when you
sample to the outside of a texel, it clamps to that texel value. What
you want it to do is to interpolate over to the neighboring texel on the
other cube face. For example, if you are on the North cube face and you
sample all the way at the West-mode edge, you want it to wrap around and
bilinear filter in with the North-most edge of the West face. Similarly,
if you sample at the Upper-West corner of the North face, you want to
get an even three-way mix of the North-West corder of the Upper face and
the Upper-North corner of the West face. The hardware can't do this, but
you can get the same effect using the "border source" stuff introduced
in Dx8. Basically what you're going to do here is supply your own
continuation of the cubemap around the corners. So, to do a 4x4 cubemap,
we're actually going to use an 8x8 cube map, with only the middle 4x4
filled out. Then, around the edges of that 4x4 area, we will copy in the
neighbors that we should have from the adjacent faces. At the very
corner of your 4x4 face you have 3 neighbors to fill out, but only 2
actual neighbors. So, you copy in the 2 neighbors for the adjacent
texels, and then for the corner texel, you put in the three-way average
of the 3 real ones. Note that you can't just sample this map yourself
correctly, the cubemap hardware has to think it's a 4x4 in order to do
the lookup right, so you have to use the bordersource extension stuff.
With very large cubemaps this isn't a problem, the seam is pretty
minimal, but with small cubemaps like 4x4 or 8x8, the seam is quite
ugly, so you need to do this. Note that this is really an issue for all
users of cubemaps, not just me, but most people just never notice it or
ignore it.

The Galaxy3 pages have some screenshots of models lit with the cubemap
lighting technique.