The Direct3D rendering pipeline

20-Dec-1998

I mean here, Direct3D Immediate mode, of course. And also, I'll be considering the pipeline of DirectX 6 only.

This page is dedicated to the Direct3D rendering pipeline. I'll try to document it as well as I can, and hopefully other people will be able to fill in the gaps of my ignorance.

The reason for writing this page is so that I can try to make some sense of all the available information about the pipeline, and to put it together in a way that will allow me (and any other reader) to understand how that pipeline works, and also, to pinpoint any deviation from this encountered in any particular imlpementation. So far, I haven't seen any literature that clearly describes this pipeline in detail.

Note that I'm not interested in the actual, real implementation of this pipeline. There can, in fact, be many different implementations (imagine different involvements of hardware in different implementations, and you'll see what I mean). The pipeline steps don't even need to be distinct, and sometimes, they can even be out of order in a particular implementation. What matters are the results. They must match the results of the documented pipeline.

For starters, there's a geometry stage in the pipeline that is, as far as I understand, simplicity itself. It can be viewed as a black-box that gets a list of vertices in an input format, and outputs the same vertices in an output format. The output formats are Transformed, Lit vertices (TLvertices), of varying amount of UV-coordinate sets. The input formats are many (considering the combinations of Flexible Vertex Format flags), but they fall into two categories: Untransformed, unlit vertices (ULvertices) and Untransformed, lit vertices (Lvertices).

The geometry pipeline, then, can be viewed as a two-stage pipeline:

• In the first stage, the vertices' diffuse and specular color values are calculated. This is accomplished by rotating every light attached to the current viewport, into local coordinates (that's rotating with the inverse of the World Matrix), and calculating the light that goes from each vertex to the viewer (camera) position. The formulas for this calculation are (disregarding the typos) actually very well documented in the DirectX SDK documentation, so I won't describe them here.
• The second stage seems to do several things. First, and most important, it transforms the vertex position into screen coordinates, by applying the World, View and Projection matrices. This generates the X,Y screen coordinates, the Z-buffer value and the one-over-W - the homogeneous coordinate, that is used for perspective correction, W-buffer and such. It also seems to be the one that does the calculation of the fog value (F), and puts it into the alpha component of the specular color. Also, it calculates clipping information.

It's important to note that all these geometry calculations are done without regard to the state of the texture pipeline, the blending modes, the filtering modes, etc... The only states that are used here are the LightStates and the Fog states.

When the time comes to actually render a primitive, Direct3D checks for clipping issues, and rearranges and reconstructs the vertices of the primitive as appropriate in order to have vertices that lay within the clipping region. It then passes the clipped TLvertices to the rasterizer. The rasterizer will then interpolate all values (coords, Z-buf, colors, ooW, UV coord sets...) throughout the primitive. That yields a full set of such values for each pixel (screen coordinates, z, oow, diffuse, alpha, sepcular, fog and U-V sets, maybe mip-map value) which are input into the pixel pipeline. This pipeline seems to have the following stages (and I say again: the order of operations doesn't necesarily have to match the one I describe here. Instead, the results must match those of the pipeline I portray):

• First, if there's a Z-buffer and it is active, the Z value to be rendered is compared against the buffer value and the pipeline exits if the proper condition (as defined in the settings) is met. This is done with the W values instead, if W-buffer is active.
• Second, the stipple comparison, if any, is done. If the pixel must not be rendered, the pipeline exits.
• Now comes the big monster: the multiple texture stages. Basically this uses the diffuse color and alpha, the factor color and alpha, and the texels read from the different textures in order to generate a unique color-and-alpha. The insides and workings of this 'texture pipeline' are well enough described in the SDK documentation, so I don't need to go to any more depth here.
• And, finally, comes the blend, fog and specular stages. I don't really know what's the correct order of execution here, but I can guess: blend first, then specular, and then, finally, fog. This would make sense. First, the pixel color is blended with the background color. Then, the specular color is applied by adding the specular color to the result of the blending stage (it can actually overwrite it completely), and finally the fog is applied (it's just a weighted average of the result of the specular stage and the fog color.
• Finally, all results are written back to the frame buffer, Z-buffer, stipple buffer and so on.

And that's it. It looks a lot clearer to me after writing all this, actually. There are still a few dark spots, like, for example, what's the color-key involvement in this, alpha testing and so on.

Also this view here has revealed a few places where the pipeline could be improved. For example, if you want to do the specular color as an environment-map, and you want the polygon to be alpha-blended, it's impossible to render it in one pass. Of course, current hardware doesn't allow that, anyway, but maybe future hardware would.

This would be accomplished by specifying four new arguments for the texture stages:

• D3DTA_CURRENT2 - this would use the result of the second previous stage.
• D3DTA_SPECULAR - this would use the specular color component.
• D3DTA_FBUFFER - this would use the current contents of the frame buffer.
• D3DTA_FOGCOLOR - this would use the current fog color.

With those arguments available, we could build up the following pipeline state, which would resolve that situation appropriately, and would also implement the blend, specular and fog within the texture pipeline:

SetTextureColorStage(dev, 0, D3DTA_DIFFUSE, D3DTOP_MODULATE, D3DTA_TEXTURE);
SetTextureColorStage(dev, 1, D3DTA_CURRENT, D3DTOP_BLENDTEXTUREALPHA, D3DTA_FBUFFER);
SetTextureColorStage(dev, 2, D3DTA_SPECULAR, D3DTOP_MODULATE, D3DTA_TEXTURE);
SetTextureColorStage(dev, 3, D3DTA_CURRENT, D3DTOP_ADD, D3DTA_CURRENT2);
SetTextureColorStage(dev, 4, D3DTA_CURRENT, D3DTOP_BLENDTEXTUREALPHA, D3DTA_FOGCOLOR);

SetTextureAlphaStage(dev, 0, D3DTA_DIFFUSE, D3DTOP_MODULATE, D3DTA_TEXTURE); // I know it's not so normal to modulate them, but I like it this way.
SetTextureAlphaStage(dev, 1, D3DTA_CURRENT, D3DTOP_SELECTARG1, D3DTA_CURRENT);
SetTextureAlphaStage(dev, 2, D3DTA_SPECULAR, D3DTOP_SELECTARG1, D3DTA_CURRENT);
SetTextureAlphaStage(dev, 3, D3DTA_CURRENT, D3DTOP_SELECTARG1, D3DTA_CURRENT);
SetTextureAlphaStage(dev, 4, D3DTA_CURRENT, D3DTOP_SELECTARG1, D3DTA_CURRENT);

Ok. This is enough. I want ideas, comments, suggestions, clarifications, corrections, jokes, or any other kind of feedback about all this. Except flames, of course.