N64 Programming/Video coprocessor

The RCP (VDP,PPU,video controller) is mainly interesting for polygon crunching and post-filter effects.

Formed from two components - RSP(Reality Signal Processor) and RDP(Reality Drawing Processor).

Texture cache and framebuffers are shared in RDRAM with the CPU.

Actual texture memory is only 4KB -- only allowed to operate on this amount at a time.

RSP(Reality Signal Processor)

RSP is your transform and lighting unit (TnL). It manipulates world data and textures.

Mathematically, lots of matrices to transform from local data -> world space -> view space (projection w/ z-perspective correction).

Transform means scaling, translating and rotating for polygons, lighting normals, and texture UVs.

Creates primitive lists of triangles and lines for the RDP to render.

RDP(Reality Drawing Processor)

RDP is the display unit.

Rasterizer, fog, environmental, color blending. Anti-aliasing effects.

Lower-level pixel handler.

uCodes

The RCP has its own language, dubbed 'uCodes' (256 microcodes).

Think of modern vertex and fragment shaders from OpenGL or vertex and pixel shaders from Direct3D - both stages combined.

R4000 coprocessor (COP2). Each uCode is a string of ASM instructions run by the RSP. Also sets up the RDP batch renderer.

Display lists are a sequence of uCodes defined by the game. This is fed to the RSP.

Note: Emulator authors choose to translate uCodes into higher-level languages.

The programmers can define their own vertex / texture formats. Lighting methods. Overdraw detection and other flexible wizardry.

The microcodes are uploaded to the RCP at run-time. So each game has its own library of drawing functions (some are like DSP1 -> DSP1A -> DSP1B and others are akin to DSP2,DSP3).

Texture variations

Games use custom formats. Furthermore, linear bitmaps can be any size (320x8, 48x13). 4/8/16/32-bpp is the norm.

The microcode libraries tend to define several 'accepted' formats.

This is a sample list from video plugin authors.

16-bit RGBA = 5551. Red,green,blue,alpha (transparency).
32-bit RGBA = 8888.

4-bit IA = 31. Grayscale intensity (luminosity,brightness) + alpha.
8-bit IA = 44.
16-bit IA = 88.

4-bit I = 40. Grayscale only.
8-bit I = 80. Grayscale.
16-bit I = (16)(0).

4-bit CI = 40. Palette lookup --> 16-bit RGBA or 16-bit IA.
8-bit CI = 80. Palette lookup --> 16-bit RGBA or 16-bit IA.

YUV = some other color format. Output is RGB (888) + Full alpha.