Baking Ambient Occlusion maps for graphics

Baking Ambient Occlusion Maps for Graphics

AO map—one of the most underestimated in PBR pipeline. Doesn't set color, doesn't define metallic, doesn't store surface relief. Does one thing: adds shadowing in places hard for light to reach—seams, grooves, contact zones. Without it model looks "plastic" and "flat" even with perfect normal map and good base texture.

In Substance Painter AO used as mask for procedural effects: dirt and wear accumulate in recesses, metal glints on protrusions. Without quality AO map these procedural masks work off low-poly geometry and give rough, unrealistic result.

What differs "correct" AO from automatic

In most tools AO baked with one button and default settings. Result—technically correct but practically inconvenient. Three specific problems:

Ray Distance. Parameter determining how far baker casts rays checking occlusion. Too large value—AO affects areas shouldn't be shadowed (different character parts shadow each other in T-pose). Too small—AO superficial, doesn't penetrate deep grooves. Correct Ray Distance picked individually per asset depending on its dimensions.

Ignore Backfaces. Off by default in most tools. Left off, back faces of interior surfaces (cavities, tubes, detail holes) cast shadows on exterior. For game models—almost always turn on. For cinematic and architectural render—depends if interior surfaces visible.

Self-shadow vs. Ground shadow. In Marmoset Toolbag can separate: object's own AO and environment AO (floor contact shadow). For weapon on white background need only self-shadow. For prop standing on surface, ground shadow useful—but only if it stays static in scene. Dynamic objects with baked ground shadow look unrealistic when moved.

AO baking settings in different tools

Marmoset Toolbag 4. In Bake → Maps → Occlusion: Ray Count 512–2048 (512 enough for production, 2048 for cinematics), Ray Distance—picked manually. Floor only for static objects. Ignore Backfaces—on. Jitter—on for banding reduction on flat surfaces.

xNormal. AO through Ambient Occlusion map: Number of Rays 256–512, Spread Angle 162 (standard), Bias 0.01–0.08 (larger bias—less darkening in tight angles—picked for visual result). Distribution: Cosine more realistic than Uniform.

Substance Painter. When baking in SP: Max Frontal Distance and Max Rear Distance set same as normal map. Ambient Occlusion in SP uses same cage as normal map—important to ensure cage settings identical. Otherwise AO and normal map misaligned.

Important Substance Painter nuance: AO map baked in SP used as reference for generator AO in layers. If it's poor quality—all layers with AO mask (Generator → Dirt, Wear, Cavity) give unrealistic results.

Combining AO with other maps

In modern PBR workflow AO rarely standalone—usually packed in other textures. Typical variants:

• ORM (Occlusion/Roughness/Metallic): R channel = AO, G = Roughness, B = Metallic. Unreal Engine packed textures standard.
• RGBA in BaseColor: sometimes A channel BaseColor used for AO multiplier, though less common.
• Separate map: for Substance Painter workflow where AO mask used, or manual compositing in engine.

With ORM important AO baked with correct bit depth: 8-bit enough for most game assets, 16-bit—for cinematic and archviz needing smooth gradients.

Common task specification mistakes

No context specified. AO for Substance Painter workflow (procedural generator mask) needs different settings than AO for packed ORM in engine. "Just bake AO"—not a spec.

No object staticness info. Floor / ground contact shadow baked only for non-moving objects.

"Remove all dark spots" expectation. AO by definition darkens recessed zones. If client wants "bright" AO—not about absent shadowing, it's about correct Ray Distance and Spread Angle. Discuss before starting.

Timeline guidelines

AO most often baked within full PBR bake (Normal + AO + Curvature + Thickness). Separate AO or full map set cost calculated individually.