Normal Map and Texture Map Baking

Baking Normal Maps and Texture Maps for Graphics

Baking is translating high-poly model geometric information into texture data. The difference between "just bake" and "bake correctly"—that's the difference between artifacts on every hard edge and a clean map that works under any lighting.

The process seems simple: load high-poly and low-poly, hit Bake in Marmoset Toolbag, get map. In practice 30% of time goes to setup, 40%—to fixing artifacts, and only 30%—to actual baking.

What artifacts arise and why

Shadowing artifacts on hard edges. Classic: normal map looks normal under direct lighting, but under side light dark stripes appear on edges. Cause—mismatch between smoothing groups on low-poly and UV seam boundaries. The rule that fixes this in 90% of cases: UV seam must match hard edge (smoothing group break). This is called matched UV/smoothing workflow and it's not optional, it's mandatory for correct baking.

Projection artifacts. When high-poly and low-poly volumes don't match, baker searches for nearest surface via ray casting and misses—especially on complex concave forms and thin elements. In Marmoset Toolbag this is solved through bake groups: each detail with large volume mismatch is set to separate group with individual cage.

Seams on normal map. Visible breaks along UV island boundaries. Cause—insufficient padding (2px instead of necessary 8px) or floating geometry that baker interprets ambiguously. In xNormal solved through Edge Padding post-process. In Marmoset Toolbag—through Dilation setting in Output parameters.

Tangent space mismatch. Normal map baked in one tangent space, engine uses another—result: incorrect lighting, highlights not where needed. Unity uses MikkTSpace, Unreal Engine—also MikkTSpace, but with UE4 vs UE5 nuances. Marmoset Toolbag and Substance Painter support tangent basis setting—must be set per target engine before baking.

Full stack and settings

Marmoset Toolbag 4—primary baking tool. Reason: Bake Groups with per-group cage, UDIM support, correct tangent basis for most engines, convenient real-time preview. Settings: Samples 256–512 for final bake, Dilation 32px for 4096 textures, Cage Offset individual for each group.

xNormal—for complex cases with custom cage and batch baking many objects. Faster than Toolbag on large scenes but weaker preview.

Substance Painter—when baking and texturing happen in one tool. Match by Mesh Name workflow: low-poly called Object_low, high-poly—Object_high. Painter auto-matches pairs and bakes separately. Critical for complex assets with dozens of details.

Standard bake maps: Normal Map (tangent space), Ambient Occlusion, Curvature, Position, Thickness, World Space Normal. Last three needed for procedural masks in Substance Painter. AO baked with Ray Distance and Ignore Backfaces settings—without this cavity details give incorrect shadows.

For characters with UDIM workflow baking is per-tile: each tile (1001, 1002, 1003...) baked accounting for its own geometry. Marmoset Toolbag 4 supports UDIM bake natively—one reason it became the character pipeline standard.

Work process

Input needed: high-poly mesh (FBX/OBJ), low-poly with ready UV, target engine info (for tangent basis), texture atlas size.

HP/LP pair check. Check volume correspondence, identify problem zones—thin details, sharp angles, concave forms.

Bake group setup. Divide model by surface type and geometry complexity. Floating geometry (rivets, overlays, details)—separate groups.

Test bake. At reduced resolution (512 or 1024) check all critical zones. Fix smoothing groups, move seams, correct cage.

Final bake. Full resolution, maximum sampling. Post-processing: edge dilation, filtering.

QA in engine. Maps checked in target engine or Marmoset Viewer under several lighting variants—key, side, backlight.

Timeline guidelines

Cost is calculated after source analysis.