AI colorization of black-and-white images with DeOldify and Stable Diffusion

AI colorization of archival footage: where the pipeline breaks

The archive is digitized, the frames are numbered — and that's where the smooth part ends. Frame-by-frame colorization of black-and-white images via DeOldify and Stable Diffusion delivers convincing results on a single frame: faces, sky, and vegetation look natural.

But when there are thousands of frames, an individual success stops meaning anything — all that matters is how evenly the color holds across the entire body of material.

Color artifacts show up specifically in the stream. The model reconstructs hue from luminance and context, so low-contrast areas and scene transitions pick up green and purple blotches, and object contours get halos.

Color balance drifts between adjacent frames, and in motion this reads as flicker: the viewer sees not a newsreel but an unstable picture.

The bottleneck here isn't the model itself but the absence of temporal consistency and a unified color policy. Each frame is processed independently: there's no anchor frame for the scene, no calibration against a reference, no check for hue jumps between neighboring frames.

Manual correction pulls the result through, but time grows along with volume — in archival restoration, this is the first thing that stops scaling.

We break down how this gets closed: how a batch image-processing pipeline is assembled, where per-scene color stability control is placed, and how to redo only the problematic frames instead of re-running the entire body of material.

Measurable effect: speed, share of manual correction, stability

Frame-by-frame colorization runs into manual labor: one pass per frame, eyeballing the result, fixing disputed frames in an editor. A managed pipeline turns this into a flow — a task queue, per-scene batches, parallel processing, and automatic quality control at the output.

Throughput grows from units of frames per hour to thousands per day, and the bottleneck becomes not the model but the acceptance of the result.

The share of manual rework is the second metric that determines whether the archive pays off. For each frame, model confidence and divergence from the scene palette are calculated; everything below the threshold automatically goes into a separate queue for correction.

On typical material the share of such frames drops from a third to a few percent, and corrections become pinpoint — skin, sky, documents, not the whole frame.

Scene color stability rests on processing not frame by frame but by whole scenes. The archive is cut into scenes, an anchor frame is selected for each, its palette is transferred to neighboring frames, and the boundaries are smoothed.

This eliminates an entire class of failures like "drifting" color and hue flicker within a scene — whole episodes used to be rejected because of them.

DeOldify in such a scheme provides a fast draft pass and a base palette, while a refinement diffusion model polishes the details.

The point isn't a specific model but the loop around it: at the output the client receives not a folder of images but a report on how many frames passed automatically and where color stability required an engineer's intervention.

Approaches to processing archival frames: comparing modes

The mode is chosen not by archive size but by the homogeneity of the corpus and the cost of an error on a specific frame. Batch processing is cheaper per frame, but a model failure multiplies across the entire export. On-demand processing gives control at every step, but barely scales.

Below are four colorization modes that projects actually choose between. The criterion is one: is there a reference against which color can be verified.

Comparison of colorization modes

In a pilot, 20–50 characteristic frames are taken and run through two or three modes — divergence in skin, sky, and uniform color is immediately visible. Then the mode is documented and fixed, and for recurring subjects fine-tuning of the model on your own set is added.

How the pipeline is built: implementation and verification stages

Archive colorization is not a one-off run of files through a model but a pipeline with checkpoints. At every step an artifact remains: a collection map, a set of settings, a reference sample — these show what changed if the result drifts.

1. Audit of the image archive. We break down the collection: frame types, volume, nature of defects — compression artifacts, grain, blown highlights, fading. The output is an archive map and a reference sample for acceptance.

2. Colorization pipeline design. We determine what goes through as a stream and what requires manual finishing: uniform series — in batches, portraits and rare subjects — in a separate branch.

3. Frame preprocessing. A unified color space, noise suppression, contrast equalization, damage restoration. On a dirty frame the model paints over defects and produces grime in the shadows.

4. Model selection. DeOldify holds stable color across large series, the diffusion model gives a lively result on individual subjects — we combine both branches.

5. Calibration on the reference sample. We select a palette for the era and subject, set rules for skin tone, sky, and uniforms. We check against the reference until the divergence becomes predictable.

6. Color quality control. Histograms, white balance, absence of "drifting" hues within a series. We check separately the frames where an object's color is inferred from context.

7. Acceptance and handover. The client reviews the sample, disputed frames go through a repeat pass. We hand over source files, settings sets, and a regulation for new arrivals.

After delivery the pipeline doesn't stop: new arrivals follow the same regulation, and accumulated corrections refine the model for the specific archive.

What we hand over: configurations and regulations

We hand over not "a model in a notebook" but a working colorization pipeline: artifacts your team can use to reproduce frame processing without our support. Below is exactly what goes into the repository and internal documentation.

• Frame processing scheme — a graph of stages from scan intake to the color file, with marked manual checkpoints and places where the result goes to a repeat run.
• Pipeline configuration files — image preparation parameters, profiles by subject type (portrait, landscape, city), confidence thresholds below which a frame is automatically flagged for operator acceptance.
• Launch regulation — deployment on your infrastructure: environment checks, model warm-up, a control run on the reference set, and the criteria by which the pipeline is considered correctly stood up.
• Archive access policies — roles of operator, acceptor, and administrator, separation of source files and results, retention periods, and the procedure for deleting materials.
• Processing quality log — metrics per batch: share of frames accepted on the first pass, distribution of typical defects, number of repeat runs after parameter corrections.
• Pipeline documentation — stages, external dependencies, and the areas where corrections are safe and where they break reproducibility of the result.
• Development guide — how to connect a new subject type or replace the colorization model without reassembling the whole process.

Everything is versioned together with the code: parameter changes are visible in the history, and reverting to a previous configuration takes minutes.

Case: restoring a newsreel, metrics before and after

The project — a newsreel archive: digitized film with grain, scratches, and unstable exposure. The material had been transferred to digital but remained black-and-white, and manual colorization of one minute took an artist about forty minutes.

With several hours of newsreel, this stretched into months and hit the archive's throughput ceiling.

What it was before moving to the pipeline

The manual pass gave accurate color on close-ups but scaled poorly: each frame was approved separately, the palette "drifted" between adjacent scenes, skin went green in the shadows.

About 18% of frames came back for rework, and the share didn't drop — corrections went in circles. Throughput held at 2–3 minutes of finished material per day.

What was changed

The first pass — DeOldify: it provides a base palette per frame and removes most of the routine. Then a diffusion model draws in textures and refines hues where the source has faded.

Between stages, automatic control runs: if a hue goes beyond the threshold relative to neighboring frames, the scene is flagged and goes to an operator as a whole, not frame by frame.

The palette is fixed by a single profile per scene, so cuts don't "flicker." After moving to the pipeline, the share of frames needing rework is 4%, the pass time for a minute is about six minutes, and archive throughput grew to 12–14 minutes per day. Manual work remained for rejection and scenes with mixed lighting.

How does the diffusion approach differ from the adversarial one?

An adversarial colorization network solves the task in a single pass: the generator predicts color from luminance, and the discriminator during training demands indistinguishability from a real color frame.

The model learns the most probable palette of the sample — hence the speed and the main risk: the hue is taken from the statistics of similar subjects, not from the meaning of a specific scene.

A diffusion colorization model takes a different path: the image is reconstructed from noise over dozens of steps, and each step can be guided by a text prompt.

This gives controllability — you set the scene palette and get a predictable hue instead of one averaged over the training set.

Resistance to film defects differs between the families. An adversarial network takes a scratch or a blown highlight for an object and confidently paints over it — the output is a colored stripe where the emulsion is damaged.

Diffusion relies on the statistics of the whole frame and suppresses such artifacts, but smooths grain: where grain is part of the historical texture, the strength of the effect has to be limited.

On a series of frames of one subject, an adversarial network is orders of magnitude faster, but the color drifts: frames are processed independently, and skin, sky, and uniforms shift in hue from frame to frame.

Diffusion, with fixed noise and a shared prompt, holds a single palette across the whole scene — at the cost of batch run time.

That's why choosing a model for an archive is not a debate about what's better but a decision based on the state of the material.

An even but faded array is more profitably run through an adversarial network in batch; collections with key frames and a requirement for historical accuracy — through a diffusion model with a prompt and selective manual review.

The difference between the families becomes visible at scale — that's where it's clear where speed turns into rework.

AI colorization: technical doubts and answers

Before a batch starts, clients ask the same set of questions: how historically accurate the color is, what to do with a mixed archive, how artifacts on faces are removed, and who owns the rights to the processing. Here are brief answers to each point.

What if the archive mixes different film stocks and scan qualities?

Images of different density, graininess, and contrast produce different tonality, so a single common setting for the whole collection doesn't work. We split the material into groups by these attributes, with its own processing profile for each. As a result the series looks uniform, without color jumps from frame to frame.

Why do colored blotches appear on faces and how do you deal with them?

The first DeOldify pass produces characteristic failures: bluish shadows on cheeks, halos along the hairline, lost lips and ears on small portraits. We cover such zones with masks and process them separately, with a repeat pass and manual finishing of disputed areas. Skin is brought to a natural tone, not a "plastic" one.

Who owns the rights to the source files and the processed images?

Processing runs in an isolated environment on your data: images don't go to external services and aren't used to train anyone else's models. Rights to the source files and to the result remain yours.

Fine-tuning on your material is possible — the model is tuned to a specific collection and never leaves your environment.

How is a batch accepted as a whole and can the palette be adjusted after the start?

Acceptance is by sample: we review control frames from each group together with you and record comments on tone and details. The palette is a configurable parameter, so corrections can be made at any stage and are recalculated uniformly across the entire batch.

Let's discuss your task and come back with an estimate

For a colorization project estimate to rest on data rather than assumptions, a few input materials are enough.

The first conversation is a discussion of the task directly with engineers: they look at the material and immediately say where automation will cope and where manual finishing is needed.

What helps speed up the review:

• 5–10 characteristic frames: portrait, shadow, mixed light, a close-up of skin — these are exactly where models fail most often.
• archive volume: number of files, formats, duration, folder structure — this determines the order of work and the scheme for storing intermediate versions.
• condition of the source files: grain, scratches, blown highlights, frame jitter — severe defects require a separate preparation stage.
• color requirements for the archive: references, brand palette, historical accuracy of costume or landscape.
• access constraints: closed environment, processing without exporting beyond your infrastructure, non-disclosure agreements.
• acceptance criteria: who approves the result and by what attribute a frame is considered ready.
• delivery order: one-off or in batches as processing proceeds, and how often you review intermediate results.

Then comes a trial run on a sample: engineers process 10–20 frames, show before and after, and break down typical failures — skin hue drift, false color on uniforms, artifacts on small details.

Based on this review we fix the share of frames for manual finishing and the composition of the stages.

Describe your task and send samples — we'll come back with a colorization project estimate and a work plan.