Why AI super resolution produces artifacts on real photos
Why does upscaling look clean on a test set but fall apart on a real photo?
Because the model is trained on pairs of “original — programmatically downscaled frame.” During downscaling, noise and grain are averaged out, high frequencies are suppressed predictably, and the network learns to reverse exactly that degradation.
In a real frame, noise is not averaged: it lies in the same frequency range as fine detail. AI super resolution does not separate grain and texture when their spectra overlap, so it amplifies both the useful signal and noise at the same time.
As a result, image upscaling brings to the surface color speckles and large blotches that the eye barely noticed in the source.
The second class is upscaling artifacts on contours. The network predicts sharpness from the statistics of training pairs, and on high-contrast boundaries — highlights, metal, dark text on a light background — light and dark halos appear.
Alongside it there is the opposite effect: fine textures (grass, fabric, skin pores) fall into the area the model considers noise and are cleaned away. This is loss of detail — resolution has increased, but no new information has been added to the frame.
In a series of frames, color divergence is added. Each frame is processed independently, so in a static scene the hue and contrast breathe slightly, and when stitched together this reads as flicker.
In this case, noise and grain in the photo must be separated from detail before upscaling, rather than fighting the consequences afterward: a noise profile for the specific camera, limiting gain on contours, temporal consistency between neighboring frames.
That is why we start not with choosing a model, but with analyzing the material — what was shot, with what, and what in the frame counts as useful detail. Below we break down how this is addressed.
What changes after implementation: timelines, quality, manual editing
Before implementation, a batch of 300–500 frames takes several working days: an engineer opens each file, assesses noise and softness, selects the upscaling factor, and manually refines edges and small text.
Publication of a catalog or delivery of material waits until the entire chain finishes, and an unsuccessful source file entails a reshoot and pushes the timeline even further.
In the first passes after launch, what changes is not an individual frame but the pipeline. Upscaling on a model of the Real-ESRGAN class raises the image to 8x and restores fine texture — fabric fibers, type on packaging, foliage.
Batch mode processes a batch in parallel, so the same frames take tens of minutes, and upscaling quality is accepted based on a control sample, not by eye.
Once parameters are calibrated for a specific type of shooting, the main metric emerges — the share of frames without manual refinement.
In product and interior photography it stays around 85–90%; editing remains for isolated frames with a complex background or heavy noise in the source.
The defect class “insufficient resolution” stops requiring a reshoot: instead of a repeat shoot, it is enough to pull the archive file and run it through the same pipeline.
Then planning changes. Processing throughput stops depending on the number of people — the stream is scaled in batches, shooting and preparation run in parallel, and the timeline for releasing material is determined by the shooting schedule, not by manual refinement capacity.
Comparison of modes: batch, streaming, and individual frames
The processing mode determines not only speed but also reproducibility of the result. With batch image processing, there is a complete file list and a repeatable run: the same archive can be reprocessed and compared with the previous one.
In streaming frame processing, the result depends on arrival order — after a failure, some frames are recovered from the buffer.
Selective upscaling is a compromise between volume and quality: a rule selects frames by source sharpness, type, and size, and upscaling is not spent on material where it will add nothing.
In practice, archive processing modes are combined: the accumulated stock goes in batches, new arrivals in a stream, and disputed areas as individual frames.
| Mode | When it fits | Where the limit is |
|---|---|---|
| Batch archive processing | Accumulated stock, reproducibility and repeat run needed | Peak load on storage; not suitable for material that is continuously replenished |
| Streaming frame processing | Continuous arrival, throughput and low latency matter | Stream interruption on failure; a buffer and resubmission rules are needed |
| Selective upscaling | Quality control, varying frame value, limited resources | Selection requires rules; as volume grows it turns into manual work |
The limitation is common to all modes: upscaling does not restore details that are not in the source — it removes compression artifacts and restores structure.
Therefore, before choosing a mode, we examine the source material: where 8x produces a result and where the limit is reached already at 2-4x.
Implementation stages: from archive audit to handover of regulations
The stages proceed strictly sequentially: upscaling quality is determined not by the model but by the condition of the source files and agreed acceptance criteria.
Until the archive is labeled, any assessment of the result remains subjective — so we start with inventory, not mass processing.
- Image archive audit. Inventory: formats, compression artifacts, duplicates, corrupted and recompressed files. For each class we record which details are recoverable and what will have to be replaced.
- Upscaling quality criteria. We define measurable benchmarks: contour sharpness, absence of “plastic” textures, readability of small text, correctness of faces. We separately describe unacceptable defects.
- Processing pipeline assembly. Preprocessing, upscaling, denoising, normalization to a single format and color profile. Parameters are versioned so the next batch is processed the same way.
- Check on a control sample. We select the heaviest and most typical frames, compare before/after pairs, adjust thresholds, and fix the reference result.
- Result acceptance. We hand over metrics and image pairs to the customer, who checks them on their own sample; disputed frames are reviewed individually, not by the archive average.
- Handover of regulations. Pipeline description, parameters, instructions for launching new batches, and acceptance rules — the team continues work without us.
Each stage is closed with a written artifact: a report, a reference pair, a configuration. This makes acceptance predictable and repeat batches reproducible.
What you keep: configurations, diagrams, access, regulations
We do not hand over the system as a “black box”: after launch, your team keeps a set of artifacts that make it possible to operate, evolve, and transfer the solution to another team without reverse engineering.
- Working configurations — files with parameters for the entire processing path: which upscaling factor to apply to which image type, which setting sets are enabled for batch and single modes.
- Processing pipeline diagram — the path from file intake to result output: stages, branches, behavior on error at each step, where the source is stored and where the finished image is stored.
- Data access policies — which employees and services can see source files, who can change settings, which data does not leave your perimeter, and how this is verified.
- Operation logging — for each image you can see the time, parameters, settings set version, and result; logs are stored separately and are not cleaned up together with working files.
- Image processing regulations — how to accept a batch, by what criteria to check quality, what counts as a defect, and what to do if sharpness drops.
- Shift instructions — step-by-step guides for operators: typical operations, diagnostics of common failures, escalation procedure.
- Update procedure — how to change the model or setting sets without stopping the pipeline and how to roll back to the previous version if quality declines.
We hand over everything together with acceptance: engineers go through each item on your data so that the artifacts do not remain a formality.
Case study: photo archive processing and moving away from manual refinement
The customer came with an archive of several tens of thousands of photos: some digitized from film, some shot on early cameras, some received from partners.
For catalog and print, such frames lacked resolution, and before publication they were refined manually in a graphic editor — on average about 12 minutes per frame.
In the workflow this turned into a bottleneck: archive processing stretched over months, and 35% of images were still deemed unsuitable after manual refinement — noise, artifacts, loss of small text and texture during upscaling. Such frames were returned to the queue or dropped from output entirely.
We addressed the archive upscaling case with a batch pipeline: upscaling to 4x and 8x based on Real-ESRGAN with the frame split into overlapping fragments — no seams appear at the boundaries.
Before upscaling, denoising and color balance correction are performed; after, automatic sharpness and artifact checks: only frames below the threshold go to a human.
Implementation result: the share of unsuitable images decreased from 35% to 5%, manual refinement remained for isolated frames instead of all, and time per photo went from 12 minutes to 40 seconds of selective review by an operator.
Throughput increased more than 20-fold: what previously took weeks of designers’ work is now completed in a shift, and new arrivals are processed by the same rules.
How super resolution differs from ordinary frame upscaling
Ordinary frame upscaling is image interpolation: a bilinear, bicubic, or Lanczos filter stretches existing pixels over a larger grid and averages neighbors. The size grows, boundaries blur, and no new information appears.
Super resolution solves the inverse problem: the model is trained on “low resolution — high resolution” pairs and reconstructs details from the statistics of similar images.
| Option | What it does to the frame | Upscaling limit | Result reliability |
|---|---|---|---|
| Interpolation | Averages neighboring pixels | 2x, beyond that blurring is noticeable | Details are not added |
| Single-image super resolution | Reconstructs details using a trained model | 4x on typical scenes | High on familiar object classes |
| Generative restoration | Synthesizes plausible structure | 8x and above | Requires control: artifacts are possible |
The boundary between restoring details and inventing them lies in the predictability of the structure.
The texture of asphalt, a strand of hair, or a letter in a known font are restored reliably by generative models such as Real-ESRGAN — they are backed by thousands of similar examples in training.
A unique object, a serial number, or a one-off crack has no such support, and the model draws something plausible instead of something correct.
That is why the upscaling limit is set by the task, not by the name of the mode: where plausibility is enough, and where factual accuracy is needed.
We compare the result with the original and check whether extra details have appeared in the frame, and on object classes where invention is unacceptable, we keep a conservative mode and confirm quality on the customer’s validation set of frames.
Technical concerns worth resolving before work starts
Concerns usually relate not to the model but to its behavior on a real archive: noise, compression traces, heterogeneous formats, thousands of files in a batch.
Therefore, before the start we fix the processing scheme and acceptance criteria — otherwise the result is hard to accept.
Below are the questions engineers ask most often.
What should be done with noisy source files?
Noise cannot be fed into upscaling directly — the model will amplify it along with details. So we add denoising with adjustable strength to the scheme: we barely touch weak grain, remove strong grain, while controlling loss of texture.
Does upscaling help on heavily compressed files?
Partially: blockiness and “ringing” around contours are inherited and become more pronounced during upscaling. We first restore artifacts and limit the factor — above the limit that the source can sustain, it is not detail that grows but mush.
Can the entire archive be processed in batch, without selection?
Technically yes, but some frames will get worse: where there are no details, the model draws something plausible. Before batch processing, we calculate sharpness metrics and split the array — strong frames go to the full factor, weak ones to a gentle one.
Will you deploy the solution in our environment?
Yes: processing runs on your hardware, images do not go to external services, and the closed segment works without the internet. We hand over configurations, launch scripts, and maintenance instructions.
Let’s discuss the task: what to send to estimate the scope of work
An accurate estimate of the scope of work relies not on a general description like “we want better quality” but on input data. The more specific the source files and requirements for the result, the fewer assumptions remain in the implementation plan.
Below is the minimum sufficient for us to come back with an estimate, processing scheme, and timeline.
- Sample photos — 10–20 files from different sources, including the “heaviest” ones: heavily compressed, blurred, with noise or compression artifacts. They show where upscaling to 2x is sufficient and where an 8x limit and a separate processing branch are needed.
- Requirements for the result — what is considered suitable: no artifacts at object boundaries, preservation of text and small details, acceptable sharpness. It is better to attach 2–3 reference before/after pairs so the criterion is not just words.
- Archive volume — number of images, average size, formats (JPEG, PNG, TIFF, RAW), share of corrupted and duplicate files. This determines the batch processing scheme and queue order.
- Data placement constraints — processing in your environment or on dedicated servers, isolation requirements, retention period for originals and finished files.
- Use case for the result — print, product cards, panoramas, video. The target resolution and output format depend on this.
- Desired launch time and mode — one-time archive processing or a continuous stream. For a stream, we separately discuss upload automation and quality control.
- Acceptance criterion — who accepts the result and on which control sample, so that disputed situations do not arise at delivery.
Let’s start with a discussion of the task: describe it and attach what you have — we will come back with an estimate, implementation plan, and clear control points.
