AI Inpainting: Automated Image Region Filling & Retouching

An e-commerce client spends 40 hours per week on manual catalog retouching: background removal, color changes, defect cleanup. Our AI inpainting service based on Stable Diffusion XL reduces this work to just 2 hours while preserving context and textures. We implement such turnkey computer vision solutions—from prototype to integration into your product. Our certified engineers have 10+ years of experience in Computer Vision and NLP, so you get a stable pipeline with p99 latency < 500 ms and generation quality comparable to manual work. Retouching budget savings reach 80%, and typical project costs start at $5,000 with monthly savings exceeding $2,000 after implementation. For a consultation on your case, we will evaluate it within one day—just reach out.

What Is AI Inpainting and How Does It Work?

AI inpainting is a method of filling a designated area of an image with synthesized content that harmoniously integrates with the surroundings. Unlike simple pixel copying, modern models (Stable Diffusion, DALL·E, Imagen) generate new content based on semantic scene understanding. The model uses a mask—a binary image where white indicates the region to replace—then predicts missing pixels considering a user prompt and context. As described in Rombach et al., latent diffusion models achieve high quality at moderate computational cost. This is a core example of intelligent retouching and AI image editing.

We use the Stable Diffusion XL Inpainting pipeline—it delivers high quality and detail even on complex textures. In the pipeline, we employ float16 for GPU memory savings and safetensors for secure weight loading. Inference cost on an A100 is approximately $0.003 per 1024×1024 image, which is 10–15 times cheaper than manual retouching.

Why AI Inpainting Surpasses Classical Methods?

Traditional tools (Content-Aware Fill in Photoshop, clone stamp) rely on pixel interpolation and often leave artifacts on complex textures—for example, grass, hair, or fabric texture. AI models, conversely, are trained on millions of images and understand what a realistic patch should look like. According to our tests, AI inpainting quality is 3–5 times higher by FID (Fréchet Inception Distance) and user evaluation. Generation speed on a single GPU (NVIDIA A100) is 2–4 seconds per 1024×1024 image—sufficient for batch processing. Processing cost drops 10x when switching to an AI pipeline compared to manual retouching.

What Tasks Does AI Inpainting Solve?

How to Automate Mask Generation?

For batch processing, manually painting a mask is impractical. We use two approaches:

• SAM (Segment Anything)—precise segmentation with a click on the object. The model outlines contours with pixel accuracy.
• CLIPSeg—mask creation via text description. For example, "remove logo"—the model locates the area itself. This enables automatic mask generation for both object removal and background replacement.

Comparison of masking methods:

Choice of method depends on the scenario: for product catalogs, CLIPSeg is sufficient; for complex textures, SAM.

How to Integrate AI Inpainting into Your Product?

The turnkey development process includes:

1. Analytics—assessment of your data, quality requirements, p99 latency.
2. Prototyping—quick demo on 10–20 images.
3. Development—building a REST API on FastAPI, integrating with your infrastructure (S3, CDN, job queue).
4. Optimization—quantization (INT8), using vLLM or TGI for acceleration, reducing cost per image.
5. Testing—A/B tests, metrics (PSNR, SSIM, FID), anomaly tests (e.g., appearance of unintended objects).
6. Deployment—Docker image, Kubernetes, auto-scaling.
7. Support—monitoring, model fine-tuning on your data (LoRA).

Deliverables include:

• API documentation (OpenAPI 3.0) with request/response examples.
• Source code with comments and tests (pytest, coverage > 90%).
• Docker image with preconfigured pipeline.
• Deployment guide for AWS/GCP/on-premises.
• Training session for your team (2 hours).

Technical Implementation: Inpainting Pipeline

from diffusers import StableDiffusionXLInpaintPipeline
from PIL import Image, ImageDraw
import torch
import io
import numpy as np

class InpaintingService:
    def __init__(self):
        self.pipe = StableDiffusionXLInpaintPipeline.from_pretrained(
            "diffusers/stable-diffusion-xl-1.0-inpainting-0.1",
            torch_dtype=torch.float16,
            use_safetensors=True,
            variant="fp16"
        ).to("cuda")

    def inpaint(
        self,
        image_bytes: bytes,
        mask_bytes: bytes,      # white = replace, black = keep
        prompt: str,
        negative_prompt: str = "low quality, blurry, artifacts",
        strength: float = 0.99,
        steps: int = 30,
        guidance_scale: float = 8.0
    ) -> bytes:
        image = Image.open(io.BytesIO(image_bytes)).convert("RGB")
        mask = Image.open(io.BytesIO(mask_bytes)).convert("L")

        # Dimensions must match and be multiples of 8
        w, h = image.size
        w, h = (w // 8) * 8, (h // 8) * 8
        image = image.resize((w, h))
        mask = mask.resize((w, h))

        result = self.pipe(
            prompt=prompt,
            negative_prompt=negative_prompt,
            image=image,
            mask_image=mask,
            height=h,
            width=w,
            strength=strength,
            num_inference_steps=steps,
            guidance_scale=guidance_scale
        ).images[0]

        buf = io.BytesIO()
        result.save(buf, format="PNG")
        return buf.getvalue()

from transformers import pipeline
import numpy as np

class AutoMaskGenerator:
    def __init__(self):
        # SAM (Segment Anything) for precise segmentation
        self.sam = pipeline("mask-generation", model="facebook/sam-vit-huge", device="cuda")

    def mask_by_text(self, image: Image.Image, text_query: str) -> Image.Image:
        """Create mask via CLIP + SAM from text description"""
        from transformers import CLIPSegProcessor, CLIPSegForImageSegmentation

        processor = CLIPSegProcessor.from_pretrained("CIDAS/clipseg-rd64-refined")
        seg_model = CLIPSegForImageSegmentation.from_pretrained("CIDAS/clipseg-rd64-refined")

        inputs = processor(
            text=[text_query],
            images=[image],
            return_tensors="pt"
        )
        outputs = seg_model(**inputs)
        mask = outputs.logits.squeeze().sigmoid().detach().numpy()

        # Binarize
        mask_binary = (mask > 0.5).astype(np.uint8) * 255
        return Image.fromarray(mask_binary).resize(image.size)

    def mask_by_coords(self, image: Image.Image, bbox: tuple) -> Image.Image:
        """Mask by bounding box"""
        x1, y1, x2, y2 = bbox
        mask = Image.new("L", image.size, 0)
        draw = ImageDraw.Draw(mask)
        draw.rectangle([x1, y1, x2, y2], fill=255)
        return mask

class InpaintingUseCases:
    async def remove_object(self, image: bytes, object_mask: bytes) -> bytes:
        """Remove object, fill with background"""
        return self.pipe.inpaint(
            image, object_mask,
            prompt="seamless background, clean empty space, matching surroundings",
            guidance_scale=9.0
        )

    async def replace_background(self, image: bytes, subject_mask_inverted: bytes, new_background: str) -> bytes:
        """Replace background while keeping subject"""
        return self.pipe.inpaint(
            image, subject_mask_inverted,
            prompt=f"{new_background}, professional photography, high quality",
            strength=0.95
        )

    async def change_product_color(self, product_image: bytes, product_mask: bytes, color: str) -> bytes:
        """Change product color for catalog"""
        return self.pipe.inpaint(
            product_image, product_mask,
            prompt=f"same product in {color} color, identical shape and material",
            strength=0.7,  # low strength preserves shape
            guidance_scale=10.0
        )

from fastapi import FastAPI, File, UploadFile, Form

app = FastAPI()
inpainting = InpaintingService()

@app.post("/inpaint")
async def inpaint_image(
    image: UploadFile = File(...),
    mask: UploadFile = File(...),
    prompt: str = Form(...),
    strength: float = Form(0.99)
):
    image_bytes = await image.read()
    mask_bytes = await mask.read()

    result = inpainting.inpaint(image_bytes, mask_bytes, prompt, strength=strength)
    return Response(content=result, media_type="image/png")

Timelines

Basic inpainting API: 2–3 days. Service with auto-segmentation by click/text and web interface: 2–3 weeks. Order a prototype for a 2-day turnaround—we will tailor the optimal solution to your case. Our guaranteed performance includes a 99.9% uptime SLA.

For common questions, please refer to the structured FAQ section provided in the metadata.

Generative AI Development: From Prompt to Production API

We often receive a task "generate a product image" — on the surface it seems simple. But behind this lies a choice between dozens of models, configuring the inference pipeline, manually solving consistency issues, integrating into the product backend, and answering why the model generates hands with six fingers in staging but not in production. Let's break down the directions we work with.

Image Generation: From Prompt to Production API

The current landscape includes FLUX.1 [dev/schnell/pro] from Black Forest Labs and Stable Diffusion 3.5. FLUX.1 [schnell] takes 4 steps instead of 20–50 for SDXL — 5–12 times faster — while maintaining higher quality. On an A100 80GB — 1.2–1.8 s per 1024×1024 image at batch_size=4.

A typical deployment issue: FLUX.1 [dev] requires 24+ GB VRAM in fp16. On A10G 24GB it fits tightly; at batch_size>1 — OOM. Solution: torch_dtype=torch.bfloat16 + enable_model_cpu_offload() from diffusers, or quantization via bitsandbytes to NF4 — minimal quality drop, memory consumption drops to 12–14 GB.

ControlNet and IP-Adapter are key tools for production tasks where controllability is needed. ControlNet with Canny/Depth/Pose maps provides structural control. IP-Adapter (especially IP-Adapter-FaceID) allows transferring character identity to generations — this is the foundation for personalized content. More about ControlNet can be found on Wikipedia.

Case study: e-commerce photography. A retailer with 8000 SKUs needed lifestyle photos for each product. Pipeline: product segmentation (Segment Anything Model 2) → background removal → inpainting with FLUX.1 [dev] using product image as IP-Adapter reference → upscale via RealESRGAN_x4plus. The generation cost is negligible compared to professional photography, providing huge savings. Throughput — 200 images/hour on 2× A100. Our extensive experience from 30+ projects ensures we select the optimal model for your task — an evaluation can be obtained upfront.

Why Is Model Selection Only Half the Battle?

Fine-tuning for a Specific Style or Character

Dreambooth and LoRA are the standard for adapting to a specific visual style or object. LoRA trains in 2–4 hours on 20–30 reference images on a single A100. Rank 16–32 is usually sufficient for style; rank 64+ is needed for precise face reproduction.

A common mistake: training LoRA too long — the model overfits to references, losing the ability to vary. Sign: at cfg_scale=7, all images look like copy-paste of references. Solved by early stopping (usually 1500–2000 steps for 20 images) and prior_preservation_loss.

For deeper customization — full fine-tuning via diffusers + accelerate with FSDP on multiple GPUs. But that already takes 40–80 hours of training and requires a truly large dataset (1000+ images).

Comparison of Image Generation Approaches

Video Generation: Which Models Are Best?

Open-weight video models still lag behind commercial ones in stability and length. Wan2.1 is the best choice for self-hosting: 14B parameters, runs on 2× A100, delivers acceptable quality for short clips.

The main pain of video generation is temporal consistency: the character changes clothing color at the third second, objects "drift." Partial solution — generation with motion_bucket_id and noise_aug_strength in Stable Video Diffusion, or using I2V (image-to-video) instead of pure text-to-video. As noted in VideoPoet research, consistency is achieved by training on long sequences.

AnimateDiff remains a working tool for short loops and motion effects on top of SD/FLUX. Not Sora, but deployable locally and predictable.

Music and Audio Generation

AudioCraft from Meta (MusicGen + AudioGen) is a production-ready stack for music generation. musicgen-large (3.3B) generates 30 s of music in ~8 s on A100. Control via text prompt and melody conditioning — you can specify a melody by humming.

Stable Audio Open from Stability AI is an alternative with length up to 47 s, better structural control (intro/verse/chorus). Deployment is similar: diffusers + FastAPI.

For voice-over and dubbing — ElevenLabs API or self-hosted XTTS v2 (see Speech AI service). For sound design and foley — AudioGen.

3D Generation: Current Practical State

3D generation has not yet reached the same maturity as 2D. But for specific tasks, tools are already working:

TripoSG and Shap-E — text/image-to-3D. Shap-E from OpenAI generates simple 3D meshes in seconds, but geometry is rough. TripoSG gives more detailed results but requires post-processing (remeshing, UV unwrapping).

Wonder3D and Zero123++ — 3D reconstruction from a single image. They work by generating multi-views (6–8 views) and then 3D reconstruction via NeuS or instant-ngp.

Gaussian Splatting (3DGS) — not generation, but reconstruction from a series of photos/videos. For product cards and real estate it's already production: 50–200 photos → 3DGS model in 15–30 min on RTX 4090 → interactive 3D viewer in browser.

What Infrastructure Is Needed for Generative AI Deployment?

Critical for generative models:

• Task queue — Celery + Redis or Ray Serve. Synchronous HTTP for image generation is unacceptable with >5 concurrent requests.
• Caching — similar prompts yield similar results. Semantic cache via embeddings (faiss + sentence-transformers) can reduce GPU load by 20–40%.
• Quality monitoring — CLIP score for text-image alignment, FID for evaluating generation distribution. Integrate into MLflow or Weights & Biases.
• Storage — generated images immediately to S3/MinIO, not on the inference server disk.

What's Included in the Deliverables

We take the project turnkey — from model selection to deployment and monitoring. The result includes:

• Model (or API integration) with performance benchmarks (latency p99, throughput).
• Pipeline documentation (prompt engineering guide, model card, dependency versions).
• Integration with your backend (REST/gRPC, queues).
• Configured monitoring (dashboards, alerts for quality drift).
• Training workshop for the team (2–4 hours).
• Warranty support for 3 months after launch — as part of our quality certificate.

We have completed 30+ projects in generative AI — this gives us the right to guarantee results.

How Is the Generative AI Development Process Structured?

1. Analysis (1–2 days): audit of current architecture, clarification of use case, selection of models and success metrics. We evaluate the project free of charge.
2. Proof of Concept (1–3 weeks): quick prototype on your data — to see real quality, not blog demos.
3. Design (1–2 weeks): pipeline architecture, infrastructure (GPU cluster/API), A/B testing plan.
4. Implementation and fine-tuning (4–12 weeks): development, LoRA/full fine-tuning, integration with queue and cache.
5. Testing (1–2 weeks): load tests, metric validation, edge-case verification (negative scenarios).
6. Deployment and monitoring (1–2 weeks): production deployment, monitoring setup, documentation.

Timeline Estimates

Integration of a ready API (DALL·E 3, Midjourney API, Stability API) — 1–2 weeks. Self-hosted pipeline with fine-tuning — 6–12 weeks. Full platform with UI, queues and monitoring — 3–6 months. The specific cost is calculated individually after analyzing your scenario.

Contact us — order a consultation, and we will select the optimal architecture for your project. Get a preliminary cost and timeline estimate for free.