Media File Migration: rsync, S3, and Automatic URL Updates

When migrating a site with a 15 GB media library and a database of 60,000 records, manual copying inevitably leads to losses. Ten years of experience show that automation with rsync and S3 cuts transfer time by three times and eliminates errors. Without automation, you risk losing data and traffic. We have developed a proven process that includes inventory, delta synchronization, cloud upload, and automatic link updates. This guarantees file integrity and zero downtime. Contact us for a project consultation to assess the scope of work.

What Problems Do We Solve?

Broken links are a common issue. After migration, up to 20% of links remain broken if all occurrences of the old URL are not updated. We replace them automatically, updating content and metadata. Uncontrolled traffic growth is another problem: without image optimization, download sizes remain large. We convert JPEG and PNG to WebP, reducing size by 60% without quality loss. This reduces server load and lowers hosting costs by 30%. Traffic savings reach 60% through CDN caching. Duplicates and garbage: inventory reveals files on disk missing from the database, and vice versa. We delete excess, saving up to 30% of disk space.

How We Do It: A Case Study from Our Practice

For our client with a WordPress site (8 GB media, 40,000 records), we performed migration to a new hosting. First step—inventory:

# Summary by size and type
find /var/www/uploads -type f | awk -F. '{print $NF}' | sort | uniq -c | sort -rn
du -sh /var/www/uploads

Next—synchronization via rsync. We ran it multiple times, using --checksum for accuracy. Synchronization took 2 hours instead of 12 hours with manual copying.

# Initial synchronization (can be run multiple times)
rsync -avz --progress --checksum user@old-server:/var/www/uploads/ /var/www/new-site/uploads/

# Delta synchronization before final switch
rsync -avz --delete user@old-server:/var/www/uploads/ /var/www/new-site/uploads/

After that, we uploaded files to S3 for CDN delivery, reducing TTFB by 40%:

aws s3 sync /var/www/uploads/ s3://company-media-bucket/uploads/ --storage-class STANDARD --exclude "*.tmp" --acl public-read

Link updates were done with a Python script that processed 12,000 records in 3 minutes:

import mysql.connector

def update_media_urls_in_content(db_conn, old_base, new_base):
    cursor = db_conn.cursor()
    tables_columns = [('posts', 'content'), ('posts', 'excerpt'), ('pages', 'body'), ('users', 'avatar_url')]
    for table, column in tables_columns:
        cursor.execute(f"SELECT id, {column} FROM {table} WHERE {column} LIKE %s", (f'%{old_base}%',))
        for row_id, content in cursor.fetchall():
            if content:
                new_content = content.replace(old_base, new_base)
                cursor.execute(f"UPDATE {table} SET {column} = %s WHERE id = %s", (new_content, row_id))
    db_conn.commit()
    print(f"Updated {cursor.rowcount} rows")

update_media_urls_in_content(db_conn, 'https://old-site.com/wp-content/uploads', 'https://cdn.new-site.com/uploads')

For WordPress, we additionally executed SQL queries and configured 301 redirects in nginx:

location ~* ^/wp-content/uploads/(.*)$ {
    return 301 /uploads/$1;
}

Comparison of Methods and Formats

WebP saves up to 70% space.

rsync is 5x faster than FTP and ensures integrity.

Work Process

1. Analytics—inventory of files and database relationships.
2. Design—choose copy strategy, redirects, and optimization.
3. Implementation—sync, cloud upload, URL updates.
4. Testing—verify integrity, check all links.
5. Deploy—configure 301 redirects, switch DNS.

How to Minimize Downtime During Migration?

Using rsync with delta-sync allows migration without taking the site offline. The final switchover takes minutes. Setting up 301 redirects ensures users do not see broken links. Thus, downtime is virtually eliminated. Contact us for precise planning.

What's Included in the Work?

• Detailed media library audit: report by file types, duplicates, unused resources.
• Writing and executing sync scripts (rsync, S3 CLI).
• Automatic URL updates in all posts and meta fields (WordPress, custom tables).
• Configuration of 301 redirects for old paths.
• Image optimization: conversion to WebP, lossless compression.
• CDN integration (CloudFront or similar) with caching.
• Documentation of the new media storage architecture.
• Technical support for one week after migration.

File Integrity Guarantee

After copying, we run verification: compare md5 hashes of original and copied files. If mismatch, the file is copied again. We also check that all files from the database exist on disk. This eliminates broken files and guarantees full correspondence. Additionally, rsync can be run with the --checksum flag for ongoing verification.

Why Use CDN for Media?

CDN reduces server load and speeds up page loading. We upload files to S3 and connect CloudFront. This reduces TTFB by an average of 40%. Moreover, traffic is saved by 60% through edge caching. When choosing a CDN, consider audience geography and egress costs. For Russian users, Cloudflare or Selectel CDN are optimal. For international, CloudFront or Fastly.

Common Mistakes in Media File Migration

Copying without checksum verification risks broken files. Ignoring files with no database links leads to disk clutter. Lack of redirects loses traffic from old URLs. Skipping meta field updates (e.g., thumbnails in WordPress) breaks site functionality. Omitting image optimization increases traffic costs. Our engineers with ten years of migration experience have successfully transferred over 500 projects. Contact us for a project consultation to assess the scope of work.

Timelines and Cost

Timelines depend on data volume. For a site up to 10 GB—2–3 business days. For larger projects—5–7 days. Cost is calculated individually after analysis and includes support for one week after migration.

Website Redesign and Migration: CMS Change, SEO Preservation

A client came to us 6 weeks after a self-attempted redesign: 'We moved from WordPress to Tilda, traffic dropped by 70%.' I opened Google Search Console — 847 pages returned 404, the URL structure had completely changed, not a single 301 redirect was in place. Yandex hadn't reindexed the new site yet, positions collapsed. Recovery took 4 months and resulted in significant revenue loss for the quarter. Our experience — over 7 years and 80+ successful migrations, we guarantee position retention with the right approach.

Why Do Migrations Break SEO?

Search engines have indexed specific URLs. If /catalog/shoes/nike-air-max-270 turned into /products/nike-air-max-270 without a 301 redirect — all the link equity, traffic, and rankings go nowhere. Google says 301 passes ~99% of PageRank, but in practice positions recover over 2–8 weeks, not instantly.

Commonly, SEO gets broken not out of malice, but because a developer doesn't view the URL structure as a public API. Here are typical breakages:

How to Recover Traffic After a Failed Migration?

If traffic dropped, act immediately:

1. Crawl the new site for 404s and compare with the pre-migration URL list.
2. Create redirects for all lost pages with traffic >0.
3. Check structured data and meta tags on a test sample.
4. Daily monitor Coverage in Search Console and positions for top 50 queries.
5. If after 2 weeks traffic does not recover — deep audit of redirects (transitivity, chains, loops).

In our practice, a large e-commerce site lost 50% of traffic when moving from Bitrix to React + Strapi. We restored 95% of redirects in three days, and within 3 weeks traffic returned to 90% of original.

What Does a Pre-Migration Audit Include?

Before starting development on the new site:

1. Full crawl of current site via Screaming Frog or Sitebulb. Get list of all indexable URLs with traffic from Google Search Console.
2. Export all pages with organic traffic >0 over the last 6 months — these are priority for redirects.
3. Record all external backlinks to specific pages — Ahrefs, Semrush.
4. Snapshot current positions for key queries — baseline for post-migration comparison.
5. Save Core Web Vitals from Search Console for the previous 90 days.

Table for recording:

Contact us for a detailed pre-migration audit — we will help identify all risks and create an action plan.

URL Mapping and Redirects

For projects with 200+ pages, we create a mapping table: old URL → new URL → status (301, merged with another page, deleted). Each row is verified: does the content actually migrate here?

In Laravel, redirects are handled via configuration file and middleware, not .htaccess — faster and more manageable. For WordPress → Next.js: redirects are set in next.config.js (static) and at the Nginx/CDN level for dynamic ones. Old .htaccess on shared hosting with 500+ lines of redirects is a special hell. Each redirect is checked sequentially, performance suffers. We move to Nginx map directive or Redis cache for dynamic lookup. More at Wikipedia: HTTP 301.

How to Migrate Content from Different CMSs?

WordPress → Headless CMS (Contentful, Strapi, Sanity): WordPress REST API or WP All Export to export posts, meta fields, media files. Migration script in Node.js: parse export, transform structure, upload via CMS API. Media files are reuploaded to new storage, links updated in content. Typical problem — shortcodes in WordPress content ([gallery id="123"]): need parser and transformation to new format.

1C-Bitrix → modern stack: Bitrix stores content in non-standard tables with IBLOCK_ELEMENT_PROPERTY. Direct SQL export via phpMyAdmin or Bitrix API. Transformation is the longest part due to specific Bitrix data structure.

Heavy WYSIWYG → structured content: Years of editing in FCKEditor/TinyMCE leave inline styles, non-standard tags, broken attributes. HTML sanitize + transformation to Markdown or Portable Text (Sanity) with manual check of problematic pages.

How to Preserve Technical SEO Elements During Migration?

Structured data (Schema.org) — if the old site had Product, Article, BreadcrumbList markup, they must be on the new site too. Google Search Console → Enhancement reports will show loss of rich snippets.

Sitemap XML: generated automatically, submitted to GSC a day after launch. Old sitemap remains until full reindexing.

hreflang for multilingual sites: if tags are lost during migration, conflicts between language versions in search results will start within weeks.

Open Graph and Twitter Card meta tags — often forgotten when changing template, pages stop displaying correctly when shared on social networks.

Launch and First Weeks Monitoring

DNS propagation: DNS switching takes up to 48 hours, plan launch with buffer. Cloudflare as DNS provider — propagation takes minutes, not hours.

After launch, monitor daily: Search Console → Coverage (indexing errors), Analytics → organic traffic, year-over-year comparison, crawl site for 404 errors.

First 2 weeks are critical. If traffic drops more than 30% — immediate audit of redirects and comparison with pre-migration crawl.

What the Service Includes

Results you receive:

1. Migration plan with URL mapping and redirects in Excel/Google Sheets format.
2. Configured 301 redirects at server level (Nginx/Cloudflare/Vercel).
3. Migrated content with integrity check: images, meta fields, links.
4. Structured data (Schema.org) on the new site, identical to old or improved.
5. SEO report: position trend at 1, 3, and 6 weeks after launch.
6. Coverage monitoring in Search Console with error notifications.
7. Guaranteed position retention: if traffic drops more than 15% within the first month — free audit and correction.

Timelines and Estimates

• Redesign with migration for a small site (up to 100 pages): 4–8 weeks.
• E-commerce migration with 500+ product pages: 8–16 weeks.
• Only technical migration part (redirects, metadata) without redesign: 1–3 weeks.

Cost is calculated individually based on scope.

Get a consultation for your project — we will respond within a day. Order a pre-migration audit of your site and receive a detailed proposal with a redirect plan. Contact us to discuss details.