Developing an Online Video Editor: Browser-Based Editing and Rendering
When a client requests browser-based video editing, the first hurdle is rendering the final output. Processing a one-minute clip on the client via FFmpeg.wasm works, but a ten-minute video will freeze even on a powerful laptop. We've tackled this on multiple projects and developed an architecture that scales. This article covers key architectural decisions: render type, timeline implementation, and file uploads.
Browser vs Server: Where to Render?
The choice between client-side and server-side rendering defines the entire stack and user experience. Here's a comparison:
| Parameter | Client-side (WebCodecs + FFmpeg.wasm) | Server-side (Remotion + FFmpeg) |
|---|---|---|
| CPU dependency | User's CPU | None |
| Max video duration | ~2 min | No limit |
| Rendering time for 10 min (i7 CPU) | 5–15 min | ~2 min on GPU |
| Scaling | One video at a time | Parallel tasks (Lambda) |
| Browser support | Chrome 94+, Firefox 130+, Safari 16.4+ | Any browser (file via link) |
| Infrastructure cost | Zero | Server costs (up to 40% savings under high load) |
Client-side rendering (WebCodecs API + FFmpeg.wasm): No server power needed; all load on the user's CPU. FFmpeg.wasm renders 10 minutes of video in 5–15 real-time minutes (depends on CPU). WebCodecs API is supported in Chrome 94+, Firefox 130+, and Safari 16.4+. Ideal for short clips up to 2 minutes—no delays, no server costs.
Server-side rendering (Remotion + FFmpeg): Rendering on GPU servers; the user gets a ready file via link. Remotion describes videos as React components, offering flexibility and reusability. Scales via AWS Lambda—rendering in parallel chunks. For a commercial product with videos longer than 2 minutes, this is the only reliable path. In our projects, we usually choose server-side: it gives predictable rendering times and doesn't drain the user's battery. But for a minimal editor for short clips, the client-side option is faster and cheaper.
Remotion official documentation: https://remotion.dev/docs/
How Is the Timeline Structured?
The central UI concept is a timeline with tracks. The data structure looks like this:
interface VideoProject {
id: string;
duration: number; // seconds
fps: number; // 24 | 30 | 60
width: number;
height: number;
tracks: Track[];
}
interface Track {
id: string;
type: 'video' | 'audio' | 'text' | 'image' | 'effect';
clips: Clip[];
muted: boolean;
locked: boolean;
volume: number; // 0–1
}
interface Clip {
id: string;
trackId: string;
assetId: string; // ссылка на загруженный файл
startTime: number; // позиция на таймлайне (секунды)
duration: number; // длительность клипа
trimStart: number; // обрезка начала исходного файла
trimEnd: number; // обрезка конца
speed: number; // 0.25 – 4.0
opacity: number;
transform?: ClipTransform;
filters?: VideoFilter[];
}
Data is stored in a state manager (Zustand or Redux) and synced with the database on save. Drag-and-drop is implemented via @dnd-kit—it reliably handles clip movement across tracks.
How to Split a Clip on the Timeline?
Split is a basic operation. Here's the step-by-step algorithm:
- Determine the split point in seconds relative to the timeline (e.g., when dragging the marker).
- Calculate splitPoint = (atTime - clip.startTime) + clip.trimStart.
- Create two new clips: left with trimEnd = splitPoint and right with trimStart = splitPoint.
- Delete the original clip and add the new ones.
Code:
const splitClip = (clipId: string, atTime: number) => {
const clip = getClip(clipId);
const splitPoint = atTime - clip.startTime + clip.trimStart;
const leftClip: Clip = { ...clip, id: uuid(), duration: atTime - clip.startTime, trimEnd: splitPoint };
const rightClip: Clip = {
...clip,
id: uuid(),
startTime: atTime,
duration: clip.startTime + clip.duration - atTime,
trimStart: splitPoint,
};
removeClip(clipId);
addClip(leftClip);
addClip(rightClip);
};
Browser Preview
For preview, we use HTML5 <video> with synchronization via currentTime. Here's a key hook:
const PreviewPlayer: React.FC = () => {
const { currentTime, isPlaying, tracks } = useEditorStore();
const videoRefs = useRef<Map<string, HTMLVideoElement>>(new Map());
useEffect(() => {
// Синхронизируем все видео-клипы с таймлайном
tracks.forEach(track => {
track.clips.forEach(clip => {
const video = videoRefs.current.get(clip.id);
if (!video) return;
const clipTime = currentTime - clip.startTime;
const isActive = clipTime >= 0 && clipTime <= clip.duration;
video.style.display = isActive ? 'block' : 'none';
if (isActive) {
const targetTime = clip.trimStart + clipTime * clip.speed;
if (Math.abs(video.currentTime - targetTime) > 0.05) {
video.currentTime = targetTime;
}
isPlaying ? video.play() : video.pause();
} else {
video.pause();
}
});
});
}, [currentTime, isPlaying]);
// ...
};
Scrubbing on the timeline is done via mousedown/mousemove—calculating the position in seconds from the container width.
Clips: Drag, Trim, Split
Dragging: on drag, calculate the delta and update startTime considering pixels per second. Constrain to project boundaries and adjacent clips.
Trim: implemented similarly to split, but without deleting the original clip—just change trimStart/trimEnd.
Server-Side Rendering with Remotion
Remotion allows describing a video as a React component. We pass the project via inputProps and render on the server:
// Компонент для рендеринга
const VideoComposition: React.FC<{ project: VideoProject }> = ({ project }) => {
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
const currentTime = frame / fps;
return (
<AbsoluteFill style={{ background: '#000' }}>
{project.tracks.flatMap(track =>
track.clips.map(clip => {
const clipTime = currentTime - clip.startTime;
if (clipTime < 0 || clipTime > clip.duration) return null;
return (
<OffthreadVideo
key={clip.id}
src={clip.assetUrl}
startFrom={Math.round(clip.trimStart * fps)}
style={{ opacity: clip.opacity }}
/>
);
})
)}
</AbsoluteFill>
);
};
We launch rendering via the API using renderMediaOnLambda for scaling. On one project, we reduced the average rendering time for a 10-minute video from 15 minutes (client-side) to under 2 minutes using this approach.
Asset Upload
Large files (video, audio) are uploaded directly to S3 via presigned URLs—bypassing the application server:
// 1. Запрашиваем presigned URL
const { uploadUrl, key } = await api.post('/api/editor/upload-url', {
filename: file.name,
contentType: file.type,
size: file.size,
});
// 2. Загружаем напрямую в S3 с прогрессом
const xhr = new XMLHttpRequest();
xhr.upload.addEventListener('progress', (e) => {
setProgress(Math.round(e.loaded / e.total * 100));
});
xhr.open('PUT', uploadUrl);
xhr.setRequestHeader('Content-Type', file.type);
xhr.send(file);
This approach offloads the server and increases upload speed. Contact us to discuss your project details.
Our Process and Timeframes
We follow a structured process: data collection → audit/analysis → design → estimation → development → testing → launch. The minimum editor (timeline, trim, preview, export via Remotion) takes 13–17 business days. Full functionality with effects, transitions, and audio mixer takes 20–25 days. The final cost is determined after a brief.
What's included in the result:
- Architecture and project code with API documentation.
- Deployment instructions (Docker Compose, .env).
- Repository access and CI/CD.
- 12-month warranty on code and support during implementation.
Here's a breakdown of development stages:
| Stage | Time |
|---|---|
| Timeline (drag, trim, split) | 4–5 days |
| Preview (video sync) | 3–4 days |
| Asset upload (S3 presigned) | 1 day |
| Text overlays, images | 2–3 days |
| Remotion rendering + task status | 3–4 days |
| Audio (volume, mute, fade) | 2 days |
| Effects and filters (brightness, contrast) | 2–3 days |
We have 10+ years of experience and have completed over 40 projects in web development, including several online editors. Request a custom development—we will accommodate all your requirements.