I built a desktop app to repair missing video frames with machine learning.
A missing frame sounds like a small problem, until it appears in the middle of an otherwise polished video, in an 13 minute short film like Nightborne, there are hundreds of these dropped frames, and even if viewers don’t see them, they will feel them.
The result is a visible jump in motion like a vehicle suddenly shifts position, a camera move stutters, or an object appears to skip forward. Duplicating one of the neighboring frames hides the gap, but it also creates a momentary freeze.
I wanted a better solution, so I built FrameFix, a desktop application that generates the missing image between two existing frames intelligently, with machine learning, but also with a color correction pass to match the frame before and after better.
I’ve also created a video showing how the application works, which I’m sharing alongside this article.
The problem
Some generated and processed videos contain predictable missing frames. In one clip, for example, the skips occurred after frames 40 and 80. In another, the problem appeared after frames 2, 6, 10, and then at irregular positions later in the video.
The application needed to handle both situations without forcing the me to manually rebuild an image sequence or construct a complicated compositing workflow.
The basic process is:
- Select one or more videos.
- Detect the missing frames.
- Generate a new frame between each selected frame and the one following it.
- Export a repaired video or image sequence.
Generating the missing image
The application uses machine-learning-based frame interpolation.
Instead of duplicating a neighboring frame, it analyzes the image before the gap and the image after it. It then estimates the motion between them and synthesizes a new midpoint frame.
This produces a much more natural transition:
Frame 40 → Generated midpoint → Frame 41
Only the missing positions are processed. The original source frames remain unchanged.
Solving the less obvious problems
Generating an image was only part of the challenge.
Color matching
The first generated frames had a slight color and brightness shift. It was subtle when viewed individually but noticeable during playback.
The application now measures the generated frame against the color midpoint of its two neighbors. It applies a clip-specific luminance and chroma correction before inserting the frame.
This correction affects only the generated images.
Batch processing
The application can process multiple videos in one operation. Each video can have its own list of missing-frame positions.
For a recent batch, I used it to repair 35 videos, all automatically! This would have taken hours and hours before, and the results would not have been as good.
What’s next
For now, the application is already solving the original problem, repairing missing frames without freezing the motion or requiring a complicated manual workflow.
If you work with generated video, animation, editorial, or VFX footage, I’d be interested to know how often you encounter dropped or missing frames, and how you currently detect and repair them.