How does converting a video to 4K actually work?

The shift from analogue film to digital video happened quickly. Before you knew it, tape-based video recorders had been replaced with digital video cameras and, later still, smartphones. 4K is now the gold standard for video creation, with 1080p left as its lower-quality sibling.

Despite widespread 4K adoption, there's still plenty of video being created in 1080p. And that's not to mention content made in the past, from home videos to old movies, available in standard definition or even lower. That can lead you to thinking about converting video to 4K, giving it a boost to make it look as sharp on a modern 4K OLED display as something you'd shoot today. Converting a video to 4K is possible, but it's a process with a lot of steps and points to consider first.

When we talk about 4K and 1080p, or ultra-high and high definition, we're talking about the amount of pixels contained within each video frame. For ultra-high definition video content, which is what most people usually mean by 4K, that means a resolution of 3,840 x 2,160 pixels, versus 1,920 x 1,080 pixels for 1080p high definition video. Standard definition video, meanwhile, sits at either 480 or 576 vertical lines depending on your region; width can vary depending on how the content was originally filmed. You can also find cinematic 4K resolutions, so-called 'true 4K' or DCI 4K, with a resolution of 4,096 x 2,160. 

You can upscale any output file to 4K — 4K is just the frame size. To move from 1080p to 4K, you're going to need three additional pixels on-screen for every original pixel that was recorded, so four times overall. Managing that without losing any quality is the issue. That is almost impossible and simply changing the resolution won't reveal a hidden version with extra clarity or sharpness, either.

Traditional upscaling software uses interpolation to manage this, scaling up each frame and figuring out what it should look like. How accurate upscaling software can be is the main question. It has to estimate what the pixels should look like, but it can't recover the real detail if it wasn't recorded in the first place.

Often, this means that sizing up will make the footage look softer. The "guess-timation" used to add extra pixels often makes it look smooth as a result. Sharpening can help around the edges, but it's a Goldilocks dilemma: too much, or not enough, can ruin it. The same is true for noise reduction, which can make the final footage look unnatural.

If you're a professional looking to convert a video to 4K, you're going to want to work from the best source data available to you: the original source files. The better that source data is, the better quality a job you'll be able to achieve.

The HD restoration of Star Trek: The Next Generation proves how tricky this can be if you want to get the process right. That upgrade, commissioned by CBS in 2011, used the show's original 35mm film negatives, which were described as having a quality equivalent to a 20 megapixel resolution, to remaster all seven seasons of the show from their original broadcast quality to 1080p for Blu-ray release. 25,000 reels of original film stock were used, with visual effects and CGI upgrades to help the process. It took over three years to complete and reportedly cost the studio over $12 million, according to one of its producers.

Ironically enough, the analog film to digital video conversion here makes this type of job easier. If you're working with an older digital master file with a set resolution, you're stuck with a fixed number of pixels to work from. The original negatives used in the Star Trek upgrade capture far more detail that wouldn't have made it into the original television broadcasts.

Unfortunately, when the best source files are already low resolution, you're not able to work from the real data. All of the missing pixels that are created are guesses. The best result is that this will look pretty convincing, but it won't be 100 percent true to form.

Unlike CBS, you're unlikely to have $12 million to convert your videos to 4K. Home users converting video to 4K will instead have to choose between traditional interpolation or newer AI-based tools. Both methods will create the additional pixels needed to fill out an enlarged 4K frame, but not in the same way.

4K upscalers using interpolation rely on good, old-fashioned math. It takes a pixel from a single frame and looks at the color and brightness of the pixels around it. From there, it'll try to calculate how the pixels surrounding it will look. At a basic level, it might achieve this by sheer duplication, but more advanced algorithms will focus on trying to preserve edges and avoid jagged lines. The end result is pretty predictable, but one problem is softness. Interpolation can't magically recreate genuine detail or textures, making it seem slightly fake. Your TV might be able to do this automatically too, with varying degrees of success.

Luckily, AI upscaling looks to solve that problem. These models are trained on examples of low and high-resolution images that allow it to identify how common features should look on your screen. Instead of estimating each pixel, the AI is predicting what a sharper image will look like.

The result can be sharper and more convincing but, like interpolation, it won't be an exact match for the real world. AI upscalers can also introduce unusual artifacts that interpolation doesn't. AI models are constantly improving, however, so the quality and consistency you'll see today might seem antiquated in just a few years' time.

If you're working with older video, you may also need to think about interlacing, where video frames were split into two halves by odd and even horizontal lines, with each line refreshing each time the screen refreshes. Modern displays don't use interlacing, but older broadcasts meant for CRT-like screens would. Converting these kinds of videos will require you to de-interlace before upscaling, although how well you'll be able to upscale a 480i NTSC video to 4K in a way that maintains a watchable video quality is questionable.

Upscaling tools often feature plenty of configuration to help you get the result you want, whether it's using AI or interpolation. If a single pass through doesn't get you the result you're looking for, you can tweak the settings to have another go (as long as you've got the disk space).

You can't just whack a 1080p video into an upscaler and expect a perfect 4K video at the end. You can achieve it in the technical sense, by increasing the size of the video frame, but getting the video to scale up without losing quality is difficult. It may take several attempts to get the result to match up to your expectations.

It's easier to achieve if you've got good source material to work from, and the smaller the leap in resolution, the better. Upscaling from 1440p to 4K, for instance, won't require as many 'fake' pixels as 1080p to 4K, or worse, 480p to 4K. 4K upscaling software can only guess what the missing pieces, those extra pixels, should look like. AI is making the process easier, and potentially better, but a like-for-like match isn't guaranteed.

Resolution isn't the only factor at play here, though. To save on disk space, video files can be compressed with a lower bitrate, meaning the amount of data saved for each second of video is lower. Upscaling to 4K won't undo that, and you can't just lose common compression artifacts like blocks or banding if you're working from a low bitrate file. It all comes down to the source, and if that's good enough, upscaling becomes a whole lot easier to handle.