Fox Sports’ new virtual studio runs on Unreal Engine

Realism and versatility thanks to a hand from Epic Games.

Sponsored Links

Billy Steele|October 16, 2018 3:00 PM

It's no secret that Epic Games has enabled a number of gaming studios to create more-realistic visuals with its Unreal game engine. Since its debut in 1998, powering the first-person shooter Unreal, the technology has evolved to power hundreds of games, from Fortnite to Street Fighter V, and with that process, the virtual has become increasingly more realistic. Now in its fourth iteration, Unreal Engine is no longer exclusively being used for gaming, as other industries have taken notice of the possibilities. Unreal Engine 4 has become a key element for film and television in recent years, and Fox Sports is using it to power its new, completely virtual studio set.

"Virtual sets have been around for quite some time, and we've done our fair share of using them," Zac Fields said. "But it's always been a struggle to give that sense of photo realism." Fields oversees Fox Sports' Graphic Technology and Integration department, which includes the addition of new gear during a studio build. He said the team started thinking about the idea of a virtual set about two years ago. Around 15 months ago, the network started getting staff familiar with the software and began tests. Then last winter, the broadcaster did a virtual show. Fields described this as a "full run-through" of a show on the virtual set that was built in Charlotte, North Carolina.

Charlotte is a NASCAR city. While the official headquarters may be in Daytona Beach, Florida, and the governing body of stock car racing has offices in other cities, Charlotte is the epicenter of all things NASCAR. The Hall of Fame and a major portion of NASCAR's offices are downtown, and a lot of the race teams are headquartered nearby. Plus, Charlotte Motor Speedway is just a few minutes north, in Concord. So it makes sense, then, that Fox Sports would coordinate its NASCAR coverage from a facility just north of downtown -- the same spot where the broadcaster made its final preparations for the World Cup earlier this year.

Inside the former home of the Speed Network, Fox Sports broadcasts its NASCAR Race Hub show every weekday. Think of it as a news hour for NASCAR. During the hour-long broadcast, hosts, commentators and experts analyze the previous weekend's action and preview upcoming races. The Race Hub studio needed a refresh this year, though, and Fox Sports Senior Vice President Zac Fields explained the network had some specific goals for the overhaul.

Realism and versatility were at the top of the list.

Unreal Engine allows production teams to expand their bag of tricks exponentially. "In the past, virtual sets were used as a cost-saving venture, but with new technology -- specifically what we're doing with Unreal Engine -- the level of realism that you can achieve opens the doors to so much more," Fields said. "You're not limited to a space, and you're not limited to graphics to convey storylines during your shows."

He noted that most of the work was behind the scenes, creating a custom application that can link what they need for television production to a video game engine. Fox Sports plans to start putting the new space through its paces in the lead up to the 2019 Daytona 500. That race is scheduled for mid February and is basically the Super Bowl of NASCAR -- even though it's the first race of the season. That's quite the debut for your production team's new crown jewel.

The most impressive thing about this virtual space is that it looks like a normal studio. Having seen the current Race Hub set, if you told me this was just a redesign of that physical space, I would probably believe you. But it's the details that really make this a showpiece. In a demo reel that previews the studio, the virtual set itself looks massive -- much larger than it is in real life. The space allows Fox Sports to "park" 10 stock cars on set, which is probably about eight more than the actual studio will accommodate.

When it's time to preview a specific event or venue, Fox Sports can easily put a section of the track -- Daytona International Speedway, for example -- with a race car rolling on it behind the host. A balcony around the the top of the virtual set gives producers even more perspectives to shoot from. The result is a regular-sized studio that's tucked into the corner of a building but appears to offer the space of an airplane hangar. In the future, if Fox Sports needs to move on to another show for the next hour, the whole thing can be swapped out in minutes, and without any literal heavy lifting.

The virtual set itself looks massive -- much larger than it is in real life.

So if a broadcaster is going to invest in its "most advanced studio," why put it in Charlotte instead of its Los Angeles headquarters? A few reasons. First, NASCAR is a huge part of the network's coverage. There's racing content broadcast every single day. The second is available space. Fox Sports had the room in North Carolina to build out this 60-foot-by-60-foot production space. And last, the Race Hub set was due for an overhaul.

"It was the perfect spot for us to experiment," Fields said.

He explained that the sport itself is very tech-heavy, which opens up all kinds of possibilities for production visuals. More specifically, where a 3D model of a dissected stock car may have been on a display before, now it will be just like a full-size car is on set with the hosts. Multiple cars can rise up through the floor and be not only on the set but also moved around easily -- a feat that's not easily achieved with actual, very heavy stock cars.

Fox Sports can also bring a remote location to the set instead of having to send a team there. In the demo reel, there are examples of things like track elevations and examining specific sections of a track in lifelike, and life-size, detail. One minute analysts are explaining key parts of a 3D model, and the next they're standing on a track.

"It creates this seamless transition from the virtual set into the 3D world," Fields described.

Fox isn't the only network to employ Unreal Engine. Marc Petit, Epic Games' general manager for Unreal Enterprise, explained that it's actually been used in television for about 18 months, starting with a French-language children's program on TFO in Canada.

"We've worked with our partners to integrate our graphics engine into their systems," Petit explained. It's a collaborative process to ensure that things like camera trackers work with broadcast systems. Once all of that is ironed out, it means broadcast production teams now have AAA game-quality graphics at their disposal.

One of the more high-profile uses on television is The Weather Channel's. After announcing that it would be adding Unreal-powered mixed-reality segments to its broadcasts, the meteorological network first harnessed the tech to showcase the dangers of tornadoes -- safely from the confines of a studio.

During the explanation, a virtual telephone pole came crashing down near meteorologist Jim Cantore. Of course, it's not the best idea to send your on-air talent into a tornado (though Cantore is regularly in the path of a hurricane), but mixed reality helps visualize and demonstrate the dangers of things like flying debris. Being told about it is one thing, but seeing a two-by-four go flying through the air is another. Another key part of the segment was a tornado on a massive in-studio display behind Cantore. It showed the formation of the cyclone as well as what happens as it gains strength. Once you realize that Unreal Engine is at work here, you can tell the visualization is something like you'd see in a big-budget gaming title.

Petit also explained that movie studios are creating renders in Unreal Engine to test with audiences before embarking on a full-scale production. We're not talking your Bradley Cooper rom-coms here. These are films that would already be heavy on special effects and rely on computer modeling -- movies like your big summer blockbusters, for example. Even still, investing a few million in a rough draft to work out any issues is much better than making an entire film that's a bust. And one that you spent $200 million on.

"Studios are willing to invest in a high-quality product that they can validate before making an entire movie," he continued.

Epic also created Siren with Unreal Engine in collaboration with Vicon, Tencent, Cubic Motion and 3Lateral. Siren is a digital personality that, quite frankly, makes it difficult to distinguish CGI from reality. That demonstration was powered by motion-capture technology that superimposed the movements and facial expressions of an actress onto a completely different digital person in real time. The implication here is that one day actors in games, movies and television may not have to travel to film certain scenes. Instead, someone else could step in and handle those duties for them, and the end result would still look like the intended person or character. Siren shows some possibilities for the uncanny valley and what's ultimately possible with Unreal Engine. But don't expect Fox Sports to go replacing its hosts with mo-cap stand-ins just yet.

This virtual studio isn't just for segments. Fields explained that the network also plans to use the versatile space for full broadcast shows."We've been making significant advances in our sets across all shows for the last eight years," he said. "When we revamped our studios in LA, we got into a lot more technology on the sets, from LED monitors to augmented reality graphics." Of course, AR was on full display during Fox Sports' World Cup coverage a few months ago, and it will be a key part of the studio in Charlotte.

Fields explained that even if you build sets properly so they can be swapped out from show to show, the space is still the same. You can add things like walls, but the shape and most of the tech remains. This means the content you're able to showcase is also mostly consistent. To sum it all up, "you're limited to that space," he said.

The goal for Fox Sports has been to make the set part of the show, so when you sprinkle in Unreal Engine to a virtual set, Fields doesn't mince words. "We realized this could be a game changer in our studio programming," he said.

In the same space and on the same set, the production team can now create something different for every show. What's more, all the features of the set can be dynamic, not just an anchor desk in the center and a sit-down interview area to the side. And everything can be changed dynamically, live on air -- right down to the shape of the set itself. Of course, the most important part is that it looks more realistic than ever.

"There are no boundaries anymore," Fields said.

Fox Sports is using Zero Density studio software to bring the capabilities of Unreal Engine to its broadcast workflow. That tool will be used on a 50-foot-by-47-foot green screen area, with the ability to shoot 360 degrees if the production team demands it. The rest of the gear list includes five Stype RedSpy-equipped cameras, a Steadicam and a jib -- the latter two, of course, are pretty common on television sets. The network will also employ Blacktrax 3D-tracking tech alongside the RedSpy optical-tracking hardware to activate events on set. Lighting can automatically follow analysts as they move around, and AR graphics can be activated when the hosts hit a certain spot. Automated lighting is already in use for things like live concerts, so it makes sense Fox Sports would want that type of tracking for a virtual broadcast set as well.

"We have tons of ideas, but we're in the position now where we have to rein them in."

Chances are you'll hear about Fox Sports building more of these studios at its other production facilities in the future. Fields admitted that he doesn't see any reason why all of the networks' production teams wouldn't want the toolset that a virtual studio running on Unreal Engine provides. "Whether it's a 100 percent virtual set or whether it's part of the set that's designated as a virtual space, once we work out the kinks, our production teams are going to realize just how helpful this will be," he continued.

Of course, even with no boundaries, it's important for Fox Sports to exercise some restraint. And doing so will be important now that the network has announced a time frame to begin using its new virtual set on the air.

"We have tons of ideas, but we're in the position now where we have to rein them in," Fields confessed. "We have to make sure we have a clear path to where we're going to start and how quickly we can get to some of these futuristic concepts."

Who knows, if Fox Sports really dazzles during the 2019 NASCAR season, it might not be long before virtual studio coverage of college sports, Major League Baseball or Major League Soccer is the norm on Fox networks.

Images: Fox Sports

Fox Sports’ new virtual studio runs on Unreal Engine