VR is a strangely fitting home for stop-motion animation

Strange, fantastical things are brewing in VR, and the Tribeca Film Festival's Immersive Arcade is a great place to find them. One of the most intriguing things I saw at the festival this year was a stop-motion experience that also uses CGI and 3D 360-degree video to craft a creepy gymnasium. Sure, stop-motion is relatively low-tech, but the combination of technologies produces something truly unique.

To be clear, Gymnasia isn't the first stop-motion piece in VR. Felix & Paul Studios had previously worked on other VR projects with stop-motion elements, like last year's Isle of Dogs: Behind the Scenes. But Gymnasia's creators Chris Lavis and Maciek Szczerbowski believe it is the first piece to be a full 360-degree experience, complete with a set and stop-motion puppets shot in the same environment without green screen or photogrammetry. "The shooting process was difficult," they told Engadget. The sets and puppets were created and hand-animated at one-third scale, then complemented with some computer-generated elements.

"Two to three seconds a day of animation is typical in our work, but what made this extraordinary was the length of the shots," Lavis said. One scene in particular involved two minutes of animation, which took months to shoot. Because the puppets were actually in a set, not in front of a green screen, details in the environment had to be meticulously maintained for continuity. "If the puppet had busted a wire, the camera rig broken down, or the animator lose his courage or concentration, the shot would have been ruined and there wasn't the time nor budget to go back," Lavis said.

The entire piece runs six minutes long, and it's an eerie experience starring characters who look like they were rejected from The Nightmare Before Christmas for being too realistic and unsettling.

You find yourself in an abandoned grade-school gymnasium, with music sheets strewn all over the floor. Ghostly shadows of kids flit around you and about two dozen basketballs start bouncing on their own. A creepy baby doll wearing a goth-schoolmarm dress riding a projector rolls in from the hallway, looking for... something. Finally, you find yourself transported to the stage where a zombie-like boy starts singing a haunting melody, as you're both watched by the projector lady-baby.

There's no real coherent storyline to Gymnasia; there's no conflict or climax. But there was plenty of atmosphere and technical beauty. It also helped that the exhibit at the Tribeca Film Festival's Immersive arcade was housed in a dark room with the same music sheets strewn all over the floor. There was even an actress on-site dressed as the projectionist, complete with oversized bobble-head (shudder).

Gymnasia is an impressive demonstration of the creators' skill. There were some stalled frames during my preview, but I wasn't sure if that was because the headset was laggy or the animation was glitchy, but the hiccups didn't take away from the experience.

What made the piece feel so complete were the details. When I was up on the stage, every time I looked down at the floor, I saw a baby's face (not the projector baby) in place of my feet. In fact, I don't have a body at all. The head looking back at me has a butterfly on it instead of facial features. Each time I bent my head to look at the strange visage, the butterfly's wings would flutter. Without fail. After awhile, a boy appeared out of nowhere, staring into my face and scaring me. These two bits of the experience are examples of what the creators called "subtle gaze-based interactions," and the boy's sudden appearance was meant to surprise the viewer.

As Gymnasia is a work in progress, it's likely the story will further develop into something more gripping and meaningful. Felix & Paul Studios worked with Clyde Henry Productions and the National Film Board of Canada, as well as the Adam Mickiewicz Institute, the Ministry of Culture and National Heritage of the Republic of Poland to put this piece together and we may see it become more widely available after some time. I would love to check it out again on a more capable headset and see if that eliminates the glitches I saw. Still, the piece is more than a mere curiosity -- it's a compelling demonstration of the potential for stop-motion animation and VR to perfectly complement each other.