The blockbuster success of James Cameron’s Avatar proved what many gamers and game developers already knew: Motion capture is the future of performance. The film’s groundbreaking use of mocap technology caught the attention of Hollywood and the game industry, raising interest in perfecting the nascent craft with a combination of new technology, refined processes, and good old-fashioned acting chops.

”We’ve developed a unique process for motion capture and performance capture for games,” UNCHARTED 3 Creative Director Amy Hennig explained during a recent tour of Naughty Dog’s cutting-edge new mocap studio in Culver City, California. “We’re kind of ahead of the pack.” Hennig described the evolution of the UNCHARTED series’ motion capture technology, beginning with the humble stage used to record UNCHARTED: Drake’s Fortune. It was an outdated, uncomfortable place to perform, equipped with a camera array that painted the room with eye-searing red light. And its poor acoustics made dialog recording impossible, so in-game speech had to be added later.

“We cast very carefully, that’s step one in any project. If you cast hastily or badly, you’re screwed forever. If you cast well, you can write any piece of crap and they’ll make it sound wonderful. Trust me, I’ve done it!”
— Amy Hennig, Creative Director, UNCHARTED 3: Drake’s Deception

Following UNCHARTED’s breakout success, Naughty Dog upgraded to the vastly more sophisticated facilities at the House of Moves for UNCHARTED 2: Among Thieves, a game that further strained the boundaries of cinematic game storytelling. The House of Moves facilities featured an integrated sound stage that enabled Naughty Dog’s team to record dialog while simultaneously capturing the actor’s physical performances — a then-unheard of process that is quickly becoming the gold standard for video-game performances.

An array of 86 infra-red cameras capture the actors’ physical performances with eerie precision. The studio’s integrated soundstage lets the actors deliver their lines seamlessly along with their movements.

Now, with UNCHARTED 3: Drake’s Deception, the team is shooting on its very own cutting-edge mocap set, the culmination of years of pioneering experimentation in game storytelling. Though the sets and props are largely virtual, Hennig and her team make frequent comparisons to shooting for television. “We don’t have the luxury of writing a big script in an ivory tower,” she explained. “We have an outline that we work from and we write the scenes as we go.” That looser approach gives Hennig and her storytelling team the flexibility to adapt to script changes or make tweaks based on a performer’s strengths. “As the writer, I get to learn the actor’s voices — their cadences, their idiosyncrasies, and I write them right in,” Hennig said. “Their characters become infused with their personalities.”

UNCHARTED 3’s high-tech motion capture process converts the actor’s movements into 3D datam which displays in real time over a simple approximation of the game environment. Later, animators and artists adjust the mocap data and drop it into the final game.

Actor Nolan North — who performs as series frontman Nathan Drake but also boasts an impressive resume of film, TV, animation and game performance work — raved about working with Hennig’s team on the UNCHARTED series. “I find that movie stars aren’t that interested in doing games, but actors are. There is a freedom and a beauty of doing this that’s like going back to the roots of black-box theater. It’s collaborative; it’s the true actor experience.” North also pointed out a rapid shift in the perception of video game performances among actors. “It’s changed. I know people who used to do video games and told me, ‘oh, eventually you’ll move on to animation.’ And now they’re trying to get back into games!”

“This might surprise some people, but we don’t just play Drake and Sully. Sometimes I’m mocapping the guy whose neck I’m breaking! Sometimes we play as the bad guys…in UNCHARTED 2’s train sequence, you’ll notice that one soldier runs with a bit of a limp. That’s me, chasing me, and getting blown up by propane tanks.”
— Nolan North (as Nathan Drake)

This humble array of sandbags and carpentry will be transformed into a realistic vehicle in the digital world of UNCHARTED 3. (Right) Naughty Dog pioneered the process of capturing motion and dialog simultaneously for video game performances, eliminating the disjointed feel found in some other game performances.

“Amy writes a great script. It’s like a strong tree, and we just branch from it,” Nolan North says. “If you let actors go too much with ad-libbing, when you don’t have a strong story structure, it becomes a mess. We do our thing, and then Naughty Dog takes it and makes magic.”