Building Xbox One: An inside look at Microsoft's play for the next generation of gaming

The engineers in Microsoft's windowless next-gen Xbox silicon lab are rattled. And understandably so. We're in their office, after all, and we have a mess of cameras in the one place you're not allowed to have cameras (or even cellphones). We're obviously outsiders on Microsoft's multi-building, security-heavy Mountain View campus, especially given our quartet of esteemed escorts: Todd Holmdahl, Ilan Spillinger, Nick Baker and Greg Williams. These four gentlemen are leading the charge on both Microsoft's next big thing and, perhaps more importantly, a major effort to internalize silicon architecture at the traditionally software-focused megacorp.

The skittish engineers aren't worried we'll film the mess of 24-inch LCD screens running video-compression tests, or the rows of desks with water hose stations used for temperature stress tests, or even the sea of circuit boards in various states of disrepair -- that's all standard for any Silicon Valley computer lab. It's really just a single chip that's causing concern: a custom-built Microsoft SoC that sits at the heart of the Xbox One. It's this SoC that has us in Mountain View, Calif. -- in Silicon Valley, literally down the road from Google -- a mere five days before Microsoft will unveil its next game console to the world. Over six hours last Friday, we learned not just about that SoC, but also how the company plans to utilize it in the new console. We spoke with its four lead hardware architects. We toured the labs where they are testing the silicon, and where the next-generation Kinect was born. What follows is more than a look behind the silicon that drives the next Xbox -- it's a deep dive into the changing approach Microsoft's taking to creating devices.%Gallery-188921%

Hidden silicon in the valley

Considering there are large Microsoft logos emblazoned on both the buildings and signs at each entrance to the campus, the company's Mountain View location isn't exactly "hidden." What the people are doing inside, however, is one of the only secrets about the company's next-generation Xbox that hasn't leaked. It's at this location that Williams and Baker -- the Xbox silicon development general manager and Xbox architecture distinguished engineer, respectively -- drew up the silicon architecture for Microsoft's Xbox One.

In the first Xbox, Intel and NVIDIA crafted the silicon. In the case of Xbox 360, it was more of a joint effort between Microsoft and ATI / IBM. Though Microsoft's still working with AMD to build out some of its chips this time around, it's also invested millions of dollars in building out verification facilities (among others) on-site in Mountain View and doubling the amount of in-house engineering dedicated to silicon. Holmdahl explains:

"In the consumer space, to control your destiny, you can't just rely on commodity components. You have to be able to make your own silicon. It helps with performance; it helps with the cost; it helps make your product smaller; it helps you create your own IP (always a good thing). I'll argue you're a lot more flexible -- you're not relying on somebody else's schedule; you make your own. So we're obviously heading that way. The stuff we've done over the last 13, 14 years is one example of that within Microsoft. And you're gonna see more and more of that, is my guess, as you go forward."

For now, silicon's a teensy component of Microsoft. Of the company's approximately 95,000 employees, the silicon team is around 200 people, or roughly 0.2 percent of the total workforce. Double that if you count the system engineers working on next-gen gaming hardware -- the console, Kinect and accessories -- and that's still nothing compared to most dedicated chipmakers. Intel employs over 100,000 people, for instance, while NVIDIA employs around 7,000.

However small, Mountain View's 200-person team has been working heads-down for the past several years to verify its silicon architecture and get it "first time right." That's not to say that the chip returns perfect from manufacturing, but that it doesn't run into any major issues -- of course, there's a small team within the larger group that's dedicated to debugging. As Xbox Hardware Group Lead Holmdahl points out, "One transistor can mess up your day really badly," setting the team back months to try and isolate the issue. Unlike software development, you can't quickly (or cheaply) iterate on silicon implementation. Worse, the equivalent of "debugging" is a guesswork process. "The fewer times you have to redo it, the faster you can put your product on the market," Holmdahl adds, with a nod to the business logic that also drives that need for "first time right."

"We booted the OS within days of getting the SoC back," Williams says; a good indicator their initial architecture design wasn't tremendously off. "I'm proud to say that, in our schedule, we didn't have any major showstoppers.

DNP Where the Xbox was built An inside look at Microsoft's play for the next generation of gaming

"It's extremely hard. It's a matter of breaking it down. There are ways you can determine 'is the right thing happening as an input into the chip, or is it illegal?' And within the chip, there are ways we can instrument on this board and with the silicon to bring signals about to say, 'Well, within the chip, we think it's maybe between these two blocks,' and to try to get access into that signal. Of course, when you're highly integrated ... that's not as easy once you've actually built it. That's part of the art, essentially, of the process."

That "art" he's speaking to applies across the entire spectrum of console development. For every generation, console manufacturers play a guessing game with internal specs, software / storage-medium support and a variety of other factors, hoping that they've prepared enough to support both aggressive initial interest and long-term attention from consumers and game developers.

Powering the living room of the (near) future: 'Xbox On'

In-house silicon is at the core of what makes the Xbox One tick. It provides the structure that enables the console to run two operating systems at once for instant multitasking, and for the new Kinect to dish out much more information than before (to the tune of 2 Gbps). Most importantly, the five pieces of custom silicon spread across the console and its new camera peripheral helped the Mountain View team support their vision of an "always-on" console. Rather than use your hands to turn it on like a baby's toy, simply saying "Xbox On" will immediately wake the new Xbox.

"The box will pop on and come to your home page or wherever you were last. In order to do that in an efficient way, you have to architect all of that into the box up front. A lot of it is in the SoC," Holmdahl says. That SoC contains both the CPU and GPU, as well as embedded ESRAM; the first two components are based on an AMD design, and custom-built into an SoC with that embedded memory. That CPU is based on the Jaguar design from AMD, with eight cores and a 4MB L2 cache, while the GPU is of the D3D11.1 (with extensions) variety, Baker tells us.

DNP Where the Xbox was built An inside look at Microsoft's play for the next generation of gaming

The console runs in multiple power states, which means it runs in a low-wattage setting when not in use. (Microsoft wouldn't give us specifics other than to say, "The system is designed for an SoC up to about 100W, but will vary on the scenario.")

"If you look at the instant app-switching, if you look at multiple OSes, if you look at power consumption -- that placed a lot of the main constraints on what we did on the silicon," Baker tells us. Some of the silicon design was derived from data center concepts, as that was the only parallel in computing available. "You're trying to make that technology seamless for the living room. It doesn't mean we need 64-bit CPU architecture. We're talking about many, many cores so you can run these tasks in parallel. We wanted to be able to support 8GB [of RAM] out of the chute, to probably support virtualization, which is what you need for running multiple operating systems. That's just a lot of making sure you have the right security systems in place so things don't stomp on each other."

In-house silicon also powers the other crucial component driving Microsoft's vision for the Xbox One: the new Kinect. Improved cameras and acoustics, not to mention a sleeker form factor, are all nice changes; it's discussing the new CMOS sensor's processor, however, that gets Spillinger, a passionate, intense gentleman (whose lovable accent sadly doesn't translate to text), visibly giddy.

"The highlight of the story is the CMOS sensor, which we developed internally," Spillinger says. "This design was done completely, 100 percent on this site. This is brand-new technology. There is discontinuity between this technology and the first Kinect; from the technology perspective that we are using for depth, for 3D measurement. So this was done here. On this one, this was a complete Microsoft custom design, where our engagement is directly with the manufacturer. It's not with any third party. We did the work. We do the qualification of the parts. We do the validation of the parts. We have done everything on this one."

If it isn't clear enough from that, he is a beaming father when it comes to the next generation of Kinect. And yes, you read that right -- your old Kinect won't work with the Xbox One.

Spillinger joined Microsoft just as the company was beginning work on the first Kinect (then "Project Natal"). He hailed from IBM, where he led the team that created the Xbox 360's CPU. At the time (early '08), he thought he was joining the Xbox hardware team to get started on a next-generation gaming console.

"First I was the design architect in Intel, then a design manager at IBM, and when I joined Microsoft, the view was 'Okay, it's about time -- early '08 -- to start to think about the next gen,'" he says. "It didn't take us five and a half years to get there, because what happened is that the moment sort of turned around and we started development of Kinect. The entire focus was about shipping Kinect, which now, if you in retrospect see, is such a great success."

The first Kinect did indeed sell very, very well for Microsoft -- 24 million as of this past February. Microsoft's decision to refocus on a new Kinect with much-improved audio recognition makes sense -- many users employ the first model solely as a HAL 9000-like order-taker. Beyond "improved acoustic models," Holmdahl says the new Kinect can track whoever is speaking using a "beam array" on the video side, which works with the audio side. It can identify, "exactly who is talking, and then be able to subtract out other people in the room so you get a really clear audio signal into your box." As time goes on, he promises that Microsoft will push out improvements to voice recognition and commands, and, "At some point, we'll be able to have conversational understanding."

If that isn't HAL, what is?

Inside the Silicon lab

DNP Where the Xbox was built An inside look at Microsoft's play for the next generation of gaming

Like so many other engineering labs, the one we're touring is pumped full of cool, conditioned air, very white and rife with LCD monitors. A visually demanding scene from The Avengers plays on loop, putting stress on various iterations of circuit boards, while very smart, very young engineers monitor nearby workstations flooded with code. A handful of monitors display a basic dolphin animation that Microsoft uses in-house -- if the dolphin's moving, you're in business (essentially). Another few are running a very pared-down version of an Xbox Dashboard, which looks similar to that of the 360 (or perhaps more apt, that of Windows 8's Live Tile arrangement) -- the system says "Hi" and seeks a user, which the engineer quickly skirts on the debug hardware using a wired Xbox 360 controller.

Black tubes protrude from the workstations, pumping hot and cold water into the lab for stress testing (and the occasional multimillion-dollar water fight, we imagine).

Black tubes protrude from the workstations, pumping hot and cold water into the lab for stress testing (and the occasional multimillion-dollar water fight, we imagine). CPU sockets sit unused, next to racks of circuit boards running the SoC through the virtual gauntlet. In one corner, a massive 4K television is seamlessly playing two HD video streams from a single box. Though the Xbox One is capable of pushing and receiving 4K signals, this test station is an illustration of how Microsoft's architecture went into the console's SoC (not a demonstration of its raw power). Spillinger says that the new console's ability to decode multiple HD streams at once is a measure of his team demanding specific silicon design from third-party partners.

"When we talk about the SoC and the general-purpose SoCs out there, this is per our requirement," Spillinger tells us while we huddle around the station. "This is our aggressiveness to have two HD streams being compressed, decompressed. It's actually a parallel circuit, it's not part of the CPU / GPU -- it's an add-on."

DNP Where the Xbox was built An inside look at Microsoft's play for the next generation of gaming

A tray full of the much-celebrated SoCs sits unceremoniously next to an electrical signal analyzer. "Definitely no photos of that one!" Williams half-jokes as he hands us a dead SoC to examine. This room and its many stations is the first of several we tour, and it's a good place to start, given what it symbolizes. The depth of hardware verification taking place in Mountain View is unprecedented for the software company, and offers further evidence of its internal silicon investment.

Making a fake video game console

Principal Design Verification Engineer Padma Parthasarathy has a bowl full of M&Ms on her desk, right next to the door, which she invites guests to indulge in. Next to that, her computer is running a software virtualization of a game console. And that's her job -- as "principal design verification engineer" -- to verify whether or not all the bits and pieces of the chips driving Microsoft's next Xbox are working as they should be. It's part of the ongoing process the company's set up to create its own silicon, and it's an important step. Without people like Parthasarathy, chip development could go from a few years to a decade or more. She is yet another fail-safe in the endless quest for "first time right."

In her little office, on a standard desktop computer, the logic behind the Xbox One is running -- from a server farm on campus -- through a variety of test scenarios. She can log issues on hardware that doesn't exist yet, essentially troubleshooting a game console being faked by software on a server farm (its components, anyway). Given the complex nature of the virtualization, tests can only be processed at a fraction of actual run speed, but it's still one of the best ways to troubleshoot hardware ahead of mass-producing millions of units.

Parthasarathy's time with Williams and others on the team goes all the way back to the 3DO. "She has been the architect of the verification team for all of these years," he tells us as we leave for another part of campus. She's been with Microsoft since 1998, when Microsoft absorbed her team (and Williams') in an acquisition. It's people like Parthasarathy that are helping bridge the gap between the company's old approach to gaming hardware and its future.

The next step in faking a game console requires a giant machine that needs its own HVAC system, of course. The spaceship-like device originates from Cadence Design Systems, and serves to run a variety of processor emulations. A peek into its innards reveals all sorts of hoses and wires, akin to opening a panel on a Willy Wonka candy machine; an even larger machine sits outside the building, powering its test conditions. Our friend the dolphin shows up once more on a monitor displaying what the emulator is up to, but his movement is imperceptible due to the speed of the emulation. Next up is a significantly smaller lab with what looks like a standalone meat locker. The meat locker has a manned workstation in front of it, where one very unlucky engineer spends his time logging issues registered by the consoles being tested within.

DNP Where the Xbox was built An inside look at Microsoft's play for the next generation of gaming

Before long, the four gentlemen who architected the Xbox One lock our video producer Ben Harrison in that meat locker -- which is actually a temperature chamber for running the console through a variety of thermal extremes. The chamber is full of hundreds of variations of prototype Xbox hardware -- today, it's set to very cold -- and is vital in determining how the Xbox One stands up to extreme thermal conditions. With laughs all around, he's freed from the icy, zebra-filled prison. Surprising no one, the various beta kits of the console itself, the controller and the new Kinect all sport zebra-pattern tape to hide their shape (as rumored). What wasn't known is that the zebra tape serves a purpose: it's used in product design specifically to hide angles from sneaking photographers, and is usually employed by the auto industry. The final version of the console looks damn similar (if not identical) in shape and design to the developer beta kits, and Microsoft is understandably leery of leaks. At this point, though, the cat's out of the bag.

The virtual console becomes reality

When Holmdahl lifts the dark cloth off a black, rectangular box, a gamepad and the new Kinect, it's the first time some of his own colleagues have seen the final designs. Not a beta kit with zebra tape, or a mockup, but a presentation-ready prototype. As we stand around the hardware placed on a little circular table, the concept of the box "slipping into the home entertainment center" is mentioned. The Xbox One is no "inhale" -- it looks like a little computer with a slot for discs. It's got a two-tone finish of alternating black matte and gloss, with a slot-loading Blu-ray disc drive and its power button greeting users out front. It's ... well, it's not such a looker. At least not thus far -- anything could change by the time it ships, though it seems doubtful the form factor will change dramatically.

When Holmdahl lifts the dark cloth off a black, rectangular box, a gamepad and the new Kinect, it's the first time some of his own colleagues have seen the final designs.

The gamepad, similar as it is to the 360's, is both smaller and chunkier all at once. Though it's designed to be sleeker all around -- shortened analog sticks, a shrunken battery pack area, a lack of screw holes -- we couldn't help but think it looked a bit rough around the edges. It has a distinct "mid-'90s vision of the future" look to it. And it's certainly not helping that the 360 gamepad is considered by many to be the best game controller ever built.

Kinect also got a visual makeover, and looks the nicest for the revisions. For starters, only one eye glares at you from the front, and the new chassis is a clean update to a clunky, aging peripheral. Unlike the Johnny Five-esque look of the original Kinect, the second version deserves the prime placement in living rooms that it demands.

The five

"In different levels, we were working on five custom-designed components. Silicon components. Three of them going to the console and two of them to the sensor," Spillinger explains. That's the SoC that drives the console, the CMOS processor in the new Kinect, I/O integrators in both Kinect and the console and a digital signal processor on the Blu-ray drive. For the four gentlemen who show us around the Mountain View campus and scads of others we don't meet, getting to the point where so much of that silicon was designed and verified in-house is the fruition of years of work.

It's a major shift away from the company's past reliance on external partners, with only AMD serving as collaborator this time around. And like any game console launch, it's another huge investment for the next... five, eight, 10 years? That's an unknown, of course, but it seems likely based on history that we'll have the Xbox One for the foreseeable future. Whatever the future dictates, it looks like we'll see internally developed chips in many of Microsoft's products going forward.