The military’s quest for autonomous drones could also yield sky taxis
So, who wants to be first to ride in a self-guided robot helicopter?
Self driving cars are only the beginning. Within a couple decades, most anything with wheels, wings, or rotors will be able to operate, not just on its own, but in concert with hundreds or thousands of similarly self-guided vehicles around it. This won't just revolutionize how current transportation systems operate, it will open up entirely new roles for unmanned vehicles -- especially aircraft.
In fact, the roles of unmanned aerial vehicles (UAVs) in military applications is continually evolving. Originally developed just after WWII as self-flying targets, UAVs today perform a variety of roles, from intelligence gathering and reconnaissance to strike missions.
Their usage changed dramatically since the September 11th attacks. "The counter terrorism activities that occurred there really became an insatiable demand for a full motion video [in military surveillance]," said John Clark, Skunk Works vice president of intelligence, surveillance and reconnaissance -- manned and unmanned. "And that full motion video, I'll argue, is the most prominent element of driving UAV growth over the last 15 years."
"This is a totally different ability to derive intelligence information by being able to look at video and actually understand how people are," he explained. "How they're behaving, what they're doing and being able to follow them. And instead of just getting a snapshot in time... you're able to derive intent."
As machine learning and artificial intelligence technologies continue to mature, the roles of military UAVs will similarly evolve just as the technologies have impacted manned aircraft. "If you look at the decision aiding capabilities that exist within an F-35 or F-22 cockpit, those decision aiding tools that are giving the pilot information," Clark said, are the same software applications that feed similar recommendations to remote UAV operators. That said, Skunk Works has a "laundry list" of projects working towards further maturing its AI systems.
For example, the company has installed autonomous control systems into an F-16 and leveraged it as a "loyal wingman," according to Clark. "The unmanned system actually senses what the manned system is doing and responds accordingly -- staying in formation with them, performing an appropriate maneuver based on what the manned system does... it becomes part of an orchestration process."
It's been a long road to get even to this point of automation, Clark points out. From the outset, control systems for UAVs have taken on a "satellite mindset" wherein "every payload has somebody tending to it." The result was a workload-intensive environment with various operators stepping all over each other.
Skunk Works instead has taken a "single seat cockpit mentality". As Clark explains, the pilot of a single-seat aircraft is responsible for handling dozens of flight subsystems. But rather than replace the pilot entirely, Skunk Works is developing systems to assist the pilot do his job more efficiently. This includes a focus on simplifying displays and interfaces to ensure that the information being presented to the pilot is timely, accurate and actionable.
And for the foreseeable future, there will be a human at the helm of the operation, even if they're not actually operating the vehicle, Clark assured me. Regardless of how fast these assistive AI's can process data, they'll always need to receive confirmation from the operator before executing tasks. "It becomes a way of keeping the human in the loop so that they understand what's going on but they're not directly responsible for controlling or guiding every facet of the system," Clark said.
This slow takeover of the cockpit by assistive AIs is only going to continue over the next few decades. As for what autonomous UAVs would look like in 20 year, Clark said, "I would assert it would be something that would be largely indistinguishable from a manned platform. I'd really like to see the systems grow and merge with all of the multi mission capability that we've seen in our manned fighters and ISR systems."
The only thing standing in Skunk Works way is, ironically, the government itself. "The government will have to develop new processes and procedures for how they certify their aircraft," Clark said. "There's just the inertia of 'how things have been done historically' that has to be overcome" before that will happen.
And if you thought that getting the DoD to sign off on new technologies was a challenge, you've never tried to certify an autonomous 11,000 pound, 12-passenger Sikorsky S76 helicopter with the Federal Aviation Administration. "These aren't the kinds of drones you can find in Brookstone," quipped Chris Van Buiten, vice president of Sikorsky Innovations.
The AI, therefore, has "got to be designed to the highest levels of integrity, redundancy, and robustness," he continued. "One, to preserve the vehicle that may cost $10 million or more, but much more importantly, all of our products have passengers on board." Van Buiten estimates that certification will take another 3 - 4 years should the process go as planned.
Still, that's a huge step up from the level of technology we saw even just 10 years ago. Back then autonomous functions were barely out of the lab. "The processing required to do high speed flight close to the ground required a supercomputer that completely filled the cabin of the helicopter," Van Buiten explained. "It physically filled the cabin, it would have fully consumed the payload and we would have had to fly with the doors open" to keep it cool. Today, the same amount of processing power fits in a box the size of a toaster. "And we've doubled the number of cores!" he exclaimed.
And for the company's self-flying research platform, the SARA (Sikorsky's Autonomy Research Aircraft), every last bit and byte those cores can produce is vital. It's a modified S76 helicopter outfitted with triplex fly-by-wire controls and is capable of taking off and landing without any human intervention. That massive processing capability is necessary because of the nature of how we use helicopters compared with cars or planes.
"In an airplane, it would be leaving kind of a groomed runway and fly on a preset trajectory. Typically they're flying at 30,000 feet where there aren't other obstacles -- you just need to make sure you're avoiding other aircraft," Van Buiten said. Helicopters on the other hand, "operate in and amongst obstacles close to the ground. The value of a helicopter is that you can land almost anywhere."
Van Buiten points out that roughly two-thirds of helicopter accidents are caused, not by mechanical failure, but rather what the industry calls "controlled flight into terrain." That is when the pilot, say, accidentally snags a rotor on an overhead power line or strikes a commercial drone (or is struck by an airborne police car).
Automation could completely prevent this sort of accident from occurring, Van Buiten argues. It would act "as a third crewmember that's hyper-vigilantly visually scanning the surroundings, looking for obstacles and risks to the mission, and provides a kind of crash-proof functionality to the crew," he said.
The company is already testing the functionality aboard SARA. Van Buiten figures that as these sorts of autonomous systems mature and gain greater acceptance amongst a pilot community that has, to date, been reticent to embrace AI, we'll see flight crews shrink until virtually all operational functions are handled by a computer. And that's when things get interesting.
We're already using autonomous helicopters to perform missions deemed too dangerous for human pilots, such as delivering supplies to forward operating bases in Afghanistan. Just wait until they take over one of the most dangerous positions for civilians: cab driving. Van Buiten envisions an "Uber-like" transportation solution for urban mobility issues.
"The really hard problem for me is the certifiable autonomy that lets those systems operate with push-button ease," Van Buiten said. "We want someone who has zero training to walk in and take a seat and generally do what they do on an elevator, press a button, and be able to say 'I'm next to this building and I want to end up at that building'."
"We call it Click and Fly flight" he continued. As with SARA, "you go in and hit a button that says 'Take Off'," and that's it. The computer handles the nitty gritty details of actually getting you off the ground, through the air, and then safely back onto the ground. Of course, this same technology can be applied to everything from Search and Rescue missions to Medivac flights and military operations.
But as with Skunk Works' autonomous UAVs, Sikorsky has no intention of handing over full flight control to an AI. "There will always be people on the loop. we're migrating from people being in-the-loop, to on-the-loop, which some people call 'supervised autonomy'." Still, he explained, "the power of the autonomy improvements and advancements will be that we can go from one human on one loop to one human on ten loops or a hundred or a thousand."
There's still quite a ways to go before we are all flying around in people-sized autonomous quadcopters. Beyond convincing pilots to let AI into their cockpits, significant changes to federal policy and certification practices will have to be enacted. And, most importantly, a cultural shift -- a fundamental adjustment in how we view, value, and trust autonomous systems -- must occur before the average person will be willing to set foot in a flying machine piloted by a thinking machine.
