The augmented reality interface in Minority Report was futuristic, but what it lacked were objects that you could physically touch, according to researchers from Queens University. The problem is that the film's virtual interface floats in mid-air, and physical objects tend to be ground-bound. To get around that, the team developed a system called BitDrones, with flying nano-drones serving several functions. "PixelDrones" have a basic LED display to show information, "ShapeDrones" are cubic objects that form a physical 3D "display," and "DisplayDrones" contain a touchscreen interface. All of them, plus the human operator, are tracked with a 3D motion capture system.
The technology is still in early stages, so the wobbly drones can't form complex objects, but the potential is there. In one case, the team used the system to browse "files," swiping drones left and right to show the contents. After the operator opened an architectural drawing, the cube-like ShapeDrones form the basic positioning of the building in 3D space. Then, a user can "drag" the drones around to adjust the building's orientation. The operator can change the parameters of the ShapeDrone by using the touchscreen on the DisplayDrone to change the LED color. You can perform other common gestures like pinching and rotating by touching multiple drones (see below).
Right now, the system supports around a dozen large drones, but the researchers are hoping to scale it up to work with "thousands" of flying bots no larger than a half-inch across. Those would act, in essence, like 3D physical pixels to form simulated objects. We'll admit it would be cool to see swarming drones form a flying 3D display, but in this case, donning a headset actually looks easier.