Georgia Tech researchers teach robots to be mechanical MacGyvers

There weren't many sticky situations that '80s television action hero MacGyver couldn't slip out of with the help of a disposable lighter, penknife and two tabs of Alka Seltzer. His unconventional application of scientific principles and outside the box thinking were what made him such a formidable opponent week after week. Now, a team of researchers from Georgia Tech's RAIL research lab is working to impart those same survival skills into robots.

The RAIL team, led by PhD student Lakshmi Nair, focused its research on teaching a robot to fabricate tools -- hammers, screwdrivers, ladles and the like -- out of whatever materials are handy. "So if a robot needs to solve the task that uses a hammer," Nair explained to Engadget. "They can combine a stick and a stone, for instance, to be able to make a hammer."

But this is much more than just showing the robot a picture of a hammer and telling it to make something that looks like it. "It's situation-specific," Nair said. "So given a particular situation, if it wants to hammer something, it figures out which objects to put together. So you're not giving it a specific example of a hammer, just telling it what the situation is."

Once the robot knows what it needs to do, it will evaluate the materials in its workspace based on their shape and how they can be attached to one another. By leveraging supervised machine learning, the RAIL team taught the system to match objects by their relative shapes and perceived functions.

"Essentially, the robot is taught to match form to function so it learns things like the concavity of bowls enables it to hold liquids, for instance," Nair said. The system is fed labeled examples of everyday objects so that when presented with a new set of items that it hasn't seen before, it can use what it learned beforehand to reason about the unknown objects in front of it. Cups and tongs combine to create ladles, those same tongs and a coin become an improvised flathead screwdriver while piercing a rectangular foam block with a poker makes a DIY squeegee.

Although the robot does use a handheld spectrometer to determine whether an item is pierceable or not, it can't effectively determine the material properties of what it's looking at which is why it fabricated a number of hammers out of foam blocks. "Right now, it just looks at shape as one of the main ways to reason about which part to use for the construction of the tool," Nair explained. "Some of our future ongoing work looks at incorporating material reasoning so that it would construct hammers or more sturdy material that can then be used in the actual application."

Nair's team is not alone in its robotic tool building research. In fact, a team from Tufts University suggested The MacGyver Test as a means of evaluating a robot's resourcefulness and creativity in 2017 -- a practical alternative to the Turing Test, which scores a robot's loquaciousness.

"The proposed evaluation framework," the Tuft's team wrote, "based on the idea of MacGyver-esque creativity, is intended to answer the question whether embodied machines can generate, execute and learn strategies for identifying and solving seemingly unsolvable real-world problems."

But does that constitute cleverness or resourcefulness? In the TV show, MacGyver's gadgets and contraptions were so effective because his plans exploited some underlying scientific or physics principle. Professor Nathan Michael, Director of the RISLab at Carnegie Mellon University argues that these systems are performing the same creative task as Richard Dean Anderson's character.

"A lot of the problem-solving challenges that arise when systems are engaging in executive-level task planning share a lot of similarities to this idea of building tools," he told Engadget. "In fact, the underlying algorithmic framework employed for the idea of problem-solving itself is one basically trying to optimize or find a solution within a particular set of constraints subject to a particular condition."

The way that these problems are handled depends on the chosen methodology, which in turn "depends largely on the nature of the problem," he continued. "What we're talking about here is kind of like a larger problem of reasoning. The question of cleverness or resourcefulness really comes down to, within that context, the ability for the system to quantify that which it can or cannot do, those resources that it can or cannot leverage, and its ability to find a way to solve that particular problem through the application of those resources."

There are plenty of places in the real world where such systems can find use Nair said, "any application that involves repair and improvisation." In addition to helping out with light home repairs, Nair anticipates that this technology will make it into space before too long. "You have space exploration where you could potentially send these robots in beforehand and then have them use the resources available to construct habitats before humans get there," she continued. However, the system still faces a number of technological challenges to overcome -- whether they're clunky manipulator arms or cameras that can't reliably detect metallic items -- before we see them in space.