Using its Xbox Kinect 3D sensor, a camera, a laptop and a laser pointer, Watch-Bot observed a week's worth of human activity in a kitchen and an office. During that time, it collected 458 videos -- about half of which included someone human deliberately "forgetting" to do something. The team then made Watch-Bot analyze the videos and use its unsupervised learning algorithm to determine which human actions were intentional and which ones -- like leaving the milk out on the counter -- were accidental. Using probabilistic learning models, Watch-Bot was able to independently figure out which actions the humans were forgetting.
When Watch-Bot does notice the clumsy human in the room forgot to do something, it quietly highlights that item with the laser pointer until it is put away or dealt with. Among the tasks that Watch-Bot was able to identify were such pet peeves as: not putting a book away after reading it, not turning a computer monitor off after using it, forgetting to put the milk away and leaving food in the microwave. Although it is only able to correctly identify those brainfarts about 60 percent of the time, according to the researchers, "most study participants thought that the robot was helpful," and not actually just a passive aggressive roommate.
One interesting thing to remember here: Watch-Bot also has no context for the objects in the room. It only acts based on probabilistic patterns it has observed and therefore has no idea what the milk even is, much less that it has an expiration date.