Disney Research taught AI how to judge short stories

It's no literary critique, but these neural networks are good at predicting what's popular.

Christopher Robbins via Getty Images

Disney researchers have been coming up with some striking new technology lately, including a method for real-time speech animation, shared augmented reality and some creepy face-projection tech for live performances. Now, researchers at Disney and the University of Massachusetts Boston have been working on neural networks that can evaluate short stories. While these AIs don't (yet) analyze story like a professional literary critic, the software tries to predict which stories will be most popular. "Our neural networks had some success in predicting the popularity of stories," said Disney Research scientist Boyang "Albert" Li in a statement. "You can't yet use them to pick out winners for your local writing competition, but they can be used to guide future research."

The researchers used social question and answer site Quora for a large database to feed into its AI algorithms. Many of the answers on Quora come in the form of stories, so reader upvotes can be used as a measure of popularity, and as "a proxy for narrative quality." The team gathered almost 55,000 answers and classified more than 28,000 of them as stories, each with an average of 369 words. Then they developed a couple of different neural networks — one to look at different sections of each story and one to take a more holistic view of a story's meaning. Each AI made predictions about the relative popularity of a given story. Both neural nets were better at choosing a story's popularity over a baseline text evaluation, but the holistic network showed an 18 percent improvement over the one that focused on sections.

It's not hard to imagine a movie studio, for example, using a future version of this type of technology to choose scripts for production, of course, but the tech is still in its infancy. Let's just hope that researchers find a way to filter stories for quality, and not just popularity. No one needs another Transformers movie.