SwiftKey for Android is now powered by a neural network

From today, the popular keyboard app SwiftKey will be powered by a neural network. The latest version of the app combines the features of its Neural Alpha, released last October, and its regular app in order to serve better predictions. It's the first major change to the main SwiftKey app since Microsoft acquired the London-based company earlier this year.

Understanding why the new SwiftKey is going to be better than what came before it requires a little effort, but the real-world benefits are definitely tangible. See, the regular SwiftKey app has, since its inception, used a probability-based language algorithm based on the "n-gram" model for predictions. At its core, the system read the last two words you've written, checked them against a large database and picked three words it thought might come next, in order of probability.

That two-word constraint is a serious problem for predicting what a user is trying to say. If I were to ask you to guess the word that comes after the fragment "It might take a," the first suggestion you come up with is unlikely to be "look." But with a two-word prediction engine, it's only looking at "take a," and "look" is the first suggestion. There had to be a better solution. Simply upping the number of words it looks at is impractical -- the database grows exponentially with every word you add -- so SwiftKey's initial solution was to boost its n-gram engine with less fallible, personalized data. If you regularly use phrases, SwiftKey uses that data to improve predictions. And you could also link social media and Gmail accounts for better predictions.

The Neural Alpha launched last year did away with all those additional layers and instead relied solely on a neural network for predictions. A neural network is a loose term that defines an algorithmic system, modeled on the way the brain processes information, that can learn from datasets. To train its network, SwiftKey used millions of complete sentences and applied "tags" to each word. These tagged terms helped the network to understand what the sentences "meant," or more accurately, how they were structured. This tagged database essentially created a broad pool of interconnected synonyms, but rather than linking words by meaning, like a thesaurus, SwiftKey's database links them by their linguistic use.

The SwiftKey that users will be updating to today is fully trained. It's trained to use its database of tags to examine entire sentences, stringing together words as though they were code to find more accurate suggestions. The sequence of tags that makes up "It might take a" will throw up more suggestions than can possibly be displayed on-screen, but the neural network puts a probability on each and displays the three most likely.

Neural Alpha's prediction system was obviously, at its core, far superior to the n-gram method. So why has it taken 11 months for SwiftKey's users to feel the benefit? Aside from the usual stability and quality-assurance side of things, there were some major hurdles to overcome. First, last year's release was powered by a phone's GPU, which drastically limited the number of devices that can run it. Second, while Neural Alpha outperforms n-gram, it didn't always outperform SwiftKey's personalized predictions engine.

Over the past year, engineers have been working on both those issues. It very quickly became apparent that using GPUs to power the network was not a viable long-term option. Yes, GPUs are better suited to running the math, but there are literally thousands of different Android phones out there, all with slightly different configurations. Executing code on the many different Android GPUs around just isn't practical, and there was no way to use cloud computing for something that needs to always be working.

What nearly every Android phone does have is an ARM or ARM-compatible processor inside. With that base level of compatibility to work from, SwiftKey reworked its engine to run entirely off the CPU. Naturally, that brings up a worry of speed being affected -- the last thing you want is a laggy keyboard. But speaking with Engadget, SwiftKey Project Manager Ben Leavett said: "Across all the devices we currently support, people will be able to use this tech and there will not be a discernible difference in speed."

As for personalized predictions, the solution it went with seems so obvious: Run both the customizable n-gram engine and the neural network simultaneously and have them compete against each other for your keyboard's affection. Leavett compared the competition as the two engines trying to see "which can shout the loudest." The more technical explanation is that both models attach a probability of their predictions being correct, and the app displays the top three.

Part of the challenge of combining these engines was that SwiftKey had to balance the "volume" of these "shouted" predictions. The n-gram system's opinion of its guessing abilities was far higher than it should've been when compared to the neural network's. In the final app, for the majority of predictions at least, the neural network will now win out. But when you're typing a phrase you use often, the n-gram system will jump in and "shout louder," and its suggestion will be the first you see.

Running local neural networks is only just becoming viable on smartphones. Apps like Google Translate can process translations using machine learning, and in recent months Prisma has added offline processing to its iOS app. But neither of these apps are used with the same frequency as you use your keyboard. And SwiftKey's Android install base is in the "hundreds of millions" range. Overnight, the app's neural network will become one of the most used in the world, and almost certainly the most used on mobile. And very few people will know.

SwiftKey decided not to make a big deal about the switch from n-gram to neural network. The team had considered some sort of notification on installing the update, but quickly realized the tough truth: The majority of users couldn't care less what's happening behind the pixels on their phone display. Informing users of the change in a disruptive way was more likely to draw ire and confusion than a celebratory applause for the engineering team. "If it makes the experience better, they're fine with it," Leavett explained. Hopefully, it will.