The Neural Alpha launched last year did away with all those additional layers and instead relied solely on a neural network for predictions. A neural network is a loose term that defines an algorithmic system, modeled on the way the brain processes information, that can learn from datasets. To train its network, SwiftKey used millions of complete sentences and applied "tags" to each word. These tagged terms helped the network to understand what the sentences "meant," or more accurately, how they were structured. This tagged database essentially created a broad pool of interconnected synonyms, but rather than linking words by meaning, like a thesaurus, SwiftKey's database links them by their linguistic use.
The SwiftKey that users will be updating to today is fully trained. It's trained to use its database of tags to examine entire sentences, stringing together words as though they were code to find more accurate suggestions. The sequence of tags that makes up "It might take a" will throw up more suggestions than can possibly be displayed on-screen, but the neural network puts a probability on each and displays the three most likely.
Neural Alpha's prediction system was obviously, at its core, far superior to the n-gram method. So why has it taken 11 months for SwiftKey's users to feel the benefit? Aside from the usual stability and quality-assurance side of things, there were some major hurdles to overcome. First, last year's release was powered by a phone's GPU, which drastically limited the number of devices that can run it. Second, while Neural Alpha outperforms n-gram, it didn't always outperform SwiftKey's personalized predictions engine.
Over the past year, engineers have been working on both those issues. It very quickly became apparent that using GPUs to power the network was not a viable long-term option. Yes, GPUs are better suited to running the math, but there are literally thousands of different Android phones out there, all with slightly different configurations. Executing code on the many different Android GPUs around just isn't practical, and there was no way to use cloud computing for something that needs to always be working.
What nearly every Android phone does have is an ARM or ARM-compatible processor inside. With that base level of compatibility to work from, SwiftKey reworked its engine to run entirely off the CPU. Naturally, that brings up a worry of speed being affected -- the last thing you want is a laggy keyboard. But speaking with Engadget, SwiftKey Project Manager Ben Leavett said: "Across all the devices we currently support, people will be able to use this tech and there will not be a discernible difference in speed."
As for personalized predictions, the solution it went with seems so obvious: Run both the customizable n-gram engine and the neural network simultaneously and have them compete against each other for your keyboard's affection. Leavett compared the competition as the two engines trying to see "which can shout the loudest." The more technical explanation is that both models attach a probability of their predictions being correct, and the app displays the top three.
Part of the challenge of combining these engines was that SwiftKey had to balance the "volume" of these "shouted" predictions. The n-gram system's opinion of its guessing abilities was far higher than it should've been when compared to the neural network's. In the final app, for the majority of predictions at least, the neural network will now win out. But when you're typing a phrase you use often, the n-gram system will jump in and "shout louder," and its suggestion will be the first you see.
Running local neural networks is only just becoming viable on smartphones. Apps like Google Translate can process translations using machine learning, and in recent months Prisma has added offline processing to its iOS app. But neither of these apps are used with the same frequency as you use your keyboard. And SwiftKey's Android install base is in the "hundreds of millions" range. Overnight, the app's neural network will become one of the most used in the world, and almost certainly the most used on mobile. And very few people will know.
SwiftKey decided not to make a big deal about the switch from n-gram to neural network. The team had considered some sort of notification on installing the update, but quickly realized the tough truth: The majority of users couldn't care less what's happening behind the pixels on their phone display. Informing users of the change in a disruptive way was more likely to draw ire and confusion than a celebratory applause for the engineering team. "If it makes the experience better, they're fine with it," Leavett explained. Hopefully, it will.