One year later, Bixby is still half-baked

Oh, Bixby. What am I going to do with you? It's been a year since we first tried the virtual assistant on the Galaxy S8 and S8 Plus, and since then we've seen relatively little in terms of substantive improvements. It certainly doesn't help that Injong Rhee, Samsung's CTO and Bixby's de facto steward, recently left the company for a gig at Google. Few things seem more disheartening for Bixby's future than that.

Now that the Galaxy S9 and S9 Plus are on sale, I thought we should take some time to get reacquainted with Samsung's ambitious virtual assistant. The sad truth is, the version of Bixby installed on the Galaxy S9 and S9 Plus isn't that much better than what shipped on last year's Samsung flagships.

The world through Bixby's eyes

Bixby does a lot of things, but some of Samsung's most fascinating work has gone into Bixby Vision, a suite of seemingly useful image-recognition tools. Here's the rub, though: They're just about all powered by third-party services, and there's often little reason to use Bixby over any of those standalone apps. Vision is legitimately useful in that it provides a single place to access these functions, but it's hard to get excited when Samsung's main selling point comes down to convenience.

Samsung does, however, deserve credit for dramatically improving Bixby Vision's overall speed. With the Galaxy S8, you launched Bixby Vision through the camera, pointed it at something, and then chose what you wanted Bixby to do with that visual information after the fact. This time, Samsung streamlined the approach: You can swipe between those different Bixby modes before pointing the camera at an object. It might not sound like a profound change, but it definitely makes the Bixby experience feel more fluid.

As I write this, there are eight Bixby Vision modes, some more interesting than others. Consider the QR code scanner: It does its job well, and it's one of the few functions I could see myself using regularly (that is, if QR codes were still a thing). And, as was the case last year, Bixby is good at identifying bottles of wine through Vivino. Would-be sommeliers will probably get some regular use out of the feature, but it still feels like a gimmick whose novelty quickly wears off.

The rest of Bixby's Vision features are more ambitious. The new makeup mode would be more impressive if rival apps hadn't been on the market for a while now, but it still did a fine job shellacking my face with faux eyeliner. It's also nice to know that whatever digital makeup gets applied to your face is (probably) available at a nearby Sephora, though I don't think I'll be rushing out to pick up some Rouge Volupte Shine Oil-In-Stick Lipstick anytime soon. Still, Makeup mode works consistently well -- I can't say the same for everything else Bixby Vision does.

Live visual translations are a great idea in theory, but Bixby often faltered in the situations when I needed it most. Mi español is high-school-level at best, and in fairness, Bixby did a fine job translating small snippets of text like store signs and posters in Barcelona. Those translations weren't always perfect because Bixby leans on Google's overly literal Translate service, but they were often enough to get by on. It even handles certain potentially tricky examples, like handwritten Mandarin, fairly well. In other situations, like when you're holding the phone sideways to translate something wider than it is tall, Bixby barely works at all.

Where Bixby can also fall short when it tries to translate lots of text at once, though your mileage may vary. I've often had trouble getting to translate segments of a book, or large swaths of a restaurant menu page -- less than ideal when you're trying to sort out lunch in Barcelona. Whether or not this is a bummer falls to you, but I couldn't help but feel disappointed. Maybe that's my fault for expecting too much of a new feature, but no matter: Google's own Translate app generally handles live translations (even weirdly oriented ones) much better, and there's no reason you shouldn't just use that.

What I've always wanted, even more than a universal live translator, is a device that can identify objects out in the world so I can buy them. Bixby isn't awful at this, but there's a stark difference in the way its two most potentially useful image recognition modes work. More often, the so-called Image mode works better than expected -- it handily surfaces photos of smartwatches when I pointed it at my Apple Watch, and was quick to offer results for things that looked like my TV's remote control. All of those image results come by way of Pinterest, and the service's broad database of tagged images really seems to come in handy. Image mode even managed to tell that the Adidas sneakers I pointed it at were specifically UltraBoosts. I can live with that.

Unfortunately, there's no connection between the results generated by Image mode and the stuff that pops up when you try to use Shopping mode. That level of granularity about product makes and models doesn't exist when you're trying to use Bixby to buy stuff -- you'll get results for Apple Watch bands instead of the Watch itself, generic Adidas kicks when the camera is pointed at those aforementioned UltraBoost. The list goes on. It's just not terribly helpful. What makes this whole thing more complicated is the fact that when this feature went live on the S8s last year, it pulled shopping results straight from Amazon's near-infinite inventory. For whatever reason, Bixby Shopping on our S9 review unit only shows results from Walmart. That's not necessarily a bad thing, and Samsung confirmed that Bixby would soon scour inventories at Sam's Club, Lowes, and Nordstrom, but Amazon's absence is still highly noticeable.

Bixby now also doubles as a calorie counter -- just not a very good one. Samsung first showed it off at CES earlier this year, and even at that early stage, things seemed amiss. Never mind the fact that Bixby can't identify multiple items on a plate -- there's no way for it to interpret the items of a single dish, making most of its calorie estimates little more than wild guesses. The S Health integration is a nice touch, but when Bixby can't even identify a bowl of noodles , the feature borders on useless. You're much better off taking the few extra seconds to log food in an app like MyFitnessPal.

Ultimately, what feels most disappointing about Bixby Vision is how great it could be. Having all of these features in one place is a powerful proposition, but it doesn't count for much when many of the features themselves fall short in their own ways. To be fair to Samsung, building a suite of sophisticated image recognition features into a smartphone is no small feat. Even Google's image-recognizing Lens feature is still a work in progress. At least Google has the courtesy to call Lens a preview. After a year, Bixby Vision still feels like one too, and most people who had hoped they could rely on it will likely come away disappointed.

Talking to Bixby

When Bixby launched on the S8s last year, you couldn't actually talk to it; you had to make do with Bixby's limited Vision functions and a homescreen panel that wasn't much more than Samsung's spin on Google Now. The ability to have a conversation with Bixby came months later as part of an update that was itself late because Samsung needed more time to test the assistant with actual English speakers. Things are thankfully better now, but not much.

For one, Bixby guides you through a setup process where you train it to understand your understand your voice, but it doesn't seem to make much difference. Even after multiple attempts, I couldn't count on a "Hi, Bixby" to consistently launch the assistant. The only way I've been able to get Bixby's voice recognition to work with any decent level of consistency was to turn up the system's audio sensitivity, which theoretically makes it more prone to false positives. Good thing there's a physical Bixby button, right? Erm, about that. Its placement on the S9's left side means I've accidentally mashed it countless times while trying to change the phone's volume. Thankfully, it's easy enough to disable the Bixby key entirely -- just jump into the Bixby Home panel and flick a switch.

Once those issues are squared away, actually talking to Bixby works well. It lags behind Google Assistant by a large margin when it comes to answering general knowledge questions -- if that's any part of your daily routine, you should really just default to Google. Bixby excels at handling more complex kinds of system commands that Google is only starting to get into with partners like LG. I spend most of my time customizing smartphones with new launchers and add-ons, and it's been great asking Bixby to "open default apps" so I can switch between different homescreen setups. Want to fire up your mobile hotspot without having to dig into the device settings? Just ask. In moments of ultimate laziness, I've also asked Bixby to "scroll all the way to the bottom" of the story I'm reading, just to see how it ends. Bixby always made it happen.

Bixby's value as a voice assistant is clear: It's capable of quickly performing actions that otherwise would've required a few taps and a few moments to remember where you had to go to do it. As a result, it feels more like a tool for power users than it does a feature that most people would bother to use on a regular basis. In that, Bixby Voice is the polar opposite of Bixby Vision -- with the latter, you might want to trust it but you can't. And the former? You can rely on it, but you probably won't.

To be clear, I don't think Bixby is a failure. The strength of its voice controls alone make me glad Samsung decided to embark on this anguishing process, and it bodes well for a future where Samsung expects us to talk to our TVs and refrigerators. With all that said, I'm running out of patience for Samsung's assistant -- at least in its current form. A little more consistency would go a long way, but for now, I just might disable that Bixby button, delete my Bixby Vision shortcut and live the Google Assistant life instead.