Pleco 2.2 Chinese Dictionary uses iPhone camera to translate text in real time (video)
By Thomas Ricker 
posted August 16th 2010 5:42AM
posted August 16th 2010 5:42AM
Can't say we've heard of the Pleco Chinese Dictionary app prior to today even though it's been around for two full version releases. Of course, we've seen plenty of Google Goggle-like smartphone apps that can snap pictures and then convert the image to text using optical character recognition (OCR) -- something that can be incredibly useful when important stacks of business cards. Pleco 2.2, however, is one of the first apps we've seen that can translate text in real time using the live image presented on your phone's camera display. The new feature, targeted for release in September, requires iOS 4 and should work with either the iPhone 4 or older 3GS. Sure, you might not need this app every day, but it might just be the difference between ordering chop suey or a thousand-year old egg garnished with a side of stinky tofu. See it in action after the break.
Wow
@rybek There is actually a food called "Thousand year old Stinky Tofu" It's spoiled Stinky Tofu. Stinky tofu good. Spoiled Tofu= Fungus tofu that's been sitting in a bucket of other spoiled tofu that's been transferred to spoil with other tofu for months.
Hmm, wonder what Engadget has been eating lately. Stinky tofu... perhaps?
@rybek Agreed - Totally WOW...
I also love that retro-tech green overlay vibe they have going on there...
I wonder if that translation would work on outdoor signs at an angle, if you can't square them up in your phone.
Damn, the future is coming super-fast in some areas. Battery life, hair-loss cures, cancer and disease solutions? Not as fast.
@aznipredroid I'll choose stinky tofu and century egg over chop suey any day.
@aznipredroid
Stinky tofu is good. It might stink but it tastes good.
@rybek
Oh, future you scary.
@aznipredroid
Kimchee, Century Egg, Stinky Tofu, Miso and Fugu Salad W/Lemongrass Lime Ponzu Vin? Ol' fermenting death gamble, an appetizer in duality.
Okay after all these years, someone from the States please explain to me what is "Chop Suey"? I'm Chinese, widely-travelled and have a pot-belly for a reason.
@darkmax I think is some kind of "mix fry vegetable" thing. I am a Cantonese(Chinese). This is basically what it said from the dictionary. "Chop Suey" is 杂碎 in Cantonese. Which is like some kind of mix fry vegetable.lol
@hey buddy Outdoor signs generally work as long as they're clearly visible and in a fairly standard font, but recognizing them at an angle is a bit of a stretch - getting the recognition engine up to 5 FPS (which is about what we're clocking it at now) required shedding some processing-intensive OCR features and skew correction is one of them. We might add an option for a slower-but-skew-correcting mode in a future release, though - we've got the code already.
This is what I want !!! But on Android PLEASEEEEEEEEEEEEEE!!!
@ewlung The dictionary itself is coming to Android (check their forums), but the OCR isn't slated for initial release with it.
@ewlung That the Farking sweetest thing I have read for a while (IMO). Man, dude, what? Agreed, to wanting this on Android.
@ewlung I think it's the most popular comment nowadays.
@rybek I don't care what's popular. I just want stuff like this on as many platforms as possible iOS4, Android, Symbian, WinMo7...Blackberry, wtv. Yay, for technology.
@aznipredroid I am an iPhone developer and to speak truth it's a big pain in the ass to port app to other platform, that has different interface, frameworks, resolutions and everything. Maybe if several people make the app it's easier, for one single developer it's a lot of work.
I don't even think of writing things for Android Market, because here in Poland it is not supported and AppStore fully is. Not very nice, Google, your're loosing a lot of potential buyers in my country.
I want this for android too. I don't have android or iOS but am planning to get an android soon. I hope this isn't going to be an iOS only app. Also, more languages?
This is sweet. How long does acquisition take? Accuracy?
@xsacha
Argh, I just found out this app feature should already be available on WM? And has been for a long time? Well I'll download that now.
Thanks
@xsacha Pleco has been on WinMo for a long time, but the OCR feature has not, and likely won't be arriving. I know they've moved some features over from the iPhone version, but active WinMo support is winding down to the essentials. The age of WinMo and Palm have passed, and the age of iOS and Android are upon us.
@rybek How could I contact you? I have one app idea.
@aznipredroid We're not promising OCR for Android yet mainly because we're worried Google might beat us to the punch - unlike Apple, they've already got voice recognition and even limited OCR (through Goggles) built into their OS, and it would kind of suck to invest lots and lots of money in porting our OCR module over to Android only to discover that Google had just released the same thing for free. (look what happened to Vlingo just yesterday)
If they drag their feet about an official OCR library for Android, or if it doesn't support Chinese / is too slow or inaccurate in general to be usable for something like this, we'll certainly consider bringing OCR to our Android software, but at the very least we'd like to wait until they announce the feature list for Android 3.0 before making a decision on that.
@xsacha More languages might come, but that depends on how quickly other apps with this same sort of capability start to emerge - if there are half a dozen Japanese dictionaries doing OCR by Christmas it probably isn't worth our time to develop yet another one :-)
For what it's worth, the app is a full-fledged dictionary/learning aid - if you're studying Chinese, this is *essential*. If you're not, it's probably does a lot more than you wanted. On the other hand, the app is free, and you just pay for the addons you want.
@ipsi It's never really been hard to just hire up a Chinese-English dictionary and copy the charatcter using handwriting recongition, most touchscreen phones can do that, but this is more convienent. Its usefulness also depends on how well it can recongize phrases and context.
@YpoCaramel That's true. I was referring more to the full set of features available in the application, which I generally consider to be essential for any Chinese learner, as opposed to just this feature.
@YpoCaramel
not true, Chinese characters aren't recognized by how it looks, instead, the handwriting software recognizes it based on the strokes used to write the character. write it wrong, and your word is not recognized.
@samleung Partially true. Using the correct stroke order helps, but is not actually necessary most of the time. Trying to use any form of HWR would be murder if you needed to be 100% correct.
@samleung yea, it's not that hard. Handwriting tablets are pretty good these days, never had a problem with non-recognition.
Real time?
NEAT
I can do that with my DSi, N900 and my 10 year old Pocket Loox pda with cf camera :p Photo OCR is very usefull in Asia!
@MAIcrosoft And now I can do it on my iPhone with this free app. No need for me to buy any of that other stuff.
The 2.2v is not released yet??
Sometimes the new technology is really incredible! Even though this first iteration may not work perfectly, I am sure in a few years it will be useful!
@Daanmannetje I'm sure some chinese people would say the exact same thing about our latin alphabet! (minus the scribbling part :)
@Daanmannetje It's actually not that hard. Requires a fair bit of memorization, but after you make it through a few you'll start seeing patterns in the characters, which makes later ones progressively easier to remember. Though those patterns can also cause confusion because they often result in very similar, but different, characters.
"when important stacks of business cards."
I assume that it should read "importing". Lately I'm seeing quite a bit of mistakes of this nature on Engadget. Does anyone bother to proofread anymore? They're pretty obvious. At least one or two more people should read the articles (shouldn't take much time since it's only one paragraph) before publishing. Engadget is not your average blog, and shouldn't have your average grammatical errors in it.
@solmar
"Lipstick on a pig, Polish on a turd," are all generally applicable here.
good times, if character recognition and voice recognition keeps improving , we might really see something in our lifetime that is close to the universal translator from star trek.
Mount it up in a pair of glasses with a heads up display. Real time translation for tourists. Think we will get there? I do.
this is nice i really like that and i think its a very useful tool.
love it! great to see real time translations in your hand from your phone coming to market using the on board camera for OCR.
@stevenf17 probably including the scribbling part!
Kick ass
Any idea of the mobile data roaming rates in China? This looks like a must have, but is it a can afford given current data costs when roaming ?
@brianM Shouldn't need to use data. My understanding from the forum thread about it is that all the logic is part of Pleco and internet access is not required.
Merci @ipsi
@stevenf17
maybe not, http://www.bbc.co.uk/news/world-asia-pacific-10982393
"Chinese jettison rare surname unrecognised by computer". there's some advantages to sticking to latin alphabet, especially with movable type(despite mt being invented in china, it's far more practical with latin alphabet, there's other advantages as well, especially in the effort it takes to learn).
@glassfin Yeah, that's actually been an interesting little trend - in the olden days it was common and even trendy for Chinese parents to dig up really really obscure characters that nobody had used for hundreds of years and use them in their children's names, but once computers came along it got to be problematic to have a name containing anything outside of the 6,763 covered by the most universally-accepted mainland character set (GB-2312) - I actually had a friend who had to have her name handwritten on airplane tickets and even on her ID card because the computers that printed them didn't support a character in it.
Goddamn, this fellow has a really steady grip, You should see me trying to scan a barcode.
@tommalota Good point! Is there a hangover remover app yet?