It's tempting to treat IBM's Watson as a cure-all: just throw some cognitive computing at the problem and you'll make everything better. That can only happen if it's well-implemented, however, and we've just seen what happens when things go awry. The University of Texas' MD Anderson Cancer Center has put its highly-touted Watson project (the Oncology Expert Advisor) on hold after an audit discovered both spending issues and an unfocused strategy that didn't meet goals.
Where tech companies often foot the bill for projects like this, MD Anderson paid the $62 million for the project itself, ducking around usual procedures in the process. Former genomic medicine chair (and wife of MD Anderson's president) Lynda Chin reportedly didn't get approval from the center's IT department, and ensured that payments to IBM were just low enough to not require board approval. To make matters worse, payments were made from donations that hadn't been received, and didn't hinge on meeting tangible goals.
The actual technology, which mines patient and research databases for recommended treatments, wasn't necessarily at fault. While the audit says Watson is "not ready" for full-fledged use, doctors also said they were impressed -- the main issue is one of constantly switching focuses and technology. The emphasis switched from leukemia to lung cancer, and a change in medical record software rendered Watson incompatible without an overhaul. Moreover, the info used to feed Watson is outdated.
This doesn't mean that the project is dead. Even though MD Anderson tells Forbes that it's soliciting competitive bids to "see where the industry has progressed," its audit stresses that this isn't an opinion on Watson's abilities. IBM tells us that it's "proud" of its work, pointing out the audit's finding that Watson managed 90 percent accuracy when supporting lung cancer treatment decisions (you can read the full statement below). However, it's clear that a lot of work is needed to make this system useful in the real world. And regardless of whether or not that happens, MD Anderson is learning an expensive lesson in using AI wisely.
"This report assessed procurement practices, not the value or functionality of the OEA system.
"IBM is proud of the early research work that was accomplished to apply Watson technology in the Oncology Expert Advisor program at MD Anderson Cancer Center. We created an oncology advisor for lung cancer and leukemia, and as reported in the state's finding, Watson achieved 90% accuracy in lung cancer decision support.
"IBM Watson Health is successfully working with businesses, scientists, researchers and governments worldwide to solve some of the biggest healthcare challenges."