Watson Gets a Job in Healthcare: A NewPublicHealth Q&A with Herbert Chase

Sep 27, 2011, 5:42 PM, Posted by

file Herbert Chase, Columbia University

Insurer WellPoint will use IBM’s “Jeopardy”-playing Watson computer system to offer diagnosis and treatment suggestions to physicians, particularly those who manage patients with complex chronic conditions, according to a news release. NewPublicHealth spoke with Herbert Chase, M.D., Professor of Clinical Medicine in the College of Physicians and Surgeons and the Department of Biomedical Informatics at Columbia University, and a consultant to the Watson project, about the potential for Watson to improve quality of care, ways they're checking up on Watson's accuracy, and what’s up next for Watson – trolling the blogosphere for patient-driven data.

NewPublicHealth: What are the hopes for Watson in informing treatment and diagnosis?

Dr. Herbert Chase: Just as background, diagnostic error represents a third of medical error – that includes diagnoses that are missed, delayed or wrong. Watson can help physician keep an open mind about the diagnostic possibilities. The classic error in differential diagnosis is to narrow thinking early on and then forget. Watson even now can develop a pretty complete list of diagnoses.

Treatment is a much bigger space because the same treatment doesn’t apply to all patients with same disease. You don’t need Watson to diagnose high blood pressure – you can do that at Walgreens.You may, however, need Watson to treat it. No human can compute all the gene, risk factor and drug interactions with the right possibilities for treatment. Treatment uses of Watson will be far more useful and compelling.

NewPublicHealth: What about the use of Watson to cull data from patients themselves?

Dr. Herbert Chase: Patients have an incredible amount of experience with treatments and feedback on their usefulness and side effects, and they share these with each other on blogs. Watson will be able to cull that in the future. The web is filled with data from all these “informal experiments” that will never be the subject for randomized controlled trials, but are useful data nonetheless.

As an example, I took Lipitor and developed insomnia, which I never had before. It took me a few days to figure out what was causing it, but when I did I went to the medical text books, and there was nothing there on insomnia as a side effect. I went to PubMed, and I’m a pretty good searcher, and there was nothing there. When I went to the blogs, though, it was full of people saying, “Don’t take Lipitor, it will keep you up at night.” What’s interesting about Watson is that it can understand natural language, so it will know that “can’t go to sleep” is the same as “insomnia.”

My colleagues in evidence-based medicine, of which I consider myself a card-carrying member, are appalled by this, but there’s tremendous potential in tapping this type of data.

NewPublicHealth: What are other potential uses for a super-computer like Watson in health care?

Dr. Herbert Chase: We’re already working on diagnosis and treatment. The next step is to take Watson into electronic health records (EHRs). Watson doesn’t need to be prompted by a human being. It can be working behind the scenes in the EHR and Watson might alert the doctor that based on these symptoms pulled from the consultation notes, you might want to consider this disease.

Other decision support systems are based on lab reporting. This reads natural language straight from the notes.

You could imagine different uses. If a patient wants a test and the doctor seeks approval from the insurance company, Watson can help find the evidence for whether or not a test is useful and accurate, and can deny payment if it’s not. Wellpoint feels it will be very helpful to give evidence from textbooks and PubMed back to the physician, whether they approve or disapprove, so they’ll know why the decision was made.

NewPublicHealth: How do we know Watson will be accurate? How was it tested?

Dr. Herbert Chase: To step back, you’re only as good as the source. There’s a concept that’s very profound for the medical profession called the evidence pyramid. At very top, with the smallest volume, is the very best evidence, which generally can be found in text books. It’s been highly vetted. Most of the time when someone has a medical question, it’s answerable in the textbooks. It’s so hard to search the textbooks alone, that you need a Watson-like 2 million pages/second machine. The worst evidence, which seems paradoxical, is in PubMed. This will include articles the drug companies wrote, articles that got nixed once they were repeated, etc. But [this category of evidence] is vast.

With Watson, we point the arrow to the top of the pyramid, then work our way down – but we only go there if we need it.

NewPublicHealth: How was Watson tested?

Dr. Herbert Chase: It’s an ongoing process, and that’s where we have our team of Columbia University medical students. We literally look at the output from Watson and sit around and decide how close is it to the accurate diagnosis or treatment options. We’re literally grinding through thousands of questions. It would be tedious to anyone else but since we love learning, it’s tremendous for us. We’re wrapping up the experiments now.

It’s important to note that unlike "Jeopardy" where there is “the best answer,” we don’t want one answer. We want a list of things that can be important. I could ask what’s the probability of this disease given this symptom, or I could ask what’s the probability of this symptom given this disease. Those are two different sets of answers. We want Watson to show us both lists because among the list might be an incredibly uncommon disease – that you might have.

NewPublicHealth: Anything else you want to add?

Dr. Herbert Chase: As we become electronically connected, on a national grid, Watson’s computing power and the ability to find like patients from which to cull information will grow exponentially. If you’re looking for a treatment and what you’ve tried has failed – blogs have some info, but if everyone served through an EHR was accessible, Watson could find patients at other institutions that might inform treatment for a particular patient.

Size matters in computers. Processing information at two million pages per second is fundamentally different from what other computers can do. The core of why Watson is so good is because it’s such a great computer. In public health we look for trends. If everyone is connected to the same network, the potential for picking up on trends in the data is tremendous, and you need a big computer to process all that data.

>>View more stories on technology in public health here.

This commentary originally appeared on the RWJF New Public Health blog.