Errors in Google Health

Google Health, the Personal Health Record (PHR), has quite a nasty problem. The Boston Globe reported that at least one patient had major errors in his record. The errors were quickly traced to Google's use of billing codes to drive its diagnosis and problem lists.

The good news is that Google 'fessed up and says it has learned from the episode. The bad news is that they may not have learned very well. Neil Versel reports that Google managers now say that "...Google Health would now allow free-text diagnoses that didn’t have to correspond to a billing code."

The underlying problem is not just that billing codes may be inaccurate, but that all data may be inaccurate, out-of-date, or misleading. Free text is notoriously difficult to work with and certainly prone to its own problems and errors. The problem of automatically constructing a patient profile from warehoused data is analogous to the problem of diagnosing a patient complaint. Very rarely does any one bit of data from history, physical exam, laboratory or imaging unambiguously make the diagnosis. Conflicting possibilities need to be resolved by assembling a (sometimes large) list of data points and finding the underlying pattern that explains them.

Likewise, the role for PHRs is not just to assemble the data about a patient. Making a diagnosis from those data requires very sophisticated algorithms that take into account the inherent quality of each data point, the process that produced it, how it relates to other data, when and where it was collected, and so forth. When you are all done, even the best systems are still going to have both false-positives (listing something not really true, like what happened to Dave deBronkart on Google) and false-negatives (missing something important).

How do we handle this on the Vermedx Diabetes Information System? The system provides very actionable advice to both providers and patients, so we need it to be very, very accurate and reliable. Before we do any contact with the patient, we do something a bit like what Google and other PHRs do: we automatically scour computerized records. In our case, the data are primarily clinical laboratory results which are generally much more reliable than billing codes. Then, we check with the primary care provider and ask them two critical questions: Is this patient truly diabetic? Are they really your patient? Only when we have two "yes" answers do we even offer to enroll the patient.

Even with this level of feedback and control, we still have false positive results. So, when we do contact the patient, we ask them to tell us if the information is incorrect. Some patients have moved, switched doctors or even died without the primary care practice knowing. And, sometimes the provider told us "yes" when they should have said "no."

The point is, we don't trust just data source. Even in this seemingly simple domain of a single diagnosis with good laboratory tests that are reliably reported, we need to combine multiple data sources to achieve high quality information. PHRs need to do more than copy a list of apparent diagnoses from one database to another. They need to explain the data's provenance, strengths and weaknesses, and help the user decide whether to accept, reject, or modify the tentative conclusions reached by the system. This is not easy, but when you do it well, there are big pay-offs!