Why Google’s AI is Limited To Diagnosing Only 26 Skin Conditions…for Now

By Vinay Ramani On Dec 3, 2019

It was recently reported that Google’s AI system can accurately diagnose 26 skin conditions on par with US board-certified dermatologists. Results were published in the paper titled “A Deep Learning System for Differential Diagnosis of Skin Disease.” Yet, one source, the American Academy of Dermatology lists 70 common skin diseases. The British Skin Foundation presents an equally extensive list.

Google’s AI System can only detect 37 percent of skin conditions using the list of 70 referenced above. So, if AI is so smart, why can’t it detect all the skin conditions?

For starters, AI smarts can be summed up in two words: “precision” and “recall.” Precision is the accuracy of the machine’s prediction for a given question. Machines are not born smart. They have to be trained with a lot of data before they start to hit accuracy levels that one might consider “human-like” – for just one use case.

The more use cases a machine can successfully evaluate and have a meaningful prediction – the wider its “recall.” The ultimate AI machines are able to maintain a high degree of precision and a wide recall.

One key thing to notice in the Google announcement is that the diagnostic precision of AI to be applied to the 26 diseases is “on par” with US board-certified dermatologists. That statement says that the precision of the diagnosis is not perfect. Just as a highly trained dermatologist will not be 100 percent right. But, the dermatologist and the machine each use limited recall to get the right diagnosis most of the time.

In the world of Machine Learning, this balance between precision and recall is what I have coined the “Automation Continuum.”

As a basis here are some definitions. Automation is the use of technology to enable a machine, process, or system to perform activities more quickly, with greater accuracy, and without biases compared to human workers.

Automation Should Be Viewed as a Continuum

Automation is a continuum. Basic automation involves machines following pre-programmed rules to perform and repeat predictable tasks. Advanced automation enables machines to simulate what humans say, think, and do, including learning and improving with experience.

Basic automation: An early example of basic automation is using robots with workflow software to machine, weld, paint, and assemble parts in certain manufacturing environments. With advances in automation technology, machines are now also capable of performing less predictable or higher-skilled tasks without human intervention. Examples include cashiering in a fast-food restaurant and helping a customer open a bank account. Smarter machines also augment and extend human capabilities, offloading many of the routine administrative tasks of knowledge workers such as accountants, lawyers, financial advisors, and sales professionals, which frees them up for higher-value activities.

Advanced automation: Artificial Intelligence with Machine Learning adds another dimension to automation. AI is the ability of a machine, process, or system to perform complex tasks that require capabilities that used to be considered exclusive to humans, including being able to respond to stimuli in ways that are consistent with traditional responses from humans. These capabilities include understanding and interacting using human speech, interpreting and matching images, recognizing patterns in data, making recommendations, predicting outcomes, and learning from experience instead of being explicitly programmed.

AI-based systems can combine information from a variety of inputs, analyze the information rapidly, act on the insights derived from the information, and self-adjust based on successes and failures. This enables them to perform higher-level activities without human intervention, or further augment and extend human capabilities in applications such as disease diagnostics and treatment, security and surveillance, credit card fraud prevention, community-driven navigation, and self-driving vehicles.

Decisions About Automation Depend on What You Need to Accomplish

The level of automation necessary in a specific application depends on the following across a spectrum of complexity: 1) the available input required to perform a task and 2) the desired output. At one end of the spectrum, where task input is defined and the desired output is to execute a comparatively straightforward task, basic automation is likely sufficient. Examples include assembling automobile parts on an assembly line or processing a self-service food order.

At the other end of the spectrum, where task input is random and the desired output is an opinion or a prediction, an AI-based system with machine learning is required. One example is a self-driving vehicle that must predict whether a person standing on a street corner is going to step out into the street in front of the car. In this example, for obvious reasons, accuracy is essential. A self-driving system that might predict what is going to happen correctly 60 percent of the time is clearly inadequate. Over time, with more data and direct experience with pedestrian behavior, the system’s accuracy could improve to near-perfect, which is required in situations where an error could result in injury or even death.

In the case of AI in dermatology, of the 26 diseases that are “learned” through AI, the machine has sufficient accuracy for these 26 use cases. Although we are asking the machine to provide an opinion, the accuracy of the opinion is better than or at par with human dermatologists. For the other 44 skin diseases that are not yet learned, the machine probably has an opinion – albeit a not so accurate one. You can see how precision and recall play with each other… Yet, over time, AI will accurately diagnose all 70 diseases on par with US board-certified dermatologists.

AI in dermatology Artificial Intelligence Automation google machine learning skin diseases Vinay Ramani