AI Scans Tongue Color to Predict Diseases

For thousands of years, traditional Chinese medicine (TCM) practitioners have checked patients’ tongues as part of a comprehensive examination, carefully scrutinizing their color, shape and coating to try to detect disease. TCM considers the color of the tongue to be particularly revealing. Today, some researchers, encouraged by recent studies pointing to a measurable association with health factors, are working to adapt this old diagnostic approach to today’s AI-based technology.

TCM remains a controversial topic within the global scientific community. The World Health Organization officially added TCM diagnoses to the 11th revision of the International Classification of Diseases, the global standard for classifying health information, in 2022. But most high-profile studies have treated the topic cautiously. “Despite the growing use of TCM and recognition of its therapeutic benefits worldwide, the lack of robust evidence for EBM [evidence-based medicine] This perspective hinders the acceptance of TCM by the Western medicine community and its integration into mainstream health care,” wrote the authors of a 2015 journal article on the perspectives of TCM. Yet pockets of strong academic interest persist.

In TCM, the color of the tongue “is closely related to the state of blood and qi.” [a Chinese term often translated into English as ‘vital energy’]making it a primary indicator for TCM practitioners in assessing a patient’s overall health,” says Dong Xu, whose research at the University of Missouri focuses on computational biology and bioinformatics and who co-authored a 2022 study on analyzing digital tongue images. But tongue testing can be very subjective: it relies entirely on an individual practitioner’s perception and analysis of colors.

On supporting science journalism

If you enjoy this article, please consider supporting our award-winning journalism by subscription. By purchasing a subscription, you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.

Frank Scannapieco, a periodontist, microbiologist and oral biologist at the University at Buffalo, says that in Western medicine, no standardized clinical system is routinely used to monitor tongue characteristics, although defined lesions on the tongue can serve as indicators for certain cancers. Some studies have linked the appearance of the tongue to particular diseases such as breast cancer and psoriasis. Elizabeth Alpert, a dental health expert at the Harvard School of Dental Medicine, adds that tongue examination is often part of routine oral cancer screening by dentists and hygienists, but its accuracy depends on providers’ training and experience in clinical settings.

However, massive developments in computer technology are leading some medical researchers inspired by TCM to take a new look at the language. The authors of a 2024 study in Technologies used machine learning models to classify tongue colors and predict several associated conditions, including diabetes, asthma, COVID, and anemia, with a testing accuracy of 96.6%.

A major challenge in previous studies of tongue imaging was perceptual bias caused by varying lighting conditions, says Javaan Chahl, co-author of the recent study, a roboticist and co-chair of sensor systems at the University of South Australia. “There have been studies where people have tried to [diagnose via tongue color] without a controlled lighting environment, but color is very subjective,” explains Chahl.

To solve this problem, Chahl and his team developed a standardized lighting system in a kiosk configuration. The patients placed their head in a box illuminated by LED lights, which emitted a stable, controllable wavelength of light, and exposed their tongue.

Chahl and his colleagues collected 5,260 images, both real photographs of tongues found on the Internet and additional color gradient images. They used them to train machine learning models to recognize seven specific colors (red, yellow, green, blue, gray, white and pink) at different saturation levels and in different lighting conditions.

Researchers have confirmed that a healthy tongue generally appears pink with a thin white film; They found that a whiter-looking tongue can indicate a lack of iron in the blood. Diabetic patients often have a bluish-yellow tongue. A purple tongue with a thick fatty coating could indicate certain cancers. The intensity of COVID (in people already diagnosed) can also influence the overall color of the tongue, they found, with pale pink in mild cases, crimson in moderate infections, and dark red in severe cases.

Next, they applied the most accurate of six machine learning models tested to 60 tongue images, all taken using the team’s standardized kiosk at two hospitals in Iraq in 2022 and 2023. They then compared the experimental diagnoses with the patients’ medical records. “The system correctly identified 58 out of 60 images,” says study co-author Ali Al-Naji, now a professor of medical engineering at the Medium Technical University of Iraq.

Al-Naji is currently working to limit the diagnosis to the center and tip of the tongue. His group is also using a new tongue dataset consisting of 750 internet images to examine tongue shape and oral conditions such as ulcers and fissures with the YOLO deep learning algorithm. Ultimately, Chahl would like to analyze more than just the tongue, perhaps the entire face.

Tongue color may potentially serve as a useful biological marker of a person’s health status, but Xu cautions that it alone may not be enough to make accurate clinical decisions. “The most fundamental limitation of current tongue imaging systems is that tongue analysis represents only one element of a comprehensive TCM diagnosis,” he says. And because image labeling isn’t widely standardized for this type of experiment, he adds, it’s harder to replicate research findings.

The team has seen commercial interest in its system, Chahl says, but collecting usable data remains the biggest limitation to scaling up research: “You have to have a lot of different people involved in the process” to collect data with a kiosk in a large hospital, for example, and to obtain consent to access patient medical records.

Scannapieco also highlights the challenges of standardizing tongue testing in a clinical or research setting. He says broad AI-based language analysis would require massive investments and huge databases of images and medical histories. “Until then, I think the field will grow through the accumulation of small studies revealing correlations between tongue appearance and specific conditions,” Scannapieco says. “Of course, many diseases show no change in the appearance of the tongue.” He adds that such a tool would be just one of many used for diagnosis.

Meanwhile, online AI tools for language analysis are gradually gaining popularity among consumers. Earlier this year, Xu, his current Ph.D. University of Missouri student Jiacheng Xie and colleagues launched a GPT-based AI application, BenCao. Users can upload tongue images and receive personalized health advice based on TCM concepts.

For now, the app is designed and marketed solely as a “wellness” tool, rather than a clinical diagnostic system, because making a medical diagnosis requires much more caution. “We only provide some diet and lifestyle recommendations,” says Xie. To take it to the next level, his team aims to collaborate with clinical doctors, comparing diagnostic results from machine learning models and human doctors to identify differences and performance gaps.