AI-Driven Differential Diagnosis: Leveraging RAG and LLMs in Intelligent Healthcare Systems

Pinna S.;Massa S. M.;Riboni D.

2025-01-01

---

Abstract

Artificial Intelligence is playing an increasingly prominent role in healthcare. For this integration to continue evolving effectively, professionals in the field, including doctors, psychologists, and nurses, must trust the technology. In the domain of Large Language Models, this trust requires transparency in the sources of information, ensuring they are verifiable and reviewed. This paper presents an intelligent diagnostic assistant designed to support differential diagnosis and disease comparison, leveraging an LLM enriched with the Retrieval-Augmented Generation technique to address trust-related challenges. The knowledge base for our system is PubMed, a biomedical article aggregator, which supplies relevant articles on the considered disorders, as well as evidence supporting or refuting potential links between them. Retrieval-Augmented Generation empowers the model to incorporate external knowledge tailored to the specific question while providing citations for its statements. These citations not only enhance the trustworthiness of the answers but also enable practitioners to explore the referenced articles in greater detail. We developed a prototype of our system, and conducted an evaluation through questionnaires administered to 12 professional psychologists. Results show that our system compares favorably with state of the art Large Language Models in providing the relevant information to distinguish between two diseases, and that the provided details are useful for supporting differential diagnosis.