Abschlussarbeit

Master Thesis

IMPLEMENTATION AND EVALUATION OF AN OPEN-SOURCE CHATBOT FOR PATIENT INFORMATION LEAFLETS

Kurzfassung / Abstract: The rapid evolution of Artificial Intelligence (AI) and chatbots presents a significant opportunity to address challenges in healthcare information accessibility. This thesis aims to bridge the gap between people with different health literacy levels and tries to facilitate the accessibility of medication information. The goal is to develop and evaluate an open-source chatbot, named MediChat, that provides users with reliable and easy-tounderstand medication information from package inserts. This was achieved by utilizing the Retrieval-Augmented Generation (RAG) approach and the open-source Large Language Model (LLM) Llama3.1 as well as the use of embeddings and a vector database. Considering this architecture, the chatbot reduces the risk of hallucinations, provides responses in simple language and the modular structure ensures adaptability for future enhancements. The holistic assessment was split into quantitative (accuracy, response time, precision, recall, F1-score) and qualitative (response time, grammar, task completion, quantity, relevance) metrics. The evaluation framework yielded an accuracy of 80% and a task completion rate of over 85% which demonstrates that, through the use of the latest AI-tools, medication information can easily be accessed and presented to people with different health literacy levels.

Zum Volltext: Download