Large language models and retrieval-augmented generation-based chatbot for adolescent mental health

Abstract

Access to fast and efficient information is crucial in today's digital era, especially for teenagers in obtaining mental health services. The manual method used by Youth Information and Counselling Centre (PIK R) to provide mental health information requires significant time and effort. This research presents an AI-based solution by developing a chatbot system using retrieval-augmented generation (RAG) and large language models (LLM). This chatbot is designed to provide accurate and effective mental health information for teenagers throughout the day. An analysis of a dataset consisting of articles on teenage mental health and data from the Alodokter website was used as the basis for the development of this chatbot. The research results show that the chatbot is capable of providing relevant and accurate information, with evaluations using the recall-oriented understudy for gisting evaluation (ROUGE) score method yielding an average of ROUGE-1 with a precision of 87.8%, recall of 83.0%, and F1-measure of 84.0%; ROUGE-2 with a precision of 82.8%, recall of 76.8%, and F1-measure of 78.2%; and ROUGE-L with a precision of 88.0%, recall of 82.6%, and F1-measure of 83.4%. These findings indicate the potential use of chatbots as an effective tool to support the mental health of adolescents.