Leveraging social media data and large language models for understanding public health behaviours in the context of infectious diseases and vaccination

This thesis explores the integration of artificial intelligence (AI) and, more specifically, large language models (LLMs) in public health practice by exploring the intersection between AI, public health and research and their impact in public health discourse in the context of infectious diseases and vaccination. It focuses on the potential of LLMs to enhance public health surveillance, risk communication and decision making. It addresses key challenges of AI implementation in public health such as gap between AI innovation and AI literacy among public health experts, reliability and use of non-traditional surveillance data sources, and the need for standardised, user-friendly and rapid LLM evaluation frameworks.

Using an expert-annotated dataset of English tweets, the first study compares LLMs and rule-based sentiment analysis tool to extract the public stance towards vaccination. The study shows that few-shot prompting with top-performing LLMs is the most accurate approach and other methods risk significant misclassification. These findings highlights the potential of LLMs as a scalable approach for monitoring public discourse.

The second study presents a real-world use case in which LLMs are used to analyse the temporal and geographical evolution of public stance towards vaccination in Brazil between 2013 and 2019 in tweets. With almost two million tweets classified by GPT-4, the findings show a decline in the positive stance towards vaccination, with a peak in negativity in 2019, coinciding with the resurgence of measles cases. The geographical analysis shows that two states consistently had a lower stance towards vaccination. These results highlight the need for near-real-time monitoring of public discourse to support risk communication and community engagement strategies, supporting public health experts to address misinformation and improve vaccination uptake.

The third study presents an open-source solution for LLM prediction, evaluation and benchmarking and its usability in four public health use cases: extracting features from official disease reports, predicting the stance towards vaccination in tweets and Facebook posts, and detecting vaccination adverse reactions in a parental forum. This study addresses the lack of structured, user-friendly and fast solution for LLM evaluation and benchmarking for public health, since context-specific evaluation frameworks are essential for ensuring adequate LLM implementation.

The last study is a comparative study that assesses the timeliness of official websites and social media from public health institutions for reporting key epidemiological indicators during the COVID-19 pandemic in the World Health Organization's regions of Europe and Africa. The findings showed that social media offered faster updates and official sources were usually less timely. However, the preference from public health institutions in which of these were used varied across regions. This highlights the importance of a