Hallucination-Hunting Season, or UX Interventions To Better Read LLM Outputs

This work-in-progress paper examines how interface-level interventions can help novice knowledge workers detect hallucinations in responses from commercial LLM-powered chatbots. These systems have brought powerful tools into professional workflows, yet their fluent human-like language can mask the errors produced by inherently probabilistic models. Focusing on RAG-powered systems, we move beyond the typical question-and-answer experience by introducing thoughtful friction and visual affordances that make users readier to spot hallucinations. We articulate a design space and corresponding hypotheses for such interventions, and outline an experiment to evaluate how these designs reshape Human-AI collaboration, shifting workers from passive consumption to more critical engagement. CCS Concepts • Human-centered computing → Graphical user interfaces; Natural language interfaces; Empirical studies in HCI ; Usability testing; User studies; Activity centered design; • Social and professional topics → Socio-technical systems.