Evaluation of Artificial Intelligence in the Medical Domain
Speech, Language and Applications
Time: Fri 2025-12-12 13.00
Location: Kollegiesalen, Brinellvägen 8, Stockholm
Video link: https://kth-se.zoom.us/j/69936124469
Language: English
Subject area: Speech and Music Communication
Doctoral student: Birger Moëll , Tal, musik och hörsel, TMH
Opponent: Dr.-Ing. Jan Alexandersson, DFKI GmbH, Saarbrücken, Germany
Supervisor: Professor Jonas Beskow, Tal, musik och hörsel, TMH
QC 20251022
Abstract
This doctoral thesis investigates the potential of advanced speech and languagetechnologies, driven by deep learning, to improve clinical diagnostics and patientcare, primarily within the Swedish healthcare context. The research encompasseseight key papers, which are presented across three main sections:(1) Data Capture and Machine Learning for Speech: This section explores the use ofmultimodal data and advanced speech processing techniques for clinical applications.It includes research on utilizing multimodal data capture (speech, gaze, and digitalpen input) from clinical interviews to identify potential digital biomarkers for theearly detection and differentiation of dementia (Paper A). It also develops anautomated deep learning system to evaluate the oral diadochokinesis test for motorspeech disorders, which demonstrates higher accuracy than human raters andproposes a human-in-the-loop clinical interface (Paper B). Furthermore, this sectionevaluates the performance of Automatic Speech Recognition (ASR) systems,comparing word error rates between native (L1) and non-native (L2) Swedishspeakers (Paper C), and investigates data augmentation techniques to improve ASRaccuracy for individuals with aphasia, demonstrating a path towards more inclusivetechnology (Paper D).(2) Evaluation of LLMs in the Medical Domain: This section focuses on establishingrobust methods for assessing Large Language Models (LLMs) within a medicalcontext. It details the development of a specialized Swedish Medical LLM Benchmark,comprising over 2600 questions across various medical domains, designed to assessLLM performance in a clinically relevant, language-specific manner (Paper E).Additionally, the medical reasoning capabilities of LLMs, such as DeepSeek R1, arerigorously assessed, focusing on their capacity for general medical diagnosticreasoning (Paper F).(3) Application and Best Practice for Working with AI in Healthcare: This sectionaddresses the practical, ethical, and user experience (UX) considerations forvimplementing AI in healthcare. It proposes a novel user interface paradigm throughan AI-powered journaling application designed for personal health management,illustrating a low-risk, user-centric approach to AI integration (Paper G).Complementing this, it develops harm reduction strategies for the thoughtful use ofLLMs in the medical domain, providing perspectives for both patients and cliniciansto maximize utility while mitigating risks, thereby establishing best practices forresponsible AI engagement (Paper H).Collectively, this work advances the field by providing new tools and methodologiesfor early disease detection using speech and multimodal data, establishing robustevaluation methods for ASR and LLMs in the medical domain, and offering pathwaysand frameworks for responsible, user-centered, and effective AI implementation inhealthcare.