Speech2Health: A Mobile Framework for Monitoring Dietary Composition From Spoken Data

Abstract

Diet and physical activity are known as important lifestyle factors in self-management and prevention of many chronic diseases. Mobile sensors such as accelerometers have been used to measure physical activity or detect eating time. In many intervention studies, however, stringent monitoring of overall dietary composition and energy intake is needed. Currently, such a monitoring relies on self-reported data by either entering text or taking an image that represents food intake. These approaches suffer from limitations such as low adherence in technology adoption and time sensitivity to the diet intake context. In order to address these limitations, we introduce development and validation of Speech2Health, a voice-based mobile nutrition monitoring system that devises speech processing, natural language processing (NLP), and text mining techniques in a unified platform to facilitate nutrition monitoring. After converting the spoken data to text, nutrition-specific data are identified within the text using an NLP-based approach that combines standard NLP with our introduced pattern mapping technique. We then develop a tiered matching algorithm to search the food name in our nutrition database and accurately compute calorie intake values. We evaluate Speech2Health using real data collected with 30 participants. Our experimental results show that Speech2Health achieves an accuracy of 92.2% in computing calorie intake. Furthermore, our user study demonstrates that Speech2Health achieves significantly higher scores on technology adoption metrics compared to text-based and image-based nutrition monitoring. Our research demonstrates that new sensor modalities such as voice can be used either standalone or as a complementary source of information to existing modalities to improve the accuracy and acceptability of mobile health technologies for dietary composition monitoring.

Publication
IEEE Journal of Biomedical and Health Informatics (JBHI), vol. 22, no. 1, pp. 252–264, January 2018
Niloofar Hezarjaribi
Niloofar Hezarjaribi
Graduate Alumni

Graduate Research Assistant.

Hassan Ghasemzadeh
Hassan Ghasemzadeh
Director

Hassan Ghasemzadeh (Zadeh) is an Associate Professor of Computer Science in the School of Electrical Engineering and Computer Science at Washington State University (WSU). Prior to joining WSU in 2014, he was a Research Manager at the UCLA Wireless Health Institute and an Adjunct Professor of Biomedical Informatics at San Diego State University. He received his Ph.D. in Computer Engineering from the University of Texas at Dallas in 2010, and spent the academic year 2010-2011 as a Postdoctoral Fellow at the West Health Institute. He was Founding Chair of Computer Science and Engineering Department at Azad University, Damavand, 2003-2006. He received his M.S. degree in Computer Engineering from University of Tehran, Tehran, Iran, in 2001 and his B.S. degree in Computer Engineering from Sharif University of Technology, Tehran, Iran in 1998. He received the 2019 WSU GPSA Academic Advisor Excellence Award, 2018 NSF CAREER Award, 2018 WSU EECS Early Career Award, 2018 WSU VCEA Outstanding Communication, Connection, and Engagement Award, 2016 NSF CRII Award, and 2011 IEEE RTAS Best Paper Award.