Hybrid Attention Model Using Feature Decomposition and Knowledge Distillation for Blood Glucose Forecasting
Abstract
The availability of continuous glucose monitors (CGMs) as over-the-counter commodities has created a unique opportunity to monitor a person’s blood glucose levels, forecast blood glucose trajectories, and provide automated interventions to prevent devastating chronic complications that arise from poor glucose control. However, forecasting blood glucose levels (BGL) is challenging because blood glucose changes consistently in response to food intake, medication intake, physical activity, sleep, and stress. It is particularly difficult to accurately predict BGL from multimodal and irregularly sampled mobile sensor data and over long prediction horizons. Furthermore, these forecasting models need to operate in real-time on edge devices to provide in-the-moment interventions. To address these challenges, we propose GlucoNet1, an AI model to forecast blood glucose patterns using sensor data about behavioral and physiological health. GlucoNet devises a feature decompositionbased lightweight transformer model that incorporates patients' behavioral and physiological data (e.g., blood glucose, diet, medication) and transforms sparse and irregular patient data (e.g., diet and medication intake data) into continuous features using a mathematical model, facilitating better integration with the BGL signals. Given the non-linear and non-stationary nature of blood glucose signals, we propose a decomposition method to extract both low-frequency (long-term) and high-frequency (short-term) components from the BGL signals, thus enabling the model to capture complex glucose dynamics for accurate forecasting. To reduce the computational complexity of transformer-based predictions, we propose to employ knowledge distillation (KD) to compress the transformer model. Our comprehensive analysis on two real-world T1D cohorts demonstrates that GlucoNet achieves a 35% improvement in RMSE, a 33% improvement in MAE, and a 62% reduction in the number of parameters over state of the art work such as PatchTST on the OhioT1DM dataset (12 patients), while additional experiments on the AZT1D dataset (25 patients), together with extensive ablation and robustness analyses, further demonstrate its generalizability and stability. These results underscore GlucoNet’s potential as a compact and reliable tool for real-world diabetes prevention and management.
Shovito Barua Soumma
Graduate Research Associate
I am a Ph.D. candidate at Arizona State University. I work as a Graduate Research Associate at Embedded Machine Intelligence Lab (EMIL) under the supervision of Dr. Hassan Ghasemzadeh.
Nooshin Taheri Chatrudi
Graduate Teaching Assistant
I am a Ph.D. student at the College of Health Solutions, Arizona State University (ASU). Currently, I am working under the supervision of Dr. Hassan Ghasemzadeh at the Embedded Machine Intelligence Lab (EMIL). My research interests include machine learning, clinical informatics, and health monitoring system development.
Hassan Ghasemzadeh
Director
Hassan Ghasemzadeh is an Associate Professor of Biomedical Informatics at Arizona State University (ASU) and a Computer Science Adjunct Faculty at Washington State University (WSU).