Feasibility and Utility of Multimodal Micro Ecological Momentary Assessment on a Smartwatch

Picture of Veronika Potter
Veronika Potter
Picture of Rithika Lakshminarayanan
Rithika Lakshminarayanan
Picture of Stephen Intille
Stephen Intille
Published at Proceedings of the Conference on Human Factors in Computing Systems (CHI) 2025

Abstract

$\mu$ EMAs allow participants to answer a short survey quickly with a tap on a smartwatch screen or a brief speech input. The short interaction time and low cognitive burden enable researchers to collect self-reports at high frequency (once every 5-15 minutes) while maintaining participant engagement. Systems with single input modality, however, may carry different contextual biases that could affect compliance. We combined two input modalities to create a multimodal-$\mu$ EMA system, allowing participants to choose between speech or touch input to self-report. To investigate system usability, we conducted a seven-day field study where we asked 20 participants to label their posture and/or physical activity once every five minutes throughout their waking day. Despite the intense prompting interval, participants responded to 72.4\% of the prompts. We found participants gravitated towards different modalities based on personal preferences and contextual states, highlighting the need to consider these factors when designing context-aware multimodal $\mu$ EMA systems.

Materials