Impact of Iterative Development and Beta-Testing on the Usability and Acceptability of a Novel Just-in-Time Adaptive Digital Physical Activity Intervention

Vandelanotte, C., Maher, C., Hodgetts, D., Imam, T., Rashid, M., To, Q. G., & Trost, S. (2025). Impact of Iterative Development and Beta-Testing on the Usability and Acceptability of a Novel Just-in-Time Adaptive Digital Physical Activity Intervention. Journal of Physical Activity and Health, 1(aop), 1-7.

[Open Access] https://journals.humankinetics.com/view/journals/jpah/22/10/article-p1315.xml

Abstract

The search for cost-effective population-based physical activity interventions continues. Therefore, we developed a novel just-in-time adaptive digital assistant supported by machine learning (ie, MoveMentor). Beta-testing is essential to evaluate both technical performance and user acceptance. The aim of this study was to assess app usability, acceptability, and technical performance through iterative rounds of beta-testing.

Insufficiently active people (age: 39.8 [10.2]; 86% female) participated in 2 rounds of beta-testing (round 1, n = 112; round 2, n = 41). Participants downloaded the digital assistant app onto their phone to use during the study period (round 1: 12 wk, round 2: 4 wk). Participants were asked complete at least 4 educational and 5 chat conversations, rate over 50 notifications, and complete an online follow-up survey at week 4 examining aspects of app usability and acceptability. Descriptive statistics and t tests were used to analyze outcomes.

Across both rounds, the app demonstrated good overall usability scores (System Usability Scale: 75.3 out of 100) but lower usefulness ratings. Round 2 participants showed increased engagement with features including action plans ( P  < .001), educational conversations ( P  < .001), and personalization features ( P  < .001), and they appreciated the educational conversations more ( P  < .05). Technical issues including data syncing problems and chat limitations persisted across both rounds. The notification system received mixed feedback, though customization options in round 2 reduced complaints (12.2%–7.3%).

The app demonstrated good acceptability and usability but low usefulness. The iterative beta-testing successfully identified areas for improvement and enabled meaningful enhancements to content and user engagement features. While some technical challenges persisted, the beta-testing provided clear direction for ongoing improvements.