A Critical Analysis of FDA Guidance for User Percentile Device Design Criteria Versus Currently Available Human Factors Engineering Data Sources and Industry Best Practices

Charles Mauro

Peter Pirolli

Chris Morley


To report on the use of an eye-tracking retrospective think-aloud for usability evaluation and to describe its application in assessing the usability of a mobile health app.

Materials and Methods

We used an eye-tracking retrospective think-aloud to evaluate the usability of a HIV prevention mobile app among 20 young men (15-18 years) in New York City, NY; Birmingham, AL; and Chicago, IL. Task performance metrics, critical errors, task completion rate per participant and task completion rate per task, were measured. Eye-tracking metrics including fixation, saccades, time to first fixation, time spent, and revisits were measured and compared among participants with/without a critical error.


Using task performance analysis, we identified 19 critical errors on four activities, and of those, two activities had a task completion rate of less than 78%. To better understand these usability issues, we thoroughly analyzed participants’ corresponding eye movements and verbal comments using an in-depth problem analysis. In areas of interest created for the activity with critical usability problems, there were significant differences in time spent (p = 0.008), revisits (p = 0.004), and total numbers of fixations (p = 0.007) by participants with/without a critical error. The overall mean score of perceived usability rated by the Health IT Usability Evaluation Scale was 4.64 (SD = 0.33), reflecting strong usability of the app.

Discussion and Conclusion
An eye-tracking retrospective think-aloud enabled us to identify critical usability problems as well as gain an in-depth understanding of the usability issues related to interactions between end-users and the app. Findings from this study highlight the utility of an eye-tracking retrospective think-aloud in consumer health usability evaluation research.

This publication uses Eye Tracking which is fully integrated into iMotions Lab

Learn more