For well over a decade, the FDA has placed intense focus on how use errors contribute to the majority of adverse medical events. This focus has driven the generation and implementation of comprehensive regulatory guidance surrounding human factors and ease-of-use and how they should be accounted for in the development process. As we move forward and new medical devices and systems enter the market each day, it is crucially important that patient safety is at the center of any device design. Yet, aspiring developers may find themselves overwhelmed by the density and complexity of guidance documents and their contents. Overcoming this begins with understanding the most prominent of human factors guidance: AAMI HE75.
The Basics of HE75
Arguably the most important of all human factors guidance documents is ANSI/AAMI HE75: Human Factors Engineering- Design of Medical Devices. Successor to HE48, a 1993 document authored by AAMI's human factors committee in collaboration with the FDA, HE75 is now recognized as a definitive industry best practice document and provides comprehensive design guidelines for the development of medical device user interfaces. These guidelines provide a steady framework for the methodical integration and study of medical device users throughout all development stages, including detailed human factors engineering and usability input for guidance, examples, checklists, and case studies.
HE75 also outlines appropriate usability methods performed onsite and in real-world scenarios to most effectively evaluate, measure, and improve usability while moving towards minimal usability errors and maximally optimal usage ceremony. Notably, the document also guides developers in identifying best practice usage and providing regulators with the required evidence for submission, demonstrating that a human factor design process has been adopted to support a device or system's usability and safety while also developing a cycle more efficiently.
Implementing HE75 in Development
When developing a device, HE75 can and should be integrated into every stage. MIDI achieves this with clients by utilizing a quality management system (QMS) with baked-in best practices, ensuring that each program pays close attention to all relevant human factors and usability criteria, influencing user performance, comfort, safety, efficiency, and risk of human error. This QMS also provides all documentation required by the FDA for device submission, i.e., Design History Files (DHFs).
At MIDI, HE75 guidance is weaved seamlessly throughout the entire development cycle. During the Design Input stage, the document may be used to pinpoint design requirements by applying the general and specific standards within HE75 sections pertinent to the human interface design. Specifically, the Test and Evaluation Methods section may be utilized in planning interface evaluations, informing developers about user interaction challenges related to design inputs early in the process. This section also assists with summative studies applicable in design validation.
Later, during MIDI's design verification phase, our QMS becomes a method of documenting best practices for human interface parameters during device development, making the criteria used to design and build the device available for straightforward reference.
The above process, and HE75 more generally, applies to all classes and types of medical device applications. The usability and human factors engineering challenges can vary significantly across devices, whether they be handheld or wearable, surgical or diagnostic, or any other number of factors. Moreover, this does not only extend to the device's physical design; all digital interfaces are subject to HE75 standards. A significant number of devices today have digital displays and graphical user interfaces, and a growing number are now integrating other user interface technologies such as mobile support applications and IoT connectivity through cloud-based software. Packaging is also of concern, particularly in the case of medical disposables. Consisting of specialists in human factors, usability, research, industrial design, and engineering, MIDI's multi-disciplinary team addresses all usability aspects of devices pre-, during, and post-development according to all relevant standards guidance, including HE75.
Usability Research, Types, and Methods
Typically, a development program at MIDI involving usability research is guided by a multi-phase process of research, development, and testing prescribed by HE75, including generative and evaluative usability research methods.
Generative Research Methods
Usability researchers use generative research methods early in development to familiarize themselves with and understand the user groups of a medical device or system to identify solutions to issues and opportunities for innovation. Developing a top-of-the-line medical device hinges upon thoroughly performed generative research, which allows challenges to be correctly identified early on, rather than simply refining the device with continuous evaluative research later on. Productive research captures the end user's behaviors, needs, desires, and opinions. There are several methods for doing so, including contextual immersion, Voice of the Customer (VOC) research, workflow mapping, and diary studies. These methods provide hard data on behaviors that may cause use errors and their root causes. The development team may then use it to identify human factors engineering opportunities and define the metric by which future designs will be tested.
Contextual Immersion studies and VOC research are standard generative research methods in which development teams engage users in their natural environments, whether they be clinics, offices, hospitals, or patients' homes. Typically, these include onsite observational research and documentation and subsequent one-on-one and group interviews with users to clarify collected data further. These methods allow researchers to bypass potentially flawed self-reporting.
Another of these methods is workflow mapping, which identifies and captures a comprehensive visualization of the user's macro and micro workflow steps to gain a thorough understanding of the device's task or process. With this data, MIDI's team then generates a Workflow Map, which will shape and inform the design process as it identifies the root causes of particular behaviors and user errors. This method also assists with identifying metrics to create a safe, effective, and efficient medical device application.
Diary studies are another crucial generative research tool, being MIDI's longitudinal qualitative user research method in which users self-report workflow activities and thoughts within their usual environment and routine. Because this study takes place over time, with users repeatedly logging responses, it constitutes an asynchronous qualitative research methodology. At MIDI, one tool used for qualitative remote user research and UX diary studies is an app and cloud-based platform offered by indeemo, allowing users to input study responses as video diaries. These compilations capture repetitive tasks and may consist of videos, images, or screen recordings. Using indeemo, the resulting data is highly contextual, occurring in the user's everyday context and being utterly raw to give a complete picture of their workflow. This points to one significant benefit to performing diary studies; because responses are recorded in real-time, it is not post-rationalized. Its high contextuality lends itself to spotting workflow, rituals, and routines effectively. Notably, the indeemo remote user research and UX diary smart software is ISO 27001 and HIPPA certified, as well as being GDPR compliant.
Evaluative Research Methods
Evaluation research, also known as evaluative research, is used post-generative studies in the assessment of medical device designs to ensure that they are grounded in the wants, needs, and desires of its end-users. With this, MIDI may test proposed designs, rapidly and effectively generating, evaluating, and maturing device solutions throughout the development lifecycle. This includes the generation of storyboards, simulations, mockups, and models, leading to higher fidelity prototypes. When used in conjunction with generative research performed throughout development, a quality feedback loop is established between MIDI researchers, designers, and engineers, allowing for continuous device design evolution. This is supported by in-house research, design, prototyping, and testing capabilities that result in advanced human factors yielding improvements in user performance, comfort, safety, and efficiency, all while providing the FDA-required DHF.
At MIDI, evaluative research is guided by the DevelopmentDNA™ process as governed by our QMS and consists of formative usability research studies and summative usability research studies with validation.
Our formative usability research studies are centered on evaluating concept designs with target users to identify strengths and weaknesses and any potential use case errors. In the hopes of identifying opportunities for refinement before design finalization, this testing is typically conducted with targeted user groups using device simulations paired with prototypes and storyboards to measure and study various usability factors. HE75's Clause 9, Usability Testing, outlines multiple types of formative evaluations useful in device development, while Annex D of IEC 62366 provides their descriptions. These include cognitive walkthroughs, heuristic evaluations, and walk-through-talk-through usability tests. Just as device types vary, formative testing can be different across programs, including qualitative observations and assessments and quantitative analysis of task performance. Because of this, it is common to conduct multiple formative research studies throughout the medical device program to maintain a continuous quality feedback loop.
Formative testing uncovers improvements to be made pre-finalization; summative usability testing is utilized after design finalization when transferring to design controls. Leveraging data points collected during formative research studies, MIDI conducts usability testing with specific end-user groups to demonstrate the medical device's ability to be used by its intended users, for its intended use, safely and effectively. Within this, individuals in a representative use environment use the device to perform critical tasks with the potential to cause medical harm. During this, any use errors or near errors are observed, documented, and examined for their root cause; this information is then used in the risk management process for analysis, evaluation, and control.
It is important to note how summative testing and usability testing differ—summative testing is not intended to be an exploratory exercise. Rather than collecting input on device design features, it is essentially a "final" demonstration of a device's usage and safety. As such, test participants are not to be interrupted, questioned, or corrected at any point during the usage ceremony. Participants should be provided with training representative of real-world conditions. Instructions should be available during testing, and dwell intervals may be employed to simulate potential real-world cognitive memory workload scenarios. Users should then perform the regular usage ceremony for all tasks categorized as high risk. Each step should be observed and classified according to a level of success, including success with difficulties like hesitation, self-correction, and potential confusion. This information should then be taken into post-summative testing reviews. The users' failures and challenges are identified and assigned a cause, whether it be an error, incorrect use, or otherwise. The result of successful summative testing should be a consensus among developers regarding the device's safety according to actual performance rather than pre-defined quantitative goals. This includes identifying any remaining patterns of use-related issues that may directly correlate to the device's design or accompanying documentation, such as labeling or instructions.
As new healthcare technologies continue to emerge each day, it is of the utmost importance that the patients they are intended for receive the highest quality, safest, and most effective treatment possible. What makes certain that this is the case are the standards, regulations, and guidelines mandated by agencies like the FDA. This attention on human factors engineering and usability research has already called attention to essential instances of design-related failures. It has expanded our understanding of how the environment impacts behavior and contributes to medical device failures. Moving ahead in this age of rapid advancement, it will be the responsibility of regulatory, standards, and guidance organizations to continuously improve and encourage human factors engineering and usability research practices. Moreover, it will be human factors professionals and medical device developers' duty to perform these methods and apply their findings to device designs. Together, they move towards a future with greater systemic solutions for patient safety that encourage new technology for safer, more effective, higher quality healthcare.