Formative vs Summative Usability Testing

Doctors utilizing medical devices

Formative vs Summative Usability Testing

Usability Testing is a key component of the Human Factors/Usability Engineering Process. Human Factors play a crucial role in identifying and reducing use errors (human errors) during the development of medical device and combination products. There are two main types of usability testing: Formative Usability Testing and Summative Usability Testing.

Learn about the difference between Formative vs Summative Usability Testing and how Improvita can guide you through the process for each type of testing.

Formative Usability Testing

The objective of a Formative Usability Testing is to identify problems and areas for improvement in the product design by involving representative users, observing their interactions with the product, and obtaining their feedback. This type of testing is typically less formal and less structured than the Summative Usability Testing, allowing more flexibility with sample sizes and data collection methods such as a cognitive walkthrough (i.e., participants are encouraged to discuss their thought process about using the product and explain difficulties or concerns). Formative Usability Testing is one tool that is used as part of the Formative Evaluation. The FDA defines Formative Evaluation as user interface evaluation conducted with the intent to explore user interface design strengths, weaknesses, and unanticipated use errors. A best practice is to conduct Formative Usability Tests early and often, beginning in the early stages of product development and at strategic intervals leading up to design validation. Formative Usability Testing prepares a product for the formal and structured Human Factors Summative Usability Testing.

Summative Usability Testing (Human Factors Validation)

The terms Summative Usability Testing and Human Factors Validation Testing are generally considered synonymous in the industry. The FDA preferred term is Human Factors Validation. The agency defines it as testing conducted at the end of the device development process to assess user interactions with a device to identify use errors that would or could result in serious harm to the patient or user. Summative Usability Testing is also used to evaluate the effectiveness of risk management measures and represents one portion of design validation. This type of testing is formal and structured compared to Formative Usability Testing, having less flexibility around sample size, data collection requirements and having a rigid protocol. Summative Usability Testing evaluates the product in a real-world setting, typically a controlled simulated use environment, with representative end users performing realistic use scenarios.

Formative Usability Testing vs Summative Usability Testing: Key Differences

Formative Usability Testing

  • Focuses in identifying areas for improvement during the product design and development
  • Conducted early and throughout product development
  • Testing is less formal and less structured
  • Study participants: 5 to 8 subjects per evaluation

Summative Usability Testing

  • Focuses on evaluating that the completed design can be used safely and effectively
  • Conducted after the product has been developed and is ready for formal validation
  • Testing is rigid and is explicitly defined
  • Study participants: minimum 15 subjects per distinct user group

The Benefits of Usability Testing

Usability testing is a primary component of the Human Factors Engineering (HFE) process and plays a vital role in the development of any product or service that is designed to be used by people. It helps to promote an end product that is user-friendly, safe, efficient, and satisfying, and provides valuable insights into the user experience.

There are several benefits to conducting usability testing, including:
1

Improving Product Design:

By testing the product during the development stage, it is possible to identify areas for improvement and make changes to the product design before it is released to the public. This leads to a better user experience and higher customer satisfaction.

2

Identifying User Needs:

Usability testing provides a better understanding of user needs, preferences, and behaviors. This helps to inform the development of products that meet the needs and expectations of the intended users.

3

Identifying Usability Issues:

Usability testing helps to identify any usability issues that may impact the user experience. These issues can be addressed early in the development process, leading to a better end product.

4

Reducing Development Costs:

By identifying and addressing usability issues early on in the development process, it is possible to reduce the costs associated with fixing problems later in the development cycle.

5

Improving User Experience:

By obtaining user feedback and making improvements through usability testing, it is possible to create an excellent product that is enjoyable, efficient, and easy to use.

FAQs

Formative Usability Testing is a form of evaluation that is conducted to identify problems and areas for improvement in the product design by involving representative users, observing their interactions with the product, and obtaining their feedback.

Summative Usability Testing is a form of evaluation that is conducted at the end of the device development process to assess user interactions with a device to identify use errors that would or could result in serious harm to the patient or user. Summative Usability Testing is also used to evaluate the effectiveness of risk management measures and represents one portion of design validation.

The main difference between Formative and Summative Usability Testing is that Formative Usability Testing focuses on identifying areas for improvement in product design and Summative Usability Testing focuses on evaluating the overall effectiveness, efficiency, and satisfaction of the product.

Why Choose Improvita for Formative and Summative Usability Testing?

By choosing Improvita’s medical device Human Factors consulting services, medical device manufacturers can benefit from our extensive experience and knowledge of conducting Human Factors engineering activities which includes but not limited to risk analysis, task analysis, Formative Evaluations, Validation Testing, and protocol and report writing. We provide support to develop a medical device that keeps usability and the needs of the user top of mind. We have a proven history of helping clients obtain FDA approval of their products by demonstrating the users can use the product in a safe and effective manner. Our services can also help with complying with relevant regulations and guidelines.

To learn more about Improvita’s Human Factors consulting services, contact us to discuss your specific needs.