Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) – Part Two

Early in 2019, the FDA proposed a framework for how software that is based on AI and ML could lead to approval as medical devices. The FDA seeks to balance the ability of AI and ML to provide new medical solutions for hospitals, doctors, and patients with the need to ensure the safety of the patients. Artificial Intelligent (AI) and machine learning (ML) rely on training from data input to essentially reconfigure their programs to adapt to the new data. Prior software was fairly static. Once it was designed and marketed, the software didn’t change. The new AI/ML software does change after it is marketed. Determining how to make sure the changes the software make are safe and helpful is a complex problem. The new framework is aimed at providing a solution to that complex problem.

Much of this article includes selected excerpts from the new framework to illustrate the FDA goals, the scope of the regulatory problem, and an example of how the framework might work for developers. AI and ML software are already changing the healthcare landscape. The FDA is working to keep pace with wearable technology, new health software, and now adaptable AI software.

A Total Product Lifecycle (TPLC) Regulatory Approach for AI/ML-Based SaMD

“In the Pre-Cert TPLC approach, FDA will assess the culture of quality and organizational excellence of a particular company and have reasonable assurance of the high quality of their software development, testing, and performance monitoring of their products.” This approach would allow for a reasonably-high guarantee of safety and effectiveness during the whole lifecycle of organizations as well as products so that patients, caregivers, healthcare professionals, and others can be confident of the products’ safety and quality. The approach also lets software products be evaluated and monitored all the way from pre-market development to post-market performance, and lets organizations demonstrate their continued excellence.

“The FDA’s proposed TPLC approach is based on the following general principles that balance the benefits and risks, and provide access to safe and effective AI/ML based SaMD:

1. Establish clear expectations on quality systems and good ML practices (GMLP);
2. Conduct premarket review for those SaMD (software as a medical device) that require premarket submission to demonstrate reasonable assurance of safety and effectiveness and establish clear expectations for manufacturers of AI/ML-based SaMD to continually manage patient risks throughout the lifecycle;
3. Expect manufacturers to monitor the AI/ML device and incorporate a risk management approach and other approaches outlined in “Deciding When to Submit a 510(k) for a Software Change to an Existing Device” Guidance18 in development, validation, and execution of the algorithm changes (SaMD Pre-Specifications and Algorithm Change Protocol); and
4. Enable increased transparency to users and FDA using post-market real-world performance reporting for maintaining continued assurance of safety and effectiveness.”

Quality Systems and Good Machine Learning Practices (GMLP)

The proposed GMLP are akin (but not identical) to the Current Good Manufacturing Practices the FDA requires for drug development. Best practices are designed to ensure “proper design, monitoring, and control of manufacturing processes and facilities.”

“AI/ML algorithm development involves learning from data and hence prompts unique considerations that embody GMLP. In this paper, GMLP are those AI/ML best practices (e.g., data management, feature extraction, training, and evaluation) that are akin to good software engineering practices or quality system practices. Examples of GMLP considerations as applied for SaMD include:

• Relevance of available data to the clinical problem and current clinical practice;
• Data acquired in a consistent, clinically relevant and generalizable manner that aligns with the SaMD’s intended use and modification plans;
• Appropriate separation between training, tuning, and test datasets;
• Appropriate level of transparency (clarity) of the output and the algorithm aimed at users.

Initial Premarket Assurance of Safety and Effectiveness

The paper suggests a framework for changes to AI/ML-based SaMD that rests on the principle of a “predetermined change control plan.” The FDA believes that with this proposed regulatory approach, their oversight allows for responsible performance enhancements in AI/ML-based technologies.

“The predetermined change control plan would include the types of anticipated modifications – SaMD Pre-Specifications – based on the retraining and model update strategy, and the associated methodology – Algorithm Change Protocol – being used to implement those changes in a controlled manner that manages risks to patients.

• SaMD Pre-Specifications (SPS): These are the types of changes the manufacturer plans to achieve when the SaMD is in use. The SPS draws a “region of potential changes” around the initial specifications and labeling of the original device. This is “what” the manufacturer intends the algorithm to become as it learns.
• Algorithm Change Protocol (ACP): The ACP is a step-by-step delineation of the data and procedures to be followed so that the modification achieves its goals and the device remains safe and effective after the modification. This is “how” the algorithm will learn and change while remaining safe and effective.

The FDA also recognizes that the kind of changes that could be pre-specified in a SPS and managed through an ACP could make it necessary to consider that SaMD individually during premarket review in order to assess its risks and benefits to patients.

Approach for modifications after initial review with an established SPS and ACP

“Learning, adaptation, and optimization are inherent to AI/ML-based SaMD. These capabilities of AI/ML would be considered modifications to SaMD after they have received market authorization from FDA. This paper proposes an approach to appropriately manage risks to patients from these modifications, while enabling manufacturers to improve performance and potentially advance patient care.

“Depending on the type of modification, the current software modifications guidance results in either

• Submission of a new 510(k) for premarket review or
• Documentation of the modification and the analysis in the risk management and 510(k) files.”

Transparency and real-world performance monitoring of AI/ML-based SaMD

To fully align with a TPLC approach of regulating AI/ML-based SaMD, manufacturers can introduce new mechanisms supporting transparency and performance monitoring in order to ensure the safety and effectiveness of their software products. Transparency regarding the functionality and modifications of medical devices is a key component of their safety. This is particularly true for devices that evolve over time, like SaMD that incorporate AI/ML.

Collecting data on the performance of the SaMD in real-world settings might help manufacturers identify the various ways their products are used, identify possible improvements, and address safety and usability issues they discover preemptively. This data collection and monitoring is a critical tool for manufacturers to leverage to minimize the risks associated with AI/ML-based modifications.

Transparency might include updates to any of several parties, including the FDA, device companies, the manufacturer’s collaborators, general users, clinicians, and patients.

An Example

The following is a hypothetical example of AI/ML-based SaMD modification that may or may not be permitted under this proposed framework,

Intensive Care Unit (ICU) SaMD

Description of SaMD: An AI/ML application intended for ICU patients receives electrocardiogram, blood pressure, and pulse-oximetry signals from a primary patient monitor. The physiologic signals are processed and analyzed to detect patterns that occur at the onset of physiologic instability. When physiologic instability is detected, an audible alarm signal is generated to indicate that prompt clinical action is needed to prevent potential harm to the patient. This SaMD AI/ML application will ‘drive clinical management’ in a ‘critical healthcare situation or condition.’

SPS: The manufacturer proposes two potential modifications for ICU SaMD:

• Modify the algorithm to ensure consistent performance across sub-populations, especially in cases where real-world monitoring suggests the algorithm underperforms; and
• Reduce false-alarm rates while maintaining or increasing sensitivity to the onset of physiologic instability.

ACP: For these modifications, the ACP details the methods for database generation, reference standard labeling, and comparative analysis along with the performance requirements and statistical analysis plan. The manufacturer follows GMLP.

Modification Scenario 1A: Increase in performance (type i modification), consistent with SPS and ACP

In accordance with the ACP, data was collected and used to modify the algorithm in a way that the manufacturer believes will lower the false-alarm rate while maintaining the sensitivity. A separate independent validation data set was collected. The manufacturer used the independent data set to perform analytical validation and found that the false-alarm rate was reduced while the sensitivity remained the same. Labeling was updated in accordance with the modified SaMD performance, and communication was provided to SaMD users. The algorithm modification may be made without additional FDA review.

Modification Scenario 1B: Increase in performance and change related to intended use (type iii modification), inconsistent with SPS and ACP

In accordance with the ACP, the manufacturer re-trained their algorithm using additional data to improve the sensitivity, however, analytical validation demonstrated that the revised algorithm has the same sensitivity and false-alarm rate as the previous version. The manufacturer noticed that the modified algorithm can maintain that same sensitivity 15 minutes prior to the onset of physiologic instability, which the previous version of the algorithm could not do.

The manufacturer would like to update their algorithm, labeling, and intended use to indicate that the alarm condition now reflects prediction of a physiologic instability within the next 15 minutes, which was not previously included in the SPS and ACP. Because the methods required for analysis, performance, and statistics are not consistent with predicting a future state, and the significance of information provided by the SaMD has changed, FDA may review a new SPS and ACP that includes this information for the algorithm modification before the manufacturer is permitted to make the change.”

Other examples in the framework guide include:

• A Skin Lesion Mobile Medical App (MMA)
• X-Ray Feeding Tube Misplacement SaMD

Questions for review and feedback

The FDA Framework seeks answers from the public to the following questions:

1. Do these categories of AI/ML-SaMD modifications align with the modifications that would typically be encountered in software development that could require premarket submission?
2. What additional categories, if any, of AI/ML-SaMD modifications should be considered in this proposed approach?
3. Would the proposed framework for addressing modifications and modification types assist the development AI/ML software?
4. What additional considerations exist for GMLP?
5. How can FDA support development of GMLP?
6. How do manufacturers and software developers incorporate GMLP in their organization?
7. What are the appropriate elements for the SPS?
8. What are the appropriate elements for the ACP to support the SPS?
9. What potential formats do you suggest for appropriately describing a SPS and an ACP in the premarket review submission or application?
10. How should FDA handle changes outside of the “agreed upon SPS and ACP”?
11. What additional mechanisms could achieve a “focused review” of an SPS and ACP?
12. What content should be included in a “focused review”?
13. In what ways can a manufacturer demonstrate transparency about AI/ML-SaMD algorithm updates, performance improvements, or labeling changes, to name a few?
14. What role can real-world evidence play in supporting transparency for AI/ML-SaMD?
15. What additional mechanisms exist for real-world performance monitoring of AI/ML-SaMD?
16. What additional mechanisms might be needed for real-world performance monitoring of AI/ML SaMD?
17. Are there additional components for inclusion in the ACP that should be specified?
18. What additional level of detail would you add for the described components of an ACP?

The FDA’s new total product lifecycle framework is not regulatory law. It is a proposal to help the FDA properly review new artificial and machine intelligence software that can help the medical profession provide better care to patients. At its core, the framework wants to:

• Establish good machine learning practices
• Examine if the software provides reasonable assurances that it will be safe and effective
• Ensure manufactures monitor the software and incorporate a risk management protocols
• Enables transparency so the FDA and users

Contact Cohen Healthcare Law Group, PC to learn more about artificial intelligence and machine learning regulation issues. In addition to FDA approval of the software, there are issues involving privacy, ownership, the practice of medicine, and other issues that the health profession needs to understand.