Artificial Intelligence and Machine Learning in Software as a Medical Device

04/17/2019

Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) - Discussion Paper and Request for Feedback available from FDA.  Some key points discussed:

Software as a Medical Device (SaMD)

Key Definitions: 

https://www.imdrf.org/docs/imdrf/final/technical/imdrf-tech-131209-samd-key-definitions-140901.pdf.

Pre-Cert Program Version 1.0 Working Model: 

https://www.fda.gov/downloads/MedicalDevices/DigitalHealth/DigitalHealthPreCertProgram/UCM629276.pdf

Software as a Medical Device (SaMD): Clinical Evaluation: 

https://www.fda.gov/downloads/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm524904.pdf

Deciding When to Submit a 510(k) for a Software Change to an Existing Device: 

https://www.fda.gov/downloads/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm514737.pdf

Deciding When to Submit a 510(k) for a Software Change to an Existing Device: 

https://www.fda.gov/downloads/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm514737.pdf.

Developing a Software Precertification Program: A Working Model; v1.0 - January 2019: 

https://www.fda.gov/downloads/MedicalDevices/DigitalHealth/DigitalHealthPreCertProgram/UCM629276.pdf.

https://jmc.stanford.edu/articles/whatisai/whatisai.pdf.

Software as a Medical Device (SaMD): Possible Framework for Risk Categorization and Corresponding Considerations: 

https://www.imdrf.org/docs/imdrf/final/technical/imdrf-tech-140918-samd-framework-risk-categorization-141013.pdf.

Non-device software functions are not subject to FDA device regulation.  In addition, as detailed in section 502(o) of the FD&C Act, software functions intended (1) for administrative support of a health care facility, (2) for maintaining or encouraging a healthy lifestyle, (3) to serve as electronic patient records, (4) for transferring, storing, converting formats, or displaying data, or (5) to provide certain, limited clinical decision support are not medical devices and are not subject to FDA regulation.

The IMDRF SaMD risk categorization framework takes a risk-based approach to categorize SaMD based on intended use, similar to traditional risk-based approaches used by the FDA. The IMDRF risk framework identifies the following two major factors as providing a description of the intended use12 of the SaMD:

1) Significance of information provided by the SaMD to the healthcare decision, which identifies the intended use of the information provided by the SaMD - i.e., to treat or diagnose; to drive clinical management; or to inform clinical management; and

2) State of healthcare situation or condition, which identifies the intended user, disease or condition, and the population for the SaMD - i.e., critical; serious; or non-serious healthcare situations or conditions.

Taken together, these factors describing the intended use can be used to place the AI/ML-based SaMD into one of four categories, from lowest (I) to highest risk (IV) to reflect the risk associated with the clinical situation and device use.

While AI/ML-based SaMD exist on a spectrum categorized by risk to patients, they also exist on a spectrum from locked to continuously learning. "Locked" algorithms are those that provide the same result each time the same input is provided. As such, a locked algorithm applies a fixed function (e.g., a static look-up table, decision tree, or complex classifier) to a given set of inputs. These algorithms may use manual processes for updates and validation. In contrast to a locked algorithm, an adaptive algorithm (e.g., a continuous learning algorithm) changes its behavior using a defined learning process. The algorithm adaptation or changes are implemented such that for a given set of inputs, the output may be different before and after the changes are implemented. These algorithm changes are typically implemented and validated through a well-defined and possibly fully automated process that aims at improving performance based on analysis of new or additional data.

The adaptation process can be intended to address several different clinical aspects, such as optimizing performance within a specific environment (e.g., based on the local patient population), optimizing performance based on how the device is being used (e.g., based on preferences of a specific physician), improving performance as more data are collected, and/or changing the intended use of the device. The adaptation process follows two stages: learning and updating. The algorithm "learns" how to change its behavior, for example, from the addition of new input types or adding new cases to an already existing training database. The "update" then occurs when the new version of the algorithm is deployed. As a Information that may be used to describe intended use for FDA purposes is set forth in 21 CFR 807.92(a)(5), 814.20(b)(3), and 860.7(b), and could be written using terminology as described in the IMDRF risk categorization framework.

Although AI/ML-based SaMD exists on a spectrum from locked to continuously adaptive algorithms, a common set of considerations for data management, re-training, and performance evaluation can be applied to the entire spectrum of SaMD. For example, the rigor of performance evaluation for both locked and continuously adaptive algorithms depend on the test methods, quality and applicability of dataset used for testing, and the algorithm's training methods. Robust algorithms typically require the availability of large, high-quality, and well-labeled training data sets. Likewise, a common set of principles can be applied to considerations about how to provide confidence in function and performance to users through appropriate validation, transparency, and claims after the modification.

Types of AI/ML-based SaMD Modifications

There are many possible modifications to an AI/ML-based SaMD. Some modifications may not require a review based on guidance provided in "Deciding When to Submit a 510(k) for a Software Change to an Existing Device."13 This paper anticipates that many modifications to AI/ML-based SaMD involve algorithm architecture modifications and re-training with new data sets, which under the software modifications guidance would be subject to premarket review. The types of modifications generally fall into three broad categories:

• Performance - clinical and analytical performance14;
• Inputs used by the algorithm and their clinical association to the SaMD output; and/or
• Intended use - The intended use of the SaMD, as outlined above and in the IMDRF risk categorization framework, described through the significance of information provided by the SaMD for the state of the healthcare situation or condition.


The changes described may not be mutually exclusive - one software modification may impact, for example, both a change in input and change in performance; or, a performance change may increase a device's clinical performance that in turn impacts the intended use. These software changes in AI/ML-based SaMD, grouped by the types of changes as described above, have different impact on users, which may include either patients, healthcare professionals, or others:

i. Modifications related to performance, with no change to the intended use or new input type: This type of modification includes improvements to analytical and clinical performance that can result from a number of changes. This may include re-training with new data sets within the intended use population from the same type of input signal, a change in the AI/ML architecture, or other means. For this type of modification, the manufacturer commonly aims to update users on the performance, without changing any of the explicit use claims about their product (e.g., increased sensitivity of the SaMD at detecting breast lesions suspicious for cancer in digital mammograms).

Deciding When to Submit a 510(k) for a Software Change to an Existing Device: 

https://www.fda.gov/downloads/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm514737.pdf.
Software as a Medical Device (SaMD): Clinical Evaluation: 

https://www.fda.gov/downloads/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm524904.pdf.

Modifications related to inputs, with no change to the intended use: These types of modifications are those that change the inputs used by the AI/ML algorithm. These modifications may involve changes to the algorithm for use with new types of input signals, but do not change the product use claims. Examples of these changes could be:

  1. Expanding the SaMD's compatibility with other source(s) of the same input data type (e.g., SaMD modification to support compatibility with CT scanners from additional manufacturers); or
  2. Adding different input data type(s) (e.g., expanding the inputs for a SaMD that diagnoses atrial fibrillation to include oximetry data, for example, in addition to heart rate data).

Modifications related to the SaMD's intended use: These types of modifications include those that result in a change in the significance of information provided by the SaMD (e.g., from a confidence score that is 'an aid in diagnosis' (drive clinical management) to a 'definitive diagnosis' (diagnose)). These types of modifications also include those that result in a change in the state of the healthcare situation or condition and are explicitly claimed by the manufacturer, such as an expanded intended patient population (e.g., inclusion of pediatric population where the SaMD was initially intended for adults ages 18 years or older); or the intended disease or condition (e.g., expansion to use a SaMD algorithm for lesion detection from one type of cancer to another). Changes related to either the significance of the information provided by the SaMD or the healthcare situation or condition may be limited in scope by the pre-specified performance objectives and algorithm change protocols.

A Total Product Lifecycle Regulatory Approach for AI/ML-Based SaMD
As envisioned in the Software Pre-Cert Program,16 applying a TPLC approach to the regulation of software products is particularly important for AI/ML-based SaMD due to its ability to adapt and improve from real-world use. In the Pre-Cert TPLC approach, FDA will assess the culture of quality and organizational excellence of a particular company and have reasonable assurance of the high quality of their software development, testing, and performance monitoring of their products. This approach
16 Developing a Software Precertification Program: A Working Model; v1.0 - January 2019: 

https://www.fda.gov/downloads/MedicalDevices/DigitalHealth/DigitalHealthPreCertProgram/UCM629276.pdf.

Questions / Feedback on the types of AI/ML-SaMD modifications:

  1. Do these categories of AI/ML-SaMD modifications align with the modifications that would typically be encountered in software development that could require premarket submission?
  2. What additional categories, if any, of AI/ML-SaMD modifications should be considered in this proposed approach?
  3. Would the proposed framework for addressing modifications and modification types assist the development AI/ML software?

To fully realize the power of AI/ML learning algorithms while enabling continuous improvement of their performance and limiting degradations, the FDA's proposed TPLC approach is based on the following general principles that balance the benefits and risks, and provide access to safe and effective AI/ML-based SaMD:

  1. Establish clear expectations on quality systems and good ML practices (GMLP); This proposed regulatory approach would apply to only those AI/ML based-SaMD that require premarket submission and not those that are exempt from requiring premarket review (i.e., Class I exempt and Class II exempt).
  2. Conduct premarket review for those SaMD that require premarket submission17 to demonstrate reasonable assurance of safety and effectiveness and establish clear expectations for manufacturers of AI/ML-based SaMD to continually manage patient risks throughout the lifecycle;
  3. Expect manufacturers to monitor the AI/ML device and incorporate a risk management approach and other approaches outlined in "Deciding When to Submit a 510(k) for a Software Change to an Existing Device" Guidance18 in development, validation, and execution of the algorithm changes (SaMD Pre-Specifications and Algorithm Change Protocol); and
  4. Enable increased transparency to users and FDA using postmarket real-world performance reporting for maintaining continued assurance of safety and effectiveness.

1. Quality Systems and Good Machine Learning Practices (GMLP):

The FDA expects every medical device manufacturer to have an established quality system that is geared towards developing, delivering, and maintaining high-quality products throughout the lifecycle that conforms to the appropriate standards and regulations.19 Similarly, for AI/ML-based SaMD, we expect that SaMD developers embrace the excellence principles of culture of quality and organizational excellence.20
As is the case for all SaMD, devices that rely on AI/ML are expected to demonstrate analytical and clinical validation, as described in the SaMD: Clinical Evaluation guidance (Figure 3).21 The specific types of data necessary to assure safety and effectiveness during the premarket review, including study design, will depend on the function of the AI/ML, the risk it poses to users, and its intended use.

AI/ML algorithm development involves learning from data and hence prompts unique considerations that embody GMLP. In this paper, GMLP are those AI/ML best practices (e.g., data management, feature extraction, training, and evaluation) that are akin to good software engineering practices or quality system practices. Examples of GMLP considerations as applied for SaMD include:

21 CFR Part 807 Subpart E or 21 CFR Part 814 Subpart B.

Deciding When to Submit a 510(k) for a Software Change to an Existing Device: 

https://www.fda.gov/downloads/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm514737.pdf.

21 CFR Part 820.

See the discussion in Developing a Software Precertification Program: A Working Model; v1.0 - January 2019: 

https://www.fda.gov/downloads/MedicalDevices/DigitalHealth/DigitalHealthPreCertProgram/UCM629276.pdf.

Software as a Medical Device (SaMD): Clinical Evaluation: 

https://www.fda.gov/downloads/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm524904.pdf.


Questions / Feedback
1. Do these categories of AI/ML-SaMD modifications align with the modifications that would typically be encountered in software development that could require premarket submission?
2. What additional categories, if any, of AI/ML-SaMD modifications should be considered in this proposed approach?
3. Would the proposed framework for addressing modifications and modification types assist the development AI/ML software?
4. What additional considerations exist for GMLP?
5. How can FDA support development of GMLP?
6. How do manufacturers and software developers incorporate GMLP in their organization?
7. What are the appropriate elements for the SPS?
8. What are the appropriate elements for the ACP to support the SPS?
9. What potential formats do you suggest for appropriately describing a SPS and an ACP in the premarket review submission or application?
10. How should FDA handle changes outside of the "agreed upon SPS and ACP"?
11. What additional mechanisms could achieve a "focused review" of an SPS and ACP?
12. What content should be included in a "focused review"?
13. In what ways can a manufacturer demonstrate transparency about AI/ML-SaMD algorithm updates, performance improvements, or labeling changes, to name a few?
14. What role can real-world evidence play in supporting transparency for AI/ML-SaMD?
15. What additional mechanisms exist for real-world performance monitoring of AI/ML-SaMD?
16. What additional mechanisms might be needed for real-world performance monitoring of AI/ML-SaMD?
17. Are there additional components for inclusion in the ACP that should be specified?
18. What additional level of detail would you add for the described components of an ACP?

Regulatory Affairs FDA Approval Medical Device IVD NDA BLA