Summary and assessment: Using Artificial Intelligence & Machine Learning in the Development of Drug & Biological Products: Discussion Paper and Request for Feedback (FDA)

The FDA discussion paper begins with a lengthy section covering potential use cases for AI/ML in drug development. There is some overlap here with a 2023 publication by CDER and CBER staffers¹, although it goes much further, considering potential use cases as well as those already seen by FDA.  

It follows with a series of questions on which it seeks stakeholder feedback. These fall under three broad headers: 

1. Human-led governance, accountability, and transparency 
2. Quality, reliability, and representativeness of data 
3. Model development, performance, monitoring, and validation 

Since the text poses questions, but does not offer an explicit viewpoint, one must infer what might be in the minds of the authors. However, we see many similar themes to those raised by EMA.

Human-led governance, accountability, and transparency

FDA’s first question is interesting: Where is more regulatory clarity needed when it comes to AI in drug development? To which one is tempted to say ‘everywhere!’. However, this may speak to a desire to draw on existing guidances where they exist, something which EMA also highlights. Questions in this section relate to definitions for transparency, and the extent to which ‘how the AI system operates’ can be understood in detail. This leans on the concept of explainability. It also asks about human oversight of functioning AI systems, and how this can be made ‘meaningful’, by which it would seem they mean, simply, how do we know it works as intended? Traceability and auditability are also raised as aspects of technology validation that are ordinarily required from technology systems deployed in clinical development. Additionally, FDA asks about pre-specification, presumably regarding AI systems that may still be being trained in-use. 

Quality, reliability, and representativeness of data

In this section, FDA ask how we ensure data used to train AI algorithms are fit-for-purpose. This indicates that an issue uppermost in their mind is the potential for biased training data to lead to biases in the AI system performance characteristics.  In terms of AI system performance, FDA also asks about practices deployed to enhance reproducibility and replicability – perhaps in part reflecting on known behaviors of generative AI models. 

They also ask what AI system developers are doing to ensure the integrity of their training data pipeline, manage issues such as missing data, and ensure the data are accurate, consistent, and complete. Like EMA, data privacy and data security are areas highlighted where FDA wants to understand how these twin issues are managed. 

Model development, performance, monitoring, and validation

This final set of questions seeks input that may help establish a shared view on relevant best practices for AI developers. These relate to system documentation, algorithm development, validation, model evaluation, model accuracy and model explainability. There is also an interesting reference to use of open-source AI, asking when its use might be appropriate. 

When can we expect to see regulatory guidance?

With so many questions to be answered, it seems unlikely that a formal regulatory guidance will be published very quickly.  However, given the complexity to be addressed, this is perhaps to be welcomed – even if the one thing sponsors and other participants in clinical development want more than anything else is regulatory clarity. 

For more on this topic, read our full blog: Regulatory acceptability of AI: Current perspectives

References

1. Liu, Q et al.  Landscape analysis of the application of artificial intelligence and machine learning in regulatory submissions for drug development from 2016 to 2021.  Clin Pharm & Therapeutics; 113(4): April 2023. 

Return to Insights Center