Adapting the validation process for PROs for rare diseases and other diseases with large unmet need and/or rapid progression

Given the challenges of validating Patient Reported Outcome (PRO) instruments in rare diseases, the FDA may consider "working through" using non-validated instruments in these populations, according to Peter Marks, FDA CBER Director.¹ In this blog, we discuss what a potential revised minimum validation effort could look like, so that patients and sponsors can proceed quickly. As well as in rare diseases, this is important when we consider the emergence of new infectious diseases (such as COVID-19) where an instrument is needed within weeks, as opposed to the years normally needed for FDA COA (clinical outcomes assessment) acceptable validation.

Scale validation ensures that the selected instrument measures what it is intended to measure, and that robust evidence is provided to accurately interpret the results. We understand the need to be flexible, particularly as we consider the large sample sizes and the time that is typically required for standard validation. However, it is important to ensure that the foundations of the PRO validation process are maintained to ensure that a true treatment effect is observed, especially for those assessing primary and secondary endpoints. 

Interpreting FDA’s Patient Focused Drug Development guidance 

Since 2020, the FDA COA Science group has outlined various paths for how sponsors can validate their instrument using the Patient Focused Drug Development (PFDD) guidance 1,2,3 and 4.^2,3,4,5

We represent the main components of an instrument validation including validity, reliability, and responsiveness in Table 1 below. Content validity, which is a prerequisite for other types of validity, does not require large sample sizes due to the qualitative methodology used. Content minimum validation ensures that:

• The diagnosed patient input is included in the content of the new PRO development
• The items in a PRO instrument are relevant to the majority of the population being treated
• The PRO is comprehensible, covers the relevant symptomatology, and interpretable data can be generated

As such, it is important to preserve the content validity, even for PROs in rare diseases and/or other diseases with high unmet patient needs and/or rapid progression. 

Potential minimum validation approach

A potential minimum validation approach for PRO development in these scenarios involve conducting a small number of interviews (5-7), reviewing existing qualitative data from other sources (survey, case reports) as well as existing real-world sources (e.g NIH studies) and conducting psychometric analysis.  

Once the disease content has been established based on patient evidence, it is then possible to build a toolbox of items that are available already and have shown promise – the PROMIS database may offer information to support the selection of items that are best suited for some symptoms and functions. This is particularly important if the instrument is included as a primary or secondary endpoint.

The use of multiple sources is important in validating instruments. For example, Parexel’s COA science group contributed to the validation of a scale designed to assess symptom severity in a rare paediatric neurological condition. We conducted 5 qualitative interviews, engaged with a panel of experts, and evaluated the psychometric properties of the scale with 39 completed questionnaires using the Classical Test Theory. 

Additionally, to facilitate designing new instruments for these populations, we suggest incorporating gold-standard outcome tools for related conditions that might provide additional validation approaches to help bring treatments to such patients. 

Table 1. Parexel COA Science Understanding and Approach to PRO Validation

New challenges and opportunities for PRO validation best practice 

It is important to continue to validate instruments to ensure with certainty and transparency that an instrument is indeed patient-focused, reliable, and valid for the context of use of a clinical trial. The implementation of PRO validation best practice in rare diseases and/or other diseases with high unmet need and/or rapid progression evaluation is evolving. 

We expect new opportunities to become available in PRO science towards shared and more robust methodologies. A growing number of experts now advocate for a more flexible approach, including a task force from the International Rare Diseases Research Consortium.^6,7 This initiative explored solutions to understand diseases and COA implementation, such as strengthening international collaboration with healthcare professionals and patient organizations, use of item banks, qualitative exit interviews, implementing iterative mixed method designs, and psychometric methods suitable to smaller samples (e.g. Rasch measurement theory), as well as combining the target populations with similar disease population to increase sample size and the use of validated disease group measures. ^7,8,9 

Although some of the suggested approaches require further discussions and clearer definitions, this approach begins to address the challenges posed by the development and evaluation of PROs in rare disease, and hope that this is the evidence that FDA and other agencies may accept.  We hope that a shorter validation effort for content validity and using existing item banks will allow us to expedite the development of instruments, as well as allow the research community to reduce the time spent in translating new instruments. 

Ready to discuss your PRO validation approach? Our COA sciences team is always available for a conversation. 

References

1.     https://pink.citeline.com/PS148238/Unvalidated-PROs-In-Rare-Diseases-US-FDA-May-Have-To-Work-Through-Them-CBERs-Marks-Says. Accessed 19 March 2024.

2.    US Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research, Center for Biologics Evaluation and Research, and Center for Devices and Radiological Health. Patient-focused drug development: collecting comprehensive and representative input: guidance for industry, food and drug administration staff , and other stakeholders. 2020. https://www.fda.gov/media/139088/download. Accessed 16 Sept 2022.

3.    US Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Reserach, Center for Biologics Evaluation and Research, and Center for Devices and Radiological Health. Patient-focused drug development: methods to identify what is important to patients guidance for industry, food and drug administration staff, and other stakeholders. 2022. https://www.fda.gov/media/131230/download. Accessed 16 Sept 2022.

4.    US Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Reserach, Center for Biologics Evaluation and Research, and Center for Devices and Radiological Health. Patient-focused drug development: selecting, developing, or modifying fit-for purpose clinical outcome assessments (draft guidance). 2022. https://www.fda.gov/media/159500/download. Accessed 16 Sept 2022.

5.    US Department of Health and Human Services, F.a.D.A., Center for Drug Evaluation and Research, Center for Biologics Evaluation and Research, and Center for Devices and Radiological Health. Guidance for industry, US Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research, Center for Biologics Evaluation and Research, and Center for Devices and Radiological Health. Guidance for industry Patient-Focused Drug Development: Incorporating Clinical Outcome Assessments Into Endpoints for Regulatory Decision-Making. Accessed 23rd Feb 2023.

6.    Sabino G, Mills A, Jonker AH, Lau LPL, Aartsma-Rus A, Aarum S, Arora J, Calvert M, Cano SJ, Denegri S, Hass S, Kelley TA, Klassen AF, Kodra Y, Légout V, Hivert V, Morel T, Payne K, Posada M, Prainsack B, Moy CS, Schmitt J, Summar M, Terry SF, Terwee CB, Vernon M, Williamson PR, Aymé S, Eds., “Patient-Centered Outcome Measures in the Field of Rare Diseases”, International Rare Disease Research Consortium, February, 2016. PCOM_Post-Workshop_Report_Final.pdf (irdirc.org) Accessed 19th March 2024.

7.    Benjamin K, Vernon MK, Patrick DL, Perfetto E, Nestler-Parr S, Burke L. Patient-reported outcome and observer-reported outcome assessment in rare disease clinical trials: an ISPOR COA emerging good practices task force report. Value in Health. 2017 Jul 1;20(7):838-55. https://doi.org/10.1016/j.jval.2017.05.015. Accessed 19th March 2024.

8.    Morel T, Cano SJ. Measuring what matters to rare disease patients–reflections on the work by the IRDiRC taskforce on patient-centered outcome measures. Orphanet journal of rare diseases. 2017 Nov 2;12(1):171. https://doi.org/10.1186/s13023-017-0718-x. Accessed 19th March 2024.

9.    Whittal A, Meregaglia M, Nicod E. The use of patient-reported outcome measures in rare diseases and implications for health technology assessment. The Patient-Patient-Centered Outcomes Research. 2021 Sep;14:485-503. https://doi.org/10.1007/s40271-020-00493-w. Accessed 19th March 2024.

Return to Insights Center