The Dawn of Real-World Evidence Frameworks: Where Does NICE Fit In?

Regulatory Frameworks

The US Food and Drug Administration (FDA) published a framework for their RWE programme in 2018.³ Broadly, the FDA framework aimed to outline what was needed in order to include and evaluate RWE in regulatory decisions about the effectiveness of drug products to accelerate access to patients. Since this framework was published, RWE has been used in regulatory decision-making for the label expansion of drug products for additional disease indications and has expanded into decision-making for medical devices. The FDA framework continues to be updated with more specific guidance documents for various types of RWD studies. Thus far, additional regulatory decision-making guidance documents have been published by the FDA on the following RWE topics:⁴

• Assessing electronic health records, claims data and registries
• Using electronic health records in clinical investigations
• Submitting documents utilising RWD to the FDA in a uniform format
• Using RWE in medical device regulatory decision-making

Following the release of the FDA framework, the Institute for Clinical and Economic Review (ICER) in the US announced their commitment to seek out increased opportunities to generate and use RWE in their clinical effectiveness and cost-effectiveness reports.⁵ Increasing awareness of available RWE data sources and increasing collaboration with patient advocacy groups (PAGs) are two key ways to expand the use of RWE in regulatory and HTA decision-making.

In the EU, the European Medicines Agency (EMA) envisions that by 2025, the use of RWE will have been enabled and its value established across the spectrum of regulatory use cases. This has been accompanied by a varietiy of initiatives such as the OPTIMAL framework in 2019, which provides guidance on the operational, technical and methodological aspects of regulatory use of RWE, the Data Analytics and Real World Interrogation Network (DARWIN EU) and the European Health Data Space (EHDS).^6-8

Reimbursement Frameworks

In Asia, the REAL World Data In ASia for HEalth Technology Assessment (REALISE) guidance was published in 2021 by a regional working group including representatives from 11 Asian health systems.⁹ It presents a detailed framework for the use of RWD and RWE to inform drug assessments following a recognised need and opportunity to use data from routine healthcare data sources in this region considering the under-representation of Asian populations in clinical trials.^{10, 11} Due to the regional scope of the document, the authors provide principles and non-binding good practice recommendations intended to be adapted to local needs, recognising that the actual implementation of the guidance will vary from country to country.¹²

Each of these frameworks provide important guidance on the use of RWE. Together, these advances play important roles in setting standards for using RWD to resolve gaps in knowledge and drive forward access to innovations for patients globally.

Key Takeaways

The final framework provides guidance on the conduct of quantitative RWE studies, how to assess data suitability and methods for real-world studies of comparative effects. This framework is a living document that NICE plans to update at least once by July 2023.

A key update between the draft guidance and final guidance is a clear stance that the framework does not set minimum acceptable standards for the quality of evidence. Other updates include guidance on the different stages of evidence generation such as results interpretation and the reporting of study limitations.

The framework is a valuable summary of best-practice in RWE and will help support the appraisal of RWE. Some particularly useful aspects are:

• Highly specific guidance on reporting of real-world studies
• Acknowledgement of the need for considerable processing or curation before analysis of RWD
• Highlights the key areas of importance when assessing a real-world dataset
• Guidance on the gold standard for assessing the accuracy of a dataset and acceptable alternatives should a gold standard approach not be feasible
• A collection of use cases where RWD has fulfilled various roles across previous NICE guidance, including characterising conditions and care pathways, populating economic models, validating economic models and measuring impact of new interventions on service delivery
• Information and recommendations on methods for real-world studies of comparative effects, including the use of RWD to generate external control arms

In future updates to the framework, we hope to see more support for the practical design and implementation of RWE studies and how RWE can be used in NICE submissions. Specifically:

• Clearer guidance on how a judgement call should be made when there are multiple alternatives and trade-offs to consider. For example, few datasets will meet all of the requirements of ‘good provenance, relevant and of sufficient quality to answer the research question’, so guidance on which areas to prioritise would be helpful. For example, if choosing between datasets with either high levels of completeness or high levels of accuracy, which would be considered more reliable?
• More discussion on the feasibility of, and barriers to, some of the best-practice recommendations. For example, the recommendation to make an analytical code available is unlikely to be practical in all studies. Some recommendations for realistic alternatives or ways of reducing the risk of information about patients becoming publicly available through code structure would be helpful, as would considerations of intellectual property
• Integration of more case studies throughout the document to illustrate how specific challenges may be overcome, and uses of RWE that have been considered by NICE to be acceptable in the past. For example, the guidance recommends a Target Trial Approach for real-world studies of comparative effects. A case study here would be helpful to illustrate how much detail is needed on aspects that will not likely be relevant for the observational study, such as cluster versus individual randomisation
• Development of supporting documents such as study design and reporting checklists to help this become a practical tool in addition to a reference document. For example, for the subsection on how contextual factors that influence the acceptability of evidence vary across NICE programmes, a flow chart, checklist or scoring matrix would be helpful to guide decisions on evidence generation planning
• Additional content on: Best practice recommendations for the development, validation and reporting of code lists; recommendations on how to best engage with stakeholders who are less familiar with RWE, particularly during the publication of peer-reviewed articles; overcoming practical barriers that will play an important role in choosing datasets, such as long delays in data access

We were delighted to participate in this important consultation, and we look forward to contributing to future updates of a framework that has the potential to play a critical role in improving trust in RWE studies and their use in decision-making.

References

1. National Institute for Health and Care Excellence. NICE real-world evidence framework. Available online here. Last accessed Jun 2022. 2022.
2. National Institute for Health and Care Excellence. NICE strategy 2021 to 2026. Available online here. Last accessed Jun 2022. 2021.
3. US Food & Drug Administration. Framework for FDA’s Real-World Evidence Program. Available online here. Last accessed Jun 2022. 2018.
4. US Food & Drug Administration. Real-World Evidence. Available online here. Last accessed Jun 2022. 2022.
5. Institute for Clinical and Economic Review. 2020-2023 Value Assessment Framework. Available online here. Last accessed Jul 2022. 2020.
6. Arlett, P; Kjær, J; Broich, K et al. Real-world evidence in EU medicines regulation: enabling use and establishing value. Clinical Pharmacology & Therapeutics. 2022;111(1):21-23.
7. Cave, A; Kurz, X; Arlett, P. Real-world data for regulatory decision making: challenges and possible solutions for Europe. Clinical pharmacology and therapeutics. 2019;106(1):36.
8. European Comission. European Health Union: A European Health Data Space for people and science. Last accessed: 5th July 2022.
9. Lin, LW; Ahn, J; Bayani, DBS et al. Use of Real-World Data and Real-World Evidence to Support Drug Reimbursement Decision-Making in Asia (Version 1.1) Available online here. Last accessed Jun 2022.
10. Fashoyin-Aje, LA; Fernandes, LL; Lemery, S et al. Asian representation in clinical trials of new drugs for the treatment of cancer. American Society of Clinical Oncology; 2017.
11. Downing, NS; Shah, ND; Neiman, JH et al. Participation of the elderly, women, and minorities in pivotal trials supporting 2011–2013 US Food and Drug Administration approvals. Trials. 2016;17(1):1-11.
12. REAL World Data In ASia for HEalth Technology Assessment in Reimbursement (REALISE). Use of Real-World Data and Real-World Evidence to Support Drug Reimbursement Decision-Making in Asia. Available online here. Last accessed Jun 2022. 2021.