The UK National Institute for Health and Care Excellence (NICE) first published its real-world evidence (RWE) framework in June 2022, with a focus on improving the quality, transparency, and applicability of real-world data (RWD) in healthcare decision-making carried out by the institute. Since publication, this guidance has been periodically updated to keep pace with the rapidly developing RWE landscape.1 The most recent update in January 2025 focuses on the planning, conduct, and reporting of qualitative research studies via a dedicated appendix (Appendix 4).2
The patient voice has long been recognised as an integral part of healthcare decision making. The NICE health technology evaluation manual outlines elements of the decision problem that are particularly suitable for qualitative evaluation.3 The standards and guidance outlined in the qualitative research appendix represent an important step to ensure stakeholders are able to formalise their qualitative data generation processes and produce research that is suitable to support decision making.2 This is important, as health technology assessment (HTA) agencies increasingly recognise the value of qualitative data for outlining stakeholder values and preferences (Figure 1).2, 4 In this article, we discuss the key features of the new guidance and consider areas where additional clarification from NICE may benefit stakeholders.
Figure 1: Different topics that qualitative techniques can generate evidence for during an HTA submission
The qualitative research appendix provides detailed guidance to support planning and conduct of the study. It covers how to develop the qualitative research question and study protocol through to best practices during the reporting and assessment of qualitative studies (Figure 2).
Figure 2: NICE qualitative research appendix structure
The qualitative research appendix provides guidance on both the formulation of a suitable qualitative research question and the content of a research protocol. NICE recognises that these protocols may iterate during the research but advise that this is subject to agreement from regulatory bodies and strict version control. Other key recommendations include:
It is encouraging to note a dedicated section providing guidance on study ethics and data protection within the qualitative research appendix. This section advises that researchers obtain appropriate ethical approvals and ensure data protection, particularly when dealing with sensitive topics.2 This NICE guidance is particularly welcome, as previous research has described the reporting of ethical issues in qualitative components of HTA submissions to be lacking.5 Indeed, qualitative research may often be erroneously considered ‘low risk’ by clinical researchers, due to its social, rather than biomedical, focus. However, contrary to this belief, qualitative research can raise a variety of ethical dilemmas that require careful consideration and management.6
We have extensive experience guiding our partners to manage ethical considerations in qualitative research studies. Specific concerns can include: maintaining continuous consent, ensuring participant anonymity, and maintaining participant and researcher safety. Additional guidance from NICE on these practical elements of ethical conduct could be valuable in future guidance, to ensure ethical best practices are followed. Our dedicated Study Management team can ensure your qualitative research study is conducted in line with current best practice recommendations.
In qualitative research, sampling typically prioritises depth of understanding through non-probability methods. NICE provides a series of recommendations when designing participant sampling methods for qualitative data generation to inform institute decision-making. These recommendations include:
We typically employ a flexible sample size target, informed by an assessment of sample size feasibility, predicted data saturation and client deadlines, when designing qualitative research studies. We have found that this approach can be effective when balancing methodological best practice with practical considerations and provides our clients with budget certainty.
Our specialised Real-World Evidence team can provide hands-on support when designing your bespoke study.
The qualitative research appendix outlines different methodologies that are commonly used to generate qualitative evidence considered by NICE, alongside best practices when employing these methods. Recommendations include:
The data generation methods considered by NICE record self-reported information. However, observation, which is commonly employed in fields like anthropology, can be a powerful tool to asses people’s actual behaviour and context-specific factors.9 We are interested in exploring the role that qualitative observational methods might have in generating mixed-methods qualitative data to supplement HTA submissions.
In addition, future NICE guidance could consider covering other best practices during qualitative question formulation, including avoiding assumptions within questions, the role of closed-ended questions and embedding multiple sub-questions within a single question. Worked examples of these scenarios can be particularly effective illustrative instruments.
Our Real-World Evidence team can generate enhanced data, by triangulating self-reported qualitative and observational data, enriching your project with more accurate insights.
Qualitative data analysis processes can vary considerably based on the methodology used. The qualitative research appendix focuses on thematic analysis, a common method used in healthcare research.2 Qualitative methods can be roughly divided into two camps: those stemming from a particular theoretical or epistemological position and those that can be applied independent of theory of epistemology.10 Thematic analysis sits firmly in the second category and therefore provides a robust but flexible research tool that fits well with the mixed-methods approaches required when integrating qualitative data into a HTA submission.10 Other recommendations include:
While thematic analysis is a highly flexible and pragmatic analysis method, there are still several considerations that need to be made. These include whether a deductive or inductive coding approach will be performed and whether themes are developed at a semantic or latent level.10 As the qualitative data in HTA submissions will provide evidence against specific questions or hypotheses, deductive coding and semantic themes are likely to be of most relevance, while more interpretative approaches are less likely to add value to the evidence submission.10
Our Real-World Evidence team can guide you on how to employ a variety of qualitative analysis methods and advise on how best to tailor each approach to suit your strategic aims.
The NICE qualitative methods guide is an essential resource for those looking to integrate mixed-methods into HTA submissions. By promoting a structured and pragmatic approach to qualitative research, it encourages the integration of insights not easily obtained through quantitative evidence into future HTA submissions. The guide’s emphasis on methodological rigor and ethical standards ensures that qualitative evidence can robustly inform health decisions. However, when considering future updates, additional information on the following elements could be valuable:
The Real-World Evidence team at Costello Medical specialise in the development and delivery of qualitative research projects to inform decision-making. Our team are therefore excited to add NICE’s qualitative research appendix to the tools at our disposal when designing and conducting high-quality qualitative research projects to support HTA submissions for our clients. With experience across various qualitative data generation methods (e.g. interviews [structured, semi-structured], focus groups, surveys and observation) and analysis methods (e.g. content, thematic, realist), our team of dedicated experts can support you with your qualitative research needs.
PAUL JÓNSSON
PROGRAMME DIRECTOR – DATA AND REAL WORLD EVIDENCE, NICE
References
If you would like any further information on the themes presented above, please get in touch, or visit our Real-World Evidence page to learn how our expertise can benefit you. Seth Francis-Graham (Consultant) created this article on behalf of Costello Medical. The views/opinions expressed are his own and do not necessarily reflect those of Costello Medical’s clients or affiliated partners.
