Analytical Methods and Technical Topics

Artificial Intelligence and Machine Learning in Health Economics and Outcomes Research (HEOR)

Artificial intelligence and machine learning are exerting a growing influence on the healthcare industry, and the field of HEOR is no exception. Machine learning had a strong presence at ISPOR 2022; ICER, in particular, dedicated an entire session to the findings from their machine learning task force on how to apply machine learning in HEOR. In this session, the task force described a few identified HEOR use cases of machine learning:

1. Data screening: Traditionally, patient screening (e.g. for study entry) and literature review are two areas that require human reviewers to manually inspect medical records or scientific publications, respectively, resulting in substantial work time. With a machine learning approach, a model can be trained to identify key terms and phrases in medical records or publications in order to help select eligible subjects or articles for research, thus reducing time and labour.
2. Model building: Human researchers often rely on explainability (the ability to provide an explanation for an association between a given predictive variable and an outcome) when choosing which variables to include in a prediction model. Yet, associations between the outcome and variables in a model may not always be causal. In such cases, explainability may be less intuitive for human researchers and variables of predictive value may therefore be missed; the growing size of datasets only compounds this problem. Adopting machine learning can overcome these issues by allowing us to explore the optimal combination of variables in a model that yields the most accurate outcome. Another use case for machine learning in supporting robust modelling would be in understanding patients’ real-world treatment pathways from complex datasets; such information could then be used to validate the real-world relevance of the structure of a Markov cost-effectiveness model, for example.

On the other hand, machine learning has long been criticised as a black box, as even the modellers themselves can’t fully explain all of the resulting properties of their models (e.g. why certain variables were selected). The ICER task force noted that we often have to trade performance (speed/accuracy) for explainability and transparency. Additionally, applying machine learning in a project also requires substantial time, and specialist skills, and therefore is only time-saving when dealing with particularly large or complex datasets or analyses. The task force report is due to be released later this year and will cover the best practices for applying machine learning in HEOR.  

Generalised Risk-Adjusted Cost-Effectiveness (GRACE) Framework

Figure 1

“A QALY is a QALY is a QALY” is a famous quotation that emphasises that all QALY gains are the same regardless of who incurs them.¹ However, this is also why standard cost-effectiveness analysis has been criticised as discriminating against individuals with severe diseases. GRACE is a new framework that aims to address this criticism by incorporating the concept of diminishing returns when estimating willingness-to-pay (WTP) thresholds.² Under this framework, QALY gains under severe diseases are valued more (i.e. there is a greater WTP level for QALY gains in severe diseases).

Operationally, the GRACE framework proposes to achieve this by modifying the traditional WTP estimate by two factors, based on the following formulae, where K represents the traditional WTP threshold and K_GRACE represents the modified WTP threshold:³

K_GRACE = K ∗ _ωH ∗ R

• The first factor, _ωH, describes how utility changes with respect to health-related quality of life (HRQoL), and incorporates diminishing marginal utility with respect to health. For example, an individual may value an improvement from 1 to 2 QALYs more than an improvement from 10 to 11 QALYs, even though 1 QALY is gained in both cases. Assuming that marginal utility is diminishing with respect to health, _ωH will be between 0 and 1, and therefore in isolation, this factor will reduce the traditional willingness-to-pay-threshold.
• The second factor, R, represents disease severity, and is equal to the ratio between the marginal utility of health for people with the disease, versus for people without the disease. R will be approximately equal to 1 for less severe diseases, but will rapidly increase with illness severity, and therefore R will increase the modified WTP threshold for more severe diseases. In contrast, traditional cost-effectiveness analysis (CEA) frameworks inherently assume this ratio is equal to 1; while this may be appropriate for less severe diseases, the GRACE method operates on the principle that the traditional approach understates the WTP threshold considerably for highly severe illnesses. By using this ratio approach, the GRACE method facilitates consideration of severity on a continuous scale; this is a notable contrast to the discrete severity thresholds recently introduced by NICE in the UK. 

Beyond severity, the GRACE framework also has the potential to incorporate a number of additional elements of value in the ISPOR value flower, including value of insurance and cost of uncertainty. However, panelists at the conference highlighted that GRACE cannot factor in externalities, such as equity or scientific spillover; the multiple-criteria decision analysis (MCDA) method may be more appropriate for incorporating these aspects.³

Quantitatively Accounting for Real Option Value in HEOR

As at previous ISPORs, the ISPOR value flower continued to influence several sessions. One session focussed on real option value from a methodological perspective, assessing two approaches for quantifying potential health benefits associated with a technology which extends life and thus creates opportunities for a patient to benefit from future advances in medicine.⁴

The “Trend Approach” aims to forecast future health improvements by using retrospective observational data reporting trends in survival or HRQoL changes over previous years. Conversely, the “Pipeline Approach” aims to identify specific future innovations from trial registries, and forecasts their treatment effectiveness, likelihood and timing of arrival, and the potential uptake level of the treatment. In order to incorporate these estimates of future health improvement, the session proposed an alternative partitioned survival model structure that includes two post-progression health states; one for patients who experience disease progression early in the model, and a second for patients who survive long enough to benefit from future advances in healthcare (Figure 2).

Figure 2

Discussions at the conference highlighted that while some research has shown real option value to represent between 5-18% of the conventional value of oncology treatments, formally incorporating real option value still faces limitations relating to uncertainty over the future costs of new innovations, as well as further research requirements surrounding the methods of forecasting future health improvements. Given this uncertainty, it remains to be seen whether real-option value becomes an additional element of value that will be quantitatively considered by any HTA agencies and payers in the coming years.