The Ongoing Challenges of Immature Data

The increasing use of flexible and accelerated regulatory pathways has prompted a reliance on immature survival data in HTA submissions. For a number of years now, ISPOR conferences have discussed approaches to handling survival data immaturity, including utilising surrogate endpoints or leveraging expert opinion. In Copenhagen this year, experts continued to explore the challenges arising from the use of these methods.

It is well established that the use of surrogate endpoints should be underscored by validation of the relationship between the surrogate outcome and final clinical outcome, based on biological plausibility, the precision of the surrogate measurement and level of correlation (Figure 1).¹ Currently, validated surrogate endpoints only exist in a limited number of indications, including some oncology and cardiovascular indications. However, even validated surrogates have been found to over- or under-predict treatment effects when compared to data generated post-approval. In Session 126,² Dan Ollendorf presented a specific case study of elevated low-density lipoprotein (LDL), an established surrogate outcome for cardiovascular risk.^{3, 4} Early modelling of treatments that are effective at significantly lowering LDL predicted significant reductions in most types of cardiovascular events based on the surrogate relationship.⁵ However, later modelling exploring the relationship based on recent randomised control trials has suggested that the relative risk of reduction in cardiovascular events was overpredicted, highlighting the need for continued research into these approaches.⁶ It will be interesting to see findings from similar case studies in oncology indications where post-approval data are now available.  

Figure 1. The Surrogate-Final Outcome Relationship

Reference: ISPOR Value & Outcomes Spotlight, November 2020 

Importantly, it was noted that further guidance on best practice use of surrogate outcomes in economic analyses is currently being developed by a group of several international organisations and HTA bodies, including NICE, SMC, CADTH, HTAi Global Policy Forum, ICER and others. This work has been ongoing since January 2023, and we hope to see some output by next year.⁷ Of note, there were calls from representatives from HTA bodies for the pharmaceutical industry to deepen the exploration of clinical trial datasets in order to discover robust correlations between measured outcomes and maximise the potential of collected data rather than solely relying on existing surrogate endpoints.

Another topic of discussion at the conference was the appropriateness of methods for incorporating external data to produce plausible lifetime extrapolations. HTA bodies such as NICE have developed methods guidance on ways to improve extrapolations by incorporating external data, such as real-world evidence (RWE) and expert opinion, but there is a need for more in-depth, standardised guidance on the application of methods, i.e., when these methods are most appropriate and how to apply them in practice.⁸ External data approaches include pooling RWE with clinical trial data; using RWE to inform baseline risk; and incorporating general population mortality in extrapolations. Which existing methods enhance the clinical plausibility of extrapolations without diminishing structural uncertainty was a matter of considerable debate in Session 229,⁹ with some experts highlighting the need for more complex methods while HTA decision-makers leaned towards a “simpler” approach. A few issues were discussed in detail:

• The ability of external data to improve generalisability: Some experts highlighted that complex methods such as left or right censoring techniques may be required to create a RWE dataset more generalisable to current practice; this is because while mature RWE is more informative, it is also often historic and this can raise generalisability issues. Decision-makers, however, indicated that it may be sufficient to incorporate external data to establish baseline risk and then apply relative effects through network meta-analyses¹⁰ 
• Complex methods for the pooling of RWE and trial data, such as Bayesian dynamic borrowing,¹¹ were suggested as appropriate methods to account for heterogeneity between the source data and target study population. However, complex methods are limited in their ability to compensate for fundamental data limitations. The decision-maker perspective emphasised that maintaining clinical plausibility regarding absolute and relative survival effects is important above all, whichever methods are selected 

The sessions at the conference tackling the challenge of immature data highlighted the trade-off between complexity and simplicity: there is at times a misalignment between the academic discourse and the need for transparency from HTA agencies (who noted that more complex methods are frequently accompanied by insufficient and/or unclear explanation). Nevertheless, there was a consensus that manufacturers should participate in early HTA consultations to verify the use of methods in the case of immature data and to establish a clinically plausible base case. With the growing influx of innovative therapies, it is vital that decision-making bodies and manufacturers are able to work together to understand which methods are likely to be accepted and the level of evidence required, and further, clear guidance from decision-making bodies would be welcome in this regard.