317P - Valuing the effect sizes hypothesized in phase 3 trials published from 2005 to 2015

Abstract

Background

We sought to determine the effect sizes hypothesized (ESH) for phase 3 trials of targeted therapies, and to compare them against frameworks for assessing the magnitude of benefit from anti-cancer treatments recently proposed by ESMO (Cherny et al Ann Onc 2015), and ASCO (Schnipper et al JCO 2015), and against recommendations for designing phase 3 trials (Ellis et al JCO 2014).

Methods

We searched Medline from 2005-15 for phase 3 trials of targeted therapies, including immunotherapies, in metastatic solid organ malignancies, designed with a superiority hypothesis. For each trial's primary endpoint, we determined the ESH in relative terms (typically a hazard ratio [HR]), and in absolute terms (typically a difference in median survival times or rates). The hypothesised effect size was assigned a clinical benefit score according to the ESMO and the ASCO frameworks for valuing the results of phase 3 trials, with scores of 1 or 2 indicating modest benefits, and higher scores indicating larger benefits.

Results

Sufficient information was available to determine both the relative and absolute ESH in 141 of the 203 included trials (69%), only the relative ESH in 53 (26%), only the absolute ESH in 0, and neither relative nor absolute ESH in 9 (4%). The 2014 ASCO recommendation that phase 3 trials should be designed with a HR ≤ 0.8 was met by the majority of trials (98/102, 96%) with a primary outcome of overall survival. The majority of these phase 3 trials had hypothesized effect sizes that were judged to reflect modest clinical benefits by both frameworks: ESMO 169/203 trials, 83% and ASCO 152/203 trials, 75% (see table).

		Primary outcome of OS N = 102 (50%)	Primary outcome of PFS or TTP N = 94 (46%)	Total number of trials that meet criteria N = 203 (100%)
ESMO preliminary magnitude of clinical benefit grade	1 Least benefit	9 (4%)	79 (39%)	88 (43%)
	2	81 (40%)	0 (0%)	81 (40%)
	3	1 (0.5%)	13 (6%)	14 (7%)
	4 Most benefit	7 (3%)	N/A	7 (3%)
ASCO criteria for clinical benefit (% improvement in outcome)	1 (> 0 – 24%) Least benefit	0 (0%)	0 (0%)	0 (0%)
	2 (25 – 49%)	87 (43%)	65 (32%)	152 (75%)
	3 (50 – 75%)	11 (5%)	25 (12%)	36 (18%)
	4 (76 – 100%)	0 (0%)	2 (1%)	2 (1%)
	5 (>100%) Most benefit	0 (0%)	0 (0%)	0 (0%)

Conclusions

Only 69% of these phase 3 trials included sufficient information to determine both the relative and absolute effects sizes hypothesized. The majority of trials were powered to show magnitudes of clinical benefit rated modest by the ESMO and ASCO frameworks. Trial protocols and reports should specify both the relative and absolute benefits hypothesized.

Clinical trial identification

Legal entity responsible for the study

The University of Sydney

Funding

Cancer Australia, Cancer Institute New South Wales, NHMRC Clinical Trials Centre Postgraduate PhD Scholarship, Sydney Catalyst PhD Top Up Scholarship, Goldman Sachs New Zealand Clinical Fellowship Award

Disclosure

All authors have declared no conflicts of interest.