Abstract 672P
Background
Open sharing of synthetic trial data could facilitate secondary analyses and synthetic control arm development.
Methods
This study included 128 experimental vs. control treatments for solid tumors from 115 phase 3 randomized clinical trials (RCTs) published in 7 high-impact journals in 2023. Original individual patient data were reconstructed from the published Kaplan-Meier curves of overall survival (OS) in the intention-to-treat population. Either both arms (fully synthetic; retaining original sample size) or only the control arm (partially synthetic; retaining original control arm size) were synthesized using statistical models (Weibull and Synthpop) and a deep generative model (SurvivalGAN). Original and synthetic RCTs were compared through the probability overlap (0: none, 1: perfect) of the 95% confidence intervals (CIs) for the hazard ratio (HR), and the true (TPR) and false positive rate (FPR) with respect to the direction of the HR and its statistical significance on the 0.05 level.
Results
The median sample size was 571 (range 82-5637) and the median HR for OS was 0.80 (range 0.39-1.14) in the original (reconstructed) RCTs. Fifty-six (43.8%) were statistically significant, all of which showing superiority of the experimental treatment. The results presented in the table indicate that the use of synthetic arms may introduce bias due to generative model misspecification (in particular for fully synthetic RCTs created by Weibull) and additional variability inherent to the data generating process (more pronounced in fully vs. partially synthetic RCTs and in deep generative vs. statistical models), leading to insufficient CI overlap for the HR and thereby impacting statistical inference. TPR ranges from 2.6% to 80.4% and 90.9% to 98.2% in fully and partially synthetic RCTs, while FPR ranges from 25.0% to 46.4% and 27.8% to 44.4%, respectively. Table: 672P
Results of the synthetic RCTs
Approach | Model | Difference in log HR: mean (sd) | 95% CI overlap: mean (sd) | TPR: % | FPR: % |
Full | Weibull | 0.33 (0.43) | 0.43 (0.39) | 2.6 | 29.5 |
Full | Synthpop | 0.01 (0.20) | 0.84 (0.20) | 80.4 | 25.0 |
Full | SurvivalGAN | -0.01 (0.73) | 0.46 (0.35) | 66.0 | 46.4 |
Partial | Weibull | -0.11 (0.15) | 0.81 (0.23) | 90.9 | 32.4 |
Partial | Synthpop | -0.10 (0.25) | 0.85 (0.26) | 96.4 | 27.8 |
Partial | SurvivalGAN | -0.14 (0.23) | 0.71 (0.31) | 98.2 | 44.4 |
Conclusions
Naive inference from RCTs with synthetic arms may lead to overly optimistic or even wrong conclusions.
Clinical trial identification
Editorial acknowledgement
Legal entity responsible for the study
Ghent University.
Funding
Has not received any funding.
Disclosure
All authors have declared no conflicts of interest.
Resources from the same session
640P - Tumor response rate for hematologic and solid cancer drugs with FDA approval supported by single-arm trials
Presenter: Daniel Michaeli
Session: Poster session 01
641P - Phase I/II clinical and pharmacokinetic study of ecubectedin in combination with irinotecan in patients with selected advanced solid tumors
Presenter: Ana Gil Torralvo
Session: Poster session 01
642P - Phase I study of SHR-A2009, a HER3-targeted ADC, in pretreated EGFR-mutated NSCLC
Presenter: Qing Zhou
Session: Poster session 01
644P - Preliminary results of GQ1005 in metastatic HER2-low expressing breast cancer and HER2 positive gastric cancer
Presenter: Ting Deng
Session: Poster session 01
645P - Persistence and resistance of HER2 amplification after T-DM1 in a basket trial for patients with solid tumors
Presenter: Dazhi Liu
Session: Poster session 01
646P - A phase I trial of AVA6000, a fibroblast activation protein (FAP)-released, tumor microenvironment (TME)-targeted doxorubicin peptide drug conjugate in patients with FAP-positive solid tumors
Presenter: Christopher Twelves
Session: Poster session 01
Resources:
Abstract
647P - EphA2-targeting bicycle toxin conjugate (BTC) BT5528 in patients (pts) with advanced solid tumors: A phase I/II study
Presenter: Elisa Fontana
Session: Poster session 01
649P - Phase I study of SHR-A2102, a nectin-4 targeted ADC, in patients with advanced solid tumors
Presenter: Hua Zhong
Session: Poster session 01
650P - Initial results from a phase I/II study of BT7480, a novel nectin-4/CD137 bicycle tumor-targeted immune cell agonist, in patients (pts) with advanced solid tumors
Presenter: Kyriakos Papadopoulos
Session: Poster session 01