176P - Enhancing immunotherapy response prediction via multimodal integration of radiology and pathology deep learning models

Background

The growing need for better predictive biomarkers in immunotherapy has driven the development of AI models across various fields. Despite this progress, most studies focus on single modalities, such as pathology or radiology. This study emphasizes the importance of multimodal biomarkers, combining radiology and pathology data, to optimize treatment outcomes and advance personalized medicine in immunotherapy.

Methods

We developed weakly-supervised deep learning (DL) models for CT and PDL1-stained immunohistochemistry (IHC) images. We trained a CT-based model to predict immunotherapy clinical benefit (PFS > 5 months) in a pan-cancer cohort from VHIO(n=369). Simultaneously, we trained a DL method to predict PD-L1 status from IHC images in an open source NSCLC cohort (n=233), validated in a pan-cancer cohort from VHIO (n=106). The model predictions of both modalities were aggregated and utilized for patient stratification in immunotherapy response. In the overlapping subset with both modalities' data (n=75), we explored the association between PFS and the combination of predicted clinical benefit by CT and predicted PDL1 status by IHC using Cox regression and log-rank test.

Results

The CT-based and IHC-based DL models exhibited effective performance in predicting PFS for immunotherapy response in the pan-cancer cohort, with hazard ratios (HR) of 0.62 [95% CI 0.5-0.78], p <0.05 and 0.66 [95% CI 0.44-1], p <0.05, respectively. In the multi-modal data subsets, patients classified as clinical benefit prediction by the CT-based model and PDL1 high by the IHC-based model (n = 20) exhibited better immunotherapy response (HR 0.44 [0.25-0.78], p <0.05) with longer PFS (median PFS 4.08 [2.70-10.91] months) compared to the rest (median PFS 1.81 [1.68-2.96] months).

Conclusions

Integrating radiology and pathology data improves response prediction compared to single modality models, emphasizing the potential of a multimodal approach for better patient stratification in immunotherapy.

Clinical trial identification

Editorial acknowledgement

Legal entity responsible for the study

The authors.

Funding

CRIS Foundation Talent Award (TALENT19-05).

Disclosure

R. Dienstmann: Financial Interests, Personal, Invited Speaker: Amgen, AstraZeneca, Boehringer Ingelheim, Ipsen, Libbs, Lilly, Merck Sharp & Dohme, Roche, Sanofi, Servier, GSK, Takeda, Janssen, Foundation Medicine; Financial Interests, Personal, Advisory Board: Bayer, Roche; Financial Interests, Personal, Full or part-time Employment, Oncoclínicas is a private healthcare provider in Brazil. I work part time as Medical Director of the Precision Medicine and Big Data Initiative. We develop molecular tests (pathology and genomics) that are offered to patients treated in the organisation as part of support programs sponsored by pharmaceutical companies and I coordinate research activities with real-world clinico-genomics cohorts: Oncoclínicas; Financial Interests, Personal, Stocks/Shares: Trialing; Financial Interests, Personal, Research Grant: Merck. E. Garralda: Financial Interests, Personal, Advisory Board: Roche, Ellipses Pharma, Boehringer Ingelheim, Janssen Global Services, Seattle Genetics, Alkermes, Thermo Fisher, MabDiscovery, Anaveon, Hengrui, F-Star Therapeutics, Sanofi, Incyte; Financial Interests, Personal, Invited Speaker: MSD, Lilly, Roche, Thermo Fisher, Novartis, Seagen; Financial Interests, Personal, Full or part-time Employment: NEXT Oncology; Financial Interests, Institutional, Funding: Novartis, Roche, Thermo Fisher, AstraZeneca, Taiho; Financial Interests, Institutional, Research Grant: BeiGene, Janssen. P.G. Nuciforo: Financial Interests, Personal, Invited Speaker: Novartis; Financial Interests, Personal, Advisory Board: MSD Oncology, Bayer; Financial Interests, Personal, Other, Consultant: Targos Molecular Pathology GmbH. J.N. Kather: Financial Interests, Personal, Invited Speaker: Fresenius, Eisai, MSD; Financial Interests, Personal, Advisory Board: Owkin, DoMore Diagnostics, Panakeia, London, UK. R. Perez Lopez: Financial Interests, Personal, Full or part-time Employment, VHIO staff (team leader of the Radiomics Group): VHIO; Financial Interests, Institutional, Research Grant, Co-PI of 3 research studies.: AstraZeneca; Financial Interests, Institutional, Research Grant, PI of a research study: Roche. All other authors have declared no conflicts of interest.