68MO - Generalization of a deep learning model for HER2 status predictions on H&E-stained whole slide images derived from 3 neoadjuvant clinical studies

Background

Due to its decisive role to select targeted treatment options in breast cancer, we investigated if machine learning (ML) models could identify HER2-positive tumours from routinely obtained haematoxylin and eosin (H&E)-stained whole slide images.

Methods

We trained a machine learning model to predict the HER2 status on 844 patients from the GeparSixto and GeparSepto studies and validated it on 1567 independent patients from GeparSixto, GeparSepto and GeparOcto. The challenge of a single label per gigapixel image without additional, granular domain-expert annotations, is addressed by aggregating learned features per patient. We trained a network with a gated attention head and patch filtering on previously segmented tumour regions. Final predictions are made by an ensemble from a 5-fold outer cross-validation. Additionally, we use selective prediction analysis to choose a subset of patients for which the model is highly certain.

Results

Our model was evaluated on 1488 patients of the validation set (excluding 5% due to automatic quality control). With respect to the centrally assessed clinical HER2 status it achieves an area under the receiver-operator curve (AUROC) of 0.81 (95% CI 0.79-0.83) and a balanced accuracy (BA) of 73.1%. In the held-out GeparOcto subcohort, we observe AUROC=0.80 (95% CI 0.77-0.83) and BA=70.4%. Taking the model's confidence into account, we show that for a subset of 13% and another set of 32% of selected cases the BA increases to 84.7% and 81.5%, respectively. Considering different thresholds illustrates the trade-off between coverage and performance.

Conclusions

The trained ML model predicts HER2 status with state-of-the-art performance. We extend on that by demonstrating the generalization of performance on a held-out clinical study (GeparOcto). In addition, our approach demonstrates that substantial performance increases can be achieved for subsets of patients based on the model's confidence. Both, the generalization to a held-out clinical study as well as the subset analysis of the most confident predictions, demonstrate the robustness of our approach.

Clinical trial identification

Editorial acknowledgement

Legal entity responsible for the study

GBG Forschungs GmbH and BIFOLD.

Funding

German Ministry for Education and Research as BIFOLD - Berlin Institute for the Foundations of Learning and Data (ref. 01IS18025A and ref 01IS18037A) and as BBDC (KS 89715122).

Disclosure

M. Hägele: Financial Interests, Institutional, Research Grant: German Ministry for Education and Research as BIFOLD -Berlin Institute for the Foundations of Learning and Data. C. Denkert: Other, Institutional, Research Grant: Myriad, Roche; Financial Interests, Personal and Institutional, Royalties: VmScope Digital Pathology Software; Financial Interests, Personal and Institutional, Advisory Role, Consulting fees: MSD Oncology, Daiichi Sankyo, Merck, Roche, Lilly, Molecular Health; Financial Interests, Personal and Institutional, Speaker’s Bureau, Consulting fees: AstraZeneca; Financial Interests, Personal and Institutional, Other, Paten for Therapy response: WO2015114146A1, WO2010076322A1; Financial Interests, Personal and Institutional, Other, Patent for Cancer Immunotherapy: WO2020109570A1; Non-Financial Interests, Personal, Ownership Interest, cofounder and shareholder until 2016: Sividon Diagnostics. A. Schneeweiss: Other, Institutional, Research Grant: BMS (Celgene), Roche, AbbVie; Financial Interests, Personal, Other, Honoraria, Travel expenses: Pfizer; Financial Interests, Personal, Other, Honoraria: AstraZeneca, Novartis, MSD, Tesaro, Lilly, Seagen, Gilead, GSK, Bayer, Amgen, Pierre Fabre; Financial Interests, Personal, Other, Travel expenses, Honoraria: BMS (Celgene), Roche. M. Untch: Non-Financial Interests, Personal and Institutional, Other: Amgen GmbH, AstraZeneca, Celgene GmbH, Daiichi Sankyo, Eisai, Lilly Int., MSD Merck, Myriad Genetics, Pfizer GmbH, Roche Pharma AG, Sanofi Aventis Deutschland GmbH, Novartis, Clovis Oncology; Financial Interests, Personal and Institutional, Other: Lilly Deutschland, Pierre Fabre, Seattle Genetics, Seagen, GSK, Gilead. P.A. Fasching: Financial Interests, Personal, Advisory Board: Roche, Novartis, Pfizer, Daiichi Sankyo, Eisai, Merck Sharp & Dohme, AstraZeneca, Hecal, Lilly, Pierre Fabre, Seagen, Agendia; Financial Interests, Personal, Invited Speaker: Novartis, Daiichi Sankyo, Eisai, Merck Sharp & Dohme, AstraZeneca, Lilly, Seagen; Financial Interests, Personal, Other, Medical Writing Support: Roche; Financial Interests, Institutional, Invited Speaker: BionTech, Cepheid; Non-Financial Interests, Member: ASCO, Arbeitsgemeinschaft für Gynäkologische Onkologie e.V., Translational Research in Oncology, Deutsche Gesellschaft für Senologie e.v. V. Mueller: Financial Interests, Personal, Other, Speaker and Consultancy Honoraria: Amgen, Daiichi Sankyo, Eisai, MSD, GSK, Gilead; Financial Interests, Personal, Other, Consultancy Honoraria: Genomic Health, Hexal, Pierre Fabre, ClinSol, Lilly; Financial Interests, Institutional, Other, institutional research support: Novartis, Roche, Seagen, Genentech; Financial Interests, Personal, Other, speaker honoraria: AstraZeneca, Pfizer, Teva; Financial Interests, Personal and Institutional, Other, speaker and consultancy honoraria, Institutional research support: Novartis, Roche; Financial Interests, Personal and Institutional, Other, speaker honoraria, Institutional research support: Seagen; Financial Interests, Institutional, Other, institutional research support: Genentech. S. Loibl: Financial Interests, Institutional, Research Grant, honorarium for Ad Boards, paid to institute: AbbVie, Celgene; Financial Interests, Institutional, Other, honorarium for Ad Boards, paid to institute: Amgen, BMS, Eirgenix, GSK, Lilly, Merck; Financial Interests, Institutional, Research Grant, honorarium for Ad Boards & Lectures, paid to institute: AstraZeneca; Non-Financial Interests, Institutional, Research Grant, honorarium for Ad Boards & Lectures, paid to institute / Medical Writing: Gilead, Novartis, Pfizer, Daiichi Sankyo, Roche; Financial Interests, Institutional, Other, honorarium for Ad Board & Lecture, paid to institute: Pierre Fabre; Non-Financial Interests, Institutional, Other, honorarium for Ad Boards & Lectures, paid to institute / Medical Writing: Seagen; Financial Interests, Institutional, Other, honorarium for Ad Board, paid to institute: Sanofi; Financial Interests, Institutional, Other, Patent for Immunsignature in TNBC, paid to institute: EP14153692.0; Financial Interests, Institutional, Other, Patent for Signature for CDK 4/6 Inhibitor, paid to institute: EP21152186.9; Financial Interests, Institutional, Other, Paten for Predicting response to an Anti-HER2 containing therapy, paid to institute: EP15702464.7; Financial Interests, Institutional, Other, Patent for GeparNuevo, paid to institute: EP19808852.8; Financial Interests, Institutional, Royalties, VM Scope GmbH, paid to institute: Digital Ki67 Evaluator. All other authors have declared no conflicts of interest.