120P - A prospective comparative evaluation of automatic trial match tools in a molecular tumor board

Background

Searching for clinical trials for a patient in Oncology requires extensive knowledge on selection criterias, molecular testings, status of the trial recruitment among other information, and for multiple trials options (https://pubmed.ncbi.nlm.nih.gov/38401694/). Automatic trial matching tools are currently developed to make this search easier and more exhaustive and mostly based on clinical information such as the cancer type, the genomic alterations and the geographical location.

Methods

We prospectively used preselected trial matching tools from 3 countries: France (Klineo, ScreenAct), Spain (Trialing) and the United Kingdom (DigitalECMT), all publically available. We analyzed sequential patients cases presented at the Molecular Tumor Board of the Centre Leon Berard. Each trial proposition result was manually reviewed for the selection criterias and the updated recruitment status. We used the Average Precision (AP) and Normalized Discounted Cumulative Gain (NDCG) metrics specifically dedicated to evaluate the performance of search engines and the performance of the ranking. We capped at 3 the number of evaluated results (AP@3 and NDCG@3).

Results

Between April 3 and June 12, 2024, we prospectively analyzed 157 patients affected by various types of metastatic cancers. Each patient had a mean of 2.19 (sd 3.13) trials proposed. The global performances of the tools were AP@3 mean 0.40, sd 0.44, and NDCG@3 mean 0.30, sd 0.34. The AP@3 and NDCG@3 scores were respectively 0.60 and 0.50 for Klineo, 0.36 and 0.25 for DigitalECMT, 0.34 and 0.24 for ScreenAct, and 0.30 and 0.23 for Trialing. We observed that 88% of the errors concerned selection criterias, and 24% concerned the status of the trial recruitment. The main limitations comprised gene variants not included in the tools, the specific type of presentation of the results, the number or types of previous treatment lines.

Conclusions

Publically available automatic Trial Matching tools are valuable help to orient patients to new therapeutic options, but may contain several erroneous results when taking full selection criterias and updated trial and cohort status, supporting a careful evaluation by the treating physician before using it for clinical decision support.

Clinical trial identification

Editorial acknowledgement

Legal entity responsible for the study

Centre Léon Bérard.

Funding

Centre Léon Bérard.

Disclosure

L. Verlingue: Financial Interests, Personal, Stocks/Shares, CEO of Resolved dedicated to treatment approval prediction: Resolved; Financial Interests, Institutional, Research Grant: Bristol Myers Squibb; Financial Interests, Institutional, Funding, Contract for bioinformatic analysis: Pierre Fabre, Servier; Non-Financial Interests, Advisory Role: Klineo; Non-Financial Interests, Institutional, Proprietary Information, As part of the Drug Development Department (DITEP) of Gustave Roussy and of the Phase 1 unit of Centre Léon Bérard, as medical doctor, LV report being: Principal/sub-Investigator of Clinical Trials for AbbVie, Adaptimmune, Aduro Biotech, Agios Pharmaceuticals, Amgen, Argen-X Bvba, Arno Therapeutics, Astex Pharmaceuticals, AstraZeneca Ab, Aveo, Basilea Pharmaceutica International Ltd, Bayer Healthcare Ag, Bbb Technologies Bv, BeiGene, Blueprint Medicines, Boehringer Ingelheim, Boston Pharmaceuticals, Bristol Myers Squibb, Ca, Celgene Corporation, Chugai Pharmaceutical Co, Clovis Oncology, Cullinan-Apollo, Daiichi Sankyo, Debiopharm, Eisai, Eisai Limited, Eli Lilly, Exelixis, Faron Pharmaceuticals Ltd, Forma Tharapeutics, Gamamabs, Genentech, GSK, H3 Biomedicine, Hoffmann-La Roche Ag, Imcheck Therapeutics, Innate Pharma, Institut De Recherche Pierre Fabre, Iris Servier, Janssen Cilag, Janssen Research Foundation, Kura Oncology, Kyowa Kirin Pharm. Dev, Lilly France, Loxo Oncology, Lytix Biopharma A: Pharmas. All other authors have declared no conflicts of interest.