1192P - Optimizing lung cancer screening: Independent verification of an AI/ML computer-aided detection and characterization software as medical device

Background

Detection and risk stratification of pulmonary nodules during a Lung Cancer Screening (LCS) examination is time-consuming, prone to false positives and missed detections. Leveraging Artificial Intelligence (AI) in such tasks has outperformed human readers in detection, and current risk models (Brock, MAYO) in risk prediction. Here, we present the results of an independent verification study to explore the software’s detection and characterization performance on an external dataset.

Methods

The model was trained on 10,872 patients (543 cancers) independently annotated from the NLST (National Lung Screening Trial) and LIDC (Lung Image Database Consortium) cohorts. 264 patients meeting the USPSTF criteria (age 50-80, Smoker) were collected from the EU (26%) and the USA (74%) for independent verification. The dataset comprised 88 cancer patients and 176 benign, with average size for all nodules of 6.2 ± 3.2 mm, and 17.5 ± 5.8 mm for cancerous nodules. To generate reference standards, two radiologists independently annotated each patient, with a third acting as an adjudicator to arrive at a consensus for location and nodule diagnosis (histopathology or ≥12 month stability).

Results

The verification AUC for risk prediction was 0.95, with a sensitivity of 93.2% and specificity of 87.5% at the Youden index. Cancer detection sensitivity was 91.2%, with an average of 0.44 false positive detections per scan. Performances were consistent across multiple technical and clinical parameters, including CT manufacturer, kernel hardness, kernel slice thickness, patient sex, data source, and nodule solidity (see the table). Table: 1192P

Conclusions

This study demonstrates that the AI software is robust to external data with similar performances versus the NLST test set (AUC: 0.97) and among subclasses. More accurate screening driven by AI promises to be beneficial to LCS, reducing unneeded exams, costs, and radiologist time.

Clinical trial identification

Editorial acknowledgement

Legal entity responsible for the study

Median Technologies.

Funding

Median Technologies.

Disclosure

All authors have declared no conflicts of interest.