Airway nodule detection for routine clinical CT scans
About
Interfaces
This algorithm implements all of the following input-output combinations:
| Inputs | Outputs | |
|---|---|---|
| 1 |
|
|
Model Facts
Summary
This deep learning based algorithm detects airway nodules in a thorax or thorax-abdomen CT scan in a routine clinical setting. The algorithm was trained and evaluated with a ten-fold cross validation procedure with 320 CT scans from Radboud University Medical Center (160 scans with airway nodules, 160 scans without airway nodules).
Mechanism
Input:
- Data type: thorax or thorax-abdomen CT scan (contrast-enhanced or non-enhanced) with a maximum slice thickness of 3 mm.
- File format: MHA or DICOM
- Target population: Adult patients (18 years or older)
Output:
- Nodule locations: world coordinates (X,Y,Z)
- Nodule likelihood: nodule likelihood probability between 0 and 1.
- Overlay: points that indicate the nodule locations with approximately 0.25 and 0.56 false positives per scan on average in negative and positive patients, respectively.
Model type: Convolutional neural network Training data location and time period: CT examinations from Radboud University Medical Center from 2004-2020. The system was pre-trained on CT examinations from the LUNA16 challenge, a subset of the LIDC-IDRI dataset. Data from LIDC-IDRI originates from seven academic centers in the United States and eight medical imaging companies (exact locations and acquisition dates unknown, acquired before 2011).
Reference standard: An experienced radiologist (32 years of experience) annotated all airway nodules in the dataset. Primary cancers were verified with either histopathological or cytopathological evidence, which were recorded in the Netherlands Cancer Registry (NCR). The malignancy status of other nodules was confirmed with bronchoscopy only or follow-up imaging CT scans if such evidence was unavailable.
Validation and Performance
The table below shows the model sensitivity at 5 predefined false positive rates (0.0625, 0.125, 0.25, 0.5, and 1 average false positives per scan in negative patients) on a Free Receiver Operating Characteristic (FROC) curve. Please note that the algorithm on Grand-Challenge is only one of the 10 algorithms used for the 10-fold cross validation procedure, thus its performance may slightly differ.
| Threshold | 0.496 | 0.434 | 0.357 | 0.274 | 0.165 |
|---|---|---|---|---|---|
| FP/s in negative scans | 0.0625 | 0.125 | 0.250 | 0.500 | 1.00 |
| FP/s in positive scans | 0.256 | 0.360 | 0.560 | 0.910 | 1.72 |
| Sensitivity - % (95% CI) | |||||
| All nodules (n=186) | 66.3 (55.1-76.0) |
71.5 (63.8-79.1) |
75.1 (67.6-81.6) |
76.1 (69.3-82.4) |
77.4 (70.8-83.4) |
| Non-tumors (n=94) | 64.8 (51.4-77.5) |
68.5 (56.9-80.0) |
71.4 (60.3-82.3) |
72.7 (62.1-83.7) |
74.7 (64.7-84.4) |
| Tumors (n=92) | 68.0 (54.3-80.0) |
74.7 (65.4-84.0) |
79.0 (70.4-86.6) |
79.6 (71.2-87.0) |
80.3 (72.0-87.6) |
| Malignant tumors only (n=79) | 66.2 (50.0-79.0) |
73.1 (62.7-83.5) |
78.1 (68.2-86.4) |
78.8 (69.4-87.2) |
79.6 (70.4-87.5) |
Abbreviations: FP/s = average number of false positives per scan, CI = 95% confidence interval.
Uses and Directions
Benefits: This model may aid the radiologists obtaining a timely diagnosis of tracheobronchial cancer and monitor metastasized cancer for adequate treatment.
Target population and use case: Any adult patient (18 years or older) who would be eligible for a thorax or thorax-abdomen CT scan at the hospital. The model may assist residents, radiologists, or other physicians by acting as a concurrent reader for the detection of airway nodules in CT scans.
General use: This model is intended for detecting airway nodules in thorax or thorax-abdomen CT scans for research purposes only. The model has not been certified for clinical use. Radiologists can decide whether follow-up is needed based on these detections and their own findings in correspondence to established nodule management guidelines.
Appropriate decision support: This model returns the locations of potential airway nodules. The referring radiologist checks for additional nodules in the scan and assesses all them along with other clinical information. The radiologist decides on appropriate follow-up recommendation for the patient.
Before using this model: Test the model retrospectively and prospectively on a diagnostic cohort that reflects the target population that the model will be used upon to confirm the validity of the model within a local setting.
Safety and efficacy evaluation: TBD
Warnings
Risks: Even if used appropriately, clinicians using this model may miss relevant lesions that turn out to be cancerous. The model may miss nodules, especially in the subsegmental bronchi. Delays in cancer diagnosis can lead to metastasis and mortality. Patients who are incorrectly treated for cancer can be exposed to risks associated with unnecessary interventions and treatment costs related to follow-ups.
Inappropriate settings: This model was trained on CT scans of the thorax or thorax-abdomen region. Do not use the algorithm for CT scans that only partly depict the lungs and airways, such as CT scans of the abdomen, neck, thoracic spine, or heart. Do not use the algorithm for processing CT scans from patients with severely affected airways (i.e., extensive aspiration pneumonia or lung metastasis with bronchial infiltration or compression).
Clinical rationale: This model does not provide a rationale for the detected airway nodule candidates. Clinical end users are expected to place the model output in context with other clinical information to make the final determination of diagnosis.
Inappropriate decision support: This model may not be accurate outside of the target population and was developed on a cohort of a routine clinical population. This model is not a diagnostic and is not designed to guide clinical diagnosis and treatment for malignant airway nodules.
Generalizability: This model was evaluated within the local setting of the Radboud University Medical Center. Do not use this model in an external setting without further evaluation.
Discontinue use if: Clinical staff raise concerns about the utility of the model for the intended use case or large, systematic changes occur at the data level that necessitates re-training of the model.
Common Error Messages
Information on this algorithm has been provided by the Algorithm Editors, following the Model Facts labels guidelines from Sendak, M.P., Gao, M., Brajer, N. et al. Presenting machine learning model information to clinical end users with model facts labels. npj Digit. Med. 3, 41 (2020). 10.1038/s41746-020-0253-3