Evaluation of the Performance of an Artificial Intelligence (AI) Algorithm in Detecting Thoracic Pathologies on Chest Radiographs

Chest radiography is relatively inexpensive and the most frequently performed radiological investigation globally. Chest radiographs play a significant role in screening and diagnosing diseases of cardiothoracic and pulmonary abnormalities [1]. Accurately recognizing abnormalities in chest X-rays (CXR) poses a notable challenge, which is primarily attributed to the scarcity of proficiently trained radiologists and the substantial workload in large healthcare facilities [2]. In addition, the increasing demand for CXRs in the emergency department may directly increase the error rate [3]. Thus, analyzing and reporting chest X-rays is still a challenging and subjective process.

This study was conducted and reported according to the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) guideline [22]. The TRIPOD checklist for prediction-model development and validation is reported in Appendix A Table A1.

The clinical validation study was performed on 900 CXR images. There was a total of 640 CXRs with no anomaly and 260 CXRs with an anomaly (consolidation = 77, pleural effusion = 60, pneumothorax = 32, cardiomegaly = 16, OAP = 37, and pulmonary nodule = 43). The performance results of the readers with and without Rayvolve aid are reported in Table 1. The average values of AUC across the readers were significantly increased by 15.94%, with Rayvolve-assisted reading compared to unaided reading (0.88 ± 0.01 vs. 0.759 ± 0.07, p < 0.001). Time taken to read the chest X-rays decreased significantly, by 35.81%, with the use of Rayvolve (14.7 ± 1.3 vs. 22.9 ± 2.3, p < 0.001). Also, the average values of sensitivity and specificity across the readers increased significantly, by 11.44% and 2.95%, with Rayvolve-assisted reading compared to unaided reading (0.857 ± 0.02 vs. 0.769 ± 0.02, p < 0.001 and 0.974 ± 0.01 vs. 0.946 ± 0.01, p < 0.001, respectively) (see Figure 3).

The findings of this study can lead to the adoption and integration of the AI tool in a real-world clinical setting. This will help automate and streamline the workflow in the radiology department, thus reducing the time and workload for radiologists. In conclusion, an AI tool, when used as a concomitant read, can significantly improve the performances of radiologists or emergency practitioners in detecting consolidation, pleural effusion, pneumothorax, APE, cardiomegaly, and pulmonary nodules and can save the time taken to analyze chest radiographs.