This study is to evaluate the performance of an FDA-cleared computer-aided detection (CADe) device for thyroid nodules and its effects on readers’ performance.
Methods and Materials
A total of 130 images of thyroid nodules were randomly selected from a case pool of 372 patients with 472 thyroid nodules. The ground truth for the presence of each sonographic feature was determined by a panel of three specialists. Detection of feature presence was evaluated by AUROC’s and t-test p-values. For assessment of the effects on readers’ performance, diagnostic decisions were compared to the pathology results (74 benign and 56 malignant). Seven physicians were recruited to read the 130 ultrasound images. The reader performance was then evaluated by the multiple reader multiple case (MRMC) study design and Dorfman-Berbaum-Metz method.
The study first showed that the computed values of the anechoic area, echogenic foci, hypoechogenicity, heterogeneity, and ill-defined margin were all tested significant in detecting their presence with p-values <0.0001 for the first three features, 0.033 and 0.0014 for the last two and the AUC=0.882, 0.788, 0.817, 0.71 and 0.686, respectively. The MRMC study showed that the average AUC (=0.796) with CADe was significantly larger (p-value=0.0077) than the average AUC (=0.724) without CADe.
The tested CADe device was shown significant in detecting presence of the five sonographic characteristics. The diagnostic interpretations by physicians reading the ultrasound images with the CADe device were also shown significantly improved when compared to interpretations without the device.