Objective: To prospectively validate the performance of the Risk of Malignancy Index (RMI), International Ovarian Tumor Analysis (IOTA) Simple Rules Risk Model (SRRisk), IOTA Assessment of Different NEoplasias in the adneXa (ADNEX) and the IOTA two-step strategy in different types of ultrasound centers in Italy. Methods: This is a multicenter prospective observational study including regional referral centers and district hospitals in Italy. Consecutive patients with adnexal mass examined with ultrasound by an IOTA certified ultrasound examiner with different levels of experience were included, provided they underwent surgery within 180 days after the inclusion scan. Ultrasound examination was performed transvaginally or transrectally and/or transabdominally based on the characteristics of the women and masses. Reference standard was the histology of the adnexal mass following surgical removal. Discrimination (area under the receiver operating characteristic curve, AUROC) calibration, and clinical utility were assessed to illustrate the diagnostic performance of the methods. The performance of the models was also evaluated in predefined subgroups based on menopausal status, type of center (oncology vs non-oncology) and ultrasound examiners experience (<500 scans performed, 500-500 scans performed, >5000 scans performed; European Federation of Societies for Ultrasound in Medicine and Biology (EFSUMB) Level 1, Level 2, Level 3). Results: 1567 patients were recruited between May 2017 and March 2020 from 23 italian centers. After data cleaning and application of exclusion criteria, our study population consisted of 1431 patients in 21 italian centers (10 oncological and 11 non-oncological). Based on histology, 995/1431 (69.5%) tumors were benign and 436/1431 (30.5%) were malignant (115/1431, 8.0% borderline, 263/1431, 18.4% primary invasive, 58/1431, 4.1% metastatic tumors). For all IOTA models (SRRisk, ADNEX with and without CA125, two step strategy with and without CA125), the AUROC was between 0.91 (95% CI 0.88-0.93) and 0.92 (0.89-0.94). The AUROC was 0.85 (0.81-0.87) for RMI. The malignancy risk was slightly underestimated by all IOTA models, but at least so by SRRisk. All IOTA models had higher net benefit than RMI at risk thresholds from 1% to 50%. AUROC was ≥0.90 for all IOTA models in all subgroups, while it ranged from 0.84 to 0.90 for RMI. Conclusions: SRRisk, ADNEX and the two step strategy with or without CA125 had similar and good ability to distinguish benign from malignant adnexal tumors in patients examined by either expert or non-expert ultrasound operators in Italy. Their discriminative performance and clinical utility was superior to that of RMI.

The authors have declared no competing interest.

This study did not receive any funding

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

Ethics committee of the Fondazione Policlinico A. Gemelli gave ethical approval for this work, IRCCS (PROT 27665/16)

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Yes

All data produced in the present study are available upon reasonable request to the authors