Background The majority of AI/ML-enabled software as a medical device (SaMD) has been cleared through the FDA 510(k) pathway, but with limited transparency on algorithm development details. Because algorithm quality depends on the quality of the training data and algorithmic input, this study aimed to assess the availability of algorithm development details in the 510(k) summaries of AI/ML-enabled SaMD. Then, clinical and/or technical equivalence between predicate generations was assessed by mapping the predicate lineages of all cleared computer-assisted detection (CAD) devices, to ensure equivalence in diagnostic function.

Methods The FDA’s public database was searched for CAD devices cleared through the 510(k) pathway. Details on algorithmic input, including annotation instructions and definition of ground truth, were extracted from summary statements, product webpages, and relevant publications. These findings were cross-referenced with the American College of Radiology–Data Science Institute AI Central database. Predicate lineages were also manually mapped through product numbers included within the 510(k) summaries.

Results In total, 98 CAD devices had been cleared at the time of this study, with the majority being computer-assisted triage (CADt) devices (67/98). Notably, none of the cleared CAD devices provided image annotation instructions in their summaries, and only one provided access to its training data. Similarly, more than half of the devices did not disclose how the ground truth was defined. Only 13 CAD devices were reported in peer-reviewed publications, and only two were evaluated in prospective studies. Significant deviations in clinical function were seen between cleared devices and their claimed predicate.

Conclusion The lack of imaging annotation instructions and signicant mismatches in clinical function between predicate generations raise concerns about whether substantial equivalence in the 510(k) pathway truly equates to equivalent diagnostic function. Avenues for greater transparency are needed to enable independent evaluations of safety and performance and promote trust in AI/ML-enabled devices.

The authors have declared no competing interest.

This study did not receive any funding

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Yes

All data produced in the present work are contained in the manuscript