The advancement of medical research and healthcare is increasingly dependent on the analysis of patient-level data, but privacy concerns and legal constraints often hinder data sharing. Synthetic data mimicking real patient data offers a widely discussed potential solution. According to the literature, synthetic data may, however, not fully guarantee patient privacy and can vary greatly in terms of fidelity and utility. In this study, we aim to systematically investigate the trade-off between privacy, fidelity and utility of synthetic patient data. We assess synthetic data fidelity in terms of statistical similarity to real data, and utility via the performance of machine learning models trained on synthetic and tested on real data. Regarding data privacy we focus on membership inference via shadow model attacks as well as singling out and attribute inference risks. In this regard, we also consider differential privacy (DP) as a possible mechanism to probabilistically guarantee a certain level of data privacy, and we compare against classical anonymization techniques. We evaluate the fidelity, utility and privacy of synthetic data generated by five different models for three distinctive patient-level datasets. Our results show that our implementations of DP have a strongly detrimental effect on the fidelity of synthetic data, specifically its correlation structure, and therefore emphasize the need to improve techniques that effectively balance privacy, fidelity and utility in synthetic patient data generation.

The authors have declared no competing interest.

This work was done as part of the NFDI4Health Consortium (www.nfdi4health.de). We gratefully acknowledge the financial support of the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - project number 442326535.

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

The used datasets are either directly openly available or available upon request at the respective data holding organizations. Access to the Alzheimer's Disease Neuroimaging Initiative (ADNI) can be requested using the IDA portal: Alzheimer's Disease Neuroimaging Initiative (ADNI) The Center for Cancer Registry Data (ZfKD) can make the validated dataset available to third parties upon request, in accordance with paragraph 5 (3) of the Federal Cancer Registry Data Act (BKRG), provided that the applicant demonstrates a legitimate interest, particularly for scientific purposes: https://www.da-ra.de/dara/study/web_show?res_id=626806&mdlang=en&detail=true The Texas Hospital Inpatient Discharge Data Public Use Data File (TEXAS) can be downloaded for public use on their homepage: https://www.dshs.texas.gov/texas-health-care-information-collection/health-data-researcher-information/texas-inpatient-public-use

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Yes

This study did not generate new unique reagents. The data supporting the findings of this study are available from the respective data holding organizations (c.f. section 2.1). Researchers interested in accessing the data should direct their requests to these organizations. The functions for fidelity evaluations utilized in this study have been made publicly available as a Python package. The source code is accessible on GitHub at https://github.com/SCAI-BIO/syndat. The code for privacy evaluations as well as for each synthetization model are available publicly in the respective cited repositories.

https://github.com/SCAI-BIO/syndat

https://zenodo.org/records/13929256