Learnings from generating synthetic health data

Two main uses

• Privacy: if synthetic data no longer counts as personal information, it can be shared more easily for research, software testing, teaching and collaboration.
• Data augmentation: adding realistic records to small datasets, adding diversity, and helping with under-represented groups.

All of his examples were tabular health data.

No single best generator

No type of generative model wins consistently on both usefulness and privacy. In practice, try three or four (for example a diffusion model, a GAN and sequential trees), evaluate them on your own data, and pick the best.

Synthetic doesn't automatically mean private

An overfit generator can copy the real data. Regulators in several countries say the same thing: measure the risk. His group helped build a consensus framework that recommends two metrics, membership disclosure and attribute disclosure. What's still missing is agreed thresholds, because the regulatory question is yes or no.

Use about ten datasets, not one

Generating data is random, so each synthetic dataset gives slightly different answers. Treat it like multiple imputation: generate about ten, analyse each, and combine the results. If you share synthetic data publicly, share several versions.

Does it give the same answers?

The key test is replicability: run the same analysis on real and synthetic data and compare the conclusions, not just how similar the data looks. His group's studies found close agreement in several settings. Other findings:

• Sending a synthetic copy of one site's data to another gave results very close to a full federated analysis, in a fraction of the time.
• Augmentation helped most on small datasets, and could make things worse on large ones.
• It helped trials that couldn't recruit enough patients, and groups with low-to-medium under-representation. Nothing fixes extreme cases where there's too little data to learn from.
• Privacy and usefulness held up as the number of variables grew, so you can synthesise a whole database once and cut cohorts from it, instead of repeating the work for every request.