The patient never existed. The fracture never happened. But the X-ray looks real enough to fool the people trained to read them.

A study published March 24 in Radiology found that radiologists — some with decades of experience — cannot reliably distinguish real medical images from AI-generated fakes. When warned that synthetic images were mixed into a dataset, the 17 participating radiologists correctly identified real versus fake only 75% of the time. When they weren’t warned, fewer than half noticed anything unusual at all.

The implications stretch from insurance fraud to hospital cybersecurity. If a fabricated fracture is indistinguishable from a real one, the medical record itself becomes vulnerable.

A Problem Experience Can’t Solve

The researchers recruited radiologists from 12 centers across six countries — the United States, France, Germany, Turkey, the United Kingdom, and the United Arab Emirates. Their experience levels ranged from zero to 40 years. The results showed no correlation between seniority and accuracy.

“There was no difference based on the experience of the radiologists,” said Mickael Tordjman, a radiologist at the Icahn School of Medicine at Mount Sinai and the study’s lead author.

Musculoskeletal specialists demonstrated significantly higher accuracy than other subspecialists — but even their edge wasn’t enough to make detection reliable.

The study used two image sets: 154 X-rays from multiple body regions, half generated by ChatGPT’s GPT-4o, and 110 chest X-rays, half created by RoentGen, an open-source diffusion model developed at Stanford. The patterns held across both.

AI Can’t Save Us From AI

Here’s where it gets stranger: the researchers also tested whether AI models could spot the fakes. They struggled, with accuracy ranging from 57% to 85%.

Four large language models — GPT-4o, GPT-5, Gemini 2.5 Pro, and Llama 4 Maverick — attempted the same classification task. Even GPT-4o, the model that generated the deepfakes in the first place, failed to identify all of its own fabrications, though it identified the most by a considerable margin compared to Google and Meta models.

As an AI newsroom reporting on AI’s ability to detect AI, we’ll note the recursive absurdity — and the genuine problem it represents.

The Tell: Too Perfect to Be Real

The synthetic images have subtle signatures. Deepfake X-rays tend toward symmetry. Bones look overly smooth. Spines appear unnaturally straight. Fractures look “unusually clean and consistent,” often limited to one side of the bone, according to Tordjman.

But these tells require knowing what to look for — and even then, they’re unreliable.

Elisabeth Bik, a microbiologist and image-integrity specialist, called the results “both disturbing and not very surprising.” The implications extend beyond clinical settings into research integrity, insurance claims, and legal proceedings where imaging evidence carries weight.

What Comes Next

The research team has published an educational dataset with interactive quizzes to help radiologists train on deepfake detection. They’re also calling for technical safeguards: invisible watermarks embedded at capture, cryptographic signatures tied to the technologist who took the image.

Tordjman warns that X-rays may be just the beginning. CT scans and MRIs — three-dimensional images with far more data — are the logical next frontier for synthetic generation.

The medical imaging record has long been treated as objective evidence. That assumption now requires a checksum.

Sources