New Study: AI-Generated X-Rays Can Deceive Radiologists, with Recognition Accuracy as Low as 41%

Image Source: The Rise of Deepfake Medical Imaging: Radiologists’ Diagnostic Accuracy in Detecting ChatGPT-generated Radiographs

DentalGoodNews | A study published on March 24, 2026, in the journal Radiology, the official publication of the Radiological Society of North America (RSNA), reveals that both radiologists and multimodal large language models (LLMs) struggle to distinguish between real X-rays and AI-generated "deepfake" images. When not informed about the presence of fake images, radiologists achieved an accuracy rate of only 41% in identifying AI-generated X-rays.

The study was conducted by a research team from the Icahn School of Medicine at Mount Sinai in New York. It involved 17 radiologists from 12 institutions across six countries, ranging from beginners to seasoned experts with up to 40 years of practice. A total of 264 X-ray images were tested, including images of various body parts generated by ChatGPT, as well as chest images generated using RoentGen, an open-source generative AI diffusion model developed by Stanford University.

Experimental data shows that when doctors were informed about the existence of synthetic images, their average identification accuracy improved to 75%. Regarding AI model performance, the accuracy rates for GPT-4o, GPT-5, Gemini 2.5 Pro, and Llama 4 Maverick ranged between 57% and 85%. Notably, although ChatGPT-4o outperformed other models, as the image generation tool itself, it still could not fully identify all the fake images it generated.

The researchers point out that these highly realistic fake images not only pose a risk of being used as evidence in fraudulent litigation but also present serious information security risks. If hackers infiltrate hospital networks and inject synthetic images, it could lead to misdiagnosis and compromise the reliability of digital medical records. Currently, fake images exhibit visually "too perfect" characteristics, such as overly smooth bones, abnormally straight spines, and excessively uniform vascular distribution. However, with future AI advancements, these telltale signs may further disappear.

Previously, DGN reported on an academic misconduct incident involving altered X-rays in a practical course at Yonsei University College of Dentistry in South Korea: some students used Photoshop to modify or replace images to cover up procedural errors. At that time, the forgery relied on manual effort, leaving traces that could be investigated. This study published in Radiology further suggests that with the involvement of generative AI, the realism of fake images may continue to increase, thereby expanding the potential risks of fraud and misinformation.

Institutional responses are also lagging. According to a previous report by DGN, New York State has proposed legislation to prohibit AI chatbots from directly providing medical advice, bringing AI risks into the regulatory purview. However, systematic protection mechanisms for the authenticity of medical images remain largely absent. Therefore, the study recommends that the medical imaging field embed invisible watermarks and cryptographic signatures at the point of image capture to verify source authenticity.

The researchers also note that the current study has certain limitations: the sample size is relatively small, and there are differences between the experimental environment and real clinical scenarios. The impact of deepfake technology on 3D imaging such as CT and MRI still requires further validation.