Artificial intelligence in medicine: what it does today, limits and rules

Artificial intelligence (AI) has entered medicine faster than scientific evidence can keep up with it. Today it reads mammograms, drafts reports and answers patients' questions. In this article we look at what it really does, what the most recent studies say about its benefits and limits, how to use a health chatbot without taking risks, and which rules apply in the European Union, the United States and the United Kingdom.

What artificial intelligence in medicine is

"Artificial intelligence" refers to programs that learn from data instead of following instructions written line by line. In medicine, three families matter most:

• machine learning: models that, starting from thousands of cases, estimate a risk or a probability, for example of a complication;
• deep learning: neural networks that recognize patterns in images, such as a suspicious lesion on an X-ray or during an endoscopy;
• large language models (LLMs): the systems behind chatbots such as ChatGPT, which generate plausible text from enormous amounts of text.

The difference matters, because the three families make mistakes in different ways. A model that reads images may fail to recognize a rare case; a language model, on the other hand, may write false information in a completely convincing tone.

What AI does in medicine today

Diagnostic imaging

This is the field with the strongest evidence. In Sweden, a randomized study of more than 105,000 women compared mammography screening with AI (which sorts the exams and flags suspicious areas) with traditional double reading by two radiologists. With AI, 6.4 cancers were found per 1,000 women versus 5.0, 29% more, mainly small invasive cancers without lymph node involvement, without increasing false alarms; radiologists' readings fell by 44%. In Germany, an observational study of more than 463,000 women in 12 screening centers found a 17.6% increase in cancers detected, again without more recalls.

In the Italian edition of med4.care we have covered other examples: AI that helps recognize esophageal cancer lesions during endoscopy, AI used to diagnose COVID-19, and a smartphone system to detect visual impairment in children early.

Clinical decision support and predictions

Many models estimate a risk from the patient's data: in Italian, we have written about those that predict hearing recovery after tympanoplasty, diagnose sleep disorders, estimate the prognosis of colorectal cancer, predict the risk of a cruciate ligament tear and estimate biological age.

One result from recent years gives pause for thought. In an American randomized study, 50 physicians worked through clinical cases with or without the help of a chatbot: diagnostic reasoning was the same in the two groups (76% versus 74%), while the chatbot on its own scored 16 points higher. Having a powerful tool is not enough: you also need to know how to use it.

Clinical documentation

A growing share of AI's work is invisible to the patient: so-called "digital scribes" listen to the visit and prepare a draft of the report. In a randomized study of 238 outpatient physicians, one of the two programs tested reduced the time spent writing by 9.5%, the other did not; both improved scores for exhaustion and workload, but physicians reported "occasional" clinically relevant errors. The report must therefore always be reread by whoever signs it.

Chatbots for patients

Chatbots pass medical licensing exams, but this does not mean they really help the people who use them. In a randomized study published in 2026, 1,298 people had to work out, with the help of a chatbot or with the sources they preferred, what the problem was in ten clinical situations and what to do. On their own, the chatbots identified the condition in 94.9% of cases; the people using them did so less than 34.5% of the time, no better than the control group. The weak point is the conversation: whoever asks does not know what information to give, and does not always understand the answer. In the Italian edition we had already discussed earlier research on the quality of ChatGPT's answers to patients' questions.

The limits of AI in medicine: errors, "hallucinations" and bias

Hallucinations. A language model can invent a piece of information and present it as true. In a 2025 study, six models were given 300 clinical cases, each with one invented detail, such as a laboratory test that does not exist: depending on the model and the instructions, the chatbots elaborated on the false detail as if it were real in 50-82% of cases. A dedicated instruction reduced the errors without eliminating them.

New models, old problems. The same group repeated the tests in 2026 on GPT-5: decisions still changed depending on the sociodemographic characteristics of the patient described, with no improvement over the previous model (some LGBTQIA+ groups were referred for a psychiatric evaluation in 100% of cases), and the hallucination rate was higher (65% versus 53%).

Still little evidence. A 2024 review counted 86 randomized studies of AI in clinical practice: 81% reported a positive result, but most were conducted in a single center, with little information on patient characteristics and probable publication bias, that is, the tendency to publish mainly favorable results.

How to use a health chatbot safely

In my clinical practice I meet more and more patients who come to the visit having already read a chatbot's answers. I do not consider this a problem, provided a few rules are followed:

• use it to understand, not to decide: it is fine for having a term or a test explained, or for preparing the questions to ask your doctor; diagnosis and treatment remain with the doctor;
• do not change or stop a treatment on the basis of a chatbot's answer;
• with severe or sudden symptoms (chest pain, difficulty breathing, weakness on one side of the body, loss of consciousness) call the local emergency number (112 in the EU, 911 in the US, 999 in the UK) immediately, without consulting the chatbot;
• check the sources: if the chatbot cites a study or a figure, check that it really exists; hallucinations look credible;
• protect your data: avoid entering your name, tax or ID number, complete medical reports or images that make you recognizable, especially in free services for the public;
• bring the conversation to your doctor: an answer that seems reassuring is no substitute for a visit.

Who oversees these technologies: the European Union, the United States and the United Kingdom

There is no single worldwide rule for AI in medicine: each country regulates it in its own way. Here are three important examples, checked on official sources on 19 October 2026.

European Union. The AI Act (Regulation (EU) 2024/1689) entered into force on 1 August 2024 and applies in stages: prohibitions from 2 February 2025, obligations for general-purpose AI models, such as those behind chatbots, from 2 August 2025, and most other rules from 2 August 2026. These include the obligation to design chatbots so that users know they are talking to an AI. AI software that is itself a medical device, or a safety component of one, and must be assessed by an independent body is a "high-risk" AI system, with the strictest obligations. In July 2026 the "Digital Omnibus on AI" (Regulation (EU) 2026/1744) postponed these obligations: for AI in medical devices they will apply from 2 August 2028. Medical software is also covered by the EU Medical Device Regulation (2017/745), and health data by the GDPR (2016/679). For a closer look at high-risk systems, see the Italian-language guide ai-act.care. ai-act.care is a guide to the AI Act for professional practices and small businesses co-authored by the author of this article.

United States. The Food and Drug Administration (FDA) states that it "does not regulate AI as such; it regulates medical devices, including AI-enabled medical devices". As of September 2026 it had authorized more than 1,600 AI-enabled medical devices. Some clinical decision support software is excluded from the definition of a medical device by the 21st Century Cures Act (section 520(o) of the Federal Food, Drug, and Cosmetic Act); the FDA explains the criteria in a guidance updated in January 2026. A final FDA guidance published in December 2024 allows manufacturers to have a "predetermined change control plan" for updating AI software authorized in advance. In August 2026 the FDA opened a public consultation on medical devices based on generative AI. The health information held by health plans and by health care providers that handle it electronically is protected by HIPAA.

United Kingdom. The UK government explains that how AI in healthcare is regulated "depends on its intended purpose, functionality and use": some AI products are medical devices under the Medical Devices Regulations 2002, overseen by the Medicines and Healthcare products Regulatory Agency (MHRA), while others fall under data protection, professional or other rules. The MHRA runs the AI Airlock, the "first exploratory regulatory sandbox for AI enabled Medical Devices". In October 2026 the government accepted the recommendations of a national commission on the regulation of AI in healthcare. Health data are "special category" data under the UK GDPR, overseen by the Information Commissioner's Office (ICO).

What this means for the patient

AI is already part of care, often without the patient noticing: in reading a mammogram, in drafting the report of a visit, in estimating a risk. The best results come where AI works together with the doctor, who checks its work, and on well-defined tasks such as reading images. Where AI instead talks to the public on its own, studies show errors, hallucinations and answers that change depending on the circumstances. A chatbot can be a good starting point for understanding, but the clinical decision remains the doctor's responsibility.

References

1. Hernström V, Josefsson V, Sartor H, Schmidt D, Larsson AM, Hofvind S, Andersson I, et al. Screening performance and characteristics of breast cancer detected in the Mammography Screening with Artificial Intelligence trial (MASAI): a randomised, controlled, parallel-group, non-inferiority, single-blinded, screening accuracy study. The Lancet Digital Health. 2025;7(3):e175-e183. doi:10.1016/S2589-7500(24)00267-X. PubMed
2. Eisemann N, Bunk S, Mukama T, Baltus H, Elsner SA, Gomille T, Hecht G, et al. Nationwide real-world implementation of AI for cancer detection in population-based mammography screening. Nature Medicine. 2025;31(3):917-924. doi:10.1038/s41591-024-03408-6. PubMed
3. Goh E, Gallo R, Hom J, Strong E, Weng Y, Kerman H, Cool JA, et al. Large Language Model Influence on Diagnostic Reasoning: A Randomized Clinical Trial. JAMA Network Open. 2024;7(10):e2440969. doi:10.1001/jamanetworkopen.2024.40969. PubMed
4. Lukac PJ, Turner W, Vangala S, Chin AT, Khalili J, Shih YT, Sarkisian C, et al. Ambient AI Scribes in Clinical Practice: A Randomized Trial. NEJM AI. 2025;2(12). doi:10.1056/AIoa2501000. PubMed
5. Bean AM, Payne RE, Parsons G, Kirk HR, Ciro J, Mosquera-Gómez R, Hincapié MS, et al. Reliability of LLMs as medical assistants for the general public: a randomized preregistered study. Nature Medicine. 2026;32(2):609-615. doi:10.1038/s41591-025-04074-y. PubMed
6. Omar M, Sorin V, Collins JD, Reich D, Freeman R, Gavin N, Charney A, et al. Multi-model assurance analysis showing large language models are highly vulnerable to adversarial hallucination attacks during clinical decision support. Communications Medicine. 2025;5(1):330. doi:10.1038/s43856-025-01021-3. PubMed
7. Omar M, Agbareia R, Apakama DU, Horowitz CR, Freeman R, Charney AW, Nadkarni GN, et al. New model, old risks: sociodemographic bias and adversarial hallucinations vulnerability in GPT-5. npj Digital Medicine. 2026;9(1). doi:10.1038/s41746-026-02584-8. PubMed
8. Han R, Acosta JN, Shakeri Z, Ioannidis JPA, Topol EJ, Rajpurkar P. Randomised controlled trials evaluating artificial intelligence in clinical practice: a scoping review. The Lancet Digital Health. 2024;6(5):e367-e373. doi:10.1016/S2589-7500(24)00047-5. PubMed
9. Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (AI Act), Articles 6, 50 and 113, Annex I. Official Journal of the European Union, L series, 12.7.2024. EUR-Lex
10. Regulation (EU) 2026/1744 of the European Parliament and of the Council of 8 July 2026 (Digital Omnibus on AI), amending Regulation (EU) 2024/1689. Official Journal of the European Union, L series, 24.7.2026. EUR-Lex
11. U.S. Food and Drug Administration. Artificial Intelligence-Enabled Medical Devices. Content current as of 22/09/2026. Accessed 19 October 2026. fda.gov
12. U.S. Food and Drug Administration. Clinical Decision Support Software. Guidance for Industry and FDA Staff, January 2026. Accessed 19 October 2026. fda.gov
13. U.S. Food and Drug Administration. Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software Functions; final guidance. Federal Register, 4 December 2024 (document 2024-28361). Accessed 19 October 2026. Federal Register
14. U.S. Food and Drug Administration. FDA seeks public feedback to inform regulatory approach for generative AI-enabled medical devices. Press announcement, 18 August 2026. Accessed 19 October 2026. fda.gov
15. 45 CFR 160.102 (HIPAA, applicability). Electronic Code of Federal Regulations, up to date as of 15 October 2026. Accessed 19 October 2026. eCFR
16. The Medical Devices Regulations 2002 (SI 2002/618). Accessed 19 October 2026. legislation.gov.uk
17. UK Government. Government response to the National Commission's recommendations on the regulation of AI in healthcare, 6 October 2026. Accessed 19 October 2026. gov.uk
18. MHRA. AI Airlock: the regulatory sandbox for AIaMD. Last updated 6 October 2026. Accessed 19 October 2026. gov.uk
19. Information Commissioner's Office. What is special category data? Accessed 19 October 2026. ico.org.uk