This week the NHS Race and Health Observatory published Towards trustworthy, equitable AI in the NHS outlining that AI systems can reproduce and amplify the racial inequalities already present in healthcare. Bias creeps in through unrepresentative data, through variables that stand in for ethnicity and through data shaped by years of unequal access and treatment. Once a biased system is deployed, it can scale that bias across thousands of decisions.
The Observatory calls for anti-racism requirements in AI procurement, for performance to be reported by ethnic group and for patients and communities to be involved from the start. These are reasonable asks and we support them. We would also add one point from our own experience: the same tools that can entrench inequality can also expose it. Models that are carefully reviewed and validated across patient groups can support fairer care.
Finding the gaps
Most of what happens to a patient is recorded as free text. Ethnicity data in structured fields is often incomplete. Natural language processing lets researchers read the full record at scale and ask who is being missed.
Clinical teams at King’s College Hospital and King’s College London have used CogStack to do exactly this. In heart failure with preserved ejection fraction, they found that a widely used diagnostic score under-diagnosed Black patients because it leans heavily on atrial fibrillation, which is less common in that group (Brown et al., JACC: Advances, 2024). In aortic stenosis, Black patients were more likely to be symptomatic and to have other conditions at diagnosis, yet were less likely to receive a formal diagnosis or a valve procedure, even within a universal healthcare system (Biswas et al., European Heart Journal Digital Health, 2025).
The same approach has been applied to research access. Comparing trial participants with the real-world patient population at King’s showed that ethnic minority groups were under-represented in research, even after adjusting for who actually uses the hospital (Ellis et al., Journal of the Royal Society of Medicine, 2025). A further study proposed using NLP to monitor trial recruitment as it happens and alert clinicians when the cohort drifts away from the population it should reflect (Breeze et al., JACC: Advances, 2025).
None of these findings would have surfaced from coded data alone.
Building models that hold up across groups
Finding a gap is the first step. The second is making sure any model built to close it works for everyone. The AIM-HFpEF model, developed at King’s and validated at Guy’s and St Thomas’, was tested on its own performance in non-White patients and in patients from the most deprived areas. Accuracy held in both groups (Wu et al., European Heart Journal Digital Health, 2025). That check was part of the design, and it is the kind of evidence the Observatory is asking procurement teams to demand.
What this means in practice
The Observatory is right that good intentions are not enough. Equity has to be built in and then checked. From our side, that means three things:
- Report model performance by ethnic group and by deprivation, and publish it.
- Keep clinicians and patients in the loop when tools are designed, tested and monitored.
- Use the full record, including free text, so that gaps in care can be seen and measured.
AI in the NHS will reflect the data it is trained on and the care it is asked to support. With honest review and validation, it can also help make that care fairer. We welcome the Observatory’s report and look forward to supporting its recommendations with our NHS partners.
References
- NHS Race and Health Observatory, Towards trustworthy, equitable AI in the NHS, September 2026 (https://nhsrho.org/)
- Brown S, Biswas D, Wu J et al. Race- and ethnicity-related differences in heart failure with preserved ejection fraction using natural language processing. JACC: Advances, 2024. https://www.jacc.org/doi/10.1016/j.jacadv.2024.101064
- Biswas D, Wu J, Brown S et al. Racial and ethnic disparities in aortic stenosis within a universal healthcare system characterised by natural language processing for targeted intervention. European Heart Journal Digital Health, 2025. https://doi.org/10.1093/ehjdh/ztaf018
- Ellis HL, Smith J, Murtagh AM, Al-Agil M, Whyte MB. Using recruitment data instead of national population ethnicity proportions in clinical trial preparation may introduce bias. Journal of the Royal Society of Medicine, 2025. https://doi.org/10.1177/01410768241299529
- Breeze J, Wu J, Carmichael J et al. Artificial intelligence methods applied to electronic health record data for health equity in clinical trials. JACC: Advances, 2025. https://www.sciencedirect.com/science/article/pii/S2772963X25006283
- Wu J et al. Artificial intelligence methods to detect heart failure with preserved ejection fraction within electronic health records: an equitable disease detection model. European Heart Journal Digital Health, 2025. https://academic.oup.com/ehjdh/article/7/1/ztaf107/8256371
- Digital Health coverage: https://www.digitalhealth.net/2026/09/observatory-urges-race-equity-to-be-central-to-future-of-ai-in-nhs/
