Patient and Public Involvement and Engagement(PPIE) for CogStack and the HDR UK National TextAnalytics project
Background
Patients have guided the development of the CogStack platform and related projects since 2017
when the research team were preparing Cogstack. King’s College Hospital hosted focus groups
with patients to discuss patient views on the use of de-identified data, the potential benefits of
using hospital records for research, concerns and information priorities, the meaning of consent,
and participating in research. These focus groups found support for using electronic health
records for research, but there were concerns around data handling, security, and using data for
commercial gains without direct NHS benefit. This work has formed the foundation of a strong
PPIE focus in the development of CogStack and related projects over recent years.
Engagement with patient-led committees/governance
King’s Electronic Records Research Initiative (KERRI) committee
South London and the Maudsley: The KERRI committee consists of 3 leading institute clinicians, 1 clinical director for data science and 3-4 patient representatives and meets every 2 months to review projects. It met throughout the pandemic to accelerate Covid research projects. The KERRI committee, which has a patient co-chair, advises on the suitability of research projects for Natural Language Processing (NLP). The committee reviews CogStack grant applications (for example a recent research proposal to use CogStack NLP for clinical coding in February 2020, with positive feedback, where ongoing pilot work has been relayed back and disseminated externally at their request).
Guy ’s and St Thomas’ Electronic Records Research Interface (CRAG) committee
Similar in function and structure to KERRI
South London and the Maudsley (SLaM) Clinical Record Interactive Search (CRIS) service user
groups
The research team regularly seeks input and advice from the SLaM CRIS service user groups and expert patients from the CogStack oversight group who review draft grant applications (e.g. AI Award, Round 2, Phase 3 draft application).
University Hospitals Birmingham NHS Trust PIONEER Data Trust Committee
The committee consists of a group of patient and public members who review all data access requests, provides public oversight of data access decisions and reviews draft grant applications. Feedback on a
recent AI Award was that members agreed there was significant need for this project, but members felt that patients and the public needed training to equip them with the skills needed to review NLP projects which we will address through our text analytics community.
Cross-centre PPIE groups
These groups have determined the structure of ongoing PPIE involvement. This includes the need for professional and clinical coordinating PPIE leads to facilitate cross-organisation PPIE events and to share learning. There is also a commitment to co- build a NLP specific suite of patient-facing tools, for use nationally.
Planned PPIE 2023 (AI Award, Round 2, Phase 3)
Patients and end-users will be involved in the project via a project-specific inter-NHS Trust Patient, Clinician and Staff Involvement Group (PCSIG). This group will be supplemented by Clinical Coding staff (two senior clinical leads at King’s College Hospital (KCH) and Guy’s and St Thomas’ NHS Trust (GSTT). The purpose of the Involvement Group will be to:
- Consolidate input and feed back to existing patient research committees of each Trust partner (CRIS, KERRI, GERRI and UHB/PIONEER) facilitated by the PPIE Lead
- Support the design of communications and broader engagement on the use of data (patient and public members)
- Help to shape the approach to insight-gathering and critical reflection (staff members) Specific activities that the PCSIG will lead on the following engagement and involvement activities include:
- Obtaining insights through surveys together with engagement with existing PPI organisations like
Understanding Patient Data to inform the production of communication materials - Co-creation of a patient and public-facing NLP toolkit, so that different patient groups can advise on local CogStack projects, as this new technology is embedded within new NHS Trusts across the country. These can be used by patient groups and Health Research Authority Research Ethics Committees, to evaluate the patient benefit and potential risk of any proposed project. The NLP toolkit would be freely available for all.
- The overall Programme Management Board will have a representative of the PCSIG to ensure PPI is
captured within the project governance.
Workshops and Focus Groups
Donated databank of clinical free text to develop and train NLP tools for research
March 2022
The research team carried out a series of stakeholder in-depth focus groups interviews, including patients and members of the public, to test early thinking around the creation of a donated databank and inform potential next steps for adopting a partner-led approach to establish a national, funded databank of free text for use by the research community.
Workshop on attitudes to the creation and use of artificial health records for research
November 2019
including synthetic text, held in conjunction with the NIHR Maudsley BRC. The 20 workshop attendees included service users, data governance leaders, researchers and clinicians
Small public consultation to inform and support the ethical approval and governance process for the use of CogStack to use local electronic health records for research
January 2019
Findings were published in the Journal of Medical Ethics
Co-production of patient and public-facing materials
CogStack information leaflet for patients
In autumn 2022 the research team led by Professor Liz Sapey at University Hospitals Birmingham NHS Foundation Trust co-designed with patients an information leaflet “What is CogStack and how can it help improve hospital care?” The leaflet was extensively tested among different patient and public communities across the UK before the final version was published. The detailed 14-page guide provides information for patients and the public on what health data are and how data are used, what is clinical coding and AI, what is CogStack, and what are the potential benefits of CogStack, in clear and simple language with user friendly infographics and examples.
“Your medical records can save lives” poster and information leaflet for patients
Patient facing materials were produced by King’s Electronic Records Research Initiative (KERRI) to
highlight use of patient data at King’s College Hospital for research.
Education, training and other engagement
HealTAC Panel Session “How does PPIE add value in text analytics research?” (June 2022)
In this special panel discussion, four different stakeholders (a public involvement expert, a patient by
experience expert, a NLP researcher and a representative from an NHS funding body) discussed how
patient and public involvement adds value to research using health care text analytics and natural
language processing. Members of the audience were then invited to discuss the issues presented
and ask questions of the expert panel. https://healtac2022.github.io/programmes/
HealTAC Patient & Public Involvement Advice Clinic “Patient and Public Involvement and Engagement (PPIE): Hands on Guidance for Clinical Text Analytics” (June 2022)
PPIE plays a vital role in helping to improve research and is now a crucial part of funding
applications. As an early career researcher, one might find it difficult to take the first step of PPIE
and to navigate through options and practices. In this tutorial, two PPIE experts, Jenny Robertson
and Natalie Fitzpatrick, and experienced public contributor, Colin Wilkinson, guided researchers
through a full journey of PPIE in relation to research projects on clinical text analytics. Hands-on
guidance on how to develop and cost PPIE sections in a grant application and what to do next when
a project is funded was also provided. https://healtac2022.github.io/programmes/
NLP STEM day (Jan 2020)
Investigators at University of Dundee led by Associate Professor Honghan Wu presented their research on NLP to S4 schoolchildren (two classes of 13 year old students) during a STEM day
Invited talks on PPIE
Nov 2022 NHS England Accelerated Access Collaborative – Artificial Intelligence (AI) Patient
Involvement and Health Equality Event, “CogStack PPIE” (Natalie Fitzpatrick)
Published papers/articles/blogs
Published paper as response to article with patient consultation at KCH around use of hospital records
“CogStack – experiences of deploying integrated information retrieval and extraction services in
a large National Health Service Foundation Trust hospital”
Paper authored by CogStack team and expert patients on the KERRI committee on sensitive area
of healthcare (end of life) “Natural language word embeddings as a glimpse into healthcare
language and associated mortality surrounding end of life”
Our work and NLP tools were featured in The Conversation article
HDR Opinion pieces
Our work supported a Data Saves Lives: Mental Health campaign in 2022. Involvement of the team is
captured in two opinion pieces published by HDR UK:
o The rise of tech in health data research: A new and evolving way of treating mental illness in
the UK (October 2022) . This featured article received 1,839 impressions on Twitter, 229 website views and was published in Today Posttimes
o Why I want researchers to study my mental health data (October 2022)
The campaign resulted in:
- Over 3,000 people visited the HDR UK mental health website hub over the course of the campaign, where we hosted blogs from our community, tools for mental health professionals, and patient-facing infographics and educational materials.
- The HDR UK community guest blogs had over 1,800 views, helping us showcase and celebrate the achievements of our health data research community and increase awareness of their expertise.
- The HDR UK Innovation Gateway had 4 new sign ups over the 3-week campaign – a 70% increase in what we would normally expect over this period – and 224 new users of our mental health collection of datasets and resources.
- Our patient-facing infographic had 30 downloads over the course of the campaign and over 3,000 impressions on social media. It will remain on our website as an ever-green tool to help the public understand how data can be used to improve mental health.
- Podcasts with our campaign partners MQ Mental Health and NHS England featuring our mental health experts were downloaded 357 times, helping us engage all-new audiences of patients and healthcare professionals in mental health data research.
- We reached 180 members of the public via public engagement events. Of 72 attendees
- who were surveyed, 81% had not heard of HDR UK before but 93% would be interested in getting involved in our work in the future.
- We improved awareness of mental health data research by 77% among an audience of 120 members of the public surveyed before and after the campaign.
- Of these people, we also positively increased perceptions around wanting to see more research using data to improve mental health (3% increase to 96%) and supporting the use of linking de-identified mental health data to non-health for research (4% increase to 86%).
- Over the whole campaign, across social media, events and partner podcasts, we reached over 300,000 people.
- Maximizing text analytics capability for health data research: key learnings from the HDR UK National Text Analytics project symposium. HDR UK Opinion Piece (Natalie Fitzpatrick and Rene Ndoyi) (November 2022)
- Paper published “Automated clinical coding: what, why, and where we are?” Hang Dong et al.
NPJ Digital Medicine (November 2022) –
Awards
HDR UK Reproducibility Award 2022
The Text Analytics Project team were commended with a Reproducibility award in recognition of the team’s commitment to reproducibility for going beyond a well-maintained GitHub repository and helping others to use their work.
HDR UK Team of the Year Award 2021
The HDR UK Text Analytics National Implementation Project Team were highly commended for our work on developing the text analytics community and tools for NLP
HDR UK Impact of the Year Award 2020
CogStack was selected by the HDR UK Impact Committee as a significant Applied Analytics impact which clearly demonstrates the value of HDR UK in our mission to unite the UK’s health data to make discoveries that improve people’s lives. This committee meets monthly to select research outputs put forward by Research Directors, Researchers, and other members of the HDR UK community – outputs are selected on the basis of research excellence, impact, and more.
HDR UK Impact case studies
- June 2022 – Patient-centric characterization of multimorbidity trajectories in patients with severe
mental illnesses: A temporal bipartite network modeling approach - December 2021 – The National Text Analytics Project
- March 2021 – CogStack information retrieval and extraction platform
- April 2020 – Analysis of text written by doctors in medical notes of patients with COVID-19
(National Text Analytics project – ACE inhibitors)
Videos / Podcasts
The team have produced various training and educational and other materials which are suitable for
a broad audience. These include:
CogStack – An introduction
the health record and assist in clinical decision making and individual direct patient care.
VIEWER: A population health management platform for mental health
Use of free-text clinical data in healthcare and research
How to use text in phenotypes
HDR UK Futures Bitesize video (Assoc Prof Honghan Wu, UCL Institute of Health Informatics)
Surveys planned for 2023
Donated Databank of Clinical Free Text
Building on our previous work which involved a series of focus group interviews with stakeholders to find out their views about a donated databank, we will use the findings of the focus groups to design and carry out a larger survey of stakeholders (patients and members of the public; clinicians; information governance leads and research ethics committee members; and NLP researchers) to explore people’s attitudes in more detail and seek their views on key areas including acceptability and concerns around
establishing, managing, accessing and using the databank. Study findings will provide valuable
insights into the acceptability of a free text databank for research and inform potential next
steps for establishing a national, funded databank for use by the research community (lead
Natalie Fitzpatrick, UCL)
Survey of knowledge of coders
Among NHS staff and patients (lead Prof Liz Sapey, University of Birmingham)