A new study supported by the NIHR BRC at UCLH has demonstrated the potential of an AI tool to predict the health trajectory of patients by forecasting future disorders, symptoms, medications and procedures.
According to the team of researchers led by King’s College London, these results indicate the tool could be used to support clinical decision-making, monitoring in healthcare settings and to improve clinical trials.
The trial involves UCL, King’s College Hospital NHS Foundation Trust and Guy’s and St Thomas’ NHS Foundation Trust.
The tool – called Foresight – is trained on existing healthcare data and uses a deep learning approach to recognise complex patterns in both the structured and unstructured data of electronic health records to produce insights and predictions.
It belongs to the same family of AI models as ChatGPT but, in contrast to ChatGPT which is trained on publicly available information, Foresight is trained on information from NHS electronic health records.
The study, published in The Lancet Digital Health, investigated the accuracy of Foresight’s medical predictions by comparing them to what actually happened to the patients as described in their records.
Using data from over 811,000 patients across King’s College Hospital NHS Foundation Trust, South London and Maudsley NHS Foundation Trust, and MIMIC-III – a publicly available dataset of patients from Beth Israel Deaconess Medical Center in the US, researchers trained three different models of Foresight.
Researchers extracted and processed the unstructured (free-text) and structured data (age, ethnicity and sex) within electronic health records (EHRs) using CogStack. The datasets were used to train Foresight and performance was measured by comparing its predictions on true outcomes in a smaller subset of these data. Foresight was trained on data under NHS and patient governance and inside the hospital NHS firewall.
Precision forecasting
When forecasting the next 10 possible disorders that could appear next in a patient timeline, the tests showed that Foresight correctly identified the next disorder 68% and 76% of the time in two UK NHS Trusts (King’s College Hospital NHS Foundation Trust and South London and Maudsley NHS Foundation Trust) and 88% of the time in the US MIMIC-III dataset. Similarly, when forecasting the next new biomedical concept which could be a disorder, symptom, relapse or medication, the precision achieved by Foresight was 80%, 81% and 91% respectively.
The accuracy of Foresight’s predictions was also assessed by clinicians. Five clinicians developed 34 mock patient timelines with simulated scenarios. When all five clinicians agreed on the forecasted medical event, the predictions Foresight provided were 93% relevant, meaning they made sense from a clinical perspective.
Uses of Foresight
Foresight can be used for real-world risk forecasting, emulating trials and clinical research to study the progression of disorders, simulate interventions and lifestyle factors and educational purposes.
Senior author Professor Richard Dobson, who is supported by the BRC at UCLH, said: “Foresight opens the door for many applications such as digital health twins, synthetic dataset generation, real world risk forecasting, longitudinal research, emulation of trials, medical education and more. It is an exciting time for AI in healthcare and to develop effective tools we must ensure that we use appropriate data to train our models and work towards a shared purpose of supporting healthcare systems to support patients.
Professor Dobson, who is Professor of Biomedical and Health Informatics at UCL, added: “Foresight will improve with more real-world data, and we are now looking for more hospitals to be involved in developing Foresight 2, a more accurate language model.”
Foresight is part of CogStack, an information retrieval and extraction platform developed by researchers at UCL and UCLH, King’s College London, King’s College Hospital NHS Foundation Trust, Guy’s and St Thomas’ NHS Foundation Trust, and South London and Maudsley NHS Foundation Trust. It uses natural language processing to harness NHS electronic health record data to support clinical decision-making and health research.
This paper was part of a programme of work that received funding from the NHS AI Lab, National Institute for Health and Care Research and Health Data Research UK.
‘Foresight - Generative Pretrained Transformer (GPT) for Modelling of Patient Timelines using EHRs: A Retrospective Study’ was published in The Lancet Digital Health.
