It has been apparent that clinicians have been engaging with generative AI tools to create and/or edit clinical notes and records, as well as students using it as part of their submitted coursework. The clinical digital leaders across the Digital Health Advisory Panels have considered this and would like to provide the following position statement on this.
Generative AI tools are excellent administrative support tools however currently have limited proven use in the creation or editing of clinical records.
There are multiple risks associated with using these tools for such purposes including:
- Potential to change the context and meaning of the content
- Misinterpretation of the assessment/clinical findings
- Copy/paste and transcription errors
- Information governance risks associated with the data sent across such platforms
Due to the severity of the harm that could ensue if these risks were realised, the lack of evidence-based guidance, and the uncertainty surrounding the liability of using such tools, it is advised that great caution is used if adopting generative AI tools in the clinical setting. The panel would also strongly recommend that there is oversight of the use of these tools through local Clinical and Information Governance processes, so that all organisations have a full understanding of their use and the associated potential risks, and that each proposed tool is assessed on its own merits.
The advisory panels will continue to monitor the situation and the development, and research into the use of such tools and update our position if the evidence base suggests such platforms can be used safely.
The NHS Information Standard DCB0160 has been in existence in one form or another since 2012 and is legislated under section 250 of the Health and Social Care Act (2012). DCB0160 provides a set of requirements to promote and ensure the effective application of clinical risk management by health organisations that are responsible for the deployment, use, maintenance or decommissioning of Health IT Systems within the health and care environment. It is evident that, historically, DCB0160 has at times received limited consideration for software within its scope, although in recent years the standard is being applied more rigorously for new products. The result is that a lot of NHS Trusts will have what is referred to as “legacy debt” of DCB0160 risk assurance, having produced no hazard logs and safety case reports for historical software that has been in their Trust for a number of years.
The issue of tackling “legacy debt” is compounded by the fact that digital growth has been exponential in the last few years and Clinical Safety Officers (CSOs) sometimes finite resources have been used to concentrate on these new systems. The result is that the legacy debt never gets tackled and there will always be an amount of clinical software within NHS Trusts that has had no risk assurance.
NHS England does not give any specific advice within the DCB0160 standard or implementation guidance on legacy debt but
The following statements within DCB0160 can be used to offer some advice:
- The extent of clinical risk management needs only to be commensurate with the scale, complexity and level of clinical risk associated with the deployment” -DCB0160
The first tasks that a Trust is able to do today is assess the size of their legacy debt and perform an exercise to identify each system and prioritise these in order of urgency, based on the clinical risk of that product.
To that end, stable, long term clinical software that has been in a Trust for a number of years without any previous issue or clinical risk assurance, the scale of clinical risk is likely lower. Therefore, the Trust could make an informed decision to produce only a focussed hazard log for this product, focussing on key aspects of the software that had the potential to cause harm to a patient.
The second important line in the DCB0160 standard is shown below:
- Top management must make available sufficient resources for clinical risk management – DCB0160
If a Trust identifies they have a number of legacy products with no DCB0160 risk assurance then this needs highlighted to “top management”; likely in the form of an entry on a Risk Register. This should then necessitate a plan for top management to ensure CSOs are well-resourced in their role and able to provide support to new products and also legacy debt. If the role of a CSO cannot be resourced adequately then this risk sits with “top management” to rectify, not with the CSO performing the role of risk assurance.
