NHS England rushes to hide software over AI hacking fears

NHS England is hastily removing all software it has written from public view due to the perceived risk of hacking from cutting-edge artificial intelligence. Security experts say the move is unnecessary and counterproductive.

Software produced by the National Health Service was previously made open source and listed on GitHub because it is created with public money. This allows other organizations to learn from it and offer better services at lower costs without duplicating efforts.

But NHS England has published new guidance for staff, which has been shared with New scientistwhich requires existing and future software to be removed from public view and kept behind closed doors. “All source code repositories should be private by default. Repositories should not be public unless there is an explicit and exceptional need and public access has been formally approved,” the new guidance states. The deadline to make the code private is May 11.

Last month, an AI created by Anthropic called Mythos was widely reported to be able to discover flaws in virtually any software, potentially allowing hackers to break into the systems running it.

NHS England guidance specifically names Mythos as the cause of the new measures. “Public repositories significantly increase the risk of unintentional disclosure of source code, architectural decisions, configuration details, and contextual information that can be exploited – especially given rapid advances in Al models capable of large-scale code ingestion, inference, and reasoning (e.g., developments such as the Mythos model),” it reads. “This red line establishes a closed default position for the code while the organization assesses the impact of these changes and ensures that any public release of the code is a deliberate, considered and justified decision.”

However, the UK government-backed AI Security Institute (AISI) investigated Mythos and found that it was capable of attacking only “small, vulnerable, weakly defended enterprise systems”, concluding that there was no indication that any truly secure software or network would be at risk.

The new measures go against the NHS service standard, which requires staff to make all software they produce open source. “Public services are built with public money. So unless there is a good reason not to, the code they are based on is [on] should be made available to other people so that they can reuse and draw inspiration from them. Open source code can save teams [from] duplicate efforts and help them build better services faster,” the previous guide states.

Open source software for utilities also creates increased trust and transparency. For example, if the code of the Horizon computer system, which led the British Post Office to prosecute innocent people for alleged theft and fraud, had been public, the scandal might not have continued for years.

Terence Eden, who has extensive experience in the UK civil service in opening up access to public data, believes the move makes no logical sense.

“Is it possible that Mythos will scan a repository and find a bug? Yes, it’s 100% likely. Will it be a bug that causes a security issue in a live NHS service somewhere? Almost certainly not,” says Eden. “I think it’s someone in NHS England who is buying into the hype that Mythos is going to bring about the end of security as we know it and is freaking out a bit.”

Eden says open source software is actually more secure because many people can check its flaws, and most NHS software is not critically security-related anyway. Importantly, given that the code has been publicly available for years, it will continue to exist in various saves and downloads anyway.

“Closing it now is largely locking the stable door after the horse leaves,” says Eden. “Myself and people I have spoken to within the NHS are completely confused as to what this is trying to achieve.”

An NHS England spokesperson said: “We are temporarily restricting access to some NHS England source code to further strengthen cybersecurity while we assess the impact of rapid developments in AI models. We will continue to release source code where there is a clear need.”