AI in healthcare: what does good evidence and regulation look like?

How are AI tools being evaluated?

So far, the UK National Institute for Health and Care Excellence (NICE) has evaluated and published reports on five AI technologies. The sixth evaluation, on the first autonomous AI tool, is due to be published later this year.

“Most of the AI tech that we have evaluated has been in the diagnostic space and are imaging based technologies,” said NICE’s HealthTech programme director Anastasia Chalkidou. “It’s still a med tech fundamentally.”

Chalkidou told NICE’s annual conference on 27 March that most AI tools are currently being evaluated through its early value assessment (EVA) process.1 To be considered for this pathway, technologies with evidence uncertainties must aim to meet an unmet need. If they are conditionally recommended for “early use in the NHS,” an evidence generation plan must then be followed to produce the “evidence that needs to be gathered while it’s in use.” Once this evidence is collected, NICE will make a full recommendation.

Priority areas for EVA consideration include mental health, cardiovascular, early cancer detection, and medical technologies that boost healthcare capacity.

One AI technology type that has been given an early recommendation through this system is AI assisted contouring for radiotherapy treatment planning. Such tools aim to save consultants’ time by marking up the organs at risk of radiation damage, as well as the site of the cancer, before radiotherapy.2

“Marking up all the images takes time and a certain level of accuracy, and it’s quite repetitive. But it needs to be done really well for the treatment to be successful. What AI autocontour promises to do is to press …