Introducing AI4H – a WHO project to help unlock the full potential of AI in health

At Ada, we believe that health AI will only achieve its full potential once it has gained the trust of clinicians, patients, and policymakers. We also believe that building – and earning – this trust is a challenge that can only be met collectively and collaboratively – no one AI company can do it alone; it will require the whole industry and all stakeholders to come together.

That is why we are taking part in an ambitious new global health initiative, AI for Health (AI4H).

Spearheaded by the World Health Organization (WHO) in partnership with the International Telecommunications Union (ITU), AI4H’s aim is to bring all stakeholders together to establish a clear method for standardized benchmarking of AI-powered solutions in healthcare so that anyone, anywhere, can have access to quality healthcare.

Benchmarking AI systems might seem like an unusual thing for the WHO to prioritize – but it is crucial for enabling a greater understanding of how these technologies can be deployed in the real world, while ensuring that companies are held to the highest medical and ethical standards. If we can establish this process for standardization on a global scale, we will be able to leverage emerging technologies as a lasting solution to global health issues, such as the shortage of global health workers – which is expected to reach 12.9 million by 2035.

Interestingly, as we’ve previously explored, effectively benchmarking AI in healthcare is often far from simple – particularly for symptom assessments, which is the area of health AI that Ada is focused on.

Why is it so challenging? Firstly, because healthcare and medicine are by their very nature incredibly complex. There are always multiple factors that must be taken into account within a clinical assessment or decision-making process, and finding a clear ‘ground truth’ is a real challenge. In addition, the information and guidance provided should be evaluated as a whole, and regional and demographic variation must be factored in.

The AI4H Focus Group is structured into 16 topic groups, with ours specifically focused on symptom assessment benchmarking. The topic group is made up of nine participating companies, with Ada taking the role as topic driver. This means we are responsible for pushing the conversation forward and striving for transparency, positive collaboration, and tangible results. In addition to the topic groups, there are also five horizontal working groups that independently advise the topic groups on overarching aspects like data and AI handling, health requirements, or regulatory considerations.

Our goal is to establish a pathway for globally consistent standardization across the sector. What does this mean in practice? It means that organizations such as the WHO, regulators, individual governments, and healthcare organizations will have an independent and reliable source of information about which AI technologies are appropriate for deployment, and in which contexts.

By paving the way for the safe and transparent adoption of these AI systems, we hope to dramatically improve access to healthcare for many people across the globe. This benchmarking initiative also marks an important step towards greater transparency and open dialogue around ethical assessment and impact, which we at Ada believe to be of critical importance in our industry.

We are currently one year into a two-year process that has the potential to provide the long-term foundations for regulators to build on in the future. The project has come a long way in a short space of time, and we’ve been hugely inspired and encouraged by the commitment and creativity that every participating organization has already shown. We spoke about some of our work at the World Health Summit in Berlin, and we were pleased to see this result in a number of positive discussions across the industry.

Our current research is focused on establishing a basic proof of concept for benchmarking on a technical level, and identifying the key obstacles for wider benchmarking protocols. Once this is done, we’ll be working together to find a non-biased, independent way of tackling those obstacles. Key considerations will include: ensuring objectivity and accurate global representation, discussing how test cases can be developed in a way that is independent and fair, and balancing the need to protect a company’s intellectual property with the broader need for transparency, to name just a few.

We are really excited to be driving the symptom assessment topic in the focus group, and over the coming weeks and months we will share more updates on the work the group is doing. We’ll soon be heading to New Delhi, India, for our next meeting in November, so you can expect to see some more developments soon. If you want to find out more, or if you have any questions for us, please do get in touch.