4.1 How accurate is Ada's assessment?

We’re constantly benchmarking Ada’s reasoning through internal stress testing using real-world clinical case scenarios, competitive comparisons, medical collaborations, external independent review, and user feedback. We advocate for robust and transparent peer-reviewed research to both showcase Ada’s superiority and advance the industry as a whole.

Ada conducted a study in collaboration with medical experts from Brown University and with advice from other academic medical experts, including from the UCL Institute of Health Informatics. We compared condition coverage, accuracy, and safety of 8 popular symptom assessment apps (including Ada) with each other and with 7 GPs. For accuracy, no app outperformed GPs but Ada came closest. For top-3 suggested condition accuracy, doctors scored a mean 82.1% and Ada, 70.5%. This was well above the app average of 38%. Ada scored 27.5% higher than the next best performing and 47% better than the worst-performing.

There was also a large variation in condition coverage between the symptom checkers. In this research, Ada had almost complete coverage at 99%, compared to the lowest-performing at 51.5%. Many of the conditions not covered by the competitors were for high service users, such as pregnant women, children, and those with mental health issues.

See: https://ada.com/enterprise/ and https://ada.com/studies/