Research and Publications
Objective and peer-reviewed research can raise the industry bar for quality, safety, and performance. Here's some research that looks at the performance and efficacy of Ada's technology.
Scroll through or navigate to a topic of interest below.
Safety and accuracy
- Breadth of coverage and accuracy of Chinese language digital symptom assessment apps for suggesting conditions and urgency advice: A clinical vignettes evaluation
Chinese Journal of General Practitioners, 2023
- China’s 8 most popular symptom checkers showed a range of coverage, safety, and accuracy, factors relevant to user and physician choice.
- Ada offered the highest coverage (99.5%) and advice safety (99.5%) compared with the average (69.6% and 90.3%, respectively).
- Ada’s accuracy was significantly higher, and more than twice the average (61.5% vs 28.7%).
- Comparison of physician and artificial intelligence-based symptom checker diagnostic accuracy
Rheumatology International, 2022
- Ada was significantly more accurate in the recognition of inflammatory rheumatic diseases (IRD) than physicians (70% vs physicians' 54%), which included experienced rheumatology physicians.
- Ada was more accurate than physicians in suggesting the correct final diagnosis (54% of cases vs physicians' 32%).
- Highlights the potential of using symptom checkers early in the patient journey and the advantages of considering complete patient information to establish a correct diagnosis.
- Shows the potential of symptom checkers to make accurate triage and diagnostic decisions in rheumatology.
- Online symptom checkers lack diagnostic accuracy for skin rashes
Journal of the American Academy of Dermatology, 2022
- Ada outperformed all other symptom checkers in suggesting the correct diagnosis for skin rashes.
- Ada's top condition suggestion matched the correct diagnosis in 73.3% of cases, vs an average of 30.8%.
- The correct diagnosis was provided within Ada's top three condition suggestions in 93.3% of instances vs an average of 43.3%.
- Safety of Triage Self-Assessment Using a Symptom Assessment App for Walk-in Patients in the Emergency Care Setting: Observational Prospective Cross-sectional Study
JMIR mHealth and uHealth, 2022
- Real-world hospital ED study of 378 patients, comparing the safety of Ada’s urgency advice to that of the Manchester Triage System (MTS).
- Ada demonstrated a high rate of safety (94.7%) over all medical specialties of the Emergency Department, with a focus on Internal Medicine, Orthopedics and Trauma, and Neurology.
- From the 3 lowest MTS categories, 43.4% could have safely accessed lower urgency care such as seeing a GP or managing their symptoms at home.
- Ada has the potential to relieve pressure on Emergency Departments by guiding patients to lower urgency care if used at home.
- Diagnostic Performance of an App-Based Symptom Checker in Mental Disorders: Comparative Study in Psychotherapy Outpatients
JMIR Mental Health, 2022
- One of Ada's top 5 condition suggestions matched therapist's formal diagnoses in 69% of cases, and Ada's top suggestion matched in 51% of cases.
- Symptom checkers have potential to complement the mental health diagnostic process, but further investigation is recommended.
- The usability of Ada was rated as high in the System Usability Scale.
- The Diagnostic Efficacy of an App-based Diagnostic Health Care Application in the Emergency Room: eRadaR-Trial. A Prospective, Double-Blinded, Observational Study
Annals of Surgery, 2022
- When used in combination with ER physicians assessments, Ada significantly increased diagnostic accuracy (87.3%) compared with an ER physician alone (80.9%).
- Patients with early diagnosis and rapid treatment allocation exhibited significantly reduced complications and length of hospital stay.
- AI tools have the potential to benefit the diagnostic efficacy of clinicians and improve quality of care.
- Evaluation of Diagnostic and Triage Accuracy and Usability of a Symptom Checker in an Emergency Department: Observational Study
JMIR Mhealth Uhealth, 2022
- One of Ada’s top 5 condition suggestions matched the correct diagnosis in 70% of cases, close to physicians' top 3 average of 68.9%.
- 90% of patients found Ada easy to use, and 83% considered Ada to be an efficient way to document their medical problems quickly.
- Ada’s correct ‘diagnoses’ were nearly identical to one of the board-certified emergency physicians'.
- Prospective Feasibility Study of a Novel Diagnostic Decision Support System (DDSS) for Medical Professionals
JMIR Formative Research, 2022
- Ada’s DDSS prototype could support doctors’ decision-making by showing them new pathways and potential diagnoses.
- Ada provided accurate decision support in the clinical inpatient setting for people presenting with dyspnea.
- Actionable absolute risk prediction of atherosclerotic cardiovascular disease: a behavior-management approach based on data from 464,547 UK Biobank participants
PLOS ONE, 2021
- Using a dataset from 464,547 patients, Ada's CVD risk prediction tool achieved superior performance to several established general CVD risk prediction models.
- Ada could be used to direct individuals to app-based behavior change recommendations for risk reduction and guidance.
- The quality of condition suggestions and urgency advice provided by the Ada symptom assessment app assessed with independently generated vignettes optimized for Australia
Australian Journal of Primary Health, 2021
- Ada identified the correct condition in the top 3 suggestions in 83% of cases, compared to the next best of 77%, including Australia-specific scenarios.
- 63% of Ada’s urgency advice matched the gold standard, compared to the next highest app’s score of 52%
- Accuracy of online symptom checkers and the potential impact on service utilisation
PLOS ONE, 2021
- Ada had 73% accuracy compared to a 38% all-app average.
- Ada's exhibited 97% disposition safety compared to an 83% all-app average.
- How accurate are digital symptom assessment apps for suggesting conditions and urgency advice?: A clinical vignettes comparison to GPs
BMJ Open, 2020
- The utility of symptom checkers relies on safety, coverage, and accuracy.
- Ada’s advice was safe in 97% of cases, on par with GPs and with 99% condition coverage for the clinical scenarios.
- Ada showed 70% accuracy for top 3 suggestion fit vs competitor average of 38%.
- Use Characteristics and Triage Acuity of a Digital Symptom Checker in a Large Integrated Health System: Population-Based Descriptive Study
Journal of Medical Internet Research, 2020
- 46.4% of 26,646 Ada assessments were completed outside primary care clinic hours.
- Ada’s recommendations were comparable to triage nurses’.
- Accuracy of a chatbot (Ada) in the diagnosis of mental disorders: Comparative case study with lay and expert users
JMIR Formative Research, 2019
- Ada could help identify mental health disorders in adults.
- From symptom to diagnosis-symptom checkers re-evaluated : Are symptom checkers finally sufficient and accurate to use? An update from the ENT perspective
- Ada ranked highly on precision out of the 24 tested in the ear, nose, and throat (ENT) discipline.
- What happened when Pulse tested symptom checker apps
Pulse Today, 2019
- Ada was the most accurate of the 4 apps tested.
- Ada was found to be fast and easy to use.
- Can you really trust the medical apps on your phone?
Wired UK, 2017
- Ada was "by far the best" app out of the 4 tested, asking clear questions and providing the best condition suggestions.
- AI-based Symptom Assessment as the Digital Front Door to Care and to Enhanced Physician/Patient Interaction: Real-World Evidence from Portugal and the Netherlands
Healthcare Information and Management Systems Society (HIMSS), 2023
- 93% of Ada’s assessments provided useful information prior to consultation with 83% of Ada’s top condition suggestions matching the physician’s primary diagnosis.
- In 52% of appointments, Ada saved more than 1 minute of primary care consultation time.
- Most physicians agreed that Ada is safe and effective, guiding patients to the correct care, providing useful information, and saving time in consultations.
- Improving emergency department patient-doctor conversation through an artificial intelligence symptom-taking tool: an action-oriented design pilot study
JMIR Formative Research, 2022
- Ada supported conversation and improved rapport for 90% of patients, 73% of physicians, and 100% of nurses.
- Patients reported high usability and understood the tool’s questions.
- Optimization of Patient Flow in Urgent Care Centers Using a Digital Tool for Recording Patient Symptoms and History: Simulation Study
JMIR Formative Research, 2021
- Ada could decrease the average triage nurse waiting times by 54%.
- Patients’ utilization and perception of an artificial intelligence-based symptom assessment and advice technology in a British primary care waiting room: Exploratory pilot study
JMIR Human Factors, 2020
- 97.8% of patients rated Ada very or quite easy to use.
- 12.8% or patients would have delayed their appointment or used lower-intensity care.
Global health and universal coverage
- Addressing the Challenges of Adapting an AI-Powered Symptom Assessment and Care Navigation Tool for South African Users
Pending publication, The Lancet, 2022
- The readability of Ada's consumer-facing medical text was reduced from grade 11, to grade 7 without compromising on medical quality.
- AI companies can reduce the burden on vulnerable healthcare systems, but should be designed and adapted to the needs of the local population.
- Efforts should be made to reduce bias in AI by using white-box systems.
- Investigating the Potential for Clinical Decision Support in Sub-Saharan Africa With AFYA (Artificial Intelligence-Based Assessment of Health Symptoms in Tanzania): Protocol for a Prospective, Observational Pilot Study
JMIR Research Protocols, 2022
- Protocol for a clinical study set in a Tanzanian district hospital outpatient clinic to investigate the usefulness of a prototype diagnosis decision support system (DDSS) in the hands of a mid-level healthcare worker.
- The diagnostic accuracy and qualitative data on the usability, usefulness, and acceptance of the prototype DDSS will provide insights on the appropriateness of the prototype DDSS interface.
- Study protocol for a pilot prospective, observational study investigating the condition suggestion and urgency advice accuracy of a symptom assessment app in Sub-Saharan Africa: the AFYA-'Health' study
BMJ Open, 2022
- Protocol for first-of-type clinical study set in a Tanzanian district hospital outpatient clinic to investigate the usefulness of symptom assessment applications in LMICs.
- Evaluation of Ada's compared to gold-standard differential diagnoses and triage advice for participants with various conditions and age groups, including children.
- Addressing the challenges of adapting an AI-powered symptom assessment and care navigation tool for South African (SA) users
- Ada’s knowledge base was adapted to account for differences in incidence and disease presentation in South Africa, including optimizing 25 maternal and 25 pediatric conditions.
- Ada’s readability was lowered from grade 11 level to below grade 8 level (7.4) while still maintaining medical accuracy to improve accessibility and uptake.
- White-box AI like Ada, based on interpretable models that can be explained and easily understood by a human, can control and reduce data bias.
- Challenges and solutions in adapting an AI-powered health management tool for South Africa
European Journal of Public Health, 2021
- AI solutions can only improve health outcomes in LMICs if they are designed and adapted with the needs of the local population in mind.
- To account for regional differences in incidence and disease presentation, 51 maternal and pediatric health conditions were optimized in Ada for South Africa.
- Ada’s content readability score has been reduced from grade 11 to below grade 8 (7.4±0.8) while still maintaining medical accuracy.
- White-box AI-approach like Ada’s makes it possible to capture the differences in underrepresented populations and low-income countriea.
- Leveraging medical-AI to speed up Cold Agglutinin Disease detection
Gesellschaft für Thrombose- und Hämostaseforschung (GTH), 2023
- The rare Cold Agglutinin Disease model was created and incorporated into Ada’s medical knowledge and reasoning engine.
- In the first 30 days, CAD was suggested among the top 3 conditions in 48 assessments.
- Integrating CAD in a symptom assessment can help speed up disease detection, potentially providing quicker access to healthcare systems.
- Medical expert knowledge meets AI: How expert knowledge can improve symptom assessment apps - a new approach in rare diseases
European Conference on Rare Disease (ECRD), 2022
- A new approach of conducting expert interviews to create 11 clinical vignettes on lysosomal storage diseases (LSDs) which were then used to update Ada’s condition models.
- 15 LSD patients and 9 LSD experts will rate both new and old disease models in Ada to compare efficacy.
- This novel approach can potentially shorten the ‘time to diagnosis’ for rare diseases.
- What is confirmed in the diagnostics of autoinflammatory fever diseases? (German language)
The Internist (Der Internist), 2021
- Ada could help detect genetic auto-inflammatory disorders, including the important group of Periodic Fever Syndromes.
- Health economic benefits through the use of diagnostic support systems and expert knowledge
BMC Health Services Research, 2021
- Retrospective analysis of the diagnostic costs of rare inflammatory systemic diseases and the impact of a prototype Ada as a diagnostic decision support system (DDSS).
- If this Ada DDSS prototype were released, it has the potential to lead to earlier diagnosis and could save between 50% and 70% of diagnostic costs.
- Zebras or horses? How common are rare diseases on a medical symptom checker, Ada?
RE(ACT) Congress - International Rare Diseases Research Consortium, Berlin, 2020
- Rare diseases were suggested as the most likely condition in 4% of Ada symptom assessments and among the top five suggestions in 17% of assessments.
- This is in line with epidemiological studies suggesting that ~2–4% of the population is living with a rare disease.
- Ada shows the potential to shorten the diagnostic odyssey of people living with rare diseases.
- Can a decision support system accelerate rare disease diagnosis? Evaluating the potential impact of Ada DX in a retrospective study
Orphanet Journal of Rare Diseases, 2019
- 89.25% of Ada’s top suggestions matched the confirmed diagnosis.
- 33.3% of participants could have received their rare disease diagnosis in the first documented clinical visit with Ada.
Public health and surveillance
- How can symptom assessment apps contribute to early cancer detection?
AI in Cancer Diagnostics: from research to clinical practice, European Association for Cancer Research (EACR), 2021
- Ada has the potential to help surface users who should seek care, including possible early cases and users who do not participate in screening programs.
- Correlations between Ada’s cancer frequency in Germany and the German cancer registry incidence data were statistically significant.
- Between 23% and 94% of Ada assessments where cancer was suggested reported no advanced signs, and between 19% and 41% reported first signs and no advanced signs.
- Novel Methods in the Surveillance of Influenza-Like Illness in Germany Using Data From a Symptom Assessment App (Ada): Observational Case Study
JMIR Public Health Surveillance, 2021
- Ada uncovered similar ILI trends to an official German surveillance system and could help identify health trends in countries without population-based monitoring systems.
- The Potential of Digital Symptom-based Screening to Reduce the Transmission of SARS-CoV-2: A Modelling Study
Pending publication, JMIR, 2021
- Digital symptom-based screening tools can substantially reduce the transmission of COVID-19.
- Applications like Ada (combined with testing and self-isolation) could prevent an additional 8% of the population being infected.
- Syndromic surveillance insights from a symptom assessment app before and during COVID-19 measures in Germany and the United Kingdom: Results from repeated cross-sectional analyses
JMIR mHealth and uHealth, 2020
- A review of data from 950,000 users found that Ada could help uncover the impact of public health policies like COVID-19 lockdown measures.
Commentary and industry
- Letter to the Editor on 'Comment on “Accuracy and usability of a diagnostic decision support system in the diagnosis of three representative rheumatic diseases: a randomized controlled trial among medical students'BMC Arthritis Research & Therapy, 2022
- Letter to the Editor on 'The quality of diagnosis and triage advice provided by free online symptom checkers and apps in Australia'Medical Journal of Australia (MJA), 2021
- Algorithm change protocols in the regulation of adaptive machine learning-based medical devices
JMIR Formative Research, 2021
- Proposals by the EU and FDA have set the regulatory groundwork for algorithm change protocols. The FDA proposals are detailed and actionable but are yet to be finalized. The EU has not yet detailed its plan. The authors recommended the EU set out an approach that is coordinated, actionable, and consultative.
- The public, manufacturers, and medical community should be engaged to inform requirements of algorithm change, with a focus on post market surveillance, RWP measurement, clinical evaluation, and labeling.
- Letter to the editor on 'Do Dr. Google and Health Apps Have (Comparable) Side Effects? An Experimental Study'Clinical Psychological Science, 2021
- Letter to the editor on 'Periodic Manual Algorithm Updates and Generalizability: A Developer’s Response. Comment on “Evaluation of Four Artificial Intelligence–Assisted Self-Diagnosis Apps on Three Diagnoses: Two-Year Follow-Up Study”'Journal of Medical Internet Research, 2021
- Rare diseases 2030: How augmented AI will support diagnosis and treatment of rare diseases in the future
Annals of the Rheumatic Diseases, 2020
- Ada could help detect genetic autoinflammatory disorders, including familial Mediterranean fever.
- Introduction of a pathophysiology-based diagnostic decision support system (DDSS) and its potential impact on the use of AI in healthcare
MedInfo, World Congress of Medical and Health Informatics, 2019
- Ada Health successfully demonstrates the feasibility of a pathophysiology-based DDSS with Medical Deep Reasoning (MDR).
- MDR extends current DDSS capabilities by utilizing pathophysiology to describe the dynamic components of disease pathogenesis to create an individualized, high-resolution model for personal healthcare.
- Facilitating the integration of this kind of health-related data will significantly advance personalized medicine.