Results show Ubie's differential diagnosis platform had a Top-5 hit accuracy of 63.4% and a Top-10 hit accuracy of 71.6%; median physician accuracy was 72.9%
Ubie is effective in providing relevant information based on symptoms,making it valuable in speeding diagnosis and care delivery for improved outcomes
The AI-assisted evaluation of prior peer-reviewed clinical vignettes represents a significant advancement over traditional static vignette studies
NEW YORK,Sept. 10,2024 --Ubie,a global healthcare AI platform that enhances the intersection of patients,providers,and life sciences,today announced data from Ubie Symptom Checker: A Clinical Vignette Simulation Study,which was published on medRxiv. The study evaluated the performance of Ubie's platform using an innovative large language model-assisted (LLM) simulation method. Results show that Ubie achieved a Top-5 hit (inclusion of the correct diagnosis) accuracy of 63.4% and a Top-10 hit accuracy of 71.6%,indicating effectiveness in providing relevant health information based on symptoms. Ubie outperformed comparator symptom checkers and compared favorably to the median physician hit accuracy (72.9%) using the same set of vignettes.
Ubie's Symptom Checker is a free,online tool trained on medical data,reviewed by medical experts and constantly updated based on real-world human feedback loops. The AI guided questionnaire collects symptom information and delivers a personalized list of the most likely related illness matching user symptoms,informing users on when to see a doctor,identifying symptom causes,providing treatment information,and guiding users on how to access appropriate medical care.
AI symptom checkers can deliver better outcomes by raising awareness of illnesses and diseases earlier in a patient's care journey. Today's patients are more engaged,researching their symptoms online,sometimes before they even reach out to a healthcare provider. Unfortunately,patients can lack the health literacy needed to interpret information,often leading to confusion,anxiety and delayed decision-making at the outset of their care journey.
By providing an effective,validated solution that is accessible anywhere to anyone,Ubie leverages artificial intelligence to fill the gaps in medical knowledge,helping patients find more relevant information,make better decisions,seek the right care for their needs sooner,and ultimately,improve long-term survival and quality of life. For patients with serious diseases,shortening the timeline to diagnosis can make a difference in available treatments,the potential to slow or stop disease progression,and length of life.
"Results of the study confirm that Ubie's Symptom Checker is highly effective at providing users with relevant disease education and information related to their symptoms,which can guide patients to the care and resources they need," said Kenji Taylor,MD,MSc,lead study author and Clinical Consultant,Supervising Physician at Ubie. "As a practicing physician,I know how important it is to quickly and properly triage patients. Having a tool with such a high accuracy will help gain the trust of healthcare providers and make sure patients get the care they need,faster. Further,it helps me understand patient concerns,and improves communication to get to the heart of the matter quickly."
Ubie can empower patients to take control of their care and become an active participant in their care visits. Being passive can cause patients to not remember issues that happen outside of their office visit,feel embarrassed bringing up concerns,or defer to their healthcare provider. Furthermore,providers are shouldering an increased workload as workforce shortages worsen with less time to spend on individual patients. Ubie provides results in a digestible form that can be easily shared with a provider,helping to get to the cause of an illness or disease sooner.
"The high accuracy of Ubie's Symptom Checker shows that it can be a valuable tool in connecting users with the right doctor,the right treatment and disease awareness information to help better guide their journey,resulting in better health outcomes," said Kota Kubo,Ubie Co-Founder and Co-CEO. "We work with clinical and pharmaceutical partners across the globe to ensure we're sharing the most up-to-date and relevant information for users."
Outcomes of the study show that Ubie provides relevant disease suggestions within its top-ranked results,which can be valuable for initial health assessments. Performance across various diseases suggests that Ubie is well-equipped to handle a diverse range of symptoms. It performed particularly well in areas such as the nervous system and respiratory conditions,though variability in accuracy was observed across different ICD groupings,highlighting areas for further refinement.
"This study marks a significant advancement in how we evaluate and improve our Symptom Checker accuracy,especially for rare diseases," said Takashi Nishibayashi,Lead Machine Learning Engineer at Ubie. "Our new AI-powered approach provides a dynamic and scalable alternative to traditional methods,which are often limited by small sample sizes and static data. Using AI simulations,we can fine-tune variables – like a patient's medical history – and directly observe their impact on diagnosis. It's like moving from a needle in a haystack to a targeted search."
The study leveraged 400 publicly available,peer-reviewed clinical vignettes,each representing a unique patient case. By mapping nearly 8,000 data points with the aid of Large Language Models (LLMs) and medical entity linking,the team created a robust and diverse dataset for the AI simulations. This allowed for extensive testing and analysis,ultimately utilizing 328 vignettes that aligned with the Ubie Symptom Checker's scope of diseases.
While the study highlights the system's strengths,it also identifies areas for improvement suggesting continued refinement and real-world testing are essential to fully realize Ubie's potential in AI-assisted healthcare.
More About Ubie's Symptom Checker
Ubie's real time AI has been trained on over 50,000 peer-reviewed publications and is connected to layers of human input,including a panel of more than 50 specialists that review the prediction pathway for each disease to ensure accuracy,and more than 1,500 healthcare providers organizations,which creates a feedback loop that ensures our AI learns and adapts based on real world outcomes. Most recently,Ubie has begun working with patient groups to further fine tune specific disease parameters,especially within rare diseases where diagnosis can be a greater challenge.
Symptom Checker has disease and symptom coverage with over 1,100 medical conditions and 3,500 question data types in its Clinical Database.
More About Clinical Vignettes
Clinical vignettes are widely used in the training and evaluation of healthcare providers (physicians,nurse practitioners,physician assistants) focused on diagnosis and treatment. Clinicians are familiar and comfortable with this methodology as a performance benchmarking tool. Vignette studies can be comprehensive and varied in scope while maintaining control over the testing environment to focus purely on Symptom Checker's performance as a user tool.
About Ubie
Founded in 2017,Ubie empowers individuals and healthcare professionals with the tools they need for better care. Leveraging cutting-edge disease prediction AI,Ubie guides 10+ million patients every month to seek appropriate medical attention through its free online Symptom Checker,and equips 1,500+ provider organizations with clinical tools that streamline workflows and support better diagnoses and health outcomes. Trained on medical data,Ubie's marketing solutions power advanced targeting capabilities and high-performing digital campaigns for 70% of the world's top life science companies.
Learn more about our vision and work at https://ubiehealth.com/company or try our free Symptom Checker at https://ubiehealth.com/.