Back to 2026 Abstracts
Torsion in the Age of AI: Not All Chatbots Are Created Equal
Noah Longton, MD1, Carol Davis-Dao, PhD2, Janelle Fox, MD, MS3, Jasilyn A. Wray-Jordan, MPH3, Malea Drummond, DNP3, Louis Wojcik, MD3, Heidi A. Stephany, MD2, Paul Kokorowski, MD4, Sarah H. Williamson, MD3.
1Macon & Joan Brock Virginia Health Sciences Eastern Virginia Medical School at Old Dominion University, Norfolk, VA, USA, 2Rady Children's Health, Orange County and University of California, Irvine, Orange, CA, USA, 3Children's Hospital of the King's Daughter Division of Urology, Norfolk, VA, USA, 4Cedars-Sinai Health Sciences University, Los Angeles, CA, USA.
Torsion in the Age of AI: Not All Chatbots Are Created EqualIntroductionTesticular torsion is a urologic emergency where time to surgical management is critical to testicular salvage. Delayed presentation to emergency care, rather than in-hospital delays, remains the main driver of orchiectomy. Traditional public health outreach strategies have had limited reach among adolescents for time-sensitive conditions like testicular torsion. AI chatbots have seen high usage among this age group, and their private, accessible nature may make them a more attractive platform for teenagers seeking information about sensitive health concerns. We sought to investigate the accuracy of AI chatbot responses to queries from teenagers with acute testicular pain.
MethodsChatbot platforms were selected based on Pew Research data identifying the most commonly used AI tools among adolescents. Patient-level prompts representing low-risk (TWIST 0-2), intermediate-risk (TWIST 3-4), and high-risk (TWIST 5-7) torsion scenarios (3 per category) were submitted to ChatGPT, Copilot, Gemini, and Meta AI three times each on separate days using Google Incognito mode with a new session per entry, yielding 108 total responses. Five board-certified urologists scored all responses independently under blinded conditions for urgency appropriateness, torsion mention, torsion prioritization, time-sensitivity, dangerous content, NPO counseling, and quality (1-5). Urgency appropriateness was defined as recommending immediate emergency department evaluation or calling 911. Consensus responses served as the primary outcome. Fisher’s exact test and Wilcoxon rank-sum test were used for comparisons.
ResultsAmong 108 total responses scored, three of four platforms demonstrated strong and consistent performance. ChatGPT, Copilot, and Gemini provided appropriate urgency recommendations in 89-100% of high-risk scenarios and mentioned torsion in 96-100% of responses, with torsion prioritized in 74-81% of cases (Table 1). NPO counseling was inconsistently provided across all platforms, present in only 36% of high-risk and 22% of intermediate-risk scenarios. Meta AI was a consistent outlier across all domains. Torsion was mentioned in only 33% of Meta AI responses compared to 96-100% for other platforms (p<0.0001), and median quality scores were significantly lower (3 vs. 4-5; p=0.0001). Dangerous responses by consensus occurred exclusively in low-risk scenarios for Meta AI (n=2) and Copilot (n=1).
ConclusionsWhile most platforms performed well, Meta AI was a consistent and concerning outlier, providing inappropriate urgency guidance across all risk categories and generating dangerous responses in low-risk scenarios. Chatbot performance is highly platform-dependent, and not all widely used tools provide safe or accurate information for time-sensitive surgical emergencies. Overall, however, AI chatbots show promise as a tool for reaching adolescents with critical health information that has historically been difficult to deliver through traditional outreach.
Table 1. Urgency Appropriateness by Chatbot and Risk Group| Chatbot | High-Risk (TWIST5-7) | Intermediate-Risk(TWIST 3-4) | Low-Risk(TWIST 0-2) |
| ChatGPT | 8/9 (89%) | 8/9 (89%) | 9/9 (100%) |
| Copilot | 9/9 (100%) | 9/9 (100%) | 8/9 (89%) |
| Gemini | 9/9 (100%) | 9/9 (100%) | 9/9 (100%) |
| Meta AI | 5/9 (56%) | 4/9 (44%) | 1/9 (11%) |
| p value | 0.04 | 0.007 | <0.0001 |
Back to 2026 Abstracts
