The Accuracy of ChatGPT in Answering Patients’ Frequently Asked Questions Regarding Jones Fractures

Articles in Press

Document Type : RESEARCH PAPER

Authors

1 University of Connecticut School of Medicine 263 Farmington Avenue, Farmington, CT 06030

2 University of Connecticut School of Medicine, 263 Farmington Avenue, Farmington, CT 06030

3 University of Connecticut, Department of Orthopedics, 120 Dowling Way, Farmington, CT 06032

10.22038/abjs.2026.90240.4095

Abstract

Objectives: This study evaluates the accuracy and readability of ChatGPT in foot and ankle surgery, particularly for fractures at the metadiaphysis of the 5th metatarsal (Jones fractures). Given that ChatGPT has over 180.5 million monthly users since May 2024, evaluating its efficacy as a patient education tool is essential. Methods: Nine frequently asked questions regarding Jones fractures were posed to ChatGPT 3.5 and 4o on June 28, 2024. Two senior authors scored the responses as “excellent response not requiring clarification” (1), “satisfactory requiring minimal clarification” (2), “satisfactory requiring moderate clarification” (3), or “unsatisfactory requiring substantial clarification” (4). Flesch Reading Ease score and Flesch-Kincaid Grade Level were used to assess length and readability. Results: The mean rating for ChatGPT 3.5 responses was 2.89 ± 0.78 compared to 1.78 ± 0.83 for ChatGPT 4o (P=0.0133), with lower scores corresponding to better quality responses. Flesch Reading Ease averaged 43.7 for ChatGPT 3.5 and 49.06 for 4o (P=0.0918), both correlating with a college reading level. Flesch-Kincaid Grade Level was an average of 11.19 and 9.27 for ChatGPT 3.5 and 4o, respectively (P=0.0214). Conclusion: ChatGPT 4o demonstrated better accuracy than 3.5, with improvements in Flesch Reading Ease and Flesch-Kincaid Grade Level, though both need further enhancement in readability.

Keywords

Main Subjects

The Archives of Bone and Joint Surgery

Articles in Press, Accepted Manuscript
Available Online from 16 May 2026

Files

History

  • Receive Date: 09 August 2025
  • Revise Date: 20 October 2025
  • Accept Date: 18 January 2026

Share

How to cite

Statistics