Interobserver Variability of Radiographic Assessment using a Mobile Messaging Application as a Teleconsultation Tool

Document Type : RESEARCH PAPER


1 Hand and Upper Extremity Service, Department of Orthopedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, USA, Yawkey Center, USA

2 Department of Surgery and Perioperative Care, Dell Medical School, The University of Texas at Austin, Austin, Texas, USA


Background: To examine whether interobserver reliability, decision-making, and confidence in decision-making in the
treatment of distal radius fractures changes if radiographs are viewed on a messenger application on a mobile phone
compared to a standard DICOM viewer.
Methods: Radiographs of distal radius fractures were presented to surgeons on either a smart phone using a mobile
messenger application or a laptop using a DICOM viewer application. Twenty observers participated: 10 (50%) were
randomly assigned to the DICOM viewer group and 10 (50%) to the mobile messenger group. Each observer was asked
to evaluate the cases and (1) classify the fracture type according to the AO classification, (2) recommend operative or
conservative treatment and (3) rate their confidence about this decision.
Results: There was no significant difference in interobserver reliability for AO classification and recommendation for
surgery for distal radius fractures in both groups. The percentage of recommendation for surgery was significantly
higher in the messenger application group compared to the DICOM viewer group (89% versus 78%, P=0.019) and the
confidence for treatment decision was significantly higher in the mobile messenger group compared to the DICOM viewer
group (8.9 versus 7.9, P=0.026).
Conclusion: Messenger applications on mobile phones could facilitate remote decision-making for patients with distal
radius fractures, but should be used with caution.


Main Subjects

1. Franko OI, Tirrell TF. Smartphone app use among
medical providers in ACGME training programs. J Med
Syst. 2012; 36(5):3135-9.
2. Consulting VW. mHealth for development: the
opportunity of mobile technology for healthcare in the
developing world. Washington DC and Berkshire, UK:
UN Foundation-Vodafone Foundation Partnership;
3. Wani SA, Rabah SM, Alfadil S, Dewanjee N, Najmi Y.
Efficacy of communication amongst staff members
at plastic and reconstructive surgery section using
smartphone and mobile WhatsApp. Indian J Plast
Surg. 2013; 46(3):502-5.
4. Molina CS, Callan AK, Burgos EJ, Mir HR. On-call
communication in orthopaedic trauma: “a picture is
worth a thousand words”--a survey of OTA members.
J Orthop Trauma. 2015; 29(5):e194-7.
5. Reponen J, Ilkko E, Jyrkinen L, Tervonen O, Niinimaki
J, Karhula V, et al. Initial experience with a wireless
personal digital assistant as a teleradiology terminal
for reporting emergency computerized tomography
scans. J Telemed Telecare. 2000; 6(1):45-9.
6. Kondo Y. Medical image transfer for emergency care
utilizing internet and mobile phone. Nihon Hoshasen
Gijutsu Gakkai Zasshi. 2002; 58(10):1393-401.
7. Johnston WK 3rd, Patel BN, Low RK, Das S. Wireless
teleradiology for renal colic and renal trauma. J
Endourol. 2005; 19(1):32-6.
8. Yaghmai V, Kuppuswami S, Berlin JW, Salehi SA.
Evaluation of personal digital assistants as an
interpretation medium for computed tomography of
patients with intracranial injury. Emerg Radiol. 2003;
9. Yamamoto LG, Williams DR. A demonstration of instant
pocket wireless CT teleradiology to facilitate stat
neurosurgical consultation and future telemedicine
implications. Am J Emerg Med. 2000; 18(4):423-6.
10. Schwartz AB, Siddiqui G, Barbieri JS, Akhtar AL, Kim
W, Littman-Quinn R, et al. The accuracy of mobile
teleradiology in the evaluation of chest X-rays. J
Telemed Telecare. 2014; 20(8):460-3.
11. Toomey RJ, Ryan JT, McEntee MF, Evanoff MG,
Chakraborty DP, McNulty JP, et al. Diagnostic efficacy of
handheld devices for emergency radiologic consultation.
AJR Am J Roentgenol. 2010; 194(2):469-74.
12. Mobasheri MH, Johnston M, Syed UM, King D, Darzi
A. The uses of smartphones and tablet devices in
surgery: a systematic review of the literature. Surgery.
2015; 158(5):1352-71.
13. Giordano V, Koch HA, Mendes CH, Bergamin A, de
Souza FS, do Amaral NP. WhatsApp messenger is useful
and reproducible in the assessment of tibial plateau
fractures: inter- and intra-observer agreement study.
Int J Med Inform. 2015; 84(2):141-8.
14. Ranschaert ER, van Ooijen PM, Lee S, Ratib O, Parizel
PM. Social media for radiologists: an introduction.
Insights Imaging. 2015; 6(6):741-52.
15. Wiggelinkhuizen M, Tilanus ME, Bollen CW, Houwen
RH. Increasement in the use of WhatsApp messenger
among medical specialists. Med Contact. 2015;
16. Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche
PC, Devereaux PJ, et al. CONSORT 2010 explanation
and elaboration: updated guidelines for reporting
parallel group randomised trials. BMJ. 2010;
17. Sim J, Wright CC. The kappa statistic in reliability
studies: use, interpretation, and sample size
requirements. Phys Ther. 2005; 85(3):257-68.
18. Walter SD, Eliasziw M, Donner A. Sample size and
optimal designs for reliability studies. Stat Med. 1998;
19. Boissin C, Fleming J, Wallis L, Hasselberg M, Laflamme
L. Can we trust the use of smartphone cameras in
clinical practice? Laypeople assessment of their image
quality. Telemed J E Health. 2015; 21(11):887-92.
20. Guitton TG, Ring D, Science of Variation Group.
Interobserver reliability of radial head fracture
classification: two-dimensional compared with
three-dimensional CT. J Bone Joint Surg Am. 2011;
21. Landis JR, Koch GG. The measurement of observer
agreement for categorical data. Biometrics. 1977;
22. Siegel S, Castellan NJ. Nonparametric statistics for the
behavioral sciences. 2nd ed. New York: McGraw-Hill;
1988. P. 399.
23. Wu R, Rossos P, Quan S, Reeves S, Lo V, Wong B, et al. An
evaluation of the use of smartphones to communicate
between clinicians: a mixed-methods study. J Med
Internet Res. 2011; 13(3):e59.
24. Hsieh CH, Jeng SF, Chen CY, Yin JW, Yang JC, Tsai HH, et
al. Teleconsultation with the mobile camera-phone in
remote evaluation of replantation potential. J Trauma.
2005; 58(6):1208-12.
25. Eze N, Lo S, Bray D, Toma AG. The use of camera
mobile phone to assess emergency ENT radiological
investigations. Clin Otolaryngol. 2005; 30(3):230-3.
26. Ploegmakers JJ, Mader K, Pennig D, Verheyen CC. Four
distal radial fracture classification systems tested
amongst a large panel of Dutch trauma surgeons.
Injury. 2007; 38(11):1268-72.
27. Neuhaus V, Bot AG, Guitton TG, Ring DC. Influence of
surgeon, patient and radiographic factors on distal
radius fracture treatment. J Hand Surg Eur Vol. 2015;
28. Blaivas M, Lyon M, Duggal S. Ultrasound image
transmission via camera phones for overreading. Am
J Emerg Med. 2005; 23(4):433-8.
29. Most popular mobile messaging apps worldwide as of
April 2016, based on number of monthly active users
(in millions). The Statistics portal. Available at: URL:
global-mobile-messenger-apps; 2016.
30. Scherschel FA. Keeping tabs on whatsapp’s encryption.
CT Magazin Fur Computer Technik. Available at: URL:
WhatsApp-s-Encryption-2630361.html; 2015.
Volume 5, Issue 5
September 2017
Pages 308-314
  • Receive Date: 14 April 2016
  • Revise Date: 06 June 2016
  • Accept Date: 18 July 2016
  • First Publish Date: 01 September 2017