ReXplain: Translating Radiology into Patient-Friendly Video Reports

Authors and Affiliations

• Luyang Luo, Jenanan Vairavamurthy, Xiaoman Zhang, Abhinav Kumar, Ramon R. Ter-Oganesyan, Stuart T. Schroff, Dan Shilo, Rydhwana Hossain, Mike Moritz, Pranav Rajpurkar
• Affiliations include Harvard University, Icahn School of Medicine, Stanford University, Los Angeles General Medical Center, University of Maryland, and SSM Health, Saint Louis University

Abstract

• Problem: Radiology reports are complex and not patient-friendly, leading to anxiety and poor health outcomes.
• Solution: ReXplain, an AI-driven system that converts radiology reports into patient-friendly video explanations using avatars, image segmentation, and language models.
• Evaluation: Feedback from board-certified radiologists suggests ReXplain accurately delivers information and simulates one-on-one consultations.
• Impact: Enhances patient engagement and satisfaction, improving patient-centered radiology communication.

Introduction

• Background: There is a shortage of radiologists and a growing demand for medical imaging.
• Patient-Centered Radiology: Involves engaging with patients and making radiology reports more accessible.
• Current Challenges: Conventional reports are complex; video reports can improve comprehension but are resource-intensive.
• AI in Radiology: Large Language Models (LLMs) and advanced image processing can simplify reports and improve understanding.

ReXplain System

• Components:
  • LLMs for medical text simplification and linking findings to anatomy.
  • Image segmentation models to highlight relevant anatomical regions.
  • Avatar generation for a personalized explanation experience.
• Integration: Combines text and image analysis into cohesive video reports simulating consultations.

Related Works

• Radiology Communication: Direct communication enhances experience but strains resources.
• AI Advances: LLMs simplify text but struggle with image interpretation.
• Video Reports: Preferred by patients but require significant effort from radiologists.

Designing ReXplain

• Video Report Concepts:
  • Simulate radiologist-patient interactions.
  • Address technical challenges like translating medical terminology and annotating images.
• Technical Implementation:
  • Use of GPT and SAT models for language and image processing.
  • Integration of normal and abnormal image comparisons.

Evaluation

• User Study:
  • Conducted with radiologists to assess video report effectiveness.
  • Positive feedback on understandability and potential for patient communication.
• Technical Performance:
  • High precision and recall in text extraction and organ localization.

Eliciting User Feedback

• Feedback:
  • Round 1: Positive results in comprehension and usability.
  • Round 2: Suggestions for improvement include better pathology highlighting and strategic pausing in videos.

Outlook

• Limitations:
  • Current organ-level segmentation needs improvement.
  • Video scrolling and complexity levels need adjustment.
• Potential Usage:
  • Pre-consultation educational tool.
  • Videos can enhance patient engagement and reduce provider burden.

Conclusion

• ReXplain presents a novel approach to patient-friendly radiology reporting.
• Encouraging initial feedback suggests the potential for significant improvements in patient understanding and healthcare communication.