🧠

Webinar on Brain-Computer Interface: Synergies between EEG and XR

Jun 13, 2024

Webinar on Brain-Computer Interface in VR: Synergies between EEG and XR

Introduction

  • Presenter: Bernard Wong
  • Hosted By: Frontiers in Virtual Reality Journal in collaboration with the Gestar European project
  • Topic: Brain-computer interface (BCI) in virtual reality (VR), synergies between EEG (Electroencephalography) and XR (Extended Reality)
  • Background of Presenter:
    • Master’s in Life Science and Technology
    • Bachelor's in Electronics Engineering
    • Experience in EEG, machine learning, and Unity since 2018
    • Developing the Unicorn Unity interface for gaming applications

Definition of Neuro Game

  • Neuro Game: Game partially or entirely controlled by the player’s brain signals
    • Old examples: 1990s’ BCI-controlled wheelchairs for disabled people
    • Current examples: Faster BCI commands in games than keyboard controls

Evolution and Importance of Games in Technology

  • History:
    • 1970s-80s: Basic arcade games (Pong, Space Invaders)
    • Portable consoles (Game Boy)
    • 3D games (PlayStation, GameCube)
    • Sensor-integrated games (Wii, Kinect)
    • Recent rise: VR/XR (Oculus Rift, Quest, Apple Vision Pro)
  • Future: Integration of BCI with 3D VR and additional sensors for immersive experiences

Applications of BCI in Games

  • Clinical Uses:
    • 2014: ADHD treatment
    • Neuro-rehabilitation for stroke patients (GTech)
  • Gaming Uses: Increasing entertainment-oriented BCI games

Structure of a BCI Game

  • Traditional Game: Controlled by keyboard/joystick
  • BCI Game: Closed-loop system involving EEG device, signal processing, feature extraction, classification, neuro game feedback
  • GTech’s Unicorn Unity Package:
    • All-in-one BCI package for Unity game development
    • Handles data acquisition, preprocessing, feature extraction, classification
    • Allows developers to integrate BCI without being experts in neuroscience

Unicorn Unity Package Examples

  • Pac-Man BCI Game: Control ghosts with BCI while navigating with arrow keys
  • BCI Puzzle Game: Mentally select and flip puzzle pieces using EEG data
  • Brain Hockey: Control bars in a pong-like game using visual flickering (within EEG data interplay)
  • Space Defender: Use EEG to deploy shields against incoming enemies

Technical Elements

  • Training Phase: Critical for robust classifier development
    • Requires EEG device calibration
    • Uses visual stimulation protocols (e.g., single stimulation, steady-state visual evoked potential)
    • Visualization of target vs. non-target responses
  • Metrics and Protocols: Task Accuracy: Combination of BCI accuracy and game logic
    • Metrics on classifier performance and user task execution
    • Includes confidence levels and timing

Real-World Implementations and Challenges

  • Importance of signal quality and proper setup
  • Training duration and feature extraction methods
  • Visual clarity in stimulation patterns
  • Avoiding interference of peripheral vision
  • Moving targets can disrupt focus

Best Practices in BCI Game Design

  • Gamify Training: Enhance user engagement
  • Combine Keyboard and BCI Controls: Hybrid models for practical usability
  • Signal Quality Checks: Ensuring consistent performance in various environments
  • Adaptation to VR: Transitioning game design from 2D to 3D/VR formats

Future Directions and Potential Challenges

  • Optimization Efforts: Faster training time, reduction of required channels, improving classifier reusability
  • Accessibility Improvements: Ensuring inclusive game designs for players with disabilities
  • Development Community: Establishment and support via GitHub and other collaborative platforms

Q&A Highlights

  • Integration with Other Headsets: Currently limited to GTech devices, with future plans for broader compatibility
  • Accessibility Benefits: Replace keyboard control for disabled players, allowing equal gaming experiences
  • Latency Considerations: Tradeoffs between classifier accuracy and response time in game interactions
  • Challenges: Necessity of robust classifier, environmental adjustments, and user adaptability

Closing Remarks

  • Upcoming Webinars: Enhancing social trust in VR with effective haptic feedback
  • Presenter’s Dream: Establish active development community for neuro games
  • Attendees Encouraged to Participate in Further Sessions

Full transcript