Lecture Notes

Introduction

• Human Interaction with Objects
  • Physical objects are fun and intuitive.
  • Gestures are a key part of interacting with objects and people.
  • Example:
    • "Namaste" as a respectful gesture.
    • Indian cricket gesture for "four runs".
• Digital Interaction Vision
  • Aim to integrate physical interaction styles with digital interfaces.
  • Started exploring around 8 years ago.
  • Initial experiment: modifying a computer mouse to create a gesture-interface device.

Early Experiments

• Gesture-Interface Device
  • Created using mouse rollers and other materials.
  • Replicated physical movements in the digital world.
• Sticky Note Digital Integration
  • Concept: Sync physical sticky notes with digital world.
  • Example uses: SMS, calendar sync, search queries printed as outputs.
• 3D Drawing Pen
  • Pen that allows drawing in three dimensions.
  • Aimed at designers and architects.
• Physical Google Map
  • Objects placed on a surface reveal related digital information.
  • Examples: Boarding pass shows flight gate, coffee cup shows coffee locations.

Moving to a Digital Approach

• Reverse Integration
  • Idea: Bring digital information into the physical world.
  • Initial concept: Head-mounted projector.
  • Later development: SixthSense device.

SixthSense Device

• Technology and Functionality
  • Combines projector and camera to create a portable digital interface.
  • Any surface can be used as an interface.
  • Camera tracks gestures and objects.
  • Color markers for enhanced gesture recognition.
  • Examples:
    • Painting on walls.
    • Using both hands for map interactions.
    • Taking photos with hand gestures.
• Applications and Capabilities
  • Interacting with digital images projected on walls.
  • Using hand for dialing phone numbers.
  • Digital identification of physical objects.
  • Real-time updates on physical media (e.g., book reviews, live weather).
  • Interactive gaming in physical spaces.
  • Extension to everyday computing tasks like browsing, movie watching, and gaming on paper.

Future and Impact

• Vision for Computing
  • Merge computing with the physical world.
  • Intuitive interactions without learning new interfaces.
• Broader Goals
  • Integrate digital information seamlessly into everyday life.
  • Make technology accessible to the masses.
  • Examples of potential uses:
    • Assistive technology for individuals with disabilities.
    • Cost-effective computing interfaces.

Discussion with Chris Anderson

• Future Plans
  • Interest from companies and NGOs.
  • Potential uses for SixthSense technology in diverse fields.
  • Plans to make the technology open source.
  • Importance of bringing technology to masses, not confined to labs.

Conclusion

• Final Thoughts
  • Aim to bridge the digital divide.
  • Encourage human connection with the physical world while utilizing digital advancements.