Creating a Real-Life Pokédex

Introduction

• Goal: Develop a real-life Pokédex for photo-based analysis and identification.
• Capabilities:
  • Identify objects (not just Pokémon).
  • Provide descriptions, statistics (e.g., weight, height), and store entries.

Inspiration

• Personal fondness for Pokémon and childhood memories.
• Pokémon's enduring popularity over 25 years.
• Previous startup experiences led to initial research.
• Concept: Analyzing text/images to produce descriptive outputs.

Research Findings

• No existing real-life Pokédex found; some 3D prints using Raspberry Pi exist.
• Comparison to Rabbit 1 (R1), a pocket companion using AI.
• Emphasis on smartphones over specialized hardware.

Plan Overview

• High-Level Plan:
  • Image scanning.
  • AI-generated descriptions, characteristics.
  • Verbal explanations via AI voice.
  • Database for entries.
• Low-Level Plan:
  • Design decisions: mobile vs. app store app.
  • Detailed page layouts.

Design Process

• Started with sketching in Figma.
• Sought inspiration from Google, Behance, Dribble.
• Key references: Nintendo's Pokémon resources and various Pokedex interface concepts.
• Design Elements: Vibrant colors, specific fonts (pixelated for authenticity), familiar aesthetics.
• Used Concepts app for sketching on iPad:
  • Login screen.
  • Camera screen.
  • Pokédex screen for viewing entries.
  • Detailed entry viewing page.
• Importance of visualizing ideas to simplify complex projects.

Initial App Design

• Tools used: Figma for mock-ups.
• Recreated Pokémon game aesthetics.
• Pulled various color schemes and typography elements.
• Fonts: Selected Poet Sans One and Inter.
• Focused on header and core camera aesthetics inspired by iPhone.
• Menu Design: Similar to Instagram/TikTok but with central camera button.
• Entries Page: Listing with color codes by type (e.g., bird = flying type).
• Labelling with Pokémon examples for initial design.

Development

• Tech Stack:
  • Used Next.js 14.
  • Astra DB for database (free up to 80GB, built on Apache Cassandra).
  • OpenAI's GPT-4 for image analysis and descriptions.
• Steps:
  • Set up database and collections in Astra DB for Pokémon entries.
  • Used image analysis capabilities of GPT-4.
  • Implementing prompts for accurate Pokémon-like descriptions and stats in JSON format.
  • Setup camera functionalities using React Webcam library.
  • Used React Image File Resizer for image size constraints.
  • Created user store with MobX and additional UI components in CSS.

Enhancements

• Added nostalgic sound effects using use sound library for camera actions.
• Graph display for stats using Chart.js radial graph.
• Voice generation for entries with fake-you API.
• Implemented OAuth login (Google) via NextAuth.
• Connectivity for user-specific Pokédex entries and leaderboard.
• Vector embeddings in DB for related search functionality.

Final Product

• Key Features:
  • Login to personal account, view entries, and access the leaderboard.
  • Take photos and get detailed analyses and statistics rendered into the app in real-time.
• Additional Information: Open-source project available, hidden Easter egg for users.

Conclusion

• An extensive journey from concept to a functional application.
• Encouragement for contributions to the open-source project.
• Gratitude expressed for viewer engagement and support.