Calculator Presentation Notes

Introduction

• Apple launched a new calculator called Math Notes for iPad.
• Transforms handwriting into digital equations, can solve problems, and graph on-the-fly.

Availability

• Currently, Math Notes is only available for iPad, raising concerns for Android and PC users.
• Plans to expand to the web and other platforms.

Features of Math Notes

• Basic arithmetic and complex calculations.
• Can analyze scenarios (e.g., car crash calculations).
• Capable of interpreting abstract concepts depicted in drawings (e.g., chicken and egg question, Captain America's shield).

Demonstration

1. Drawing Captain America’s Shield:

   • Colors used: Red, White, Blue.
   • Result: "A shield is a symbol of freedom and Justice."

2. Pythagoras Theorem Example:

   • Draw a tree and shadow; calculate hypotenuse.
   • Example values: 8m length and 6m shadow; hypotenuse expected to be 10m.

3. Car Crash Simulation:

   • Draw a tree (100m tall) and a car moving at 160 km/h to calculate crash time.

4. Abstract Question:

   • Draw people saluting a flag to represent patriotism.
   • Result: "Patriotism equals patriotism."

5. Cricut Wagon Wheel:

   • Draw various runs (6s, 4s, singles) and get results back.

6. Newton and the Apple:

   • Draw an apple falling on Newton to illustrate gravity discovery.

Technical Setup - Project Creation

• Use Vite for project setup with React and TypeScript.
• Initialize Tailwind CSS for styling.

Configurations

• Update tsconfig.json and tailwind.config.js files.
• Set up project structure with a screens directory for website screens.

Canvas Drawing Implementation

1. Canvas Setup:

   • Use ref to create a canvas for drawing.
   • Implement mouse events for "Start Drawing" and "Stop Drawing".

2. Drawing Logic:

   • Draw only when the user is clicking and holding the mouse button.
   • Set brush color and use lineTo for drawing paths.

3. Reset Functionality:

   • Implement buttons for resetting and calculating drawings.

Component Libraries Used

• Shadcn for customizable components (buttons, forms, etc.).
• Mantine for color swatch component.

Backend Development

• Create a Python backend using FastAPI.
• Use Pydantic for data validation and defining schemas.
• Enable CORS to avoid origin issues.

API Functionalities

1. Image Analysis with Gemini API:

   • Send canvas image to backend for analysis.
   • Handle responses and display results in frontend.

2. Environment Configuration:

   • Store sensitive information (API keys) in a .env file.

Final Touches

• Use MathJax to render mathematical expressions beautifully in LaTeX format.
• Implement draggable components for expression positioning.

Conclusion

• Successfully built a comprehensive calculator application.
• Features include hand-drawn inputs, complex calculations, results display in LaTeX, and a user-friendly interface.
• Encouragement to continue developing and testing the application.

Next Steps

• Further enhancements and potential local model implementations for image processing.