Intro to Nutrify: A Day in the Life of a Machine Learning Engineer

Welcome to Nutrify HQ

• Presenter: Daniel Burke, Machine Learning Engineer and Co-founder of Nutrify
• App Introduction: Nutrify is an iOS application that helps users learn about Whole Foods.
• Team Introduction:
  • Josh: Head iOS Engineer

Office Tour

• Planning Station: Two whiteboards used for planning
• Data Labeling Station:
  • Machine Learning concept: Draw boxes around food to train models
  • Current model: Image Classification, transitioning to Object Detection
• Personal Office:
  • Notable book: Designing Machine Learning Systems by Chip Hillen

Daily Routine

• Morning: 7:00-8:00 AM
  • Make breakfast; focus on Whole Foods
  • Morning meeting outside to discuss plans
• Start Work: 8:15 AM
  • List top 3-5 tasks for the day
  • Read articles/papers on machine learning
  • Exercise/snack breaks every 45 minutes (pull-ups, push-ups, squats)

Model Training and Data Labeling

• 9:00-10:30 AM: Data labeling for Nutrify Food Vision model
• Train models using a custom script
  • Parameters and logging through Weights & Biases
  • Model failed due to dimensionality issue (torch.ARG Max issue)

Lunchtime at Nutrify HQ

• Menu: Ground beef, roasted sweet potato, guacamole, broccolini stalks
• Guest Introduction: Joey and Pizza Dad
  • Joey: Working on transitioning from image classification to object detection
  • Pizza Dad: Assists in computer vision teaching

Afternoon Session

• Tasks:
  • Joey: Food box labeling
  • Josh: Working on text detection and image matching
  • Daniel: Refining data pipeline

Recommended Reading

• Overton paper by Apple (2019) – Useful for understanding roles in data and model management

Model Training Continued

• Using Two Computers:
  • Older PC: NVIDIA Titan RTX – typically used for training models
  • Newer PC: NVIDIA RTX 4090 – used for inference (Named Entity Recognition over large text dataset)
  • Goal: Continuous model training and inference

Key Concepts and Workflow

• Build muscle analogy: Run 1000 experiments, many will fail but some will succeed
• Continuous model training and evaluation
• Regular testing against real-life scenarios: Geni Jutsu – Real things, real places

Closing

• Predict and evaluate model performance to ensure it works in real-world scenarios
• Josh's Summary of Startup Experience: Fun

Emoji: 📈