Training an AI to Beat Usain Bolt's Record

Introduction

• Year: 2009
• Event: Usain Bolt sets a world record by running 100 meters in 9.58 seconds.
• Challenge: Attempt to break this record using an AI-controlled ragdoll.

AI Configuration

• Initial State: Untrained and naive AI.
• Training Environment: A 100-meter test running track for thousands of hours and episodes.
• Ragdoll Specifications:
  • Weight: 70 kilos
  • Height: 6 feet
  • Core Component: Neural network mimicking brain functions.

Neural Network Details

• Inputs and Outputs:
  • Inputs: Euler angles of 16 joints, fed into the neural network as a vector.
  • Outputs: Control over the joints.
• Hidden Layers: Determines AI's IQ. Using 256 nodes (10% of jellyfish neurons).

Reward Function

• Objective: Use reinforcement learning to incentivize desired behaviors.
• Target Speed: 11 meters per second (Bolt’s average during record: 10.4 m/s).
• Reward System:
  • Approaching target speed gets higher rewards.
  • Deviation from speed lowers reward.
  • Additional small reward for running in a straight lane.

Initial Training Results

• Outcome: AI face-plants and exhibits a zombie-like stride.
• Problem: Dominant leg syndrome causing inefficiency.

Solution: Randomization and Trial Environment

• New Training Setup: Cubic training environment.
• Training Parameters:
  • Random orientation each episode.
  • Face a randomly spawned cube to earn rewards.
  • Target velocity set at 3 meters per second for stability.
• Goal: Ensure AI uses both legs effectively.

Improved Training Results

• Outcome: AI walks properly using both legs.
• Next Steps: Place AI back into the original setting, increase target speed back to 11 m/s.

Further Enhancements

• Penalties:
  • For falling over.
  • For merely existing (to encourage urgency in completing tasks).
• Adjustments:
  • Increase AI's height to match Usain Bolt (6 foot 5).

Final Training Phase

• Implement penalties and height adjustments.
• Resume intensive training.