Lecture Notes: Neuromorphic Computing and Brain-like Architectures

Introduction to Neuromorphic Computing

• Concept of computing inspired by brain's principles: Neuromorphic Computing.
• Differences from traditional digital computing:
  • Digital computing is reaching performance limits.
  • Neuromorphic systems tend to be more distributed and parallel.

Characteristics of Neuromorphic Systems

• Continuum of Neuromorphic Models:
  • Digital Models: Similar to digital computers, but optimized for brain-like neuron message sending.
  • Analog Models: Circuits behave more like neurons, still based on silicon technologies.
  • Beyond Semiconductors: Exploring new hardware from first principles, moving away from conventional computing.

Key Principles of Brain-like Computation

• Neurons and Synapses:

  • Sophisticated Devices: Neurons can adapt their state based on activity.
  • State Adaptability: Adaptation can occur on various time scales.

• Network Structure:

  • Brain has a highly interconnected network (especially in neocortex and prefrontal cortex).
  • Major reason for dense connections: rapid information communication across the network.

Fractal and Scale Invariance

• Concept of Statistical Patterns:
  • Brain's network exhibits fractal dynamics across spatial scales.
  • Probability of neuron connections varies as a power law, indicating long-distance connections are possible but rare.
  • Similar patterns persist regardless of the scale examined.

Temporal Dynamics

• Non-Defined Temporal Activity:
  • Neurons do not oscillate at a singular frequency; temporal dynamics are also power law distributed.
  • Thoughts and neural activity span multiple time scales, from milliseconds to a lifetime.

Integration of Spatial and Temporal Dynamics

• Information Integration:
  • Neural activity patterns are influenced by both spatial and temporal scales.
  • Faster local connections vs. slower distributed processing.

Architectural Principles

• Importance of architecture resembling the Thalamocortical Complex:
  • Neocortex: Thick structure with billions of neurons and many connections.
  • Hippocampus: Randomly structured, essential for spatial and temporal memory.
  • Thalamus: Coordinates communication between the neocortex and hippocampus.

Memory Mechanisms in Neuromorphic Systems

• Types of Memory:
  • Based on Hopfield networks for working memory using dynamic patterns.
  • Memory formation involves adapting synaptic weights.

Learning Mechanisms

• Supervised vs. Unsupervised Learning:

  • Supervised: External input adjusts synaptic weights.
  • Unsupervised: Internal dynamics adjust weights without external input.

• Memory Formation:

  • Learning primarily through synaptic adaptations rather than forming new connections.
  • Synaptic plasticity and weight updates are crucial.

Important Mechanisms

• Short-term Synaptic Plasticity: Adjusts synaptic efficacy after bursts of activity.
• Metaplasticity: Change in how quickly weights adjust based on learning context.
• Homeostasis: Neurons regulate their firing rates to stay within a useful range.

Conclusion

• Goal of neuromorphic computing: Capture brain's dynamics in hardware.
• Need for devices to integrate and process information the way the brain does, using principles of spatial and temporal fractals.