Lecture Notes on Multiprocessing

Introduction

• Lecture focused on multiprocessing, discussing bottleneck acceleration, multiprocessors, and memory ordering.
• Covered topics such as asymmetric multicore, accelerating critical sections, and bottlenecks.

Bottleneck Acceleration

• Key Concepts:
  • Acceleration of serial bottlenecks by placing them on a large core to increase speed.
  • Asymmetric multicore: Different cores for different tasks.
  • Critical sections can be accelerated similarly.
• Challenges:
  • Identifying and scheduling bottlenecks.
  • Decisions on acceleration must consider utility.

Bottleneck Identification and Scheduling

• Approach:
  • Bottlenecks (e.g., barriers, critical sections) can be identified and accelerated.
  • Focus on segments of a program, seeing them as stages.
• Utility-Based Approach:
  • Utility of acceleration metrics combine speedup and criticality.
  • Methodical approach to determine which bottlenecks to prioritize.

Asymmetry in Multiprocessing

• Dynamic vs. Static Asymmetry:
  • Dynamic asymmetry involves changing core characteristics at runtime.
  • Static asymmetry involves pre-defined core characteristics.
• Utility and Decision Making:
  • Utility metrics help decide which segments to accelerate.

Memory and Data Locality

• Challenge:
  • Private data locality issues when moving instructions between cores.
• Solutions:
  • Use of data marshalling to improve data locality by pre-fetching data needed by next segment.
  • Timely data transfer through hardware-software codesign.

Implications of Parallelism

• Challenges:
  • Parallelism introduces overhead, including synchronization and resource contention.
  • Amdahl's Law: Highlights the impact of non-parallelizable portions on overall speedup.

Hardware-Based Multithreading

• Types:
  • Coarse-grained: Time multiplexing threads on a single core.
  • Fine-grained: Cycle-by-cycle thread switching.
  • Simultaneous: Multiple threads executing simultaneously.

Parallel Processing Metrics

• Utilization:
  • Measures how much processing capability is used.
• Redundancy:
  • Measures extra work done in parallel processing compared to serial.
• Efficiency:
  • Measures resource usage compared to single processor execution.

Conclusion

• Emphasized the complexity of parallel programming and the importance of optimizing both hardware and software for efficient parallel execution.
• Future directions include overcoming bottlenecks and making parallel programming more accessible.