Online MPI Course Lecture Notes

Introduction to Archer

• Archer is a UK National supercomputer managed by EPCC (Edinburgh Parallel Computing Centre).
• Located at the University of Edinburgh.
• Training and support provided for free to academics.
• Archer uses a Cray XC30 supercomputer.

EPCC Overview

• Established in 1990, now a self-funding center within the College of Science and Engineering.
• Funded through commercial and research contracts.
• Offers postgraduate programs in High Performance Computing (HPC) and Data Science.
• Involved in various academic and commercial projects.

Key Resources and Training

• Course materials available online.
• Feedback forms encouraged to improve services.
• Information about other courses and scholarships available for students.

Course Structure and Access

• Participants have access to the Sirius machine for practical tasks.
• Accounts for Sirius will be available temporarily during the course.
• Instructions on HBC practices provided, including how to avoid misuse.

Message Passing Concepts

Programming Model

• In parallel programming, processes are isolated and communicate through messages.
• Concepts include:
  • Processes
  • Send and Receive operations
  • Collective communications
• Message passing libraries such as MPI (Message Passing Interface) are widely used.

Message Passing Operations

• Send Operations:
  • Blocking send: Waits until the message is received.
  • Non-blocking send: Returns immediately and continues execution.
• Receive Operations:
  • Always blocking; waits for a message.
• Collective Communications:
  • Broadcast: Sends data from one process to all others.
  • Scatter: Distributes data from one process to all others based on a defined structure.
  • Gather: Collects data from all processes to one.

Key Functions in MPI

• MPI Init: Initializes MPI environment.
• MPI Finalize: Cleans up MPI environment.
• MPI Barrier: Synchronizes all processes at a certain point.
• MPI Reduce: Combines values across processes to a root process.
• MPI Allreduce: Combines values and distributes them to all processes.

User-defined Reduction Operators

• Allows custom operations to be defined for reduction processes.
• Important characteristics:
  • Must be associative (order of operations doesn't change result).
  • Must match the operation expected (e.g., summation).

Non-blocking Communications

• Allows overlapping of communication and computation.
• Requires explicit matching of requests and checks.
• Can improve performance in complex applications.

Collective Communication Types

1. Barrier: All processes must reach a certain point before proceeding.
2. Broadcast: One process sends data to all processes.
3. Scatter: Distributes subsets of data to each process from a single source.
4. Gather: Collects data from all processes to one.
5. Reduce: Combines data from all processes into a single value on a specified root process.
6. Allreduce: Combines data and sends the result to all processes.

Practical Applications and Exercises

• Exercises involve implementing message passing to calculate Pi using both point-to-point and collective communications.
• Performance comparisons between different methods encouraged.
• Exploring user-defined operations and their implications in MPI.

Conclusion

• Understanding MPI's message-passing model is vital for developing efficient parallel applications.
• Collective communications offer powerful tools for synchronizing and combining data across processes.
• Encouraged to explore further through practical applications and additional MPI resources.