↓ Skip to main content
The Login Node
The Login Node
  1. Talks & Workshops/

Introduction to Parallel Computing

Hpc Parallel-Computing Workshop
Will Paik
Author
Will Paik
I optimize large-scale GPU clusters for AI/ML workloads. Outside of work, I build a mini-supercomputer from consumer hardware and document every step of it here.
Table of Contents

Date: April 28, 2026 Venue: Northeastern University, Boston, MA

Overview
#

A hands-on workshop for university researchers who want to scale computation beyond a single CPU core. This session walks through core parallel computing concepts, real benchmark results, and working code examples that can be run directly on the cluster.

Topics Covered
#

  • Serial vs. parallel execution: pipelining and data parallelism
  • Flynn’s Taxonomy: SISD, SIMD, MISD, MIMD
  • Shared vs. distributed memory models and when to use each
  • Amdahl’s Law, Gustafson’s Law, and strong vs. weak scaling
  • CPU parallelism in practice: Conway’s Game of Life (serial, OpenMP, MPI+OpenMP)
  • GPU computing fundamentals: CUDA workflow and memory model
  • Scaling ML workloads with PyTorch: single GPU, multi-GPU, and multi-node DDP
  • Parallel tools for Python, R, and MATLAB
  • Mapping parallelism to Slurm: --ntasks vs. --cpus-per-task

Materials
#