Parallel and Distributed Computing 

Parallel computing is a type of computing architecture in which several processors execute or process an application or computation simultaneously. It is the breaking down larger problems into smaller, independent parts that can be executed simultaneously by multiple processors. 

A distributed system is a collection of autonomous computing elements that appears to its users as a single coherent system. It provides parallel computing in physically dispersed environment. The notion of middleware play an important role in distributed systems. 

Recommended Books:

•  Distributed Systems, principles and paradigms, 2nd & 
• 3rd Editions, Andrew S. Tanenbaum & Maarten van Steen

•  Distributed and Cloud Computing, From Parallel Processing to the Internet of Things, Kai Hwang, Geoffrey C. Fox, Jack J. Dongarra, Publisher: Morgan Kaufmann (Elsevier), ISBN: 978-0-12-385880-1

• Designing and Building Parallel Programs, Ian Foster, 
                            http://www-unix.mcs.anl.gov/dbpp/text/book.html
  

• Parallel Algorithms
  

• Programming on Parallel Machines,
  
  http://neilrieck.net/misc/pdf/computer-docs/parallel-programming-book.pdf
  

Lecture Handout Files:

• Introduction to Parallel Computing

• Introduction to Distributed Systems

• Parallel Computer Memory

• Parallel Programming Models, Designing Parallel Programs

• MPI, Multithreaded Programming

• Parallel Algorithms

• GPU

• Fault Tolerance, Concurrency Control