Parallel and Distributed Computing
Objectives:
On completion of the course, the student should be able to:
• understand and account for models, limitations and fundamental concepts within parallel and distributed computations, and apply this understanding to the analysis of concrete systems and algorithms.
• adapt and develop algorithms and applications for execution on parallel and distributed systems and analyse them for correctness, reliability, and performance.
Skills:
Study, design, analysis, progamming and execution of parallel and distributed computations.
Prerequisites:
Computer programming, algorithms and data structures
Content:
Parallel and distributed systems: Shared and distributed memory systems, multi-core processors, general purpose graphical processing units, memory hierarchy, interconnection networks, networking infrastructure, clusters, grids and clouds.
Operating systems and interprocess communication: processes, threads, mutual exclusion and synchronization, locks, semaphores, condition variables, monitors, messages and message queues, client-server and remote procedure call.
Models, environments and techniques of parallel and distributed computing: multithreding, message passing, sockets, remote procedure call and remote objects, service-oriented, channels and actors, general purpose graphical processing unit programming, functional programming, big data. Parallel and distributed algorithms: data and functional parallelism, master-workers, task graphs, pipelining (data flow),
Parallel and distributed algorithms: data and task parallelism, master-workers, task graps, pipelining (data flow), pool of tasks, functional models. Metrics and experimental performance evalation of parallel and distributed applications.
Textbooks:
Introduction to Parallel Computing (2nd edition). Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar, 2003, ISBN 78-8131708071
Parallel and Distributed Computation D.Bertsekas and J.Tsitsiklis, 1989, ISBN 0-13-648700-9
Parallel Programming in C withMPI and OpenMP, Michael Quinn, 2004, ISBN 007-282256-2
Parallel Programming (2nd edition). B.Wilkinson, M.Allen, 2005, ISBN 0-13-140563-2
Progamming Massively Parallel Processors, Second Edition, D.Kirk, W.Hwu, 2013, ISBN 978-0-12-415992-1
Distributed Systems: Concepts and Design (5th edition). George Coulouris, Jean Dollimore, Tim Kindberg and Gordon Blair,2011, ISBN 0-13-214301-1
Distributed Systems (3rd edition). Maarten van Steen, Andrew S. Tanenbaum, 2017, ISBN 978-1543057386
Concurrency - State Models & Java Programs (2nd edition). J. Magee, J. Kramer, 2006, ISBN 978-0470093559
Distributed and Cloud Computing: From Parallel Processing to the Internet of Things Kai Hwang, Jack Dongarra and Geoffrey C. Fox, 2011, ISBN 9780123858801
Assessment:
50% Laboratory exercises
50% Final written examination
Students should achieve at least ‘pass’ grade in both Laboratory exercises and Final written examination.
