.. _books: Books ===== This is a very incomplete list, permanently under construction, of books about parallel computing. General ------- - G. Hager and G. Wellein. `Introduction to high performance computing for scientists and engineers `_. Chapman & Hall, 2010. This book first introduces the architecture of modern cache-based microprocessors and discusses their inherent performance limitations, before describing general optimization strategies for serial code on cache-based architectures. It next covers shared- and distributed-memory parallel computer architectures and the most relevant network topologies. After discussing parallel computing on a theoretical level, the authors show how to avoid or ameliorate typical performance problems connected with OpenMP. They then present cache-coherent nonuniform memory access (ccNUMA) optimization techniques, examine distributed-memory parallel programming with message passing interface (MPI), and explain how to write efficient MPI code. The final chapter focuses on hybrid programming with MPI and OpenMP. - V. Eijkhout. `Introduction to high performance scientific computing `_. 2011. This is a textbook that teaches the bridging topics between numerical analysis, parallel computing, code performance, large scale applications. It can be freely downloaded from `the author's page on the book `_ (though you have to respect the copyright of course). - A. Grama, A. Gupta, G. Kapyris, and V. Kumar. `Introduction to parallel computing (2nd edition) `_. Pearson Addison Wesley, 2003. ISBN 978-0-201-64865-2. A somewhat older book, but still used a lot as textbook in academic courses on parallel computing. - C. Lin and L. Snyder. `Principles of parallel programming `_. Pearson Addison Wesley, 2008. ISBN 978-0-32148790-2. This books discusses parallel programming both from a more abstract level and a more practical level, touching briefly threads programming, OpenMP, MPI and PGAS-languages (using ZPL). - M. McCool, A.D. Robinson, and J. Reinders. `Structured parallel programming: patterns for efficient computation `_. Morgan Kaufmann, 2012. ISBN 978-0-12-415993-8 Grid computing -------------- - F. Magoules, J. Pan, K.-A. Tan, and A. Kumar. `Introduction to grid computing `_. CRC Press, 2019. ISBN 9780367385828. MPI --- - A two-volume set in tutorial style: - W. Gropp, E. Lusk, and A. Skjellum. `Using MPI: portable parallel programming with the Message-Passing Interface, third edition `__. MIT Press, 2014. ISBN 978-0-262-57139-2 (paperback) or 978-0-262-32659-9 (ebook). This edition of the book is based on the MPI-3.0 specification. - W. Gropp, T. Hoeffler, R. Thakur and E. Lusk. `Using advanced MPI: modern features of the Message-Passing Interface `_. MIT Press, 2014. ISBN 978-0-262-52763-7 (paperback) or 978-0-262-32662-9 (ebook). The books replace the earlier editions of "Using MPI: Portable Parallel Programming with the Message-Passing Interface" and the book "Using MPI-2: Advanced Features of the Message-Passing Interface". - A two-volume set in reference style, but somewhat outdated: - M. Snir, S.W. Otto, S. Huss-Lederman, D.W. Walker, and J. Dongarra. `MPI: the complete reference. Volume 1: the MPI core (2nd Edition) `_. MIT Press, 1998. ISBN 978-0-262-69215-1. - W. Gropp, S. Huss-Lederman, A. Lumsdaine, E. Lusk, B. Nitzberg, W. Saphir, and M. Snir. `MPI: the complete reference, Volume 2: the MPI-2 extensions `_. MIT Press, 1998. ISBN 978-0-262-57123-4. The two volumes are also available as one set with `ISBN number 978-0-262-69216-8 `_. OpenMP ------ - B. Chapman, G. Jost, and R. van der Pas. `Using OpenMP - portable shared memory parallel Programming `_. The MIT Press, 2008. ISBN 978-0-262-53302-7. - R. Chandra, L. Dagum, D. Kohr, D. Maydan, J. McDonald, and R. Menon. `Parallel programming in OpenMP `_. Academic Press, 2000. ISBN 978-1-55860-671-5. GPU computing ------------- - M. Scarpino. `OpenCL in action `_. Manning Publications Co., 2012. ISBN 978-1-617290-17-6 - D.R. Kaeli, P. Mistry, D. Schaa, and D.P. Zhang. `Heterogeneous computing with OpenCL 2.0, 1st Edition `_. Morgan Kaufmann, 2015. ISBN 978-0-12-801414-1 (print) or 978-0-12-801649-7 (eBook). A thourough rewrite of the earlier well-selling book for OpenCL 1.2 that saw 2 editions. Xeon Phi computing ------------------ - R. Rahman. `Intel Xeon Phi coprocessor architecture and tools: the guide for application Developers `_. Apress, 2013. SBN13: 978-1-4302-5926-8. This is a free book that is aimed at the Knights Corner generation of Xeon Phi processors. The newer Knights Landing generation has a reworked vector instruction set, but most principles explained in this book remain valid also for the newer generation(s). - J. Jeffers, J. Reinders, and A. Sodani. `Intel Xeon Phi processor high performance programming, 2nd Edition (Knights Landing Edition) `_. Morgan Kaufmann, 2016. ISBN 978-0-12-809194-4. Errata and downloadable code examples for this book and other books by Jeffers and Reinders are maintained on the blog `lotsofcores.com `_. Case studies and examples of programming paradigms -------------------------------------------------- - J. Reinders and J. Jeffers (editors). `High performance parallelism pearls. Volume 1: multicore and many-core programming approaches `_. Morgan Kaufmann, 2014. ISBN 978-0-12-802118-7 - J. Reinders and J. Jeffers (editors). `High performance parallelism pearls. Volume 2: multicore and many-core programming approaches `_. Morgan Kaufmann, 2015. ISBN 978-0-12-803819-2 *Please mail further suggestions to geertjan.bex@uhasselt.be*