Documents

Parallel Programming

Description
Description:
Categories
Published
of 692
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Share
Transcript
  Is Parallel Programming Hard, And, If So, WhatCan You Do About It? Edited by:Paul E. McKenneyLinux Technology CenterIBM Beavertonpaulmck@linux.vnet.ibm.comJanuary 2, 2017  ii Legal Statement This work represents the views of the editor and the authors and does not necessarily represent the view of their respective employers.Trademarks:ã  IBM, zSeries, and PowerPC are trademarks or registered trademarks of Interna- tional Business Machines Corporation in the United States, other countries, or both.ã Linux is a registered trademark of Linus Torvalds.ã  i386 is a trademark of Intel Corporation or its subsidiaries in the United States, other countries, or both.ã  Other company, product, and service names may be trademarks or service marks of such companies. The non-source-code text and images in this document are provided under the terms of the Creative Commons Attribution-Share Alike 3.0 United States license. 1 In brief, you may use the contents of this document for any purpose, personal, commercial, or otherwise, so long as attribution to the authors is maintained. Likewise, the document may be modified, and derivative works and translations made available, so long assuch modifications and derivations are offered to the public on equal terms as the non-source-code text and images in the srcinal document. Source code is covered by various versions of the GPL. 2 Some of this code isGPLv2-only, as it derives from the Linux kernel, while other code is GPLv2-or-later. See the comment headers of the individual source files within the CodeSamples directory in the git archive 3 for the exact licenses. If you are unsure of the license for a given code fragment, you should assume GPLv2-only.Combined work © 2005-2016 by Paul E. McKenney. 1 http://creativecommons.org/licenses/by-sa/3.0/us/ 2 http://www.gnu.org/licenses/gpl-2.0.html 3 git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/perfbook.git  Contents 1 How To Use This Book 1 1.1 Roadmap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Quick Quizzes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Alternatives to This Book . . . . . . . . . . . . . . . . . . . . . . . . 31.4 Sample Source Code . . . . . . . . . . . . . . . . . . . . . . . . . . 41.5 Whose Book Is This? . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2 Introduction 7 2.1 Historic Parallel Programming Difficulties . . . . . . . . . . . . . . . . 72.2 Parallel Programming Goals . . . . . . . . . . . . . . . . . . . . . . 92.2.1 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . 92.2.2 Productivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.2.3 Generality . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.3 Alternatives to Parallel Programming . . . . . . . . . . . . . . . . . . 142.3.1 Multiple Instances of a Sequential Application . . . . . . . . 152.3.2 Use Existing Parallel Software . . . . . . . . . . . . . . . . . 152.3.3 Performance Optimization . . . . . . . . . . . . . . . . . . . 152.4 What Makes Parallel Programming Hard? . . . . . . . . . . . . . . . 162.4.1 Work Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . 172.4.2 Parallel Access Control . . . . . . . . . . . . . . . . . . . . . 182.4.3 Resource Partitioning and Replication . . . . . . . . . . . . . 182.4.4 Interacting With Hardware . . . . . . . . . . . . . . . . . . . 192.4.5 Composite Capabilities . . . . . . . . . . . . . . . . . . . . . 192.4.6  How Do Languages and Environments Assist With These Tasks?  192.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 3 Hardware and its Habits 21 3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.1.1 Pipelined CPUs . . . . . . . . . . . . . . . . . . . . . . . . . . 213.1.2 Memory References . . . . . . . . . . . . . . . . . . . . . . 233.1.3 Atomic Operations . . . . . . . . . . . . . . . . . . . . . . . 243.1.4 Memory Barriers . . . . . . . . . . . . . . . . . . . . . . . . 253.1.5 Cache Misses . . . . . . . . . . . . . . . . . . . . . . . . . . 263.1.6 I/O Operations . . . . . . . . . . . . . . . . . . . . . . . . . 263.2 Overheads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.2.1 Hardware System Architecture . . . . . . . . . . . . . . . . . . 273.2.2 Costs of Operations . . . . . . . . . . . . . . . . . . . . . . . 283.3 Hardware Free Lunch? . . . . . . . . . . . . . . . . . . . . . . . . . 30iii  iv  CONTENTS 3.3.1 3D Integration . . . . . . . . . . . . . . . . . . . . . . . . . . 313.3.2 Novel Materials and Processes . . . . . . . . . . . . . . . . . . 313.3.3 Light, Not Electrons . . . . . . . . . . . . . . . . . . . . . . 323.3.4 Special-Purpose Accelerators . . . . . . . . . . . . . . . . . 323.3.5 Existing Parallel Software . . . . . . . . . . . . . . . . . . . 333.4 Software Design Implications . . . . . . . . . . . . . . . . . . . . . . 33 4 Tools of the Trade 35 4.1 Scripting Languages . . . . . . . . . . . . . . . . . . . . . . . . . . 354.2 POSIX Multiprocessing . . . . . . . . . . . . . . . . . . . . . . . . . 364.2.1 POSIX Process Creation and Destruction . . . . . . . . . . . 364.2.2 POSIX Thread Creation and Destruction . . . . . . . . . . . 384.2.3 POSIX Locking . . . . . . . . . . . . . . . . . . . . . . . . . 394.2.4 POSIX Reader-Writer Locking . . . . . . . . . . . . . . . . . 424.2.5 Atomic Operations (gcc Classic) . . . . . . . . . . . . . . . . 454.2.6 Atomic Operations (C11) . . . . . . . . . . . . . . . . . . . . 464.2.7 Per-Thread Variables . . . . . . . . . . . . . . . . . . . . . . . 474.3 Alternatives to POSIX Operations . . . . . . . . . . . . . . . . . . . . 474.3.1 Organization and Initialization . . . . . . . . . . . . . . . . . . 474.3.2 Thread Creation, Destruction, and Control . . . . . . . . . . . . 474.3.3 Locking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494.3.4 Atomic Operations . . . . . . . . . . . . . . . . . . . . . . . . 514.3.5 Per-CPU Variables . . . . . . . . . . . . . . . . . . . . . . . . 514.3.6 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . 534.4 The Right Tool for the Job: How to Choose? . . . . . . . . . . . . . . 53 5 Counting 55 5.1 Why Isn’t Concurrent Counting Trivial? . . . . . . . . . . . . . . . . 565.2 Statistical Counters . . . . . . . . . . . . . . . . . . . . . . . . . . . 585.2.1 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 585.2.2 Array-Based Implementation . . . . . . . . . . . . . . . . . . 595.2.3 Eventually Consistent Implementation . . . . . . . . . . . . . 605.2.4 Per-Thread-Variable-Based Implementation . . . . . . . . . . 635.2.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . 645.3 Approximate Limit Counters . . . . . . . . . . . . . . . . . . . . . . 645.3.1 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 645.3.2 Simple Limit Counter Implementation . . . . . . . . . . . . . 655.3.3 Simple Limit Counter Discussion . . . . . . . . . . . . . . . . 715.3.4 Approximate Limit Counter Implementation . . . . . . . . . 725.3.5 Approximate Limit Counter Discussion . . . . . . . . . . . . 725.4 Exact Limit Counters . . . . . . . . . . . . . . . . . . . . . . . . . . 725.4.1 Atomic Limit Counter Implementation . . . . . . . . . . . . 735.4.2 Atomic Limit Counter Discussion . . . . . . . . . . . . . . . . 775.4.3 Signal-Theft Limit Counter Design . . . . . . . . . . . . . . . 775.4.4 Signal-Theft Limit Counter Implementation . . . . . . . . . . 795.4.5 Signal-Theft Limit Counter Discussion . . . . . . . . . . . . 855.5 Applying Specialized Parallel Counters . . . . . . . . . . . . . . . . 855.6 Parallel Counting Discussion . . . . . . . . . . . . . . . . . . . . . . 865.6.1 Parallel Counting Performance . . . . . . . . . . . . . . . . . 865.6.2 Parallel Counting Specializations . . . . . . . . . . . . . . . . 87
Search
Tags
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks
SAVE OUR EARTH

We need your sign to support Project to invent "SMART AND CONTROLLABLE REFLECTIVE BALLOONS" to cover the Sun and Save Our Earth.

More details...

Sign Now!

We are very appreciated for your Prompt Action!

x