Code Jamming Everyday IT problems, solutions and discussions

Resources to learn and understand parallel programming. The hard way

There’s no way other than the hard way. (c)

Parallel programming is considered as not easy or even advanced topic by many programmers. It’s the starting point for even more advanced stuff like distributed computations, reliability, CAP theorem, consensus problems and much more. Besides, deep understanding of how CPU and operating system works can help you to write less buggy software and parallel programming can help you with that too.

In this post I will focus on books describing parallel programming using 1 computer and 1 CPU using classical approaches. Neither they contain SSE instructions guides nor you will find matterials on CUDA or OpenCL. Similary you will find no resourced about Hadoop and/or MapReduce technologies and nothing about technologies supporting parallel programming out of the box like Go or Erlang.

So I will go now through all the resources which I find more or less useful. I’m not going to stick to any technology in general - the point is to understand the topic from different perspectives. The materials I’m refering to in general should not be considered as entry-level -  they require fair amount of knowledge, but nevertheless, list goes sorted starting from “easier” things.

Personally I find books like “Parallel programming in C++/C#/Python” not quite useful since they are more skill-based and 95% can be found in the official documentation by people knowing what they are looking for.

If you know any other interesing and valuable books on the topic, you’re more than welcome to post them in the comments.

comments powered by Disqus