So you want to be a DASH coder.

Instant DASH

We maintain a VirtualBox VM images with a fully equipped DASH developer environment so you can start right away:

Put the SSH key you sent us in ~/.ssh/ on your local machine and you are good to go.

You are encouraged to improve the VM image to your needs, but remember to make a snapshot before major configuration changes.


The source code repository has been moved to Github recently: The old repository can still be found at

The most relevant branch for contributors is development, so clone a working copy using:

git clone -b development ./dash-development

All code modifications are first committed to branches. There are three types of branches:

Feature branches
For test implementations, new features and interface changes. Feature branch names have prefix feat-.
Bug branches
For test cases used to reproduce bugs and bug fixes. Bug branch names have prefix bug-.
Documentation branches
For Doxygen and user guide documentation. Documentation branch names have prefix doc- and do not trigger CI.
Support branches
For changes in the CI and build system, helper scripts and the like. Support branch names have prefix sup-.

First, configure git in your work environment. We recommend:

$ git config --global "Your Name"
$ git config --global ""
$ git config --global color.ui true         # use color in git output to terminal
$ git config --global push.default simple   # or 'tracking' with older versions of git
$ git config --global rerere.enabled 1      # auto resolve of same conflicts on rebase!

When working on any code-related task, you work in a dedicated branch. For example, when starting your work on a feature named “Turing Test”:

(~) git clone -b development ./dash-feat-turing-test
(~) cd dash-feat-turing-test
(~/dash-feat-turing-test) git checkout -b feat-turing-test
# The working copy operates on a dedicated feature branch now.
# Start implement the feature and commit your changes by pushing to the feature branch:
(~/dash-feat-turing-test) git push origin feat-turing-test

When the changes in your branch are stable, merge them back to development:

# Clone the development branch if not already done:
(~) git clone -b development ./dash-development
# In the development branch, apply the changes in your feature branch:
(~) cd dash-development
(~/dash-development) git pull origin feat-turing-test
(~/dash-development) git push origin development

To keep your branch in sync with development simply do:

(~) cd dash-feat-turing-test
(~) git merge development

There are more elegant ways to merge branches, but this variant always works and we don’t care too much about a streamlined git history for now.

Also, we don’t really follow an industry-style development procedure. We don’t need to and we don’t want to, because procedure bureaucracy just distracts from actual work.

Note: Good software developers don’t need software operation procedures. Know that agile methods are favored in the industry because they get average results from bad (ergo: cheap) coders, but they cripple the workflow of good software engineers (like us). Yes, seriously.

But we have some minimalistic workflow guidelines to coordinate our work. These are explained in the DASH Readme.

Getting Familiar with the DASH Code Base

The DASH code base might seem overwhelming if you never worked on a library project, but DASH is actually a pretty small library.

There is a Doxygen API documentation of DASH, but its audience are rather experienced DASH coders that already know what to look for.

So let’s just have a look at the code itself.

DASH Examples

Forget about the DART implementation (dart-if, dart-impl) for now, you don’t need them yet.

It’s probably a good idea to start from the user’s perspective:

How is DASH used to implement HPC applications?

Start by browsing the DASH example applications in dash/examples.

The example applications illustrate common use cases of DASH but don’t show all the gritty detail. Once you got familiar with the DASH style in the examples a bit, you can dive deeper and study the DASH unit tests.

A unit test implementation is not exactly self-explanatory, but it shows the full usage and peculiarities of a specific DASH type. For example, dash/test/ illustrates all available allocation methods provided by dash::Array.

DASH Implementation

Most of DASH is implemented in headers. Once you are somewhat familiar with the examples and some unit tests and want to know how the containers and algorithms are actually implemented, browse the headers in dash/include/dash/.

To get familiar with the DASH library code, start with the basic stuff: a one-dimensional, static container, namely dash::Array, and the algorithms using it.

The relevant sources are:

  • dash/include/dash/Array.h: The DASH distributed array implementation
  • dash/include/dash/algorithm: Implementation of DASH algorithms like dash::min_element, dash::for_each, etc.

Ignore the implementation of dash::GlobIter, dash::GlobPtr etc. for now.
They are iterators / pointers that can point to elements in memory at remote machines. That’s all you need to know for now.

Study the implementation of dash::Array and dash::for_each a bit. You will be delighted how simple things actually are. Well, at least for the most primitive data structure and the most basic algorithm.

Log Analysis

Coloring log messages by unit

Coloring log messages by unit

  # For 16 color terminals:
  COL_NORM="$(tput setaf 9)"
  COL_0="$(tput setaf 2)"
  COL_1="$(tput setaf 3)"
  COL_2="$(tput setaf 5)"
  COL_3="$(tput setaf 6)"
  # For 256 color terminals:
  # print color code map for reference:
  # for code in {0..255}; do
  #   echo -e "\e[38;05;${code}m $code: Test";
  # done
  sed "s/^\[    0 \(.\+\)\?$/${COL_0}\[    0 \1 ${COL_NORM}/" | \
  sed "s/^\[    1 \(.\+\)\?$/${COL_1}\[    1 \1 ${COL_NORM}/" | \
  sed "s/^\[    2 \(.\+\)\?$/${COL_2}\[    2 \1 ${COL_NORM}/" | \
  sed "s/^\[    3 \(.\+\)\?$/${COL_3}\[    3 \1 ${COL_NORM}/"

Log output of DASH is formatted to simplify processing with standard command line tools like awk, sed and grep. This shell function can be used to print log messages of up to 4 units in different colors:


Bugs in the DASH code are extremely rare of course as we conduct a formal proof of every line of code before committing it. In the unlikely event of unexpected behaviour, we describe a practicable way to debug your code using gdb in the following (via this article, a digest of this tutorial).

For a fun and impressive demo of gdb we recommend to watch this talk.

Build DASH in Debug mode:

$ ./

or by setting the flag -DCMAKE_BUILD_TYPE=Debug for the cmake build manually.

Insert the following code to make it wait for gdb attachment during runtime:

int DebugWait = 1;
while (DebugWait);

Launch the parallel program using mpiexec, make sure you know those computers on which processes are running.

Login to a computer, use the following bash commands to get a process id, and start gdb to attach to this process:

ps aux | grep program_name
gdb program_name process_id

First, enter <CTRL-X 1> or <CTRL-X 2> for a nice text-based GUI. As an alternative to gdbtui, we highly recommend the GDB Dashboard

The debugger will be stuck in the infinite loop, as intended.

Use the following gdb commands to break the waiting and continue executing the process.

<gdb> set DebugWait = 0
<gdb> continue

You might want to set a break point first to prevent the program from running to completion.

<gdb> b 123     # sets breakpoint at line 123 in current file
<gdb> n         # short for 'next', steps through program by single operations
<gdb> c         # short for 'continue', continues regular program run, will halt at next break point
<gdb> print foo # show current value of variable 'foo'

There are many settings for printing variables and data.

DASH Built-In Debugging Helpers

To let all processes enter a blocking loop in dash::init, right after the communication backend has been initialized, set the environment variable DASH_INIT_BREAKPOINT like:

$ DASH_INIT_BREAKPOINT=on mpirun -n 2 ./bin/dash-test-mpi --gtest_filter=MyTest.TheTestCase

Now, attach to one of the MPI processes in gdb and continue at dash::init:

(gdb) break dash::internal::wait_breakpoint()
(gdb) continue # should halt at blocking loop
(gdb) call "blockvar.set(0)"