MPI shared memory

[hmenke]

All tests were run with multiple nodes and different number of slots on each node:

$ cat hostfile 
pcscqm04 slots=4
pcscqm05 slots=3
pcscqm06 slots=5
pcscqm07 slots=2
$ mpirun -hostfile ./hostfile build/test/c++/mpi_window

Some ideas:

• MPI allocator

  Similar to std::allocator implement a shared_allocator and a distributed_shared_allocator, such that one can use e.g. std::vector<double, mpi::shared_allocator<double>>

  Questions:

  • Should this be in mpi or in nda?
  • For shared memory one can probably use a best guess and use split_shared() on the default communicator, but for distributed shared memory there needs to be internode communication and that is not easily guessed from the default communicator. Maybe a global hash table with allocation information is needed?
  • For shared memory race conditions must be prevented somehow.
  • On top of that, for distributed shared memory access must be fenced and broadcasted between nodes. That's not so easy to abstract away.

[hmenke]

Can we make this non-MPI compatible?

[hmenke]

@Wentzell LGTM, only minor nitpick.

[Wentzell]

Thank you @hmenke for this massive feature upgrade! The PR was already in great condition. I have iterated once on it, and from my side it is now ready for merge. Please confirm once the two most recent changes in e45281f and c40fbee

[Wentzell]

Thank you also @Thoemi09 for your improvements

[hmenke]

LGTM, but could you please not squash on merge? This makes it much easier to bisect later if something is wrong. You can clean up the history and rebase on unstable if you want.

[Wentzell]

LGTM, but could you please not squash on merge? This makes it much easier to bisect later if something is wrong. You can clean up the history and rebase on unstable if you want.

I would have been ok with a full squash, but if you want group things into a few commits along the way, feel free to create a suggested a grouping.