Skip to content

Launch policies for algorithms #272

@devreal

Description

@devreal

At the moment, accesses to global memory of DASH containers through subscript operators are serialized using blocking put operations (dart_put_blocking) in GlobRef, potentially resulting in poor performance for naive users. I understand that this is useful to avoid synchronization problems in the general case, but allowing asynchronous access through subscript operators would provide an easy way of improving performance of remote access to multiple elements while using the same (simple to use) interface.

Hence, we could introduce asynchronous epochs, which have an explicit begin and end, e.g.,

dash::Array<int> arr(100);
arr.start_epoch();
if (dash::myid() == 0) {
  for (int i = 0; i < arr.size(); ++i) {
    arr[i] = i; // not immediately visible
  }
}
arr.end_epoch(); 
// all changes visible now

Within an epoch, GlobRef will issue non-blocking put (+get?) operations and data transfers are guaranteed to complete only after the end of the epoch.

Note: I am aware of dash::put_value_async in OneSided but I think it's not intuitively usable and quite wordy, e.g.,

dash::Array<int> arr(100);
if (dash::myid() == 0) {
  for (int i = 0; i < arr.size(); ++i) {
    dash::put_value_async(i, arr[i]); // is this correct?
  }
}
dash::fence(arr[0]); // sufficient to fence on the first element?
// all changes visible now

The documentation is not clear on the use of these methods and there are no tests/examples either. I also cannot efficiently mix dash::put_value_async and operator[] because the latter causes a flush. The idea of asynchronous epochs closely resembles the model employed by MPI-RMA (MPI_Put and MPI_Flush), just with a nicer interface :)

Please let me know what you think. Maybe there have been discussions about it earlier?

Metadata

Metadata

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions