-
Notifications
You must be signed in to change notification settings - Fork 45
Description
At the moment, accesses to global memory of DASH containers through subscript operators are serialized using blocking put operations (dart_put_blocking
) in GlobRef
, potentially resulting in poor performance for naive users. I understand that this is useful to avoid synchronization problems in the general case, but allowing asynchronous access through subscript operators would provide an easy way of improving performance of remote access to multiple elements while using the same (simple to use) interface.
Hence, we could introduce asynchronous epochs, which have an explicit begin and end, e.g.,
dash::Array<int> arr(100);
arr.start_epoch();
if (dash::myid() == 0) {
for (int i = 0; i < arr.size(); ++i) {
arr[i] = i; // not immediately visible
}
}
arr.end_epoch();
// all changes visible now
Within an epoch, GlobRef
will issue non-blocking put
(+get
?) operations and data transfers are guaranteed to complete only after the end of the epoch.
Note: I am aware of dash::put_value_async
in OneSided
but I think it's not intuitively usable and quite wordy, e.g.,
dash::Array<int> arr(100);
if (dash::myid() == 0) {
for (int i = 0; i < arr.size(); ++i) {
dash::put_value_async(i, arr[i]); // is this correct?
}
}
dash::fence(arr[0]); // sufficient to fence on the first element?
// all changes visible now
The documentation is not clear on the use of these methods and there are no tests/examples either. I also cannot efficiently mix dash::put_value_async
and operator[]
because the latter causes a flush. The idea of asynchronous epochs closely resembles the model employed by MPI-RMA (MPI_Put
and MPI_Flush
), just with a nicer interface :)
Please let me know what you think. Maybe there have been discussions about it earlier?