Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Example for running MPI for ReplicaExchangeSampler? #472

Open
mrshirts opened this issue May 28, 2020 · 4 comments
Open

Example for running MPI for ReplicaExchangeSampler? #472

mrshirts opened this issue May 28, 2020 · 4 comments

Comments

@mrshirts
Copy link

mrshirts commented May 28, 2020

Is there a good example for a script that parallelizes the runs generated by ReplicaExchangeSampler over cores via MPI (or other means)? Note that are systems are small enough (<100 particles) that we don't appear to be getting that much performance benefit of GPU over CPU. We are having a hard time getting this work, and we didn't find anything in the official documentation.

Thanks!

@andrrizzi
Copy link
Contributor

ReplicaExchangeSampler should have automatic detection of MPI and it should parallelize the replicas over MPI processes accordingly. So it should be possible to just run mpirun [mpirun_options] my_script_with_rex.py.

To make it run over CPU rather than GPU, you should be able t set the platform in the ContextCache object used to generate the OpenMM Contexts. By default, ReplicaExchange uses a global context cache that you configure at the beginning of your script like this

from openmmtools.cache import global_context_cache
global_context_cache.platform = openmm.getPlatform('CPU')

Note however this bug: #449. This is giving us a hard time, and I am short on dev time right now, unfortunately.

@mrshirts
Copy link
Author

Thanks so much for the info! I suspect we will either be running on 1 GPU at a time, or parallelized on CPU's, so we hopefully will be OK.

@andrrizzi
Copy link
Contributor

Parallelizing over CPU might indeed work since I don't think we have tested the bug on CPUs. So far, however, the problem seems to be with MPI rather than the GPU platform in particular.

@mrshirts
Copy link
Author

mrshirts commented Jul 6, 2020

BTW, it's working, but we are still sorting out how to optimize what it does.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants