Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Issue 85 spawn #8

Open
wants to merge 327 commits into
base: master
Choose a base branch
from
Open

Issue 85 spawn #8

wants to merge 327 commits into from

Conversation

pkopta
Copy link
Collaborator

@pkopta pkopta commented Nov 3, 2020

The start method for multiprocessing python's package is set to 'fork' only when not initialized before. For some environments, like Jupyter the 'spawn' method might be more usable, but such settings should be done in block protected with 'if name == "main"' as documentation suggest (with 'spawn' method the initialization block will be called also for spawned process but with different 'name'). So different settings must be done in scripts that uses QCG-PJ LocalManager class.

pkopta and others added 30 commits September 4, 2019 16:56
@pkopta
allocation of core for QCG-PJM as an configurable option (false by de…
56fb292 
…fault)
@pkopta
moved default location of API log file to the .qcgpjm subdirectory
a285d03
@pkopta
bugfix - missing launcher package during installation
1ff9dc7
@pkopta
support for ZMQ interface dynamic port binding
55a023e
@pkopta
code reformat
c7f72b7
@pkopta
Merge branch 'issue_33_node_launcher' of https://github.com/vecma-pro…
39bf440 
…ject/QCG-PilotJob into issue_33_node_launcher
@pkopta
bugfix: related to the random port binding
2b785f1
@pkopta
support for handling slurm CPU binding
4f7714e
@pkopta
Merge branch 'issue_33_node_launcher' of https://github.com/vecma-pro…
863c306 
…ject/QCG-PilotJob into issue_33_node_launcher
@pkopta
a new resources parameter that force resource information source
31357b6
@pkopta
fixed bug with node launcher agent's working directory
67cd1ae
@pkopta
removed unnecessary imports
95aec2a
@pkopta
introduced resource type (local/slurm), replaces execution schema
79f0b0b
@pkopta
execution schema is not configurable and depends on resources type
e0c668f
@pkopta
removed execution schema as configurable option
38940d2
@pkopta
'--file' argument is implied if '--file-path' is used
81ff434
@pkopta
version 0.5
593b367
@pkopta
documentation updated
a5e5175
@pkopta
automatically enable '--net' when '--net-port' or port range is used
8914e95
@pkopta
bugfix: stderr file not created for jobs executed via node launcher
f83cf4a
@pkopta
extended timeout waiting in localmanager for service start
7fcd850
@pkopta
more verbose exception handling
dc3ebe8
@pkopta
bugfix: the proper exit function
9733a41
@pkopta
job's environment generation configuration - don't create a slurm hos…
66c2f3c 
…t file when its not necessary
@pkopta
bugfix: environment generation options not visible in child objects
5d4b438
@pkopta
documentation updated with the 'performance issues' section
1980060
@bartoszbosak
Merge pull request #35 from vecma-project/issue_33_node_launcher
6da4f82 
Issue 33 node launcher
@pkopta
support for modules and pyton's virtual env in jobs
805ad5b
@pkopta
example scripts for modules and virtual environment; cleanup of pyc f…
b315c3a 
…iles
@pkopta
write network interface address to the file in .qcgpjm directory
4d93a82
pkopta and others added 30 commits September 7, 2020 10:54
@pkopta
support for hyper-threading configurations
bdc3ba9
@pkopta
support for launching mpi with srun - srunmpi execution model
f42216d
@pkopta
docstring update
47da927
@pkopta
creating sandbox moved outside core preprocess method
3a57a94
@pkopta
tests updated due to the change of node's core list type from int to …
7424259 
…string
@pkopta
temporary fix for governor tests
ab96bb1
@pkopta
hostname in output
27e0274
@pkopta
documentation about execution models
ed4f09a
@pkopta
removed default impi modules - valid only for eagle
4d5134d
@pkopta
bugfix: not passing application arguments for intelmpi model
27280cc
@pkopta
creating env.log with qcg-pj environment for debugging purposes
c060396
@pkopta
added 'mem-per-cpu=0' slurm parameter as without it on some systems t…
af046fd 
…he jobs has stalled in slurm
@pkopta
loading modules and virtual envs from login bash
5604a19
@pkopta
added slurm's 'mem-per-cpu=0' for IntelMPI hydra proxy starting as wi…
b6d78ae 
…thout it on some systems intel's mpirun has stalled
@pkopta
Merge pull request #76 from vecma-project/issue_75_partial_nodes
ac38c37 
documentation about partial node allocations
@pkopta
Merge pull request #78 from vecma-project/issue_77_intelmpi
629f047 
Issue 77 intelmpi
@bartoszbosak
Fix: branch name - devel -> develop
c7913ef
@pkopta
0.9 rc1
9f6fac8
@pkopta
Merge pull request #79 from vecma-project/release_0.9rc1
ba7ded6 
0.9 rc1
@pkopta
version bumped to 0.9.0
1635a1b
@pkopta
Merge pull request #80 from vecma-project/release_0.9
7725096 
version bumped to 0.9.0
@pkopta
Merge branch 'develop' into issue_58_jupyter
664f64c
@pkopta
nest_asyncio for jupyter notebooks
bb39a96
@pkopta
Revert "nest_asyncio for jupyter notebooks"
5f0d876 
This reverts commit bb39a96.
@pkopta
changed method for launching new processes (pilotjob in the background)
1fd046d
@pkopta
nest_asyncio for jupyter notebooks
05f4e74
@pkopta
always change forking method
4ca9e77
@pkopta
forced change mp model
42f4840
@pkopta
removed nest_asyncio - testing jupyter
fb579ed
@pkopta
the mp start method is set to fork only when not initialized before
0285842
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@pkopta @bartoszbosak