Code

Issue 46 agent start timeout#7

Code

Open

pkopta wants to merge 131 commits intocompat-project:mastercompat-project/QCG-PilotJob:masterfrom psnc-qcg:issue_46_agent_start_timeoutpsnc-qcg/QCG-PilotJob:issue_46_agent_start_timeoutCopy head branch name to clipboard

Commits

Commits on Jan 22, 2019

• replaced SLURM_TASKS_PER_NODE with SLURM_JOB_CPUS_PER_NODE env variable as description of available cpus in allocation

  pkopta

  committed
  8c762cf

  Copy full SHA for 8c762cf

Commits on Jan 23, 2019

• command line script for starting service

  pkopta

  committed
  e5bf038

  Copy full SHA for e5bf038

• Merge pull request #2 from vecma-project/issue_1_slurm_resources

  Show description for 46a59e4
  pkopta

  authored
  46a59e4

  Copy full SHA for 46a59e4

Commits on Jan 24, 2019

• tracking order of appending job's to list; sending job's to manager in order they were appended

  pkopta

  committed
  afcd6ea

  Copy full SHA for afcd6ea

• bugfix

  pkopta

  committed
  9fa7bf3

  Copy full SHA for 9fa7bf3

• qcg.appscheduler.api included in package list; fixed url to github

  pkopta

  committed
  c105c22

  Copy full SHA for c105c22

• fixed doc strings

  pkopta

  committed
  bc66c91

  Copy full SHA for bc66c91

• bugfix

  pkopta

  committed
  0753c86

  Copy full SHA for 0753c86

• conversion of string 'after' argument to list

  pkopta

  committed
  8b5b931

  Copy full SHA for 8b5b931

• more descriptive exception message

  pkopta

  committed
  d524a20

  Copy full SHA for d524a20

• conversion of job arguments to string

  pkopta

  committed
  5e7108e

  Copy full SHA for 5e7108e

• Merge pull request #5 from vecma-project/issue_3_workflow_api_deps

  Show description for b66bab8
  pkopta

  authored
  b66bab8

  Copy full SHA for b66bab8

• Merge pull request #8 from vecma-project/issue_7_api_in_def_package

  Show description for 1b5ab16
  pkopta

  authored
  1b5ab16

  Copy full SHA for 1b5ab16

Commits on Mar 15, 2019

• fixed comments

  pkopta

  committed
  249c7d7

  Copy full SHA for 249c7d7

• rewrited configuration

  pkopta

  committed
  7246ce3

  Copy full SHA for 7246ce3

• configurable execution environment

  pkopta

  committed
  3dab6d2

  Copy full SHA for 3dab6d2

• detecting slurm execution environment

  pkopta

  committed
  5cc3299

  Copy full SHA for 5cc3299

• local execution mode

  pkopta

  committed
  832aefe

  Copy full SHA for 832aefe

• fixed comments; rewrited configuration; configurable execution environment

  pkopta

  committed
  8309e07

  Copy full SHA for 8309e07

• dockerfile for local mode testing

  pkopta

  committed
  01cac1e

  Copy full SHA for 01cac1e

• docker compose files for local mode testing

  pkopta

  committed
  f4fdf0e

  Copy full SHA for f4fdf0e

• scripts for building docker and running compose files

  pkopta

  committed
  9eaa826

  Copy full SHA for 9eaa826

• example for date command

  pkopta

  committed
  527a124

  Copy full SHA for 527a124

• example for env (through bash) command

  pkopta

  committed
  93b6b7c

  Copy full SHA for 93b6b7c

• example for sleep command

  pkopta

  committed
  18e9be0

  Copy full SHA for 18e9be0

• requirements file

  pkopta

  committed
  cf6b53e

  Copy full SHA for cf6b53e

Commits on Mar 18, 2019

• ducumentation updated with LOCAL mode description

  pkopta

  committed
  4993909

  Copy full SHA for 4993909

• support for json job's report file format

  pkopta

  committed
  510b0d9

  Copy full SHA for 510b0d9

• fixed formating

  pkopta

  committed
  50d0abf

  Copy full SHA for 50d0abf

• moved module files to src subdirectory

  pkopta

  committed
  a577072

  Copy full SHA for a577072

• support for pytest test units

  pkopta

  committed
  432c685

  Copy full SHA for 432c685

• testunit tests moved to different directory

  pkopta

  committed
  df00c30

  Copy full SHA for df00c30

• Merge branch 'issue_11_fixing_local_mode' of https://github.com/compat-project/QCG-PilotJob into issue_fixing_local_mode

  pkopta

  committed
  d4e5eae

  Copy full SHA for d4e5eae

• merge

  pkopta

  committed
  d9afdf8

  Copy full SHA for d9afdf8

• installation documentation

  pkopta

  committed
  4d313f2

  Copy full SHA for 4d313f2

Commits on Mar 19, 2019

• validation check against job's with the same name in single submit request

  pkopta

  committed
  2f707da

  Copy full SHA for 2f707da

• fixed reporting runtime statistics in json format

  pkopta

  committed
  cc6791c

  Copy full SHA for cc6791c

• fixed closed event loop bug when service is stoped and started in the same process

  pkopta

  committed
  09c3d86

  Copy full SHA for 09c3d86

• additional tests of local mode

  pkopta

  committed
  1f5b7b5

  Copy full SHA for 1f5b7b5

• replaced SLURM_TASKS_PER_NODE with SLURM_JOB_CPUS_PER_NODE

  pkopta

  committed
  5231cde

  Copy full SHA for 5231cde

• typo

  pkopta

  committed
  b12a5e4

  Copy full SHA for b12a5e4

• instruction for installing pip & virtualenv

  pkopta

  committed
  b5638f3

  Copy full SHA for b5638f3

• Merge pull request #15 from vecma-project/issue_fixing_local_mode

  Show description for 916c3f3
  pkopta

  authored
  916c3f3

  Copy full SHA for 916c3f3

Commits on Mar 20, 2019

• handle missing job in dependencies

  pkopta

  committed
  415af45

  Copy full SHA for 415af45

• local mode workflow tests

  pkopta

  committed
  2d8828f

  Copy full SHA for 2d8828f

• travis description with some pytests

  pkopta

  committed
  1234d14

  Copy full SHA for 1234d14

• lowering load tests and relaxing time constraints

  pkopta

  committed
  a8b1759

  Copy full SHA for a8b1759

• debugging

  pkopta

  committed
  dfdcb0e

  Copy full SHA for dfdcb0e

• bugfix

  pkopta

  committed
  ca581c9

  Copy full SHA for ca581c9

• removed absolute paths from executables

  pkopta

  committed
  e61518d

  Copy full SHA for e61518d

• documentation updated with the service run command

  pkopta

  authored
  a470998

  Copy full SHA for a470998

Commits on Mar 26, 2019

• major scheduler optimization - schedule only when there are free resources; profile support

  pkopta

  committed
  9216307

  Copy full SHA for 9216307

• none reporter as an option

  pkopta

  committed
  e512979

  Copy full SHA for e512979

• support for loging configuration; profile support

  pkopta

  committed
  5ef9cd4

  Copy full SHA for 5ef9cd4

• profile support

  pkopta

  committed
  6264730

  Copy full SHA for 6264730

• basic statistical analysis of job's runtime based on json report file

  pkopta

  committed
  2134d77

  Copy full SHA for 2134d77

• large iteration job test

  pkopta

  committed
  30e5d54

  Copy full SHA for 30e5d54

• changed detection of profiler

  pkopta

  committed
  cedcffd

  Copy full SHA for cedcffd

• section about profiling

  pkopta

  committed
  85f8dd6

  Copy full SHA for 85f8dd6

Commits on Apr 9, 2019

• Merge pull request #23 from vecma-project/more_local_mode_tests

  Show description for c8ecb2d
  pkopta

  authored
  c8ecb2d

  Copy full SHA for c8ecb2d

Commits on Apr 17, 2019

• fix: names() -> keys()

  bartoszbosak

  committed
  80a9837

  Copy full SHA for 80a9837

Commits on Apr 24, 2019

• support for launching pilot job manager from the api

  pkopta

  committed
  010d5c4

  Copy full SHA for 010d5c4

• Merge branch 'master' into issue_18_start_manager_from_api

  pkopta

  authored
  56dbadd

  Copy full SHA for 56dbadd

• Merge pull request #25 from vecma-project/issue_18_start_manager_from_api

  Show description for 3c1d384
  pkopta

  authored
  3c1d384

  Copy full SHA for 3c1d384

Commits on May 8, 2019

• Updated README.md with the example run

  pkopta

  authored
  c0467a1

  Copy full SHA for c0467a1

Commits on May 16, 2019

• fixed regexp warnings

  pkopta

  committed
  5e90405

  Copy full SHA for 5e90405

• wait4all method test with localmanger

  pkopta

  committed
  cffe096

  Copy full SHA for cffe096

• Merge pull request #28 from vecma-project/issue_wait4all_localmanager_test

  Show description for 3cba717
  pkopta

  authored
  3cba717

  Copy full SHA for 3cba717

Commits on Jun 10, 2019

• version bumps

  pkopta

  committed
  e37ebde

  Copy full SHA for e37ebde

Commits on Jul 17, 2019

• assignment specific cores to jobs

  pkopta

  committed
  3e3d270

  Copy full SHA for 3e3d270

• bugfix

  pkopta

  committed
  7d57d77

  Copy full SHA for 7d57d77

• unnecessary file

  pkopta

  committed
  590b240

  Copy full SHA for 590b240

Commits on Sep 3, 2019

• reserve one of the core for the QCG-PJM

  pkopta

  committed
  c1e0465

  Copy full SHA for c1e0465

• bugfix

  pkopta

  committed
  5bdebef

  Copy full SHA for 5bdebef

Commits on Sep 4, 2019

• mechanism for launching single core jobs on remote resources; detailed tracing of job processing; log and reports files moved to separate directory

  pkopta

  committed
  5b0209a

  Copy full SHA for 5b0209a

• bugfix - not creating stdout & stderr files in local mode

  pkopta

  committed
  eaff2ff

  Copy full SHA for eaff2ff

• bugfix - changed location of report file

  pkopta

  committed
  2eccc1d

  Copy full SHA for 2eccc1d

• allocation of core for QCG-PJM as an configurable option (false by default)

  pkopta

  committed
  56fb292

  Copy full SHA for 56fb292

• moved default location of API log file to the .qcgpjm subdirectory

  pkopta

  committed
  a285d03

  Copy full SHA for a285d03

Commits on Sep 5, 2019

• bugfix - missing launcher package during installation

  pkopta

  committed
  1ff9dc7

  Copy full SHA for 1ff9dc7

• support for ZMQ interface dynamic port binding

  pkopta

  committed
  55a023e

  Copy full SHA for 55a023e

• code reformat

  pkopta

  committed
  c7f72b7

  Copy full SHA for c7f72b7

• Merge branch 'issue_33_node_launcher' of https://github.com/vecma-project/QCG-PilotJob into issue_33_node_launcher

  pkopta

  committed
  39bf440

  Copy full SHA for 39bf440

• bugfix: related to the random port binding

  pkopta

  committed
  2b785f1

  Copy full SHA for 2b785f1

Commits on Sep 6, 2019

• support for handling slurm CPU binding

  pkopta

  committed
  4f7714e

  Copy full SHA for 4f7714e

• Merge branch 'issue_33_node_launcher' of https://github.com/vecma-project/QCG-PilotJob into issue_33_node_launcher

  pkopta

  committed
  863c306

  Copy full SHA for 863c306

Commits on Sep 9, 2019

• a new resources parameter that force resource information source

  pkopta

  committed
  31357b6

  Copy full SHA for 31357b6

• fixed bug with node launcher agent's working directory

  pkopta

  committed
  67cd1ae

  Copy full SHA for 67cd1ae

• removed unnecessary imports

  pkopta

  committed
  95aec2a

  Copy full SHA for 95aec2a

• introduced resource type (local/slurm), replaces execution schema

  pkopta

  committed
  79f0b0b

  Copy full SHA for 79f0b0b

• execution schema is not configurable and depends on resources type

  pkopta

  committed
  e0c668f

  Copy full SHA for e0c668f

• removed execution schema as configurable option

  pkopta

  committed
  38940d2

  Copy full SHA for 38940d2

• '--file' argument is implied if '--file-path' is used

  pkopta

  committed
  81ff434

  Copy full SHA for 81ff434

Commits on Sep 10, 2019

• version 0.5

  pkopta

  committed
  593b367

  Copy full SHA for 593b367

• documentation updated

  pkopta

  committed
  a5e5175

  Copy full SHA for a5e5175

• automatically enable '--net' when '--net-port' or port range is used

  pkopta

  committed
  8914e95

  Copy full SHA for 8914e95

• bugfix: stderr file not created for jobs executed via node launcher

  pkopta

  committed
  f83cf4a

  Copy full SHA for f83cf4a

• extended timeout waiting in localmanager for service start

  pkopta

  committed
  7fcd850

  Copy full SHA for 7fcd850

Commits on Sep 11, 2019

• more verbose exception handling

  pkopta

  committed
  dc3ebe8

  Copy full SHA for dc3ebe8

• bugfix: the proper exit function

  pkopta

  committed
  9733a41

  Copy full SHA for 9733a41

• job's environment generation configuration - don't create a slurm host file when its not necessary

  pkopta

  committed
  66c2f3c

  Copy full SHA for 66c2f3c

• bugfix: environment generation options not visible in child objects

  pkopta

  committed
  5d4b438

  Copy full SHA for 5d4b438

• documentation updated with the 'performance issues' section

  pkopta

  committed
  1980060

  Copy full SHA for 1980060

Commits on Sep 12, 2019

• Merge pull request #35 from vecma-project/issue_33_node_launcher

  Show description for 6da4f82
  bartoszbosak

  authored
  6da4f82

  Copy full SHA for 6da4f82

• support for modules and pyton's virtual env in jobs

  pkopta

  committed
  805ad5b

  Copy full SHA for 805ad5b

Commits on Sep 20, 2019

• example scripts for modules and virtual environment; cleanup of pyc files

  pkopta

  committed
  b315c3a

  Copy full SHA for b315c3a

• write network interface address to the file in .qcgpjm directory

  pkopta

  committed
  4d93a82

  Copy full SHA for 4d93a82

• updated packages - mainly prompt_toolkit; updated pjm interface

  pkopta

  committed
  28d26bc

  Copy full SHA for 28d26bc

• the status request - returns the global state of service

  pkopta

  committed
  10c1a61

  Copy full SHA for 10c1a61

• new command line client - for batch jobs

  pkopta

  committed
  4e17fb7

  Copy full SHA for 4e17fb7

• status request

  pkopta

  committed
  076d15d

  Copy full SHA for 076d15d

• client bugfixes

  pkopta

  committed
  b48860a

  Copy full SHA for b48860a

• status request bugfixes

  pkopta

  committed
  b960ffc

  Copy full SHA for b960ffc

Commits on Sep 21, 2019

• list->status command; log file in .qcgpjm

  pkopta

  committed
  01f1404

  Copy full SHA for 01f1404

• executable command for the client in setup

  pkopta

  committed
  7a44b7e

  Copy full SHA for 7a44b7e

• default valueu for zmq_address in manager

  pkopta

  committed
  9b71069

  Copy full SHA for 9b71069

• bugfix - error in file interface when zmq_address not initialized

  pkopta

  committed
  b2d2b3d

  Copy full SHA for b2d2b3d

• bugfix: click setup integration

  pkopta

  committed
  1a5da3c

  Copy full SHA for 1a5da3c

• bugfix: function name

  pkopta

  committed
  8db21fe

  Copy full SHA for 8db21fe

• bugfixes

  pkopta

  committed
  5c9e43e

  Copy full SHA for 5c9e43e

Commits on Sep 22, 2019

• changed names in status response - (upper letter)

  pkopta

  committed
  0a08326

  Copy full SHA for 0a08326

Commits on Sep 24, 2019

• bugfix: wrong calculation of finished jobs

  pkopta

  committed
  177d405

  Copy full SHA for 177d405

Commits on Sep 25, 2019

• write the status at the end of service runtime; client will read this status when connection to the service timeout

  pkopta

  committed
  cca4e4f

  Copy full SHA for cca4e4f

Commits on Sep 27, 2019

• optionally cpu binding - only when information about specific cores is avaialble; local execution code rewrite

  pkopta

  committed
  f7d0f0a

  Copy full SHA for f7d0f0a

Commits on Oct 4, 2019

• bugfix

  pkopta

  committed
  e08fcb8

  Copy full SHA for e08fcb8

Commits on Oct 30, 2019

• explicitly close zmq sockets

  pkopta

  committed
  f9bfb87

  Copy full SHA for f9bfb87

Commits on Dec 12, 2019

• Merge pull request #44 from vecma-project/issue_37_monitoring

  Show description for d1f10c6
  pkopta

  authored
  d1f10c6

  Copy full SHA for d1f10c6

Commits on Dec 18, 2019

• version update

  pkopta

  committed
  56e3543

  Copy full SHA for 56e3543

• Merge branch 'master' of https://github.com/vecma-project/QCG-PilotJob

  pkopta

  committed
  c1370ab

  Copy full SHA for c1370ab

Commits on Dec 19, 2019

• updated documentation

  pkopta

  committed
  03a5954

  Copy full SHA for 03a5954

Commits on Feb 12, 2020

• increased timeout to 60s for agents start

  pkopta

  committed
  bed4370

  Copy full SHA for bed4370