Skip to content

Commit

Permalink
[AIRFLOW-3502] Update config template to reflect supporting different…
Browse files Browse the repository at this point in the history
… Celery pool implementation (apache#5477)

The support to different Celery pool implementation has been added
in apache#4308.

But it's not reflected in the default_airflow.cfg yet, while it's
the main portal of config options to most users.
  • Loading branch information
XD-DENG authored and ashb committed Jun 26, 2019
1 parent 8481bb7 commit dd08ae3
Show file tree
Hide file tree
Showing 2 changed files with 18 additions and 0 deletions.
11 changes: 11 additions & 0 deletions UPDATING.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,17 @@ assists users migrating to a new version.
## Airflow Master


### `pool` config option in Celery section to support different Celery pool implementation

The new `pool` config option allows users to choose different pool
implementation. Default value is "prefork", while choices include "prefork" (default),
"eventlet", "gevent" or "solo". This may help users achieve better concurrency performance
in different scenarios.

For more details about Celery pool implementation, please refer to:
- https://docs.celeryproject.org/en/latest/userguide/workers.html#concurrency
- https://docs.celeryproject.org/en/latest/userguide/concurrency/eventlet.html

### Removal of `non_pooled_task_slot_count` and `non_pooled_backfill_task_slot_count`

`non_pooled_task_slot_count` and `non_pooled_backfill_task_slot_count`
Expand Down
7 changes: 7 additions & 0 deletions airflow/config_templates/default_airflow.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -417,6 +417,13 @@ ssl_key =
ssl_cert =
ssl_cacert =

# Celery Pool implementation.
# Choices include: prefork (default), eventlet, gevent or solo.
# See:
# https://docs.celeryproject.org/en/latest/userguide/workers.html#concurrency
# https://docs.celeryproject.org/en/latest/userguide/concurrency/eventlet.html
pool = prefork

[celery_broker_transport_options]
# This section is for specifying options which can be passed to the
# underlying celery broker transport. See:
Expand Down

0 comments on commit dd08ae3

Please sign in to comment.