Skip to content

Timeout setting for Opensearch and Elasticsearch #408

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 2, 2025
Merged

Timeout setting for Opensearch and Elasticsearch #408

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
4bac5b2
Timeout setting for Opensearch and Elasticsearch
z-mrozu Jun 26, 2025
e212509
changelog update
z-mrozu Jun 26, 2025
662f0bb
Elasticsearch request_timeout param instead of timeout
z-mrozu Jun 26, 2025
a7bb668
Added ES_TIMEOUT to configuration reference in readme and cleaned up …
z-mrozu Jun 27, 2025
fb2c57f
Configuration reference timeout update
z-mrozu Jul 1, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.

## [Unreleased]

### Added

- Added the ability to set timeout for Opensearch and Elasticsearch clients by setting the environmental variable `ES_TIMEOUT` [#408](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/408)

## [v6.0.0] - 2025-06-22

### Added
Expand Down
17 changes: 9 additions & 8 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -207,26 +207,27 @@ You can customize additional settings in your `.env` file:
| `ES_PORT` | Port for Elasticsearch/OpenSearch. | `9200` (ES) / `9202` (OS)| Optional |
| `ES_USE_SSL` | Use SSL for connecting to Elasticsearch/OpenSearch. | `false` | Optional |
| `ES_VERIFY_CERTS` | Verify SSL certificates when connecting. | `false` | Optional |
| `ES_TIMEOUT` | Client timeout for Elasticsearch/OpenSearch. | DB client default | Optional |
| `STAC_FASTAPI_TITLE` | Title of the API in the documentation. | `stac-fastapi-<backend>` | Optional |
| `STAC_FASTAPI_DESCRIPTION` | Description of the API in the documentation. | N/A | Optional |
| `STAC_FASTAPI_VERSION` | API version. | `2.1` | Optional |
| `STAC_FASTAPI_LANDING_PAGE_ID` | Landing page ID | `stac-fastapi` | Optional |
| `STAC_FASTAPI_LANDING_PAGE_ID` | Landing page ID | `stac-fastapi` | Optional |
| `APP_HOST` | Server bind address. | `0.0.0.0` | Optional |
| `APP_PORT` | Server port. | `8080` | Optional |
| `ENVIRONMENT` | Runtime environment. | `local` | Optional |
| `WEB_CONCURRENCY` | Number of worker processes. | `10` | Optional |
| `RELOAD` | Enable auto-reload for development. | `true` | Optional |
| `STAC_FASTAPI_RATE_LIMIT` | API rate limit per client. | `200/minute` | Optional |
| `BACKEND` | Tests-related variable | `elasticsearch` or `opensearch` based on the backend | Optional |
| `ELASTICSEARCH_VERSION` | Version of Elasticsearch to use. | `8.11.0` | Optional | |
| `OPENSEARCH_VERSION` | OpenSearch version | `2.11.1` | Optional
| `ENABLE_DIRECT_RESPONSE` | Enable direct response for maximum performance (disables all FastAPI dependencies, including authentication, custom status codes, and validation) | `false` | Optional
| `RAISE_ON_BULK_ERROR` | Controls whether bulk insert operations raise exceptions on errors. If set to `true`, the operation will stop and raise an exception when an error occurs. If set to `false`, errors will be logged, and the operation will continue. **Note:** STAC Item and ItemCollection validation errors will always raise, regardless of this flag. | `false` Optional |
| `DATABASE_REFRESH` | Controls whether database operations refresh the index immediately after changes. If set to `true`, changes will be immediately searchable. If set to `false`, changes may not be immediately visible but can improve performance for bulk operations. If set to `wait_for`, changes will wait for the next refresh cycle to become visible. | `false` | Optional |
| `BACKEND` | Tests-related variable | `elasticsearch` or `opensearch` based on the backend | Optional |
| `ELASTICSEARCH_VERSION` | Version of Elasticsearch to use. | `8.11.0` | Optional |
| `OPENSEARCH_VERSION` | OpenSearch version | `2.11.1` | Optional |
| `ENABLE_DIRECT_RESPONSE` | Enable direct response for maximum performance (disables all FastAPI dependencies, including authentication, custom status codes, and validation) | `false` | Optional |
| `RAISE_ON_BULK_ERROR` | Controls whether bulk insert operations raise exceptions on errors. If set to `true`, the operation will stop and raise an exception when an error occurs. If set to `false`, errors will be logged, and the operation will continue. **Note:** STAC Item and ItemCollection validation errors will always raise, regardless of this flag. | `false` | Optional |
| `DATABASE_REFRESH` | Controls whether database operations refresh the index immediately after changes. If set to `true`, changes will be immediately searchable. If set to `false`, changes may not be immediately visible but can improve performance for bulk operations. If set to `wait_for`, changes will wait for the next refresh cycle to become visible. | `false` | Optional |
| `ENABLE_TRANSACTIONS_EXTENSIONS` | Enables or disables the Transactions and Bulk Transactions API extensions. If set to `false`, the POST `/collections` route and related transaction endpoints (including bulk transaction operations) will be unavailable in the API. This is useful for deployments where mutating the catalog via the API should be prevented. | `true` | Optional |

> [!NOTE]
> The variables `ES_HOST`, `ES_PORT`, `ES_USE_SSL`, and `ES_VERIFY_CERTS` apply to both Elasticsearch and OpenSearch backends, so there is no need to rename the key names to `OS_` even if you're using OpenSearch.
> The variables `ES_HOST`, `ES_PORT`, `ES_USE_SSL`, `ES_VERIFY_CERTS` and `ES_TIMEOUT` apply to both Elasticsearch and OpenSearch backends, so there is no need to rename the key names to `OS_` even if you're using OpenSearch.

## Interacting with the API

Expand Down
4 changes: 4 additions & 0 deletions stac_fastapi/elasticsearch/stac_fastapi/elasticsearch/config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,10 @@ def _es_config() -> Dict[str, Any]:
if (u := os.getenv("ES_USER")) and (p := os.getenv("ES_PASS")):
config["http_auth"] = (u, p)

# Include timeout setting if set
if request_timeout := os.getenv("ES_TIMEOUT"):
config["request_timeout"] = request_timeout

# Explicitly exclude SSL settings when not using SSL
if not use_ssl:
return config
Expand Down
4 changes: 4 additions & 0 deletions stac_fastapi/opensearch/stac_fastapi/opensearch/config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,10 @@ def _es_config() -> Dict[str, Any]:

config["headers"] = headers

# Include timeout setting if set
if timeout := os.getenv("ES_TIMEOUT"):
config["timeout"] = timeout
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this right for Opensearch? I am not sure.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be, from OpenSearch Client parameters:
"kwargs (Any) – any additional arguments will be passed on to the Transport class and, subsequently, to the Connection instances."
and then Connection has the timeout parameter:
"timeout (int) – default timeout in seconds (float, default: 10)"

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for looking into this, I was going through the opensearch-py code a little and this makes sense for sure.


# Explicitly exclude SSL settings when not using SSL
if not use_ssl:
return config
Expand Down