diff --git a/README.md b/README.md index a8604eac8a9..667cd2a9933 100644 --- a/README.md +++ b/README.md @@ -156,6 +156,7 @@ With a little effort, they all could be extracted and used outside. - [Troubleshooting](/docs/troubleshooting.md) - Configuration - [Configuration](/docs/configuration.md) + - [Environment variables](/docs/environment-vars.md) - [CLI: `ais config`](/docs/cli/config.md) - [Feature flags](/docs/feature_flags.md) - Observability diff --git a/api/env/README.md b/api/env/README.md index 8e77f4ec1c9..0a77d278c16 100644 --- a/api/env/README.md +++ b/api/env/README.md @@ -8,6 +8,8 @@ As such, the `env` package is, effectively, part of the API: the names defined h * `ais.go`: AIS environment * `authn.go`: AuthN environment -* `debug.go`: DEBUG environment (build and command-line) -For the list of private system filenames (aka "filename constants"), see also: cmn/fname/fname.go +## See also + +* List of _all_ [environment variables](https://github.com/NVIDIA/aistore/blob/master/docs/environment-vars.md) +* List of [system filenames ("filename constants")](https://github.com/NVIDIA/aistore/blob/master/cmn/fname/fname.go) diff --git a/docs/cli/aws_profile_endpoint.md b/docs/cli/aws_profile_endpoint.md index 103fa6c6d08..fe14ca5a129 100644 --- a/docs/cli/aws_profile_endpoint.md +++ b/docs/cli/aws_profile_endpoint.md @@ -9,7 +9,7 @@ redirect_from: AIStore supports vendor-specific configuration on a per bucket basis. For instance, any bucket _backed up_ by an AWS S3 bucket (**) can be configured to use alternative: -* named AWS profiles (with alternative credentials and/or region) +* named AWS profiles (with alternative credentials and/or AWS region) * s3 endpoints (**) Terminology-wise, when we say "s3 bucket" or "google cloud bucket" we in fact reference a bucket in an AIS cluster that is either: diff --git a/docs/docs.md b/docs/docs.md index 1061b311d1d..7c58753192a 100644 --- a/docs/docs.md +++ b/docs/docs.md @@ -80,6 +80,7 @@ redirect_from: - [Troubleshooting](/docs/troubleshooting.md) - Configuration - [Configuration](/docs/configuration.md) + - [Environment variables](/docs/environment-vars.md) - [CLI: `ais config`](/docs/cli/config.md) - [Feature flags](/docs/feature_flags.md) - Observability diff --git a/docs/environment-vars.md b/docs/environment-vars.md index 94796809719..9d1db07674d 100644 --- a/docs/environment-vars.md +++ b/docs/environment-vars.md @@ -1,12 +1,48 @@ -Following is a brief summary of **all** currently used **environment variables**. +--- +layout: post +title: Environment Variables +permalink: /docs/environment-vars +redirect_from: + - /environment-vars.md/ + - /docs/environment-vars.md/ +--- -Note the common rules that apply across the board: +## Introduction -* all environment are **optional** -* if specified, environment variable will **override** (ie., take precedence over) the corresponding persistent **configuration** if the latter exists. +Generally, aistore configuration comprizes several sources, including: + +1. cluster (a.k.a. global) and node (or, local) configurations, the latter further "splitting" into local config per se and local overrides of the inherited cluster config; +2. `aisnode` [command line](/docs/command_line.md); +3. environment variables - this document; +4. assorted low-level constants (also referred to as "hardcoded defaults") that almost never have to change. + +This enumeration does _not_ include buckets. In aistore, buckets inherit a part of the cluster config that can be further changed on a per-bucket basis - either at creation time or at any later time, etc. + +> In effect, cluster configuration contains cluster-wide defaults for all AIS buckets, current and future. + +For additional references, please see the [last section](#references) in this document. The rest of it, though, describes only and exclusively **environment variables** - the item number 3 above. + +## Rules + +But first, let's state two common rules that, in fact, apply across the board: + +* all environment settings are **optional** +* if specified, environment variable will **override**: + - the corresponding default *constant* in the code (if exists), and/or + - persistent *configuration* (again, if exists) + +For example: + +* persistent (and replicated) node ID can be overridden via `AIS_DAEMON_ID` (below) +* `AIS_READ_HEADER_TIMEOUT`, if specified, will be used instead of the `apc.ReadHeaderTimeout` [constant](https://github.com/NVIDIA/aistore/blob/master/api/apc/const.go) +* `AIS_USE_HTTPS` takes precedence over `net.http.use_https` value from the [cluster configuration](/docs/configuration.md) + +and so on. ### Table of Contents +The remainder of this text groups aistore environment variables by their respective usages, and is structured as follows: + - [Primary](#primary) - [Network](#network) - [Node](#node) @@ -14,19 +50,25 @@ Note the common rules that apply across the board: - [Local Playground](#local-playground) - [Kubernetes](#kubernetes) - [Package: backend](#package-backend) -- [Package: memsys](#package-memsys) + - [AIS as S3 storage](#ais-as-s3-storage) - [Package: stats](#package-stats) +- [Package: memsys](#package-memsys) - [Package: transport](#package-transport) -and separately, authenication server's: +separately, there's authenication server config: - [AuthN](#authn) +and finally: +- [References](#references) + ## Primary +Background: in a running aistore cluster, at any point in time there's a single _primary_ gateway that may also be administratively selected, elected, reelected. Hence, two related variables: + | name | comment | | ---- | ------- | -| `AIS_IS_PRIMARY` | at startup, tells the (starting-up) proxy to assume _primary_ role; e.g. usage: 'export AIS_IS_PRIMARY=true' | -| `AIS_PRIMARY_ID` | at startup, tells _all_ starting-up proxies which one is the _primary_; e.g. usage: 'export AIS_PRIMARY_ID=wfGrerUUk' | +| `AIS_IS_PRIMARY` | at startup, tells _one_ of the (starting-up) proxies to assume the _primary_ role; e.g. usage: 'export AIS_IS_PRIMARY=true' | +| `AIS_PRIMARY_ID` | at startup, tells _all_ starting-up proxies that the one of them with a given ID _is_, in fact, the _primary_; e.g. usage: 'export AIS_PRIMARY_ID=foo-bar' | ## Network @@ -34,15 +76,19 @@ and separately, authenication server's: | ---- | ------- | | `AIS_ENDPOINT` | http or https address of an arbitrary AIS gateway (proxy) in a given cluster | | `AIS_CLUSTER_CIDR` | ais cluster [CIDR](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing); often can be understood/approximated as the cluster's subnet; when specified will be used to differentiate between clients within the same subnet vs outside | -| `AIS_READ_HEADER_TIMEOUT` | maximum time to receive request headers; e.g. usage: 'export AIS_READ_HEADER_TIMEOUT=10s' | +| `AIS_READ_HEADER_TIMEOUT` | maximum time to receive request headers; e.g. usage: 'export AIS_READ_HEADER_TIMEOUT=10s', and note that '0s' (zero) is also permitted | ## Node | name | comment | | ---- | ------- | | `AIS_DAEMON_ID` | ais node ID | -| `AIS_HOST_IP` | ais node IPv4 | -| `AIS_HOST_PORT` | ais node TCP port | +| `AIS_HOST_IP` | node's public IPv4 | +| `AIS_HOST_PORT` | node's public TCP port (and note the corresponding local config: "host_net.port") | + +See also: +* [three logical networks](/docs/performance.md#network) +* [K8s deployment: custom resource 'AIStore'](https://github.com/NVIDIA/ais-k8s/blob/master/docs/walkthrough.md#deploy-an-aistore-cluster) ## HTTPS @@ -58,9 +104,29 @@ and separately, authenication server's: | name | comment | | ---- | ------- | -| `NUM_TARGET` | (development scipts; automation) | +| `NUM_TARGET` | usage: development scripts, local playground automation | | `NUM_PROXY` | (ditto) | +See also: +* [deploy/scripts/clean_deploy.sh](https://github.com/NVIDIA/aistore/blob/master/deploy/scripts/clean_deploy.sh) + +Or, just run the script for quick inline help: + +```console +$ deploy/scripts/clean_deploy.sh --help +NAME: + clean_deploy.sh - locally deploy AIS clusters for development + +USAGE: + ./clean_deploy.sh [options...] + +OPTIONS: + --target-cnt Number of target nodes in the cluster (default: 5) + --proxy-cnt Number of proxies/gateways (default: 5) +... +... +``` + ## Kubernetes | name | comment | @@ -69,33 +135,92 @@ and separately, authenication server's: | `K8S_NODE_NAME` | Kubernetes node name | | `POD_NAMESPACE` | Kubernetes namespace | +See also: +* [AIS K8s Operator: environment variables](https://github.com/NVIDIA/ais-k8s/blob/master/operator/pkg/resources/cmn/env.go) + ## Package: backend +AIS natively supports 3 (three) [Cloud storages](/docs/providers.md). + +The corresponding environment "belongs" to the internal [backend](https://github.com/NVIDIA/aistore/tree/master/ais/backend) package and includes: + | name | comment | | ---- | ------- | -| `S3_ENDPOINT` | e.g. usage: 'S3_ENDPOINT=https://swiftstack.io'; see also [aws profile](/docs/cli/aws_profile_endpoint.md) and [s3 compatibility](/docs/s3compat.md) | -| `AWS_PROFILE` | TODO | -| `GOOGLE_CLOUD_PROJECT` | TODO | -| `GOOGLE_APPLICATION_CREDENTIALS` | TODO | -| `AZURE_STORAGE_ACCOUNT` | TODO | -| `AZURE_STORAGE_KEY` | TODO | -| `AIS_AZURE_URL` | TODO | +| `S3_ENDPOINT` | global S3 endpoint to be used instead of `s3.amazonaws.com` | +| `AWS_PROFILE` | global AWS profiles with alternative account credentials and/or AWS region | +| `GOOGLE_CLOUD_PROJECT` | GCP account with permissions to access your Google Cloud Storage buckets | +| `GOOGLE_APPLICATION_CREDENTIALS` | (ditto) | +| `AZURE_STORAGE_ACCOUNT` | Azure account | +| `AZURE_STORAGE_KEY` | (ditto) | +| `AIS_AZURE_URL` | Azure endpoint, e.g. `http://.blob.core.windows.net` | -## Package: memsys +Notice in the table above that variables `S3_ENDPOINT` and `AWS_PROFILE` are designated as _global_. + +The implication: it is possible to override one or both of them on a per-bucket basis: + +* [configuring buckets with alternative S3 andpoint and/or credentials](/docs/cli/aws_profile_endpoint.md) + +### AIS as S3 storage + +Environment `S3_ENDPOINT` is _important_, and may be also be a source of minor confusion. The reason: aistore itself provides S3 compatible interface. + +For instance, on the aistore's client side you could say something like: + +```console +export S3_ENDPOINT=https://10.0.4.53:51080/s3 +``` + +and then run existing S3 applications against an aistore cluster at `10.0.4.53` - with no changes (to the application). + +Moreover, configure aistore to handle S3 requests at its "/" root: + +```console +$ ais config cluster features Provide-S3-API-via-Root +``` + +and specify `S3_ENDPOINT` environment that looks even better (some would maybe say): + +```console +export S3_ENDPOINT=https://10.0.4.53:51080 +``` + +**To recap**: + +* use `S3_ENDPOINT` to override the `s3.amazonaws.com` default; +* specify `AWS_PROFILE` to use a non-default (named) AWS profile + +and separately: + +* you could run existing S3 apps (with no changes) against aistore by using `S3_ENDPOINT` on the client side + +See also: +* [AIS buckets](/docs/cli/bucket.md) +* [Bucket configuration: AWS profiles](/docs/cli/aws_profile_endpoint.md) +* [Using aistore as S3 endpoint](/docs/s3compat.md) + +## Package: stats + +AIStore is a fully compliant [Prometheus exporter](https://prometheus.io/docs/instrumenting/writing_exporters/). + +In addition and separately, AIStore supports [StatsD](https://github.com/etsy/statsd), and via StatsD - Graphite (collection) and Grafana (graphics). + +The corresponding binary choice between StatsD and Prometheus is a **deployment-time** switch controlled by a single environment variable: **AIS_PROMETHEUS**. + +Namely: | name | comment | | ---- | ------- | -| `AIS_MINMEM_FREE` | TODO | -| `AIS_MINMEM_PCT_TOTAL` | TODO | -| `AIS_MINMEM_PCT_FREE` | TODO | +| `AIS_PROMETHEUS` | e.g. usage: `export AIS_PROMETHEUS=true` | +| `AIS_STATSD_PORT` | use it to override the default `8125` (see https://github.com/etsy/stats) | +| `AIS_STATSD_PROBE` | a startup option that, when true, tells an ais node to _probe_ whether StatsD server exists (and responds); if the probe fails, the node will disable its StatsD functionality completely - i.e., will not be sending any metrics to the StatsD port (above) | -## Package: stats +## Package: memsys | name | comment | | ---- | ------- | -| `AIS_PROMETHEUS` | TODO | -| `AIS_STATSD_PORT` | TODO | -| `AIS_STATSD_PROBE` | TODO | +| `AIS_MINMEM_FREE` | TODO | +| `AIS_MINMEM_PCT_TOTAL` | TODO | +| `AIS_MINMEM_PCT_FREE` | TODO | ## Package: transport @@ -117,3 +242,13 @@ and separately, authenication server's: | `AIS_AUTHN_PORT` | TODO | | `AIS_AUTHN_TTL` | TODO | | `AIS_AUTHN_USE_HTTPS` | TODO | + +## References + +* `env` package [README](https://github.com/NVIDIA/aistore/blob/master/api/env/README.md) +* [system filenames](https://github.com/NVIDIA/aistore/blob/master/cmn/fname/fname.go) +* [cluster configuration](/docs/configuration.md) +* [local playground: cluster and node configuration](https://github.com/NVIDIA/aistore/blob/master/deploy/dev/local/aisnode_config.sh) +* [`ais config` command](/docs/cli/config.md) +* [performance tunables](/docs/performance.md) +* [feature flags](/docs/feature_flags.md) diff --git a/docs/index.md b/docs/index.md index d664b00bb7e..d9cbb39f39b 100644 --- a/docs/index.md +++ b/docs/index.md @@ -165,6 +165,7 @@ With a little effort, they all could be extracted and used outside. - [Troubleshooting](/docs/troubleshooting.md) - Configuration - [Configuration](/docs/configuration.md) + - [Environment variables](/docs/environment-vars.md) - [CLI: `ais config`](/docs/cli/config.md) - [Feature flags](/docs/feature_flags.md) - Observability diff --git a/docs/s3compat.md b/docs/s3compat.md index 39add112d69..00ccb29472f 100644 --- a/docs/s3compat.md +++ b/docs/s3compat.md @@ -348,11 +348,11 @@ In this section, we use all 3 (three) clients: # 1. Upload via `s3cmd` => `aistore` $ s3cmd put $(which aisnode) s3://ais-aa --multipart-chunk-size-mb=8 -upload: '/root/gocode/bin/aisnode' -> 's3://ais-aa/aisnode' [part 1 of 10, 8MB] [1 of 1] +upload: 'bin/aisnode' -> 's3://ais-aa/aisnode' [part 1 of 10, 8MB] [1 of 1] 8388608 of 8388608 100% in 0s 233.84 MB/s done ... 8388608 of 8388608 100% in 0s 234.19 MB/s done -upload: '/root/gocode/bin/aisnode' -> 's3://ais-aa/aisnode' [part 10 of 10, 5MB] [1 of 1] +upload: 'bin/aisnode' -> 's3://ais-aa/aisnode' [part 10 of 10, 5MB] [1 of 1] 5975140 of 5975140 100% in 0s 233.39 MB/s done ```