Elasticsearch

Provisions a managed Elasticsearch cluster and declares its schema - indices, aliases, index templates - plus least-privilege API keys for each consumer, all from one service entry.

Use this module when a tenant needs search as platform infrastructure instead of a hand-created cluster: the cluster, its indices, and the credentials its apps consume are all reviewed config diffs.

Supported Features

• Elastic Cloud provisioning (provider: elasticcloud) with version, region, size, HA zone count, autoscaling ceiling, and optional Kibana.
• AWS OpenSearch provisioning (provider: aws-opensearch) with fine-grained access control, a generated master password vaulted to Secret Manager, encryption at rest and in transit, a business-level size that maps to instance classes (vendor.aws.instance_type overrides), and an optional source-IP allowlist (vendor.aws.allowed_cidrs).
• Vendor-neutral schema layer: index templates, indices, aliases, and mappings (written as YAML, applied to whatever endpoint the cluster layer produced).
• On elasticcloud, per-consumer security roles and scoped API keys; the elastic superuser is vaulted for break-glass only. On aws-opensearch, access is rejected at plan time (OpenSearch has no API-key API); consumers use the vaulted master credential or IAM/SigV4.
• Consumer credentials written to the tenant GCP project under stable names (<PREFIX>_ENDPOINT, <PREFIX>_CLOUD_ID, <PREFIX>_<CONSUMER>_API_KEY, <PREFIX>_SUPERUSER_PASSWORD) so a k8s-configmap entry survives a vendor switch unchanged.
• Index deletion protection on by default: plans that would destroy an index fail unless deletion_protection: false is set explicitly.

Prerequisites

• An Elastic Cloud API key stored in Secret Manager (ELASTIC_CLOUD_API_KEY in the central Rabbit project by default; overridable with api_key_project/api_key_secret).
• Standard R2A tenant grants on client_project_id.

Example

---
name: 'UDX dev search'
id: 'elasticsearch-udx-dev-staging'
module: 'elasticsearch'
configurations:
  provider: 'elasticcloud'
  cluster_name: 'udx-dev-search-staging'
  version: '9.1.0'
  region: 'gcp-us-east1'
  size: '4g'
  zone_count: 1
  client_project_id: 'udx-dev-project'

  indices:
    leads-v1:
      aliases: ['leads.read', 'leads.write']
    projects-v1:
      aliases: ['projects.read']

  access:
    explorer:
      read: ['leads.read', 'projects.read']
      write: ['leads.write']

Produces ES_UDX_DEV_SEARCH_STAGING_ENDPOINT, ES_UDX_DEV_SEARCH_STAGING_CLOUD_ID, and ES_UDX_DEV_SEARCH_STAGING_EXPLORER_API_KEY in udx-dev-project.

Important Notes

• Keep cluster entries on long-lived lifecycles (staging/production). R2A destroys branch-scoped infrastructure when a branch is deleted; an Elasticsearch cluster with data should never be branch-scoped.
• Mappings are YAML maps; the module jsonencodes them. Elasticsearch rejects breaking mapping changes on existing fields, so evolving a schema means adding a new index version and moving its aliases in the same diff.
• Changing creation-only index settings (for example number_of_shards) forces index replacement; the default deletion protection turns that into a failed plan, which is the intended guard rail.
• On the very first apply, schema-layer resources plan as (known after apply) because the cluster endpoint does not exist yet; subsequent runs produce complete plans.
• k8s-eck is reserved in the schema but not yet implemented; provider rejects it at plan time with a clear message.