`--generate-update-payload --kind` returns identical payloads for different resource types (`elasticsearch`, `kibana`)

[TomonoriSoejima]

Description:
According to the ecctl documentation example, using the --kind flag with --generate-update-payload should generate an update payload specific to the chosen resource (e.g., elasticsearch, kibana, apm, etc.).

However, the following commands produce the same payload output for both Elasticsearch and Kibana:

Steps to Reproduce:

ecctl deployment show f8c26c744a09459586bfd141e3d2fd6c --generate-update-payload --kind elasticsearch > elasticsearch.json
ecctl deployment show f8c26c744a09459586bfd141e3d2fd6c --generate-update-payload --kind kibana > kibana.json
diff elasticsearch.json kibana.json

Expected Behavior:
Each --kind should scope the payload to the corresponding resource only:

• --kind elasticsearch → only Elasticsearch plan/settings
• --kind kibana → only Kibana plan/settings

Actual Behavior:
The resulting JSON files (elasticsearch.json and kibana.json) are identical, which contradicts the expected behavior described in the documentation.

Impact:
This makes it impossible to generate accurate update payloads for individual resources, blocking granular updates or validation workflows in automation pipelines.

Related Documentation:

• https://www.elastic.co/guide/en/ecctl/1.0/ecctl_deployment_show.html#_examples_4

[nick-benoit]

Hi @TomonoriSoejima Unfortunately this looks to be a case where the documentation is unclear. I will make a point to get those updated.

• Shows apm resource information from a given deployment with a specified ref-id.
  ecctl deployment show --kind apm --ref-id apm-server
• Return the current deployment state as a valid update payload.
  ecctl deployment show --generate-update-payload > update.json

In the above examples --kind and --generate-update-payload are actually mutually exclusive. When you add --generate-update-payload it ignores --kind. So this is actually working as expected, though what you are asking for would be handy.

At current I think the best way to accomplish what you're looking for (an update payload that contains only the resource you're trying to update) is to use --generate-update-payload and use something like jq to filter out resources you want to exclude.

IMPORTANT
double check prune_orphans: false, otherwise resources not included in your update will be terminated

something like the following:

# Approximation of what it seems like you're trying to do with:
# ecctl deployment show f8c26c744a09459586bfd141e3d2fd6c --generate-update-payload --kind kibana > kibana.json

ecctl deployment show ${DEPLOYMENTID} --generate-update-payload  > ${DEPLOYMENTID}_tmp.json

jq "." ${DEPLOYMENTID}_tmp.json | \
jq ".prune_orphans = false" | \
jq "del(.resources.elasticsearch)" | \
jq "del(.resources.apm)" | \
jq "del(.resources.enterprise_search)" | \
jq "del(.resources.appsearch)" | \
jq "del(.resources.integrations_server)" > ${DEPLOYMENTID}.json

[louisong]

@nick-benoit We are looking for a way to generate the payload for specific resource if its not enabled.

If we are unable to generate a resource-specific JSON using the --kind flag (e.g., for kibana/elasticsearch), the fallback seems to be generating the full deployment JSON, which includes all components like Kibana, integrations_server, Elasticsearch, and Enterprise Search.

Let’s say Enterprise Search is currently not enabled, but I want to enable and configure it via the payload — is there a supported way to generate the necessary payload structure for adding Enterprise Search to an existing deployment?

[nick-benoit]

Hi @louisong Unfortunately I don't think such a thing exists. Any of the show utilities are focused for showing the current state of the deployment. The idea of --generate-update-payload is that it provides the payload that could be edited and then used to update the current state of a deployment. There aren't really any helpers to manipulate the json into a state that may be desired.

In part this is because there is no single way to enable a component. Eg when adding integrations server to a deployment you would add something like this, where you notably need to decide what size and how many zones you would like.

"integrations_server": [
  {
    "region": "aws-eu-west-1",
    "ref_id": "main-integrations_server",
    "elasticsearch_cluster_ref_id": "main-elasticsearch",
    "plan": {
      "cluster_topology": [
        {
          "instance_configuration_id": "aws.integrationsserver.c5d",
          "size": {
            "value": 1024,
            "resource": "memory"
          },
          "zone_count": 1
        }
      ],
      "integrations_server": {},
      "transient": {
        "strategy": {
          "autodetect": {}
        }
      }
    }
  }
]

In the general case some information about the deployment is required to determine what the above payload should look like. Eg region, elasticsearch_cluster_ref_id, instance_configuration_id, size, zone_count.

Without knowing too much about the constraints of your use case you could also consider investigating the elastic cloud terraform provider which could help simplify getting a deployment into your desired state programmatically.

[louisong]

Thanks for confirming! @nick-benoit

As a workaround, we currently retrieve the full deployment JSON and manually updating the relevant sections (e.g., Elasticsearch or Kibana YAML) based on our requirements.

Raised this because the https://www.elastic.co/docs/reference/ecctl/ecctl_deployment_show mentions that it’s possible to generate payloads using the --kind flag, for example:

ecctl deployment show <deployment-id> --kind kibana ecctl deployment show <deployment-id> --kind apm --ref-id apm-server
But it doesn't seem to return the expected resource-specific payload but rather the full deployment JSON.

Assuming this behavior is expected, can we update the documentation to reflect that? --kind doesn’t generate a scoped payload — just to avoid confusion.

[nick-benoit]

Hi @louisong

Yes this is the expected behavior. I will take a note to update that documentation, I certainly agree it is confusing as is.

[louisong]

@nick-benoit Is there a public facing github ticket we can follow along for the doc update?

[nick-benoit]

Hi @louisong I suppose this ticket serves that purpose? Apologies on the delay, we are currently migrating to a new docs system which has caused a few snags and has delayed these updates a bit. I think those changes should land later this week.

[nick-benoit]

@louisong What do you think of these updates PR. You can see the docs preview linked here

[nick-benoit]

I merged the above PR to update the docs. I'm going to close this issue with those changes.