Skip to content

Conversation

@toppercodes
Copy link
Contributor

@toppercodes toppercodes commented Oct 21, 2025

Summary

Axiom is working on adding regional ingest endpoints.
This PR introduces a new region field in AxiomConfig to specify the Axiom regional edge domain for data ingestion, and enhances the url field with intelligent path detection.

Configuration priority: url > region > default cloud endpoint

Key features:

  • region: Domain-only format (e.g., mumbai.axiom.co) - automatically builds https://{region}/v1/ingest/{dataset}
  • url: Smart path handling
    • URLs with custom paths are used as-is
    • URLs without paths maintain backwards compatibility by appending /v1/datasets/{dataset}/ingest
  • Fully backwards compatible with existing configurations

Vector configuration

Using regional edge:

[sinks.axiom]
type = "axiom"
token = "${AXIOM_TOKEN}"
dataset = "my-dataset"
region = "eu-central-1.aws.edge.axiom.co"
# Result: https://eu-central-1.aws.edge.axiom.co/v1/ingest/my-dataset

Using URL with custom path (super special case):

[sinks.axiom]
type = "axiom"
token = "${AXIOM_TOKEN}"
dataset = "meh"
url = "http://localhost:3400/ingest"
# Result: http://localhost:3400/ingest (used as-is)

Using URL without path (backwards compatible):

[sinks.axiom]
type = "axiom"
token = "${AXIOM_TOKEN}"
dataset = "qoo"
url = "https://api.eu.axiom.co"
# Result: https://api.eu.axiom.co/v1/datasets/qoo/ingest

Traditional configuration (unchanged):

[sinks.axiom]
type = "axiom"
token = "${AXIOM_TOKEN}"
dataset = "foo"
# Result: https://api.axiom.co/v1/datasets/foo/ingest

URL takes precedence over region:

[sinks.axiom]
type = "axiom"
token = "${AXIOM_TOKEN}"
dataset = "test"
region = "mumbai.axiom.co"
url = "http://localhost:3400/ingest"
# Result: http://localhost:3400/ingest (url wins)

How did you test this PR?

  • Added comprehensive unit tests covering all scenarios:
    • ✅ Regional edge endpoint generation (domain-only format)
    • ✅ URL with custom path (used as-is)
    • ✅ URL without path (backwards compatibility with legacy path format)
    • ✅ URL takes precedence over region when both are set
    • ✅ Default cloud endpoint when neither is set
    • ✅ Support for production, staging, and dev environments
    • ✅ Trailing slash handling
  • All existing tests continue to pass, confirming full backward compatibility

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the no-changelog label to this PR.

Changelog fragment added: changelog.d/axiom_regional_edges.feature.md

References

Closes #24038

Notes

  • Please read our Vector contributor resources.
  • Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
  • Some CI checks run only after we manually approve them.
    • We recommend adding a pre-push hook, please see this template.
    • Alternatively, we recommend running the following locally before pushing to the remote branch:
      • make fmt
      • make check-clippy (if there are failures it's possible some of them can be fixed with make clippy-fix)
      • make test
  • After a review is requested, please avoid force pushes to help us review incrementally.
    • Feel free to push as many commits as you want. They will be squashed into one before merging.
    • For example, you can run git merge origin master and git push.
  • If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
    run make build-licenses to regenerate the license inventory and commit the changes (if any). More details here.

@toppercodes toppercodes requested a review from a team as a code owner October 21, 2025 15:06
@github-actions github-actions bot added the domain: sinks Anything related to the Vector's sinks label Oct 21, 2025
@toppercodes toppercodes force-pushed the feat-support-axiom-edges branch 2 times, most recently from c448a3c to 06e0f8e Compare October 21, 2025 18:26
Introduces optional `region` field for regional edge domains and enhances
`url` field with intelligent path detection. URLs with custom paths are
used as-is, while URLs without paths maintain backwards compatibility by
appending the legacy path format.

Priority: url > region > default cloud endpoint

Fully backwards compatible with existing configurations.
@toppercodes toppercodes force-pushed the feat-support-axiom-edges branch from 06e0f8e to 9bf1a13 Compare October 21, 2025 19:09
Copy link

@ronoc ronoc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good !

@toppercodes
Copy link
Contributor Author

Hey @jszwedko can I get a review on this please, we're hoping to get this into the next release

- Moved the Axiom sink configuration and integration tests to a new module `axiom` under the `sinks` directory.
- Implemented `UrlOrRegion` struct to manage Axiom endpoint configurations, ensuring validation between URL and region settings.
- Moved the integration tests to a new module `integration_tests` under the `axiom` module.
- Updated documentation for the Axiom sink configuration in the generated CUE files to reflect new fields and usage examples.
@toppercodes toppercodes requested a review from a team as a code owner October 30, 2025 19:44
@github-actions github-actions bot added the domain: external docs Anything related to Vector's external, public documentation label Oct 30, 2025
@toppercodes toppercodes requested a review from pront October 30, 2025 19:54
Copy link
Member

@pront pront left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @toppercodes

@toppercodes
Copy link
Contributor Author

@pront Is there any other actions required from my side?

@pront pront enabled auto-merge November 5, 2025 17:57
@pront
Copy link
Member

pront commented Nov 5, 2025

@pront Is there any other actions required from my side?

As long as the tests pass, this is good to go.

auto-merge was automatically disabled November 5, 2025 19:41

Head branch was pushed to by a user without write access

@toppercodes
Copy link
Contributor Author

@pront Is there any other actions required from my side?

As long as the tests pass, this is good to go.

Tests should be passing now

@pront pront enabled auto-merge November 5, 2025 19:53
@pront pront added this pull request to the merge queue Nov 5, 2025
Merged via the queue into vectordotdev:master with commit c9537a0 Nov 5, 2025
44 checks passed
elkh510 pushed a commit to elkh510/vector that referenced this pull request Nov 19, 2025
…ectordotdev#24037)

* feat(axiom sink): Add regional edge support with smart URL handling

Introduces optional `region` field for regional edge domains and enhances
`url` field with intelligent path detection. URLs with custom paths are
used as-is, while URLs without paths maintain backwards compatibility by
appending the legacy path format.

Priority: url > region > default cloud endpoint

Fully backwards compatible with existing configurations.

* cargo fmt

* refactor(axiom): Axiom sink configuration and integration tests

- Moved the Axiom sink configuration and integration tests to a new module `axiom` under the `sinks` directory.
- Implemented `UrlOrRegion` struct to manage Axiom endpoint configurations, ensuring validation between URL and region settings.
- Moved the integration tests to a new module `integration_tests` under the `axiom` module.
- Updated documentation for the Axiom sink configuration in the generated CUE files to reflect new fields and usage examples.

* cargo fmt

* fix(axiom): Update integration tests to include SinkConfig and UrlOrRegion

* Update website/cue/reference/components/sinks/generated/axiom.cue

Co-authored-by: domalessi <[email protected]>

* fix: Fix wording

* fix(axiom): Make AxiomConfig fields public

---------

Co-authored-by: Pavlos Rontidis <[email protected]>
Co-authored-by: domalessi <[email protected]>
elkh510 pushed a commit to elkh510/vector that referenced this pull request Nov 19, 2025
…ectordotdev#24037)

* feat(axiom sink): Add regional edge support with smart URL handling

Introduces optional `region` field for regional edge domains and enhances
`url` field with intelligent path detection. URLs with custom paths are
used as-is, while URLs without paths maintain backwards compatibility by
appending the legacy path format.

Priority: url > region > default cloud endpoint

Fully backwards compatible with existing configurations.

* cargo fmt

* refactor(axiom): Axiom sink configuration and integration tests

- Moved the Axiom sink configuration and integration tests to a new module `axiom` under the `sinks` directory.
- Implemented `UrlOrRegion` struct to manage Axiom endpoint configurations, ensuring validation between URL and region settings.
- Moved the integration tests to a new module `integration_tests` under the `axiom` module.
- Updated documentation for the Axiom sink configuration in the generated CUE files to reflect new fields and usage examples.

* cargo fmt

* fix(axiom): Update integration tests to include SinkConfig and UrlOrRegion

* Update website/cue/reference/components/sinks/generated/axiom.cue

Co-authored-by: domalessi <[email protected]>

* fix: Fix wording

* fix(axiom): Make AxiomConfig fields public

---------

Co-authored-by: Pavlos Rontidis <[email protected]>
Co-authored-by: domalessi <[email protected]>
@thecraftman
Copy link

Hi @pront can you please confirm if this was included in the 0.51.1 release? Thanks.

@pront
Copy link
Member

pront commented Dec 7, 2025

Hi @pront can you please confirm if this was included in the 0.51.1 release? Thanks.

👋 Since the PR was merged after v0.51.1, it will be included in the next release (likely v0.52.0 or similar).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain: external docs Anything related to Vector's external, public documentation domain: sinks Anything related to the Vector's sinks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add axiom regional endpoint configuration

5 participants