Feature description

Traefik currently supports Let's Encrypt certificate generation via HTTP-01 and TLS-ALPN-01 challenges, which work well for public-facing services. However, this can be limiting in cases where:

• The service is not publicly accessible (e.g., internal networks, Kubernetes clusters).
• Users need to generate wildcard certificates, which require DNS-01 challenge validation.
• Developers of Nebari running nebari locally, wanting to run nebari with valid certificates.

This feature request proposes adding support for DNS challenge validation for Let’s Encrypt certificate generation in Traefik configuration.

This will also help with avoid skipping TLS check in api calls in tests.

Ref:

• https://letsencrypt.org/docs/challenge-types/#dns-01-challenge
• https://doc.traefik.io/traefik/user-guides/docker-compose/acme-dns/

Value and/or benefit

• Enables fully automated SSL certificate provisioning in non-public or private environments.
• Allows issuing wildcard certificates directly through Traefik without requiring external ACME clients.
• Improves security (for internal deployments) by eliminating the need for an exposed HTTP challenge endpoint

Anything else?

I have a draft branch partially implementing this: https://github.com/nebari-dev/nebari/tree/dns-challenge
We don't need to implement for all the DNS providers, I would suggest to implement for Cloudflare for just now and add support for others by simply parsing environment variables in future.