Add support for generating ssl certs via DNS challenge (via Cloudflare)

[aktech]

Reference Issues or PRs

Fixes #2956

Documentation PR to follow up after this PR gets approval.

Valid certs, after deploying with the dns challenge configuration:

Secure https:

Certificate:

What does this implement/fix?

Put a x in the boxes that apply

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds a feature)
• Breaking change (fix or feature that would cause existing features not to work as expected)
• Documentation Update
• Code style update (formatting, renaming)
• Refactoring (no functional changes, no API changes)
• Build related changes
• Other (please describe):

Testing

• Did you test the pull request locally?
• Did you add new tests?

How to test this PR?

1. Set the following environment variables:

• CLOUDFLARE_EMAIL (Cloudflare login email)
• CLOUDFLARE_TOKEN (Token generated from cloudflare dashboard with permissions to access the domain for which you need to generate certs)

2. Create a DNS entry to your cloudflare account pointing the domain you want to deploy Nebari on, to the IP of the load balancer on the docker network (e.g: 172.18.1.100) - This step is optional if you already have enabled dns auto provision.

3. Add the following to the nebari-config.yml

certificate:
  type: lets-encrypt
  acme_email: <YOUR-EMAIL>
  acme_server: https://acme-v02.api.letsencrypt.org/directory
  acme_challenge_type: dns

and deploy Nebari locally and verify valid certificates are generated.

Any other comments?

[dcmcand]

I think for CI we want to use the staging server so we don't hit rate limits

[dcmcand]

@aktech why do we need CLOUDFLARE_DNS_API_TOKEN when we already have CLOUDFLARE_TOKEN? Couldn't we just use the same env var? What is the difference?

[aktech]

I think for CI we want to use the staging server so we don't hit rate limits

We can create subdomains per run, like say: nebari-{commit-sha}.nebari.dev to tackle this. Rate limits are per domain and also staging certs will still throw ssl errors.

@aktech why do we need CLOUDFLARE_DNS_API_TOKEN when we already have CLOUDFLARE_TOKEN? Couldn't we just use the same env var? What is the difference?

I was trying to use the recommended environment variable for dns challenge, but since both of these token need similar permissions, that's a fair point, we can re-use it.