feat: install script to migrate sentry.conf.py config to use pgbouncer

[aldy505]

This PR may only be merged after #3884 has been merged

[aldy505]

I will need to remove this flag after #3884 is merged.

[aldy505]

Hopefully it's okay to not clean sentry/config.yml and relay/config.yml.

[aldy505]

Test results are ok:

Test 1 (pre 25.9.0 release)
Migrating Postgres config to PGBouncer...
  Migrating sentry/sentry.conf.py to use PGBouncer
  Migrated sentry/sentry.conf.py to use PGBouncer
DATABASES section is correct
Test 2 (post 25.9.0 release)
Migrating Postgres config to PGBouncer...
  Found pgbouncer in sentry/sentry.conf.py, I'm assuming you're good! :)
DATABASES section is correct
Test 3 (custom postgres config)
Migrating Postgres config to PGBouncer...
  ⚠️ You don't have standard configuration for Postgres in sentry/sentry.conf.py, skipping pgbouncer migration. I'm assuming you know what you're doing.
DATABASES section is correct
migrate-pgbouncer-test.sh - Success 👍

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.45%. Comparing base (67c32e8) to head (fc8a61a).
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #3898   +/-   ##
=======================================
  Coverage   99.45%   99.45%           
=======================================
  Files           3        3           
  Lines         183      183           
=======================================
  Hits          182      182           
  Misses          1        1           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[frederikspang]

@aldy505 LGTM

[hubertdeng123]

We historically have not done a sed to modify the sentry.conf.py files, as we chose not to modify people's config files without permission.

I wonder if this is something that we should have a very loud announcement about, and keep it at that and have users modify their config files themselves.

[aldy505]

Often times there are GitHub issues complaining why certain things are broken, and the problem is that they didn't change the config file. I believe it's fine for this one, since we're trying to make everyone's instance better.

But, if we're still not wanting to go down this route, I'm fine with it.

[hubertdeng123]

@BYK what do you think here?

[BYK]

@hubertdeng123 I'm the one who pushed for this migration as I think it will make things quite a bit better. We do a grep-based check with a warning for memcached backend here:

self-hosted/install/check-memcached-backend.sh

Line 4 in 67c32e8

echo "PyMemcacheCache found in $SENTRY_CONFIG_PY, gonna assume you're good."

My argument for automatically doing this is:

1. If you are running a custom/complex setup, you probably are not using the install script anyway
2. If you are running the default setup, then we not only will be helping your setup transparently, we would also avoid running pgbouncer in the background for no good reason.

So if we don't want this auto migration then we should:

1. Hide the pgbouncer container and relevant stuff behind a (possibly new) docker compose profile
2. We should at least add the grep-based check and warning we have memached and steer people in the right direction.

Which approach we take depends on our risk apetite and I cannot really force either way as I won't be the one who will be bearing the consequences 😅

[aldy505]

I vote for merging this :)

[hubertdeng123]

I hear all the arguments and there is no perfect solution here, as we'd want to make this as seamless as possible for folks. However, I'd still prefer if this wasn't merged given the reasons:

1. I wouldn't feel comfortable directly editing people's config files and setting precedent that it is fine to do in the future
2. Higher risk, as we have mentioned above

Dev-infra talked about reworking how the sentry.conf.py config file looks in sentry since the sentry.conf.py file contains a lot of defaults, not just custom settings. To me, it makes sense to have the defaults for self-hosted live in sentry in a separate file so we wouldn't need to do this. Then, people's custom sentry.conf.py can override those defaults. This would pave a future where these sort of upgrades are seamless, even though this wouldn't provide relief immediately.

This would help us make the dev environment nicer and improve the self-hosted config story. Is this something you're interested in helping out with @aldy505?

[aldy505]

This would help us make the dev environment nicer and improve the self-hosted config story. Is this something you're interested in helping out with @aldy505?

I don't have much experience with Django. I will leave this for dev-infra.