Update to the [email protected] interface

[Red-Portal]

This PR aims to update Turing's Variational module to match AdvancedVI's new interface starting from v0.3. I will try not to change the interface too much, but given the new features in AdvancedVI, I think breaking changes will be inevitable. Though the focus will be to provide a good default setting rather than to expose all the features.

Currently proposed interface:

using Turing
using AdvancedVI

d = randn(100)
Turing.@model function model()
   x ~ MvNormal(randn(100), 1)
   y ~ InverseGamma()
end
m = model()
q = Turing.Variational.q_fullrank_gaussian(Random.default_rng(), m)
n_iters = 1000
Turing.Variational.vi(m, q, n_iters)

Closes #2507
Closes #2508
Closes #2430

[yebai]

@Red-Portal, can you fix the tests before I take a look?

[Red-Portal]

@yebai I marked the PR as a draft so that we can first agree on an interface, and then I flesh out the implementation and the tests. Do we wish we proceed in another way?

[yebai]

Let's address the interface later or in a separate PR since that might require more discussions. For this PR, let's try to keep the VI interface non-breaking where possible.

[yebai]

@Red-Portal can you take a look at the following error:

ERROR: LoadError: UndefVarError: `turnprogress` not defined in `AdvancedVI`

[codecov]

Codecov Report

Attention: Patch coverage is 60.29412% with 27 lines in your changes missing coverage. Please review.

Project coverage is 82.68%. Comparing base (05110bd) to head (f62e7b8).

Files with missing lines	Patch %	Lines
src/variational/deprecated.jl	0.00%	16 Missing ⚠️
src/variational/VariationalInference.jl	78.84%	11 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2506      +/-   ##
==========================================
- Coverage   84.27%   82.68%   -1.59%     
==========================================
  Files          21       21              
  Lines        1456     1461       +5     
==========================================
- Hits         1227     1208      -19     
- Misses        229      253      +24     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[yebai]

Thanks @Red-Portal. I left some comments below.

One high-level comment: I suggest we unify vi_fullrank_gaussian and vi_meanfield_gaussian into a single function, q_distribution(...; gaussian=true|false, fullrank=true|false), to reduce code redundancy.

Note on CI errors:

• CI compains about missing ADVI() and TruncatedADAGrad().

[yebai]

Can we get the dimentionality via num_params = length(varinfo_linked) instead of length(varinfo_linked[:])?

cc @mhauru

[Red-Portal]

This currently doesn't work, so I'll leave it as is for now

[mhauru]

Once VarInfo will be based on VarNamedVector we will have a function called length_internal for this purpose. We can swap that in here once the VarInfo makeover is done.

[Red-Portal]

Sorry for the delay! I've been traveling in the past weeks, but will start working on this now

[coveralls]

Pull Request Test Coverage Report for Build 15099815167

Details

• 41 of 68 (60.29%) changed or added relevant lines in 2 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage decreased (-1.6%) to 82.683%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
src/variational/VariationalInference.jl	41	52	78.85%
src/variational/deprecated.jl	0	16	0.0%

Totals
Change from base Build 14925677192:	-1.6%
Covered Lines:	1208
Relevant Lines:	1461

---

💛 - Coveralls

[github-actions]

Turing.jl documentation for PR #2506 is available at:
https://TuringLang.github.io/Turing.jl/previews/PR2506/

[mhauru]

I'm confused at the CI failure of test/mcmc/Inference.jl. It seems unrelated to this PR. I'm unable to reproduce it locally. The checks are just a tiny bit outside our allowed margin of error, but I also just find it troubling that different AD backends are giving different results at all. Maybe our seed fixing is failing. I'll rerun CI to see if it's deterministic.