Add Enzyme tests

[wsmoses]

now that the type instability is fixed, it would probably be useful to add Enzyme tests to make sure there isn't another regression in the future (and also give some assurances of support to users)

[yebai]

Duplicate of #739

We already have a PR opened against Enzyme: EnzymeAD/Enzyme.jl#1813. We can simply mirror the test set. It’s important to run Enzyme test as an isolated integration test.

[yebai]

As I explained elsewhere, I don't think Enzyme is mature enough:

1. lack of comprehensive coverage of Julia syntax, notably type instability
2. lack of debugging tools: the only way at the moment is based on guessing then minimising
3. still lots of segfaults
4. lack of integration test with most of Julia numerical ecosystem, thus supposed unbreaking Enzyme releases break Turing.jl models.

Some of these issues can be fixed by adding integration tests on both sides (Enzyme + DynamicPPL), but lots of other issues can't and require significant improvements to Enzyme itself.

[wsmoses]

The error was a type instability in this repo........... for which adding some tests would help give some notion of assurances, right?

In any case, if you have any issues open an issue. I'm unaware of any segfaults from turing, or issues with numerics. Adding integration tests (as proposed here), would help find/fix these, yes?

[yebai]

I'm unaware of any segfaults from turing or issues with numerics. Adding integration tests (as proposed here), would help find/fix these, yes?

Enzyme tests against Bijectors and AdvancedVI are constantly broken. Some issues have been reported on Enzyme.jl. Due to the frequency of Enzyme failures, it has now become a maintenance burden. Enzyme's fragility, coupled with the difficulty of debugging Enzyme failures, means that CI failures about Enzyme are often ignored.

Adding integration tests (as proposed here) would help find/fix these, yes?

My point is that libraries like Turing.jl should not be responsible for discovering and reporting Enzyme bugs. Asking for more integration tests on Turing.jl will help Enzyme, but it would also add more maintenance burden for us. It is more of the autodiff dev's responsibility to catch bugs via better unit and integration tests. A more systematic approach to bug fixing will help the Enzyme dev team.