Integrate LAHMC with PyMC3

[twiecki]

Really liked your paper. What are you thoughts on including this into PyMC3 where we already have HMC and NUTS (see here for the code: https://github.com/pymc-devs/pymc/blob/master/pymc/step_methods/nuts.py). It should be straight forward as the interface is very simple as you can see and you already have most of the code.

This way the sampler would be accessible to a much wider audience, the logp and the logp gradient are computed via theano so it's fast and automatic.

[Sohl-Dickstein]

I think this is a really good idea! Yes, I will do this.

Question - does PyMC support running multiple particles in parallel (eg, with each column in the state matrix being a different particle, and the energy function returning a 1d array with the energy for each particle)? From a quick perusal it looks like not? I don't want to strip out this functionality though if it's actually supported.

[twiecki]

Great!

Yes, currently there's no particle support in PyMC3 but it is something we should definitely add. I imagine it would be a new class at the same leve as ArrayStep (maybe ParticleStep) that can run multiple particle-chains.

I think theano is powerful enough to parallelize this then (either via threads or the GPU) so the burden would not be on us.

How would you imagine to merge this if (i) there was no particle support and (ii) if there was? Is there a way to first do it without and then add it once it's supported?