Build timeouts on ReadTheDocs

[shoyer]

A significant fraction of our doc builds have started running up against ReadTheDocs's 900 second timeout for builds:

Investigating a few of these, it seems that the main culprit is installing/downloading conda packages in the conda env create and conda install steps. In some cases, these take over 700 seconds combined, leaving little time for Sphinx, which on its own takes only a couple minutes (but RTD runs it three times for some reason).

[shoyer]

We see the same issue with our builds on Travis-CI.

Here's our latest doc build on Travis:
https://travis-ci.org/pydata/xarray/jobs/386509884

Notice that it took 411 seconds to run conda env create!

I'm not quite exact what the underlying issue is here (e.g., download vs install time), but I'm sure it's somehow related to our large number of dependencies. If download time is the issue, then perhaps caching downloaded conda packages would help, e.g., readthedocs/readthedocs.org#3261

[ocefpaf]

Notice that it took 411 seconds to run conda env create!

If you are using conda-forge bare in mind that our package index is huge and conda is not very smart about it. We are looking into possible solution. Pinging @pelson who have some ideas in mind on how to address this problem.

[pelson]

Pinging @pelson who have some ideas in mind on how to address this problem.

The ideas relate to the fetching of the index, which will take orders of magnitude less time than the resolve and download stages in conda. They aren't entirely unrelated though, as a smaller index (the proposal) would result in fewer options for the conda solver to have to work through. No matter what we do, caching the binaries will have the same impact, though it is a challenge to cache sensibly without having a really large cache... You may find that caching an environment.yaml actually has more of an impact than caching the binaries themselves (i.e. this means you continue to download the binaries each time, but don't do a conda resolve each time).

[fmaussion]

Closed via readthedocs/readthedocs.org#4432