Migrate CI build-doc to Meson #39973

tobiasdiez · 2025-04-19T09:53:04Z

Make it possible to build the documentation with meson via

meson compile -C builddir doc-html

For this a couple of changes was necessary in the docbuilder:

Make sage_docbuild independent of sage so that meson can use sage_docbuild during config time to construct all the docbuild targets (otherwise one needs to first install sage, and then could configure the docbuild)
Properly handle input and output dirs for the docbuild, without relying on magic sage env variables

📝 Checklist

The title is concise and informative.
The description explains in detail what this PR is about.
I have linked a relevant issue or discussion.
I have created tests covering the changes.
I have updated the documentation and checked the documentation preview.

⌛ Dependencies

user202729 · 2025-04-19T10:16:41Z

Looks like some file permission goes wrong? (symbolic link get converted to normal file?)

user202729 · 2025-04-19T10:18:41Z

src/sage_docbuild/builders.py

+    docs = [doc[1] for doc in sorted(documents)]
+    # Put the bibliography first, because it needs to be built first:
+    docs.remove(Path('reference/references'))
+    docs.insert(0, Path('reference/references'))


Why is such a large change necessary in this file?

To be honest, I got carried a way a bit: the strictly necessary change is to remove mentions of import sage, then I used the chance to use pathlib, and to catch all those paths vs strings issues added typing info. Oh, and there were a few old deprecated stuff in there which was easier to remove than to migrate to pathlib 🦊

…tory-reference

github-actions · 2025-05-17T03:51:02Z

Documentation preview for this PR (built with commit 3408887; changes) is ready! 🎉
This preview will update shortly after each push to this PR.

tobiasdiez · 2025-05-17T10:28:32Z

Doc build is working now, and the changes relative to the last develop build seem to be minimal and mostly due to using a newer sphinx version now.

Marking it as blocker since currently the docbuild CI for other PRs is completely failing.

dimpase · 2025-05-17T15:56:44Z

https://github.com/sagemath/sage/actions/runs/15081532937/job/42398821050#step:9:1

fails while building docs the old way - is it normal?

tobiasdiez · 2025-05-17T16:40:21Z

It fails with

ImportError: libSingular-4.4.0.so: cannot open shared object file: No such file or directory

https://github.com/sagemath/sage/actions/runs/15081532937/job/42398821050#step:9:3982 so this is the same problem as for the other PRs.

dimpase · 2025-05-17T23:56:18Z

no, my question was - why does that CI run build docs the old way?

antonio-rojas · 2025-07-10T14:30:07Z

but then I hit the second error:

Sorry, I've never seen that error before and it also looks a bit like an internal sphinx error.

I figured out the issue: this is (again) caused by missing pyx sources. Specifically, if the cachefunc.pyx source file where the @cached_method decorator is defined is not accessible, then autodoc chokes when parsing any method decorated with it.

I guess this is another point in favor of installing pyx files.

sagemathgh-39030: Use meson in sage-the-distro       Replace the old setuptools-based build by the new meson-based one in sage-the-distro. Delete most of the old stuff that is no longer needed now. ### 📝 Checklist  - [ ] The title is concise and informative. - [ ] The description explains in detail what this PR is about. - [ ] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation and checked the documentation preview. ### ⌛ Dependencies - sagemath#40133 - sagemath#39973    URL: sagemath#39030 Reported by: Tobias Diez Reviewer(s): Dima Pasechnik

sagemathgh-40379: Fix docbuilding on release tag       After sagemath#39973, some necessary steps during the docbuild on release tags are no longer run (see https://github.com/sagemath/sage/actions/runs/161 00707903/job/45429461356). Sorry - that was my fault. This is fixed here. Test run: https://github.com/tobiasdiez/sage/actions/runs/16323033505/jo b/46105684396 ### 📝 Checklist  - [ ] The title is concise and informative. - [ ] The description explains in detail what this PR is about. - [ ] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation and checked the documentation preview. ### ⌛ Dependencies    - sagemath#40383 URL: sagemath#40379 Reported by: Tobias Diez Reviewer(s): Dima Pasechnik

user202729 · 2025-08-09T01:07:48Z

bad news. CI doc build on develop is failing.

FAILED: [code=1] src/doc/htmlfunctions 
/usr/share/miniconda/envs/sage-dev/bin/python3.11 /home/runner/work/sage/sage/src/build-docs.py --no-pdf-links reference/functions html -o src/doc --source /home/runner/work/sage/sage/src/doc
Deleting empty directory /home/runner/work/sage/sage/src/doc/en/reference/function_fields/sage/rings/function_field
Error building the documentation.
Traceback (most recent call last):
  File "/home/runner/work/sage/sage/src/build-docs.py", line 11, in <module>
    main()
  File "/home/runner/work/sage/sage/src/sage_docbuild/__main__.py", line 543, in main
    os.rmdir(dirpath)
FileNotFoundError: [Errno 2] No such file or directory: '/home/runner/work/sage/sage/src/doc/en/reference/function_fields/sage/rings/function_field'

which makes the documentation diff on each pull request so large it's almost useless.

https://github.com/sagemath/sage/actions/runs/16763031702/job/47462517197

some race condition maybe?

sagemathgh-39030: Use meson in sage-the-distro       Replace the old setuptools-based build by the new meson-based one in sage-the-distro. Delete most of the old stuff that is no longer needed now. ### 📝 Checklist  - [ ] The title is concise and informative. - [ ] The description explains in detail what this PR is about. - [ ] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation and checked the documentation preview. ### ⌛ Dependencies - sagemath#40133 - sagemath#39973 - sagemath#40071    URL: sagemath#39030 Reported by: Tobias Diez Reviewer(s): Dima Pasechnik

jhpalmieri · 2025-08-23T22:49:52Z

Was there a particular reason for breaking "sage --docbuild -D" and "sage --docbuild -F"? Please fix this.

jhpalmieri · 2025-08-23T22:52:48Z

And maybe add some doctests to tests/cmdline.py to catch this.

dimpase · 2025-08-23T23:05:04Z

the main reason to break it is #39030 - the sooner we move all that to Meson, the better.

jhpalmieri · 2025-08-23T23:22:19Z

Fix it and then you can remove it once #39030 is in place.

dimpase · 2025-08-23T23:26:25Z

@jhpalmieri - please feel free to review #40588 - which does just this.

Replace the old setuptools-based build by the new meson-based one in sage-the-distro. Delete most of the old stuff that is no longer needed now. ### 📝 Checklist  - [ ] The title is concise and informative. - [ ] The description explains in detail what this PR is about. - [ ] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation and checked the documentation preview. ### ⌛ Dependencies - sagemath#40133 - sagemath#39973 - sagemath#40071 - sagemath#40597    URL: sagemath#39030 Reported by: Tobias Diez Reviewer(s): Dima Pasechnik, François Bissey, Tobias Diez

sagemathgh-39030: Use meson in sage-the-distro       Replace the old setuptools-based build by the new meson-based one in sage-the-distro. Delete most of the old stuff that is no longer needed now. ### 📝 Checklist  - [ ] The title is concise and informative. - [ ] The description explains in detail what this PR is about. - [ ] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation and checked the documentation preview. ### ⌛ Dependencies - sagemath#40133 - sagemath#39973 - sagemath#40071 - sagemath#40597    URL: sagemath#39030 Reported by: Tobias Diez Reviewer(s): Dima Pasechnik, François Bissey, Tobias Diez

sagemathgh-40686: Install pyx sources with meson Currently the meson build doesn't install the pyx source files, as opposed to the setuptools build. This has caused/is causing several issues: - The method `sage_getsource` as well as ipython's own source viewer `??` don't work for cython methods. This creates an inconsistency with python methods, and breaks several tests when the sage source is not available (eg. distro packages). - It breaks the doc builder sagemath#39973 (comment) - It used to cause issues when accessing some methods (sagemath#39735), now worked around. The only reason why these issues are not present in sage-the-distro is that cython wrongly embeds the absolute paths of source files, which masks the issues as long as the source is present in its original location. But as soon as cython/cython#6755 is released this will affect all types of sage build. In any case, I believe being able to inspect the methods implementation from the sage interface itself is a valuable feature for mathematicians, which are its main user base. I have only done one dir so far so I don't waste my time in case this is rejected or a different implementation is proposed. I will do the rest when I get the OK. Fixes sagemath#39874 URL: sagemath#40686 Reported by: Antonio Rojas Reviewer(s): Tobias Diez

user202729 reviewed Apr 19, 2025

View reviewed changes

Meson: add building of documentation as target

10fea7f

tobiasdiez force-pushed the meson-docs branch from c90e95d to 10fea7f Compare April 19, 2025 10:27

tobiasdiez added 11 commits April 20, 2025 16:09

Fix meson doc build

d7c3b6e

Fix various issues

a805a97

Fix ci

e881cf1

Fix path to sage_docbuild

af4cb32

Add shell specification for storing old documentation

0470c2f

Try with editable install

199be4e

Fix directory creation

dd44ea7

add copybutton conda pkg

5ee3c51

Work with absolute paths everywhere

7550538

Add dependency for 'reference_top' on other reference docs

230fd8f

Exclude 'reference_top' from the list of other documents in doc-inven…

ee77f03

…tory-reference

tobiasdiez mentioned this pull request May 10, 2025

instaling sage cschwan/sage-on-gentoo#804

Closed

tobiasdiez added 4 commits May 16, 2025 14:19

Merge remote-tracking branch 'origin/develop' into meson-docs

b07c0a6

Improve dependency handling

f71e282

Add dependency of 'top' docs on all other references

327a7bb

Use ccache to build sage

f6d186b

Don't create neasted doc directories

5a3c014

tobiasdiez marked this pull request as ready for review May 17, 2025 10:23

github-actions bot added the s: needs review label May 17, 2025

tobiasdiez requested a review from dimpase May 17, 2025 10:28

tobiasdiez added the p: blocker / 1 label May 17, 2025

user202729 mentioned this pull request Aug 15, 2025

Fix sage --docbuild -D #40588

Draft

5 tasks

antonio-rojas mentioned this pull request Aug 25, 2025

Install pyx sources with meson #40686

Merged

user202729 mentioned this pull request Sep 8, 2025

Faster Golay code graph construction #40695

Merged

antonio-rojas mentioned this pull request Sep 29, 2025

Fix missing conf.py files during docbuilding #40904

Merged

5 tasks

user202729 mentioned this pull request Oct 4, 2025

Migrate pdf-doc build to meson #40597

Merged

5 tasks

Uh oh!

Migrate CI build-doc to Meson #39973

Migrate CI build-doc to Meson #39973

Uh oh!

Conversation

tobiasdiez commented Apr 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📝 Checklist

⌛ Dependencies

Uh oh!

user202729 commented Apr 19, 2025

Uh oh!

user202729 Apr 19, 2025

Choose a reason for hiding this comment

Uh oh!

tobiasdiez Apr 19, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented May 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tobiasdiez commented May 17, 2025

Uh oh!

dimpase commented May 17, 2025

Uh oh!

tobiasdiez commented May 17, 2025

Uh oh!

dimpase commented May 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

antonio-rojas commented Jul 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

user202729 commented Aug 9, 2025

Uh oh!

jhpalmieri commented Aug 23, 2025

Uh oh!

jhpalmieri commented Aug 23, 2025

Uh oh!

dimpase commented Aug 23, 2025

Uh oh!

jhpalmieri commented Aug 23, 2025

Uh oh!

dimpase commented Aug 23, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

tobiasdiez commented Apr 19, 2025 •

edited

Loading

github-actions bot commented May 17, 2025 •

edited

Loading

dimpase commented May 17, 2025 •

edited

Loading

antonio-rojas commented Jul 10, 2025 •

edited

Loading