DOC, STET, and Passages update #247
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Languages whose repos are laid out one directory per chapter get deeper localization of book names and chapter names with this update.
TS will now optionally provide an additional column where the words that should be bolded in the source text are listed.
abu language has repeated USFM chapter markers
Added column containing words to bold
DOC, STET, and Passages should be at the same level and separate
Put DOC, STET, and Passages modules at same level
So that is doesn't clash with system package by same name
Not currently using focus mark on these tests
First language title for display on first page of generated document only included the first book of potentially many chosen. Fixed.
To clarify that failure was intended
This is important because in the passages app, the user might select to add NT Survey Reviewer's Guide passages en masse more than once in the UI and we would want the passages to be duplicated
In case we want to serialize to JSON a full RGBook in the future using FastAPI/Pydantic auto-encoding.
USFM layout in repo comes in mostly two forms: - one file per book, e.g., 1-GEN.usfm, or - one directory per chapter, with one file per verse span in that directory (which we pull together into one file per book) The latter layout projects a little more localization using <root>/front/title.txt and <root>/<chapter>/title.txt files which in turn results in often different (more complete) use of USFM chapter labels. Because of this, the two types have to be handled differently when splitting into chapters as the regex that is splitting hinges on is different for each case.
Requested by TS team
Also update a few python and node packages
Working to enforce consistency in generated USFM whether that USFM is the one file per book variety or the file per verse variety.
This is so because worker threads in Celery in a Docker environment have their own copy of the container including its file system. If you first request assets, via git, in a non-worker thread and then subsequently have to request them in a later step, e.g., get languages, get books, if those actions do not share the same filesystem, then you will have to clone the repos again whereas if they are both worker threads they will execute in the copy of the container used by the worker and thus git will not have to clone the repos since they were already acquired in an earlier step in the UI.
This plays better with the splitting algo by simplifying it. Now getting consistent localized chapter labels when USE_CHAPTER_LABELS is True and when USE_LOCALIZED_CHAPTER_LABEL is True.
Because docx layout is configurable, there is currently no consistent docx styled element that a template header could count on being there. This would mean that when such an element is not there, you get an error in the header instead. Eventually maybe we will have dynamic header logic for docx.
Same reason as given in the commit log for the other docx template two commits ago
Some languages, e.g., byn-reg-dan, have incorrect information in their <chapter_dir>/title.txt files: \c 1, which should have been the text (sans USFM marker) for the chapter label.
Added the ability to toggle chapter labels in DOC UI under a new optional settings area.
This should help performance both in terms of speed and in memory use because it is costly in Python to spin up a process for every repo clone we need to do when instead we can batch them into one cli call, i.e., into one process dispatch.
github actions server is slooooow
PurpleGuitar
approved these changes
Apr 4, 2025
github-project-automation
bot
moved this from In progress
to Done
in DOC Project Management
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
DOC:
STET:
Passages: