Skip to content

feat(api): sync reading progress bidirectionally between KOReader and Grimmory web reader#305

Open
Michael-Tanzer wants to merge 1 commit intogrimmory-tools:developfrom
Michael-Tanzer:feat/koreader-bidirectional-progress-sync
Open

feat(api): sync reading progress bidirectionally between KOReader and Grimmory web reader#305
Michael-Tanzer wants to merge 1 commit intogrimmory-tools:developfrom
Michael-Tanzer:feat/koreader-bidirectional-progress-sync

Conversation

@Michael-Tanzer
Copy link
Copy Markdown

@Michael-Tanzer Michael-Tanzer commented Mar 31, 2026

Summary

  • Adds bidirectional reading progress sync between KOReader devices and Grimmory's built-in web reader. When a user reads in the web reader, the progress is converted from EPUB CFI to
    KOReader XPointer format and written to the KOReader progress columns, so KOReader picks it up on next sync.
  • Renames the sync toggle endpoint from /sync-progress-with-booklore to /sync-progress-with-grimmory, fixing the PATCH 405 error (Sync reading progress with Grimmory eBook Reader PATCH error #191) caused by the frontend calling the new name
    while the backend only had the old one.
  • Removes the legacy /sync-progress-with-booklore endpoint and the frontend's catchError fallback to it.

What changed

Backend (4 production files):

File Change
KoreaderUserController.java Renamed endpoint to /sync-progress-with-grimmory, removed legacy /sync-progress-with-booklore
ReadingProgressService.java Added syncProgressToKoreader() — converts CFI → XPointer via EpubCfiService.convertCfiToProgressXPointer() and writes to koreader_* columns when
the web reader saves EPUB progress. Conversion failures are caught and logged without interrupting the save.
KoreaderUser.java Renamed DTO field syncWithBookloreReadersyncWithGrimmoryReader
KoreaderUserMapper.java Added @Mapping to bridge entity field name (syncWithBookloreReader) to DTO field name (syncWithGrimmoryReader)

Frontend (3 files):

File Change
koreader.service.ts Removed catchError fallback to legacy endpoint, removed syncWithBookloreReader from interface
koreader-settings-component.ts Removed ?? koreaderUser.syncWithBookloreReader fallback
koreader.service.spec.ts Replaced fallback test with direct endpoint test

No database migrations. All required columns already exist.

How it works

The forward sync (KOReader → web reader) already existed — KoreaderService.saveProgress() converts XPointer → CFI when the sync toggle is enabled. This PR adds the reverse path:

  1. User reads in the web reader → ReadingProgressService.updateReadProgress() saves CFI + percentage
  2. If the user has a linked KOReader account with syncWithBookloreReader = true:
    • Converts CFI → XPointer via EpubCfiService.convertCfiToProgressXPointer()
    • Writes to koreader_progress, koreader_progress_percent, koreader_device = "Grimmory", koreader_last_sync_time = now
  3. On next KOReader sync, GET /api/koreader/syncs/progress/{hash} returns the web reader's position

Position conversion is chapter-approximate, not character-exact. This is documented in the existing UI description for the toggle.

Test output

Backendjust api test (3496 tests, 0 failures):
BUILD SUCCESSFUL in 2m 24s
6 actionable tasks: 4 executed, 2 up-to-date

Frontendjust ui check (1421 tests, 0 failures):
Test Files 271 passed | 106 skipped (377)
Tests 1421 passed | 173 skipped (1594)

New test coverage (6 tests added)

Test Verifies
updateReadProgress_withKoreaderSyncEnabled_shouldWriteKoreaderColumns Reverse sync writes XPointer, percentage/100, device="Grimmory", timestamp
updateReadProgress_withKoreaderSyncDisabled_shouldNotWriteKoreaderColumns Toggle off → no conversion, no KOReader writes
updateReadProgress_withNoKoreaderAccount_shouldNotAttemptSync No KOReader account → no-op
updateReadProgress_conversionFailure_shouldNotBreakSave CFI→XPointer failure is caught, save succeeds
toggleSyncProgressWithGrimmory_returnsNoContent New endpoint returns 204
koreader.service.spec: patches the Grimmory sync-progress toggle Frontend calls correct endpoint

Known limitations

  • Position sync is EPUB-only. PDF/CBX/audiobook sync is percentage-only (no position conversion).
  • convertCfiToProgressXPointer() existed but was previously unused — validated by new tests.
  • The entity field is still named syncWithBookloreReader (DB column: sync_with_booklore_reader). A follow-up Flyway migration to rename it is tracked as TODO(grimmory-rename) in
    the mapper.
  • The reverse sync reads epubProgress / epubProgressPercent which are @Deprecated on the entity. Tracked as TODO(deprecated-fields) in the sync method.

Summary by CodeRabbit

  • New Features

    • Added automatic EPUB reading progress synchronization with Grimmory reader device.
    • Reading progress now syncs conditionally based on user preferences.
  • Changes

    • Updated device sync settings to reference Grimmory reader.
    • API endpoints and user interface updated to support Grimmory synchronization.

… Grimmory web reader

Add reverse sync path: when the web reader saves EPUB progress, convert
CFI to XPointer via EpubCfiService.convertCfiToProgressXPointer() and
write to koreader_* columns on UserBookProgressEntity. KOReader picks up
the updated position on next sync.

Rename the sync toggle endpoint from /sync-progress-with-booklore to
/sync-progress-with-grimmory, fixing the PATCH 405 error (grimmory-tools#191). Remove
the legacy endpoint and the frontend's catchError fallback.

Rename the DTO field syncWithBookloreReader to syncWithGrimmoryReader
with a MapStruct @mapping to bridge the entity's unchanged column name.

Closes grimmory-tools#225
Fixes grimmory-tools#191
@coderabbitai

This comment was marked as resolved.

coderabbitai[bot]

This comment was marked as resolved.

@balazs-szucs
Copy link
Copy Markdown
Member

Did you test this? Besides automated testing, I mean?

@balazs-szucs
Copy link
Copy Markdown
Member

Seems suspiciously lean PR considering what it the PR message claims to have achieved.

Are you sure we are not overgeneralising this? Intuitively, I feel like there is much more to this than what's proposed here.

I'm 100% ready to be proven absolutely wrong. But this PR may need more consideration than "LGTM! -> Merge"

Copy link
Copy Markdown
Member

@zachyale zachyale left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As progress conversion is a bit lossy, and can fall back to chapter-level accuracy instead of page-level accuracy, have you experienced a loss in position accuracy when using just the web reader with this approach?

For ex:

  • Open web reader
    • Progress is pulled from latest koreader progress
  • Progress is completed during the course of the reading session
  • Web reader is closed
  • Some time later (without having read on another device), the user opens the web reader
    • Progress is pulled from latest Koreader progress

In this case would progress not be lost/degraded by using the newly updated Koreader position instead of the more accurate Grimmory stored CFI?

public ResponseEntity<Void> toggleSyncProgressWithBooklore(
public ResponseEntity<Void> toggleSyncProgressWithGrimmory(
@Parameter(description = "Enable or disable sync progress") @RequestParam boolean enabled) {
koreaderUserService.toggleSyncProgressWithBooklore(enabled);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you're going to rename/complete the deprecation of the booklore related endpoint, since this endpoint is the only implementation of the toggleSyncProgressWithBooklore method we should either also rename it here, or create an issue to rename the implements within the KoreaderUserService

@Michael-Tanzer
Copy link
Copy Markdown
Author

So sorry, had a bit of a fallout at work and I will test it more thoroughly and reply to all comments during the weekend

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants