Feature request: sync clippings and bookmarks with external library servers (BookLore)

[rodgui]

Summary

Would it be possible to add a generic synchronization interface for CrossInk clippings and bookmarks, initially targeting integration with self-hosted library servers such as BookLore?

CrossInk already has strong local support for My Clippings and Bookmarks. Allowing these records to be synchronized would make them available outside the device and enable users to keep annotations consistent between CrossInk and their self-hosted library.

Use case

I use an Xteink X4 with CrossInk and host my ebook library in BookLore.

The desired workflow would be:

1. Create a clipping/highlight or bookmark on the X4.
2. Connect the device to Wi-Fi and run synchronization.
3. Import the clipping into the corresponding book's Notes in BookLore.
4. Import the bookmark into BookLore's bookmark system.
5. Optionally receive changes made in BookLore back on the X4.

A useful first version could be upload-only. Bidirectional sync could be added later.

Why a generic interface instead of a BookLore-only integration

The same foundation could support other self-hosted tools and personal knowledge systems, including BookLore, Kavita, Komga, Calibre-Web, custom annotation servers, and personal knowledge bases.

This would keep CrossInk independent from any specific server while enabling adapters on the server side.

Suggested scope

Phase 1: export/sync infrastructure

• Stable UUID for each clipping and bookmark
• Book identifier independent of the local file path
• Creation/update timestamp
• Deletion tombstones
• JSON serialization
• Incremental synchronization using a cursor or last-sync timestamp
• Manual Sync now action
• Device identifier

Phase 2: bidirectional synchronization

• Download remote changes
• Update and delete local records
• Basic conflict resolution
• Sync status and error reporting
• Optional automatic sync when Wi-Fi is available

Suggested architecture

CrossInk could act as the HTTP client and send changes to a configurable endpoint:

CrossInk/Xteink X4
        |
        | HTTPS + JSON
        v
External sync endpoint
        |
        +-- BookLore adapter
        +-- Kavita adapter
        +-- Komga adapter
        +-- Custom server

Having the device initiate the request avoids requiring the external server to discover the X4 on the local network.

Possible API payload

This is only an example to illustrate the data needed; the final contract could be much smaller.

{
  "deviceId": "xteink-x4-example",
  "cursor": "previous-sync-cursor",
  "books": [
    {
      "bookIdentifier": "urn:isbn:9780000000000",
      "title": "Example Book",
      "author": "Example Author",
      "fileName": "example.epub",
      "clippings": [
        {
          "id": "clip-uuid",
          "operation": "upsert",
          "spineIndex": 4,
          "paragraphIndex": 18,
          "startWordIndex": 5,
          "endWordIndex": 22,
          "chapterTitle": "Chapter 4",
          "text": "Selected text...",
          "createdAt": 1784211690,
          "updatedAt": 1784211690
        }
      ],
      "bookmarks": [
        {
          "id": "bookmark-uuid",
          "operation": "upsert",
          "spineIndex": 4,
          "paragraphIndex": 18,
          "progress": 0.4265,
          "chapterTitle": "Chapter 4",
          "snippet": "Beginning of paragraph...",
          "updatedAt": 1784211900
        }
      ]
    }
  ]
}

Book identification

Matching only by filename may create errors after renames or when different editions exist. A possible priority order would be:

1. EPUB package identifier / ISBN from content.opf
2. File hash calculated once and cached
3. Normalized title + author + file size
4. Manual association on the server

The exact matching could remain the server's responsibility; CrossInk would only expose enough metadata.

EPUB position interoperability

CrossInk currently has useful location context such as spine index, paragraph index, word indexes, progress, chapter title, and snippets. External EPUB readers commonly use EPUB CFI.

Possible approaches:

1. Keep the generic CrossInk location model and let each server adapter resolve it.
2. Optionally expose EPUB CFI in the future if the reader/parser can generate it reliably.

The first approach would avoid making CFI generation a prerequisite for the initial feature.

Local storage and compatibility

Because bookmark and clipping structures are persisted on disk, this feature should avoid silently breaking existing files. Possible options:

• Increment the record/file format version and provide migration, or
• Store synchronization metadata in sidecar files under a dedicated sync directory.

Example sidecar responsibilities:

• local UUID to remote ID mapping
• local/remote revision
• deletion state
• last successful sync cursor

This complements the record-versioning work discussed in #301.

Conflict handling

A simple initial policy would probably be sufficient:

• Independent creations: keep both
• Same record edited on both sides: latest updatedAt wins
• Delete versus edit: latest operation wins
• Deduplicate by stable UUID, not by selected text
• Retain deletion tombstones for a limited period

Resource constraints

Since this runs on an ESP32-class device, the implementation could use:

• small batches, for example 10–20 records
• incremental cursors
• streaming or bounded JSON serialization
• no full-history upload on every sync
• configurable request timeout

Authentication

A simple per-device bearer token would be enough for self-hosted servers:

Authorization: Bearer <device-token>
X-CrossInk-Device-ID: <device-id>

Credentials could be configured together with the server URL.

Minimum viable implementation

The smallest useful feature would be:

1. Assign stable IDs to local clippings.
2. Export only changed clippings as JSON.
3. POST them to a configured HTTPS endpoint.
4. Store the returned sync cursor/mapping.
5. Avoid uploading the same clipping twice.

Bookmarks and bidirectional synchronization could follow after this flow is proven.

Expected benefit

This would turn My Clippings and Bookmarks into portable user-owned data rather than device-only records. It would also create a reusable integration point for the broader self-hosted ebook ecosystem without coupling CrossInk to one application.

I would be interested in helping test this workflow with an Xteink X4 and a self-hosted BookLore instance.