Skip to content

Rename misleading metadata.fileSize field to textLength #77

Description

@shinpr

Problem

metadata.fileSize in VectorChunk records stores text.length (character count of parsed text content), not the actual file size in bytes. This naming is misleading.

The field is set identically in two places:

  • src/server/index.ts (handleIngestFile, L322)
  • src/cli/ingest.ts (ingestSingleFile)

Both use:

metadata: {
  fileName: ...,
  fileSize: text.length,  // ← this is text character count, not file size
  fileType: ...,
}

Proposed solution

Rename fileSize to textLength (or contentLength) to accurately reflect what the field stores.

Migration consideration

Existing records in LanceDB already have fileSize in their metadata. Options:

  • Add textLength alongside fileSize for new records, read both on query (backward-compatible)
  • Schema migration to rename the field in existing records
  • Accept inconsistency between old and new records (metadata is informational, not used for search/filtering)

Impact

  • src/server/index.ts — handleIngestFile metadata construction
  • src/cli/ingest.ts — ingestSingleFile metadata construction
  • src/vectordb/types.ts — if metadata type is defined
  • Tests referencing fileSize in metadata

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions