Skip to content

Releases: lance-format/lance-duckdb

v0.4.1

04 Jan 07:31
v0.4.1
da349df

Choose a tag to compare

Added

  • lance_vector_search now supports the nprobs and refine_factor parameters.

Fixed

  • Fixed vector search to accept query vectors as DuckDB ARRAY types (e.g. FLOAT[dim], DOUBLE[dim]) in addition to LIST types.
  • Fixed COPY ... (FORMAT lance) to normalize LIST vector columns so they can be used with vector search and indexing.

Commits

Full Changelog: v0.4.0...v0.4.1

v0.4.0

29 Dec 16:15
v0.4.0
2292a51

Choose a tag to compare

Added

  • Transactional DELETE support (honors BEGIN/COMMIT/ROLLBACK).
  • Support UPDATE without a WHERE clause to update all rows.
  • Predicate pushdown for string functions: starts_with, ends_with, contains.
  • Support CREATE INDEX ... ON ns.main.tbl(col) USING BTREE on attached namespaces.

Changed

  • Breaking: Object store credential/configuration now uses CREATE SECRET (TYPE LANCE, ...) (instead of TYPE S3).
  • Aligned directory and REST namespace behavior.

Fixed

  • UPDATE ... SET col = DEFAULT now works correctly.

Commits

  • docs: Polish our project goal by @Xuanwo in #106
  • feat: Allow update the full table without where by @Xuanwo in #107
  • fix: Fix UPDATE with DEFAULT support by @Xuanwo in #108
  • feat: Add starts_with/ends_with/contains pushdown by @Xuanwo in #110
  • feat: Add transaction support for delete by @Xuanwo in #112
  • feat: Add support for CREATE INDEX ... ON ns.main.tbl(col) USING BTREE by @Xuanwo in #111
  • refactor!: Use lance secrets instead by @Xuanwo in #113
  • refactor: Align namespace behavior between dir and rest by @Xuanwo in #114
  • Bump to verson 0.4.0 by @Xuanwo in #116

Full Changelog: v0.3.0...v0.4.0

v0.3.0

28 Dec 14:50
v0.3.0
d80a34b

Choose a tag to compare

Added

  • COPY ... TO ... (FORMAT lance, ...) writer for creating and appending Lance datasets.
  • Directory and REST namespaces via ATTACH ... (TYPE LANCE, ...).
  • DDL support for attached namespaces: CREATE TABLE, CTAS, DROP TABLE, ALTER TABLE (schema evolution and comments).
  • DML support for attached namespaces: INSERT, UPDATE, DELETE, TRUNCATE TABLE.
  • Index DDL on dataset URIs: CREATE INDEX, SHOW INDEXES, DROP INDEX (vector, scalar, and full-text).
  • Additional scan pushdown and planning improvements: LIMIT/OFFSET, TABLESAMPLE SYSTEM, LIKE, regexp_matches,
    IS (NOT) DISTINCT FROM, and expanded type coverage (including timestamp/decimal/struct filters).
  • DuckDB rowid / Lance _rowid support for scans, enabling point-lookup and rowid-based optimizations.
  • Basic scan statistics for improved planning.

Changed

  • Scan and maintenance helper table functions are internal-only; the user-facing surface is replacement scan, namespaces,
    search functions, and the lance copy format.
  • Improved parallelism for writing datasets.
  • Refined test coverage and organization for sqllogictests.

  • test: Add big ann tiny test by @Xuanwo in #50
  • feat: Add basic lance write support by @Xuanwo in #53
  • chore: Set dev panic=abort to avoid FFI unwind by @Xuanwo in #55
  • docs: add DuckDB write examples by @Xuanwo in #56
  • refactor: Support multi threads writing by @Xuanwo in #57
  • feat: Add filter for timestamp/decimal/struct by @Xuanwo in #58
  • feat: Add Lance Namespace support by @Xuanwo in #54
  • feat: Add CREATE TABLE support by @Xuanwo in #64
  • feat: Add INSERT INTO VALUES support by @Xuanwo in #65
  • feat: Add DROP TABLE support by @Xuanwo in #66
  • refactor: Always call lance namespace instead by @Xuanwo in #68
  • feat: Add DELETE FROM support by @Xuanwo in #69
  • feat: Add UPDATE support by @Xuanwo in #70
  • feat: Add TRUNCATE TABLE support by @Xuanwo in #72
  • refactor: Cleanup duplicated logic in DELETE by @Xuanwo in #73
  • feat: Add pushdown support for LIMIT and OFFSET by @Xuanwo in #74
  • feat: Add CREATE/DROP INDEX support by @Xuanwo in #76
  • feat: Add ALTER TABLE ADD/DROP/ALTER COLUMN support by @Xuanwo in #77
  • feat: Add LIKE pushdown support by @Xuanwo in #78
  • feat: Add basic statistics support by @Xuanwo in #81
  • feat: Refactor SHOW INDEXES by @Xuanwo in #82
  • feat: Add take rowid supports by @Xuanwo in #85
  • feat: Implement pushdown for regexp_matches by @Xuanwo in #91
  • feat: Implement TABLESAMPLE SYSTEM for lance by @Xuanwo in #87
  • feat: Add duckdb native rowid support by @Xuanwo in #93
  • feat: Add IS (NOT) DISTINCT FROM pushdown support by @Xuanwo in #94
  • refactor: Don't expose REINDEX as it's not part of duckdb SQL by @Xuanwo in #95
  • docs: Add a full references to SQL by @Xuanwo in #97
  • refactor: Never expose unneed table functions by @Xuanwo in #98
  • ci: Refactor sqllogictests by @Xuanwo in #96
  • Bump to version 0.3.0 by @Xuanwo in #99

Full Changelog: v0.2.0...v0.3.0

v0.2.0

25 Dec 09:38
v0.2.0
5d3c43d

Choose a tag to compare

Added

  • Filter and projection pushdown for scans on Lance datasets.
  • S3 authentication via DuckDB Secrets for s3://... dataset paths.
  • EXPLAIN (FORMAT JSON) diagnostics for Lance scans (bind-time and runtime plan details).
  • lance_vector_search(path, vector_column, vector, ...) table function.
  • lance_fts(path, text_column, query, ...) table function.
  • lance_hybrid_search(path, vector_column, vector, text_column, query, ...) table function.

Changed

  • Optimized SELECT COUNT(*) on Lance datasets.
  • Improved error propagation across the Rust FFI boundary.
  • Upgraded the Lance dependency to v1.0.0 (including Hugging Face backend support).

Detailed PRs

  • feat: Support filter pushdown to lance by @Xuanwo in #24
  • feat: Bump lance to version 1.0 by @Xuanwo in #23
  • refactor: Handling ffi errors correctly by @Xuanwo in #25
  • feat: Add last error API for ffi by @Xuanwo in #26
  • docs: Add docs for FFI ownership by @Xuanwo in #27
  • docs: Add docs for install from community by @Xuanwo in #28
  • feat: Optimize for SELECT COUNT(*) by @Xuanwo in #29
  • test: Add more push down support by @Xuanwo in #30
  • feat: Enable huggingface support by @Xuanwo in #33
  • docs: Polish README by @Xuanwo in #34
  • test: Add TPC-H test cases by @Xuanwo in #32
  • refactor: Build a simple filter expr ir to replace SQL by @Xuanwo in #38
  • feat: Add duckdb secrets support by @Xuanwo in #37
  • feat: Add big ann test cases by @Xuanwo in #39
  • docs: Add section for formatting by @Xuanwo in #40
  • feat: Add explain support for lance by @Xuanwo in #43
  • feat: Add lance_search support for {vector,full-text,fuzzy} search by @Xuanwo in #45
  • refactor: Re-org the cpp and rust code by @Xuanwo in #46
  • chore: Cleanup old lance_knn functions by @Xuanwo in #47
  • refactor: Redesign lance's search API by @Xuanwo in #48
  • Bump to version 0.2.0 by @Xuanwo in #49

Full Changelog: v0.1.0...v0.2.0

v0.1.0

22 Dec 23:30
v0.1.0
a1c75b4

Choose a tag to compare

What's Changed

  • feat: add basic read support and test by @Xuanwo in #1
  • feat: Implement replacement rule correctly by @Xuanwo in #2
  • refactor: Migrate to cpp based duckdb layout by @Xuanwo in #5
  • feat: make select * from x.lance work by @Xuanwo in #8
  • docs: Polish README and AGENTS.md by @Xuanwo in #9
  • ci: Remove not needed protoc wrapper by @Xuanwo in #11
  • feat: Support duckdb 1.4.3 instead by @Xuanwo in #10
  • refactor: Cleanup not used dead code by @Xuanwo in #14
  • feat: Enable global runtime for duckdb by @Xuanwo in #16
  • ci: Don't build parquet extension by @Xuanwo in #17
  • ci: Enable mold for linking by @Xuanwo in #18
  • ci: Include build config in to cache key by @Xuanwo in #19
  • feat: Support concurrent executing at fragments level by @Xuanwo in #20
  • chore: prepare v0.1.0 by @Xuanwo in #21
  • ci: Try fix distribution by @Xuanwo in #22

New Contributors

  • @Xuanwo made their first contribution in #1

Full Changelog: https://github.com/lance-format/lance-duckdb/commits/v0.1.0