Skip to content

Latest commit

 

History

History
1081 lines (545 loc) · 52.4 KB

CHANGELOG.md

File metadata and controls

1081 lines (545 loc) · 52.4 KB

CHANGELOG

v0.2.0 (2024-06-03)

Chore

  • chore: use variable instead of constant in setup.cfg (#22)

Co-authored-by: ds-sebastian-chwilczynski <[email protected]> (3745054)

  • chore: cleanup after release process (1bf88e9)

Documentation

  • docs: update guides and API refs (#38) (bc912c1)

  • docs: update roadmap (adfd479)

Feature

  • feat: freeform text2sql with static configuration (#36)

Co-authored-by: Ludwik Trammer <[email protected]> Co-authored-by: Michał Pstrąg <[email protected]> (8f7a166)

  • feat: Add elastic store (#34)

  • add elastic search store

  • feat add vector search store

  • update documentation (edd6de9)

  • feat(embedding): add litellm embeddings (#37) (2fb275f)

  • feat(llm): integrate LLMClient with litellm (#35) (c06b11d)

  • feat(audit): add audit events for SimilarityIndexes (#33)

  • feat(audit): add audit events for SimilarityIndexes

  • Apply suggestions from code review

Co-authored-by: Michał Pstrąg <[email protected]>

Co-authored-by: Michał Pstrąg <[email protected]> (59e08ac)

Fix

  • fix: AttributeError in LangSmithEventHandler (#32) (c8cfe97)

Refactor

  • refactor(views): api for freeform views (#41) (6d6cdcd)

  • refactor: refactor prompt builder (#30) (61f1066)

  • refactor: add LLMOptions merge (#29) (7191c3f)

  • refactor: refactor LLMOptions (#28) (3f5c4bf)

  • refactor: move prompt templates to modules which use them (#26)

  • move prompt templates to modules which use them

  • reorganize imports (e0f17a7)

  • refactor: add monotonic clock (#25) (9b31299)

v0.1.0 (2024-04-25)

Documentation

  • docs(views): add documentation of freeform views (#21)

Add documentation of freeform views, comparision of structured & freeform views.

Co-authored-by: Mateusz Hordyński <[email protected]> Co-authored-by: Mateusz Hordyński <[email protected]> (668b2ce)

  • docs: minor issues with classes names (#19)

Co-authored-by: ds-sebastian-chwilczynski <[email protected]> (4a26cbb)

Feature

  • feat: add text2sql freeform view (#20)

Add new freeform view: text2sql - direct creation of SQL query to answer user question together with initial setup of database discovery. (d6dc713)

  • feat(views): allow for freeform views (#15)

Ground work for supporting a new type of views: freeform. Freeform views are not bounded by IQL and can answer natural language questions directly. (61ab8c9)

  • feat: automatic versioning (#16) (b7ffa25)

Fix

  • fix: add exception handling to LangGraph notebook

Co-authored-by: ds-sebastian-chwilczynski <[email protected]> (6b607d6)

Unknown

  • Feat: Integration with Chromadb (#18)

  • feat: chroma db support

  • fix: hash issues

  • test: integration chroma testing

Co-authored-by: ds-sebastian-chwilczynski <[email protected]> Co-authored-by: Mateusz Hordyński <[email protected]> (e455d53)

v0.0.2 (2024-04-17)

Fix

Unknown

  • Bump version: 0.0.2-dev → 0.0.2 (669973a)

  • Bump version: 0.0.1 → 0.0.2-dev (ec46c92)

  • Remove the use_* global functions (#13)

  • Remove the use_* global functions

  • Remove use_openai_llm from notebooks (e3bcf06)

  • Integrate tutorials with the repository. (#14)

  • upload tutorials

  • provide link to collab

  • remove raw tutorials info

Co-authored-by: ds-sebastian-chwilczynski <[email protected]> (211337d)

  • API Reference refactor

Co-authored-by: ds-sebastian-chwilczynski <[email protected]> (7c16a24)

  • Allow injecting dependencies in calls to create_collection (#12)

  • Allow injecting dependencies in calls to create_collection

  • Make the create_collection description more detailed (b3251df)

  • Add Collection methods to detect/update similarity indexes (#11)

  • Add Collection methods to detect/update similarity indexes

  • Handle exceptions during index update (bb53811)

  • Use full company name in license (93c601d)

  • Fix tests (#10)

A combination of two merged commits that worked separately broke tests.

This fixes the problem. (7e8736f)

  • Add API and CLI command for updating multiple indexes at one (#4)

  • Add API and CLI command for updating multiple indexes at one

  • Add documentation and additional tests (a2e37b5)

  • Merge pull request #9 from deepsense-ai/mh/move_examples

Move examples out of dbally package (3191f24)

  • Separate view-level and collection-level results (#8)

  • Separate view-level and collection-level results

  • Add tests for Collection.ask

  • Add documentation (89062dc)

  • Move examples out of dbally package (9b9effe)

  • Add validation of view builder during registration (#6) (74892b9)

  • Fix imports in README example (#5) (6a4c2df)

  • Api Reference

  • Add API references

  • Add changes in the mkdocs.yml file

  • Add changed mkdocs file

  • Fix pylint

  • Small fix

  • Add faiss store

  • Small fix

  • Add LLMViewSelector

  • Small fix

  • Add reference docs

  • start refactoring documentation

  • update roadmap with assistants

Co-authored-by: alicja.kotyla <[email protected]> Co-authored-by: Mateusz Hordyński <[email protected]> Co-authored-by: ds-sebastian-chwilczynski <[email protected]> (8288802)

  • Extra configuration for docs (6a6a189)

  • Add how to guides on custom similarity fetchers nad stores (#3)

  • Add how to guides on custom similarity fetchers nad stores

  • Add info about async

  • async -> await (e2a4047)

  • Add gcp env to docs build process (ee6f6ca)

  • Bump version 0.0.1-dev -> 0.0.1 (0a51910)

  • Add source to setup.cfg (0f8be41)

  • Change license in setup.cfg (82d2c79)

  • Fix docs build script (978f45d)

  • Fix build script (161d100)

  • Add environment (e491e76)

  • Docs build on GH actions (a468479)

  • Merge pull request #1 from deepsense-ai/mh/github_actions

Setup github actions (7a2a7a2)

Sc/tutorial

See merge request deepsense.ai/g-internal/db-ally!75 (d62a61d)

  • OpenAI Assistants API tutorial (56206a4)

  • Merge branch 'mh/pypi_final' into mh/github_prep (f7eb52f)

  • Merge branch 'lt/dataframe' into 'main'

Introduce Pandas DataFrame base view

See merge request deepsense.ai/g-internal/db-ally!74 (380c4b7)

  • Introduce Pandas DataFrame base view (77131f1)

  • Merge branch 'sc.openai_integration' into 'main'

OpenAI assistants adapter

See merge request deepsense.ai/g-internal/db-ally!69 (c396073)

  • OpenAI assistants adapter (f96c041)

  • Merge branch 'lt/custom_views_how_to' into 'main'

How-To: Write Custom Views

See merge request deepsense.ai/g-internal/db-ally!72 (dc5c8f9)

  • How-To: Write Custom Views (dbe3874)

  • Merge branch 'mh/fix_docs_feedback2' into 'main'

Fix after feedback

See merge request deepsense.ai/g-internal/db-ally!73 (e028497)

  • Fix after feedback (b26f4f1)

  • Merge branch 'mh/fix_transformers_dep' into 'main'

Fix transformers dependency

See merge request deepsense.ai/g-internal/db-ally!71 (f7d3ae2)

  • Add transformers to dev packages (c76160c)

  • Fix circular import (9d4cb49)

  • Fix transformers dependency (3e37723)

  • Merge branch 'mh/docs_dark_view' into 'main'

Add dark theme

See merge request deepsense.ai/g-internal/db-ally!68 (903adab)

  • Add dark theme (3abba3f)

  • Merge branch 'mh/docs_fixes_after_feedback' into 'main'

Fix links

See merge request deepsense.ai/g-internal/db-ally!67 (1440fa3)

  • IQL explained (49f3203)

  • Fix links (ef9dff8)

  • Merge branch 'mh/clean_docs_nav' into 'main'

Remove not finished docs from nav

See merge request deepsense.ai/g-internal/db-ally!66 (4465cd1)

  • Remove not finished docs from nav (a3d0543)

  • Merge branch 'mh/roadmap' into 'main'

Add roadmap

See merge request deepsense.ai/g-internal/db-ally!65 (66bad62)

  • Apply suggestions (770a5cb)

  • Add roadmap (b4e7f58)

  • Merge branch 'az/add-readme' into 'main'

Add readme

See merge request deepsense.ai/g-internal/db-ally!56 (bff2451)

  • Fix extensions (52887b5)

  • Merge branch 'main' into az/add-readme (00748c9)

  • Fix whitespace (2f70004)

  • Merge branch 'sc/tutorial' into 'main'

Tutorial

See merge request deepsense.ai/g-internal/db-ally!53 (e9b28b8)

  • First tutorial (92a2479)

  • Apply 1 suggestion(s) to 1 file(s) (be0499d)

  • Add motivation and make sure that README is ready (26e3001)

  • Pypi setup (2593cf3)

  • Merge branch 'main' into az/add-readme (4d7d6e1)

  • Merge branch 'az/add-readme' into mh/pypi_final (5ececeb)

  • Merge branch 'lt/how_to_sql' into 'main'

How-to: Use SQL databases with db-ally

See merge request deepsense.ai/g-internal/db-ally!63 (a221468)

  • How-to: Use SQL databases with db-ally (30bd21e)

  • Merge branch 'ak/fix-typo' into 'main'

Fix typo

See merge request deepsense.ai/g-internal/db-ally!64 (a6fff56)

  • Fix typo (b13ccfe)

  • Merge branch 'main' into az/add-readme (6089736)

  • Merge branch 'ak/cleanup-package-build' into 'main'

Cleanup package build

See merge request deepsense.ai/g-internal/db-ally!62 (30d0a3e)

  • Cleanup package build (4bfd0c5)

  • Merge branch 'mh/event_handler_docs' into 'main'

Event Handlers documentation

See merge request deepsense.ai/g-internal/db-ally!57 (f3d7808)

  • Merge branch 'lt/concepts' into 'main'

Documentation: concepts

See merge request deepsense.ai/g-internal/db-ally!59 (d532c08)

  • Documentation: concepts (2db6bbe)

  • Merge branch 'ak/nl-responder-concept' into 'main'

Add NL Responder docs

See merge request deepsense.ai/g-internal/db-ally!61 (57c136a)

  • Add NL Responder docs (e6787a6)

  • Fix wording (9f3bc95)

  • Add create custom event handler guide (5357303)

  • Merge branch 'mh/fix_packaging' into 'main'

Add init files

See merge request deepsense.ai/g-internal/db-ally!60 (62d9b58)

  • Add init files (ea70146)

  • Merge branch 'ak/add-build-docs-stage' into 'main'

Build docs while running CI/CD pipeline

See merge request deepsense.ai/g-internal/db-ally!58 (8345e28)

  • Build docs while running CI/CD pipeline (bd61756)

  • Add EventHandler reference (dd6a231)

  • README formatting (10c25d4)

  • update readme file (45cbd17)

  • Add LangSmith how-to (e14dd0a)

  • initial readme intro (bf828b1)

  • Merge branch 'lt/quickstart' into 'main'

Add the Quickstart guide

See merge request deepsense.ai/g-internal/db-ally!54 (d022a1c)

  • Fix a typo (e25a91a)

  • Mention the tutorial (62f2810)

  • Small improvements to the text (34983e0)

  • Merge branch 'main' into 'lt/quickstart'

Conflicts:

mkdocs.yml (20c1938)

  • Merge branch 'mh/docs_nav' into 'main'

Mock docs navigation

See merge request deepsense.ai/g-internal/db-ally!55 (8489940)

  • Merge branch 'ak/tweak-iql-prompt' into 'main'

Tweak IQL prompt (and remove actions)

See merge request deepsense.ai/g-internal/db-ally!51 (ad1c6ff)

  • Tweak IQL prompt (and remove actions) (f5ba341)

  • Mock docs navigation (6be0a0c)

  • Move quickstart to a separate category (b455ab6)

  • Add expected output (7148374)

  • Review changes (92e24f6)

  • Merge branch 'sc/example' into 'main'

Initial dbally example

See merge request deepsense.ai/g-internal/db-ally!49 (15f45b8)

  • Initial dbally example (7259e33)

  • Add the Quickstart tutorial (55d4699)

  • Merge branch 'mh/annotated_similarity' into 'main'

Annotated similarity index syntax

See merge request deepsense.ai/g-internal/db-ally!48 (b3cb41f)

  • Merge branch 'mh/mkdocs' into 'main'

Basic mkdocs setup

See merge request deepsense.ai/g-internal/db-ally!52 (d6b92a4)

  • Basic mkdocs setup (f435f2e)

  • Merge branch 'lt/faiss' into 'main'

FAISS-based similarity store + OpenAI embedding client

See merge request deepsense.ai/g-internal/db-ally!50 (567191e)

  • FAISS-based similarity store + OpenAI embedding client (cfcc95e)

  • Use Annotated from typing_extensions; parser -> processor (6ee3e4a)

  • Merge branch 'ak/tweaks-in-nl-responder' into 'main'

Tweaks in NLResponder

See merge request deepsense.ai/g-internal/db-ally!47 (0504675)

  • Change function name (2d26d17)

  • Annotated similarity for python 3.8 (edba405)

  • Fix tests (6575eef)

  • Undo lint fixes (37b6b4c)

  • Add tweaks in NLResponder (8395291)

  • More changes (d2a7cb3)

  • Initial changes (37b7d04)

  • Merge branch 'sc/fix-gitlab-test' into 'main'

fix: gitlab evaluation

See merge request deepsense.ai/g-internal/db-ally!46 (71e644c)

  • Merge branch 'lt/async_views' into 'main'

Support for async indexes and views

See merge request deepsense.ai/g-internal/db-ally!45 (ca7f521)

  • Support for async indexes and views (df7cb0f)

  • Merge branch 'lt/initial_similarity' into 'main'

Initial implementation of similarity index

See merge request deepsense.ai/g-internal/db-ally!41 (48e3bdb)

  • Merge branch 'sc/multidb_benchmarking' into 'main'

Multi-db benchmarking

See merge request deepsense.ai/g-internal/db-ally!43 (baf6cb8)

  • Multi-db benchmarking (79ac204)

  • Merge branch 'lt/cli' into 'main'

Setup click-based CLI

See merge request deepsense.ai/g-internal/db-ally!44 (42a65a8)

  • Setup click-based CLI (6874903)

  • Merge branch 'ak/iql-benchmark' into 'main'

IQL benchmark

See merge request deepsense.ai/g-internal/db-ally!39 (f90048a)

  • IQL benchmark (f8a7b50)

  • Initial implementation of similarity index (7f8e3dd)

  • Merge branch 'mh/better_iql_preprocessing' into 'main'

Better IQL preprocessing

See merge request deepsense.ai/g-internal/db-ally!38 (ba29bee)

  • Merge branch 'ak/hotfix-response-format' into 'main'

Add response_format argument for OpenAI's models

See merge request deepsense.ai/g-internal/db-ally!40 (4bd176f)

  • Add comment (c19b439)

  • Add response_format argument for turbo models (53decb7)

  • Better IQL preprocessing (8393908)

  • Merge branch 'sc/fallback' into 'main'

Fallback

See merge request deepsense.ai/g-internal/db-ally!36 (21ba589)

  • add: Fallback mechanism (68ee750)

  • Merge branch 'mh/cast_types' into 'main'

Cast basic types in IQL functions

See merge request deepsense.ai/g-internal/db-ally!35 (7cd605b)

  • Merge branch 'mh/better_iql_generation_prompt' into 'main'

Better IQL example

See merge request deepsense.ai/g-internal/db-ally!37 (8568abd)

  • Better IQL example (b109fc6)

  • Change reasons in ValidationErrors (052ec9d)

  • Fix literal test (5d25d42)

  • Cast basic types (8bbd0f4)

  • Merge branch 'pk/benchmark-adjustment' into 'main'

Benchmark calculate_exec_acc improvement, superhero example improvement

See merge request deepsense.ai/g-internal/db-ally!32 (a87743b)

  • Benchmark calculate_exec_acc improvement, superhero example improvement (c722104)

  • Merge branch 'sc/pretty-printing' into 'main'

Pretty printing

See merge request deepsense.ai/g-internal/db-ally!34 (4472a28)

  • Pretty printing (948216a)

  • Merge branch 'mh/iql_validation' into 'main'

Validate IQL methods / arguments in IQLParser

See merge request deepsense.ai/g-internal/db-ally!33 (dec9a50)

  • Merge branch 'ak/add-nl-responder' into 'main'

Add natural language reasoning component

See merge request deepsense.ai/g-internal/db-ally!26 (4ff6acd)

  • Merge branch 'main' into ak/add-nl-responder (1a5bc86)

  • Limit rows and columns (cac7144)

  • Fix mypy (56667db)

  • Validate IQL methods / arguments in IQLParser (73fcd0b)

  • Merge branch 'lt/collection_builder' into 'main'

Tests for using a collection with a view builder

See merge request deepsense.ai/g-internal/db-ally!31 (fa601cc)

  • Change prompt (d1597cb)

  • Tests for using a collection with a view builder (5d39338)

  • Docstring improvement after comment (da9dbee)

  • Remove redundant print (7d26241)

  • More changes (fb0fa4e)

  • More changes after rebase (20ff30f)

  • Draft of changes after rebase (a6f390b)

  • Fix docstrings (cd831bc)

  • Don't limit dataset (5e531be)

  • Fix mypy and benchmarks (95d2d79)

  • Add tests and fix requirements (c5de7b5)

  • Include metadata in answer (641b3dc)

  • Small fix (f032e50)

  • Add natural language reasoning component (9a35f8b)

  • Add initial changes in examples (57196b1)

  • Merge branch 'lt/execute' into 'main'

Better integrate view execution with the rest of the library

See merge request deepsense.ai/g-internal/db-ally!27 (1410df6)

  • Better integrate view execution with the rest of the library (920a654)

  • Merge branch 'lt/passing_view_parameters' into 'main'

Support for passing view-specific parameters

See merge request deepsense.ai/g-internal/db-ally!25 (9426327)

  • Support for passing view-specific parameters (459c69d)

  • Merge branch 'mh/langsmith_event_handler' into 'main'

Langsmith event handler

See merge request deepsense.ai/g-internal/db-ally!23 (abb915d)

  • Merge branch 'ak/add-dbally-benchmark' into 'main'

Integrate dbally with benchmark

See merge request deepsense.ai/g-internal/db-ally!20 (4c50194)

  • Add psycopg2 to whitelist (856d23f)

  • Fix benchmark stage (11d0c5a)

  • Fix tests (6363a32)

  • Fix mypy (8873275)

  • Set of fixes (9a248e5)

  • Remove tokens from config (8ce941d)

  • Add first version of benchmark (e35b4c6)

  • Add tokens to LLMEvent (3404e46)

  • Add langsmith handler (4099889)

  • Add collection_name to request start (da79170)

  • Remove extra typing (9c59313)

  • Use generics in event handlers (15821a4)

  • Fix typings (f02f1e3)

  • Make event handlers stateless (cfdecab)

  • Merge branch 'ak/auditability-framework' into 'main'

Auditability framework

See merge request deepsense.ai/g-internal/db-ally!22 (061fe2c)

Benchmark in CI/CD

See merge request deepsense.ai/g-internal/db-ally!19 (854fb6e)

View selection

See merge request deepsense.ai/g-internal/db-ally!17 (e82801b)

  • Fix tests (openai missing) (f927075)

  • Merge branch 'mh/add_retail_shop' into 'main'

Add clothes retail database

See merge request deepsense.ai/g-internal/db-ally!18 (4b58f3d)

  • Add clothes retail database (efa7612)

  • Fix test (36abc94)

  • Integrate IQL generator and view selector with a pipeline (a5ab9b1)

  • unit test (2bbd8d8)

  • DefaultViewSelector implementation (4238dcf)

  • superhero view2 tweak (388cb45)

  • add another view to superhero example (2ee15e4)

  • adjust iql generator to return filters and actions separately, like expected in collections (2c65ab1)

  • Merge branch 'ak/implement-llm-client' into 'main'

Implement OpenAI API client

See merge request deepsense.ai/g-internal/db-ally!14 (3e78269)

  • Add LLM client

Add missing init file

Fix mypy

Fix lint again

Fixes after review

Fix lint

Fixes after rebasing

Fix lint

Fix mypy

Small fix (9f3fad4)

  • Merge branch 'pk/iql-generator-mypy-fix' into 'main'

IQLGenerator tests - add ignoring mypy

See merge request deepsense.ai/g-internal/db-ally!16 (3e1f7ed)

  • ignore mypy (51aa497)

  • Merge branch 'pk/llm-iql-generation' into 'main'

IQLGenerator

See merge request deepsense.ai/g-internal/db-ally!11 (fd99707)

  • IQLGenerator added llm_output_parser and output_format to PromptTemplate IQLPromptTemplate (fbd9bfe)

  • Merge branch 'lt/remove_registry' into 'main'

Remove view registry (replace with collection)

See merge request deepsense.ai/g-internal/db-ally!15 (06367cd)

  • Remove view registry (replace with collection) (0b15cf0)

  • Merge branch 'mh/add_collections' into 'main'

Add collections / initial question answering flow

See merge request deepsense.ai/g-internal/db-ally!12 (b5a010e)

  • Add sql execution to example (bc663b3)

  • Rename DBAllyCollection -> Collection (841c114)

  • Merge branch 'lt/mypy_tests' into 'main'

Add mypy type checking to tests

See merge request deepsense.ai/g-internal/db-ally!13 (549c896)

  • Fix errors in typing definitions (f698234)

  • Add mypy type checking to tests (5c68959)

  • Add collections / initial question answering flow (6698e31)

  • Merge branch 'mh/integrate_query_parsing_with_query_building' into 'main'

Integrate query parsing with query building

See merge request deepsense.ai/g-internal/db-ally!10 (3691758)

  • Merge branch 'pk/superhero-example' into 'main'

Add superhero example

See merge request deepsense.ai/g-internal/db-ally!5 (079d1a3)

  • Add superhero example (c03b500)

  • Fix tests (0a74e51)

  • Integrate query parsing with query building (8728327)

  • Move iql test (04e57e6)

  • Merge branch 'ak/text2sql-benchmark' into 'main'

Text2SQL benchmark

See merge request deepsense.ai/g-internal/db-ally!6 (44f19de)

Basic tests for views

See merge request deepsense.ai/g-internal/db-ally!9 (7fab89f)

  • Basic tests for views (e9e1a24)

  • Merge branch 'pk/prompt-builder' into 'main'

Add PromptBuilder, PromptTemplate, tests

See merge request deepsense.ai/g-internal/db-ally!8 (f23d340)

  • Add PromptBuilder, PromptTemplate, tests (4a2a781)

  • Merge branch 'lt/views' into 'main'

Introduce view-based SQL generation

See merge request deepsense.ai/g-internal/db-ally!4 (9785574)

  • Introduce view-based SQL generation (784876b)

  • Merge branch 'DBALL-7-handle-iql-parsing' into 'main'

Initial IQL definition and parsing

See merge request deepsense.ai/g-internal/db-ally!7 (b6be39e)

  • Remove IQL namespace; make syntax module public (0ccc374)

  • Apply lint rules (0a22aba)

  • Add unit tests for IQL module (b2ef1a8)

  • IQL module index (a979c0b)

  • Add IQLQuery container class (b3bbe55)

  • Implement IQLParser (ce1c6b9)

  • Define IQL Syntax elements (748dde0)

  • Merge branch 'fk/terraform-database-setup' into 'main'

Terraform GCP databases setup

See merge request deepsense.ai/g-internal/db-ally!2 (adb815a)

  • Terraform GCP databases setup (2c262eb)

  • Merge branch 'pk/python-version' into 'main'

python version 3.8

See merge request deepsense.ai/g-internal/db-ally!3 (33a11da)

  • set runner image to python:3-8 (102c6b0)

  • add python version 3.8 (754e38b)

  • Merge branch 'mh/template' into 'main'

Setup ds-template

See merge request deepsense.ai/g-internal/db-ally!1 (561a0c6)