master merge for 1.6.0 release #2257
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ed types handling for sqlalchemy backend (#2177)
* fix links (#1977) * correctly converts dict arrow types into dlt types * drops dbt compat code for versions below 1.5 * ignores encoding errors when reading from process pipe --------- Co-authored-by: adrianbr <[email protected]>
* fix: unravel Optional to inner generic arg from instance * test: remove dependency on Incremental in common * refactor: use extract_inner_type * refactor: remove redundant conditional
- Add `ensure_pendulum_datetime_non_utc` to parse datetime strings into non-UTC datetime objects. - Add `_datetime_obj_to_str` to preserve the colon in the timezone when converting datetime objects back to strings. - Skip writing back state if no valid rows are found for `last_value` in the transformer, which may otherwise cause incorrect behavior.
…2095) * Updated SQL server documentation * Updated as per comments
…2184) * do not normalize dates and datetimes in from / to arrow scalar * uses the same logic for naive datetimes in arrow and object incrementals
…ple disable python 3.8 (#2185) * import correct typeddict version for use in pydantic, disallow use of usual python typeddict imports * add test * update import for examples * fixed some imports * remove python 3.8 lint and test for now * always use typeddict from typing_extensions pin poetry in tests to 1.8.5
…#2194) * add python 3.12 linting * update locked versions to make project installable on py 3.12 * update flake8 * downgrade poetry for all tests relying on python3.8 * drop python 3.8 * enable python3.13 * copy test updates from python3.13 branch * update locked sentry version * pin poetry to 1.8.5 * install ibis outside of poetry * rename to workflows for consistency * switch to published alpha version of dlt-pendulum for python 3.13 * fix images * add note to readme
… Python" (#2182) * Added section Destination configuration via Python * Updated * Updated * Updated for snippet error * Updated * Updated * move example to advanced --------- Co-authored-by: Alena <[email protected]>
…ed (#2209) * bumps dlt to 1.6.0a0 * bumps dlt-pendulum
* fix unclickable expander; use native logo * fix linting issue --------- Co-authored-by: zilto <[email protected]>
* Declare pip dependency for cli extras The `pipdeptree` package includes references to `pip` but does not explicitly require it. To ensure `pip` is installed in the environment, declare it as a dependency. * Include pip in dev dependencies Some tests require pip.
…tination (#2213) * adds severa not found error message patterns to sqlalchemy destination * tests single file glob for all bucket types
* update the performance page * Update docs/website/docs/reference/performance.md Co-authored-by: Alena Astrakhantseva <[email protected]> * Update docs/website/docs/reference/performance.md Co-authored-by: Alena Astrakhantseva <[email protected]> * small changes * fix link * Update docs/website/docs/reference/performance.md --------- Co-authored-by: Alena Astrakhantseva <[email protected]>
* fixes inserted_at to datetime * adds pipeline configuration into (pipelines, name) section * registers custom destinations via synthesized types * adds destination registry, adds autoimport to registries, refactors common destination files * sets destination_name to callable name for custom destination * fixes sources registry fixture * bumps dlt to 1.6.0a0 * adds global dir path * adds plus info to anon tracker * checks if dlt can be imported in alpine container * adds top level module to run context * stores top level plugin modules to resolve shorthand references * adds destination references with shothand expansion * adds preferred table formats to destination caps * uses types as reference to sources, creates DltResource instance in from_reference * tests plugin disovery with references * tests plus plugin telemetry * converts dict arrow types before sending to delta or iceberg * updates deps, incl duckdb * adds plug/unplug callbacks for run context * improves reading snapshots on iceberg * fixes tests and deps * allows to push and pop context on stack, fixes some tests * plugs and unplugs content on reload * always refresh views on abfss + sql client filesystem * fixes some tests
* added exception for unsupported arrow types * improved exception handling without reraising --------- Co-authored-by: zilto <[email protected]>
* make argparse output nicer * small update for auto docs generation * fix invoke test * add docs command for autogenerating cli docs * add generated cli page to docs * add make commands and checks for outdated docs * adds nicer listing of args * only check docs output on py 3.11 * update rest api pokemon tests * add some developer notes and make debugging more convenient * add anchor links to subcommands * adds inheritance information for each command * add link to cli docs to default help. * update sidebar layout * Update docs/website/docs/reference/command-line-interface-generated.md Co-authored-by: Akela Drissner-Schmid <[email protected]> * Update docs/website/docs/reference/command-line-interface.md Co-authored-by: Akela Drissner-Schmid <[email protected]> * merge generated and old cli page * add additional warning * re-order help and description * fix linting * put arguments and options into collapsible * remove dlt+ mention * post merge lock --------- Co-authored-by: Akela Drissner-Schmid <[email protected]>
* does not install prereleases into venv if uv is detected * enables and fixes clickhouse and synapse dbt adapters * removes dbt logger related code from dbt runner and enables dbt-core 1.9 * allows pipelines without dataset to be run in dbt runner * tests custom cs naming convention in snowflake
* databricks: enable local files * fix: databricks test config * work in progress * added create and drop volume to interface * refactor direct load authentication * fix databricks volume file name * refactor databricks direct loading * format and lint * revert config.toml changes * force notebook auth * enhanced config validations * force exception * fix config resolve * remove imports * test: config exceptions * restore comments * restored destination_config * fix pokema api values * enables databricks no stage tests * fix databricks config on_resolved * adjusted direct load file management * direct load docs * filters by bucket when subset of destinations is set when creating test cases * simpler file upload * fix comment * passes authentication directly from workspace, adds proper fingerprinting * use real client_id in tests * fixes config resolver to not pass NotResolved hints to config providers --------- Co-authored-by: Marcin Rudolf <[email protected]>
* adds exasol enity not found cases to recognized exceptions * makes sqlalchemy indexes optional and off by default
* takes dlt-pendulum from pypi * use precise timer when getting timestamps, fixes test for py 3.13 * fixes test
✅ Deploy Preview for dlt-hub-docs canceled.
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
master merge for 1.6.0 release