Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] docs generate - retrieve column descriptions from DB when available - for Sources #10476

Open
3 tasks done
eilone opened this issue Jul 23, 2024 · 3 comments
Open
3 tasks done
Labels
dbt-docs [dbt feature] documentation site, powered by metadata artifacts enhancement New feature or request triage

Comments

@eilone
Copy link

eilone commented Jul 23, 2024

Is this your first time submitting a feature request?

  • I have read the expectations for open source contributors
  • I have searched the existing issues, and I could not find an existing issue for this feature
  • I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

Describe the feature

I am requesting an enhancement to the dbt docs generate command to automatically retrieve column descriptions for sources from the database (e.g., BigQuery) when they are not provided in the YAML files. This functionality is already partially in place as dbt docs generate accesses the database to retrieve metadata for building the catalog.json. Extending this capability to fetch column descriptions would streamline the documentation process, especially for sources managed by external teams.

Describe alternatives you've considered

1.	Manually maintaining column descriptions in YAML files for all sources.
2.	Developing custom scripts to extract and merge database column descriptions into the dbt project.

Both alternatives add significant manual overhead and are prone to errors and inconsistencies.

Who will this benefit?

This feature will benefit dbt users who manage projects with external data sources, particularly in environments where:

•	Sources are managed by other teams.
•	Column descriptions are maintained within the database itself.
•	Consistent and up-to-date documentation is critical.

Example use case: A data engineering team using BigQuery with tables owned by different departments. Automatically retrieving column descriptions will ensure accurate and current documentation without duplicating efforts across teams.

Are you interested in contributing this feature?

Not really, but I'd love to help if needed

Anything else?

dbt docs generate documentation

This feature will leverage the existing infrastructure in dbt docs generate to enhance the documentation capabilities, providing a more seamless and integrated experience for users.

@eilone eilone added enhancement New feature or request triage labels Jul 23, 2024
@dbeatty10 dbeatty10 added the dbt-docs [dbt feature] documentation site, powered by metadata artifacts label Jul 23, 2024
@fcavallaro-reply
Copy link

that would be super useful!

@eilone
Copy link
Author

eilone commented Nov 13, 2024

Any update?

@henry-yu-gce
Copy link

This would be a great feature and very useful for us as well!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dbt-docs [dbt feature] documentation site, powered by metadata artifacts enhancement New feature or request triage
Projects
None yet
Development

No branches or pull requests

4 participants