Support both yaml
and yml
file extension for project config files
#11261
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #5002, #8738
Problem
From relevant faq:
This behaviour is potentially confusing to new users and runs against the official recommendations in the yaml faq. I was certainly confused by this when I first started working with dbt and ran into errors finding dbt project configuration fils such as
profiles.yaml
anddbt_project.yaml
.Solution
Failing tests are currently due to hardcoded file names in assertions where tests are expected to fail.
This PR introduces a new helper function
get_yaml_path
in theyaml_helper
module. Given the path to a stem (i.e the path to a yaml file without any extension), this function attempts to finding a matching yaml file ending in either of the accepted extensions, i.e..yml
or.yaml
. If no match is found, an empty string is returned.So as to minimise the changes required to existing code, I also removed the
.yml
extension from the relevant constants in thedbt/constants
module.I initially considered implementing this somewhere in
dbt_common
following previous guidance provided on #5002 but the structure of the project appears to have changed since those comments were made. Additionally, there is some variance in how paths to config files are resolved.For example, some project file paths are defined as constants whereas
profiles.yml
is passed directly as a string toos.path.join
inprofiles.yml
. In the majority of cases, the path to project files appears to be passed around as a string, but intasks/base.py
.Using the new helper function as a wrapper to existing calls to resolve the path for relevant config files seemed the least likely to break existing behaviour.
See also comments on #5002.
2e938d7 is an unrelated change, but I could not get pre-commit hooks to work correctly without this change. I believe the underlying issue relates to changes in
metadata
lib in python 3.12Checklist