Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix code to work with mashumaro 3.15 #11051

Open
wants to merge 46 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
261e124
Fix NodeVersion definition, make_semantic_model utility
gshank Nov 25, 2024
37be156
Bump up mashumaro version again
gshank Nov 25, 2024
52fa7c0
Try pinning to >=3.15,<4.0
gshank Nov 26, 2024
354af6e
specify dbt-adapters branch
gshank Nov 26, 2024
7876edb
Update v12.json schema
gshank Nov 26, 2024
c0ea26e
validate in "update_from"
gshank Nov 26, 2024
dfca4e5
error classes
gshank Nov 26, 2024
b81632e
Comments
gshank Nov 27, 2024
e47bed3
Put back test_disabled_model.py
gshank Nov 27, 2024
9d6daa1
Use older mashumaro
gshank Nov 27, 2024
baf42c4
Again
gshank Nov 27, 2024
9df05a8
Put back default for validate in _update_from_config
gshank Nov 27, 2024
524ce54
Fix NodeVersion in artifacts test
gshank Nov 27, 2024
df23c7d
Skip a couple of tests until mash 3.15
gshank Nov 30, 2024
42a1afb
Make specialized 'calculate_node_config' methods
gshank Nov 30, 2024
9c6883b
Change order of Union in source definitions partitions
gshank Dec 2, 2024
25899ca
Change order of external.partitions schema elements
gshank Dec 2, 2024
5b4af17
Changie
gshank Dec 2, 2024
86791e0
Remove unnecessary "base" parameter
gshank Dec 2, 2024
d5616de
passes unit tests
gshank Dec 2, 2024
4554eb3
comment
gshank Dec 2, 2024
b4f72e4
Use dbt-common branch
gshank Dec 3, 2024
54adabe
Fix a few tests, remove unnecessary finalize_and_validate calls
gshank Dec 3, 2024
bbfc6e6
rename config to context_config
gshank Dec 3, 2024
1afb2bb
fix hooks before validation
gshank Dec 3, 2024
7d78b47
Bump mashumaro again
gshank Dec 3, 2024
1c709d2
Remove skips from tests
gshank Dec 3, 2024
13105cd
fix expected_manifest version: "2"
gshank Dec 3, 2024
3f7ee0e
Remove unnecessary ConfigSource
gshank Dec 3, 2024
c16cbe0
Rename ContextConfig ConfigBuilder
gshank Dec 3, 2024
c0fd389
rename model_configs resource_configs
gshank Dec 3, 2024
049418e
More cleanup
gshank Dec 3, 2024
e6fc9b0
Remove unnecessary mangle_hooks, other comments and cleanup
gshank Dec 3, 2024
31ce086
More comments
gshank Dec 4, 2024
923b8b6
add test for serialization of ExternalPartitions in ExternalTable
gshank Dec 9, 2024
1a30f64
Update test_serialization to simplify for mashumaro, add test_graph_s…
gshank Dec 10, 2024
c137041
Merge branch 'main' into mashumaro_fixes
gshank Dec 10, 2024
f011789
Remove base parameter from get_config_for
gshank Dec 11, 2024
0fb57ee
Update return type of generate_node_config
gshank Dec 11, 2024
e919144
Merge branch 'main' into mashumaro_fixes
gshank Dec 12, 2024
ebc2359
Remove requirements for 3.15
gshank Dec 12, 2024
4585bd0
formatting
gshank Dec 16, 2024
715ad2f
dev-requirements
gshank Dec 19, 2024
d1274b1
Merge branch 'main' into mashumaro_fixes
gshank Dec 19, 2024
5f94de5
Put back Union order changes, use dbtClassMixin in test_serialization.py
gshank Dec 19, 2024
a0e32c8
Merge branch 'main' into mashumaro_fixes
gshank Jan 9, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge branch 'main' into mashumaro_fixes
gshank committed Dec 19, 2024
commit d1274b1655e477a75bd4c793322e33e5eaa84ee9
4 changes: 2 additions & 2 deletions core/dbt/parser/schemas.py
Original file line number Diff line number Diff line change
@@ -298,8 +298,8 @@ def _add_yaml_snapshot_nodes_to_manifest(
)
snapshot_node = parser._create_parsetime_node(
block,
self.get_compiled_path(block),
parser.initial_config_builder(fqn),
compiled_path,
parser.initial_config(fqn),
fqn,
snapshot["name"],
)

Unchanged files with check annotations Beta

elif resource_type == NodeType.Exposure:
resource_configs = unrendered.get("exposures")
elif resource_type == NodeType.Unit:
resource_configs = unrendered.get("unit_tests")

Check warning on line 244 in core/dbt/context/context_config.py

Codecov / codecov/patch

core/dbt/context/context_config.py#L244

Added line #L244 was not covered by tests
else:
resource_configs = unrendered.get("models")
if resource_configs is None:
# it's ok to have a parse context with no config builder, but you must
# not call it!
if self.config_builder is None:
raise DbtRuntimeError("At parse time, did not receive a config builder")

Check warning on line 401 in core/dbt/context/providers.py

Codecov / codecov/patch

core/dbt/context/providers.py#L401

Added line #L401 was not covered by tests
# Track unrendered opts to build parsed node unrendered_config later on
if get_flags().state_modified_compare_more_unrendered_values:
if not hooks:
return
if not context:
context = self._context_for(parsed_node, config_builder)

Check warning on line 420 in core/dbt/parser/base.py

Codecov / codecov/patch

core/dbt/parser/base.py#L420

Added line #L420 was not covered by tests
for hook in hooks:
get_rendered(hook.sql, context, parsed_node, capture_macros=True)
if isinstance(experimental_sample, dict):
model_parser_copy = self.partial_deepcopy()
exp_sample_node = deepcopy(node)
exp_sample_config_builder = deepcopy(config_builder)
model_parser_copy.populate(

Check warning on line 299 in core/dbt/parser/models.py

Codecov / codecov/patch

core/dbt/parser/models.py#L298-L299

Added lines #L298 - L299 were not covered by tests
exp_sample_node, exp_sample_config_builder, experimental_sample
)
# use the experimental parser exclusively if the flag is on
# but we can't really guarantee that going forward.
model_parser_copy = self.partial_deepcopy()
jinja_sample_node = deepcopy(node)
jinja_sample_config_builder = deepcopy(config_builder)

Check warning on line 322 in core/dbt/parser/models.py

Codecov / codecov/patch

core/dbt/parser/models.py#L322

Added line #L322 was not covered by tests
# rendering mutates the node and the config
super(ModelParser, model_parser_copy).render_update(
jinja_sample_node, jinja_sample_config_builder
node: ModelNode,
config_builder: ConfigBuilder,
) -> List[str]:
result: List[Tuple[int, str]] = _get_sample_result(

Check warning on line 499 in core/dbt/parser/models.py

Codecov / codecov/patch

core/dbt/parser/models.py#L499

Added line #L499 was not covered by tests
sample_node, sample_config_builder, node, config_builder
)
node: ModelNode,
config_builder: ConfigBuilder,
) -> List[str]:
result: List[Tuple[int, str]] = _get_sample_result(

Check warning on line 517 in core/dbt/parser/models.py

Codecov / codecov/patch

core/dbt/parser/models.py#L517

Added line #L517 was not covered by tests
sample_node, sample_config_builder, node, config_builder
)
) -> List[Tuple[int, str]]:
result: List[Tuple[int, str]] = []
# look for false positive configs
for k in sample_config_builder._config_call_dict.keys():
if k not in config_builder._config_call_dict.keys():

Check warning on line 539 in core/dbt/parser/models.py

Codecov / codecov/patch

core/dbt/parser/models.py#L538-L539

Added lines #L538 - L539 were not covered by tests
result += [(2, "false_positive_config_value")]
break
# look for missed configs
for k in config_builder._config_call_dict.keys():
if k not in sample_config_builder._config_call_dict.keys():

Check warning on line 545 in core/dbt/parser/models.py

Codecov / codecov/patch

core/dbt/parser/models.py#L544-L545

Added lines #L544 - L545 were not covered by tests
result += [(3, "missed_config_value")]
break
arguments. This includes the implicit "model" argument.
"""
# make a base context that doesn't have the magic kwargs field
context = self._context_for(node, config_builder)

Check warning on line 361 in core/dbt/parser/schema_generic_tests.py

Codecov / codecov/patch

core/dbt/parser/schema_generic_tests.py#L361

Added line #L361 was not covered by tests
# update it with the rendered test kwargs (which collects any refs)
add_rendered_test_kwargs(context, node, capture_macros=True)
You are viewing a condensed version of this merge commit. You can view the full changes here.