Skip to content

Deduplicate changelog entries from generated types module#48109

Merged
msyyc merged 8 commits into
mainfrom
fix-changelog-types-module-dup
Jul 17, 2026
Merged

Deduplicate changelog entries from generated types module#48109
msyyc merged 8 commits into
mainfrom
fix-changelog-types-module-dup

Conversation

@msyyc

@msyyc msyyc commented Jul 17, 2026

Copy link
Copy Markdown
Member

Problem

When running azpysdk breaking . --changelog --use-apistub, newly added models can appear twice in the changelog, e.g.:

### Features Added
  - Client `ComputeLimitMgmtClient` added operation group `trusted_host_subscriptions`
  - Added model `TrustedHostSubscription`
  - Added model `TrustedHostSubscriptionsOperations`
  - Added model `TrustedHostSubscription`   <-- duplicate

Root cause

TypeSpec generated libraries emit a types module (e.g. azure.mgmt.computelimit.types) containing TypedDict input aliases that shadow the real models defined in the sibling models module (azure.mgmt.computelimit.models). A newly added model is therefore defined in both modules, so the changelog checker reports an AddedClass feature for each. Because the Added model \{}`` message only uses the class name (not the module), the two entries render identically and show up as a visible duplicate.

Fix

Add a ShadowTypesModuleChecker post-processing step (registered in POST_PROCESSING_CHECKERS).

A types-module entry is dropped only when the sibling models module reports the same change — i.e. the same change type and arguments, with the module substituted. This targeted match avoids silently dropping a change that exists only on the types side (which would otherwise disappear from the breaking-change / changelog output).

When building the match key, class names and semantic values (type strings, defaults) are compared exactly; only the member-name position is normalized to snake_case, because the types TypedDicts expose wire names (e.g. serviceTreeId) while the models classes expose the Python attribute names (e.g. service_tree_id). This lets a genuine member-level duplicate match across the two naming styles while preserving unmatched, types-only changes and avoiding false collisions on class names.

The sibling models match keys are indexed once so each types entry is checked with an O(1) lookup (no per-entry rescans).

Tests

Added tests in tests/test_changelog.py:

  • test_shadow_types_module_cleanup — a model added to both models and types is reported once (from models).
  • test_shadow_types_module_kept_without_models_sibling — a standalone types module (no sibling models) is not stripped.
  • test_shadow_types_module_kept_when_class_missing_in_models — a types class with no models counterpart is preserved.
  • test_shadow_types_module_breaking_change_preserved_for_unmatched_member — an unmatched types breaking change is preserved while a mirrored addition is deduped.
  • test_shadow_types_module_member_change_preserved_when_not_mirrored — an unmatched member change on a class present in both modules is preserved.
  • test_shadow_types_module_member_change_deduped_across_naming — a member change reported in both modules is deduped despite the serviceTreeId vs service_tree_id naming difference.
  • test_shadow_types_module_class_name_matched_exactly — class names are matched exactly (no snake_case normalization), so differently-named classes that would collide under normalization are not deduped.

And a scalability test in tests/test_performance.py:

  • TestShadowTypesModuleCheckerPerformance::test_shadow_types_cleanup_performance_large — 10,000 entries are processed in well under a second (verifies the O(n) set-based lookup).

Relevant suites (test_changelog.py, test_apiview_converter.py, shadow-types perf test) pass locally: 34 passed, 1 skipped.

The TypeSpec generated 'types' module contains TypedDict input aliases that shadow the real models in the sibling 'models' module. Newly added models were therefore reported twice in the changelog (e.g. 'Added model Foo'). Add a ShadowTypesModuleChecker post-processing step that drops changelog/breaking-change entries originating from a shadow 'types' module, and add tests.
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
9 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@msyyc
msyyc marked this pull request as ready for review July 17, 2026 06:06
@msyyc
msyyc requested a review from danieljurek as a code owner July 17, 2026 06:06
Copilot AI review requested due to automatic review settings July 17, 2026 06:06
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
9 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds post-processing to deduplicate changelog entries emitted by generated types modules.

Changes:

  • Adds and registers ShadowTypesModuleChecker.
  • Adds tests for shadowed and standalone types modules.
  • However, filtering is overly broad and linked issue #48107 remains unresolved.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
scripts/breaking_changes_checker/checkers/shadow_types_module_checker.py Filters reports from shadow types modules.
scripts/breaking_changes_checker/supported_checkers.py Registers the new checker.
scripts/breaking_changes_checker/tests/test_changelog.py Tests types-module cleanup behavior.

Comment thread scripts/breaking_changes_checker/checkers/shadow_types_module_checker.py Outdated
Comment thread scripts/breaking_changes_checker/supported_checkers.py
Address review feedback: only drop a types-module entry when the sibling models module actually contains a class of the same name, so a types-only change (no models counterpart) is preserved in the breaking-change/changelog output. Add regression tests for the class-existence guard.
Copilot AI review requested due to automatic review settings July 17, 2026 06:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Comment thread scripts/breaking_changes_checker/checkers/shadow_types_module_checker.py Outdated
Copilot AI review requested due to automatic review settings July 17, 2026 06:24

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Comment thread scripts/breaking_changes_checker/checkers/shadow_types_module_checker.py Outdated
Only drop a types-module entry when the sibling models module reports the same change type and arguments (with member names normalized snake_case to bridge the types wire-name vs models attribute-name difference). This avoids silently dropping an unmatched types-only member change while still deduping the real property/model duplicates. Add member-level regression tests.
Copilot AI review requested due to automatic review settings July 17, 2026 06:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Comment thread scripts/breaking_changes_checker/checkers/shadow_types_module_checker.py Outdated
Compare class names and semantic values exactly and normalize only the member-name position to snake_case, avoiding false collisions on class names, type strings, or defaults. Add a regression test asserting exact class-name matching.
Copilot AI review requested due to automatic review settings July 17, 2026 06:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

Comment thread scripts/breaking_changes_checker/checkers/shadow_types_module_checker.py Outdated
Pre-index sibling models match keys once so each types entry is checked with a constant-time set lookup instead of rescanning the whole change list (avoids an O(n^2) regression on large diffs). Handle list-valued change args when building keys. Add a scalability test. Also aligns the PR description with the exact class-name matching behavior.
Copilot AI review requested due to automatic review settings July 17, 2026 07:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Copilot AI review requested due to automatic review settings July 17, 2026 07:15

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

@msyyc
msyyc merged commit 362d68d into main Jul 17, 2026
17 checks passed
@msyyc
msyyc deleted the fix-changelog-types-module-dup branch July 17, 2026 07:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants