Skip to content
This repository was archived by the owner on May 22, 2026. It is now read-only.

Commit a25b2e2

Browse files
msyycmsyyc
authored
Update test directory paths and requirements syncing (#3420)
* Update test directory paths and requirements syncing * Update sync_from_typespec.py * Update sync_from_typespec.py * add docs * skip apiview until eng issue is fixed
1 parent c8620f2 commit a25b2e2

3 files changed

Lines changed: 240 additions & 43 deletions

File tree

‎eng/pipelines/generated-code-checks-template.yml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ steps:
2323
condition: and(eq(variables['PythonVersion'], '3.10'), or(contains('${{ parameters.folderName }}', 'version-tolerant'), eq('${{parameters.package}}', 'typespec-python')))
2424

2525
# docs/apiview only for typespec-python (removed from autorest.python to reduce CI time)
26-
- script: npm run test -- --env=docs --flavor=${{ parameters.folderName }}
27-
displayName: ApiView ${{ parameters.folderName }} - Python $(PythonVersion)
28-
workingDirectory: $(Build.SourcesDirectory)/autorest.python/packages/${{parameters.package}}
29-
condition: and(eq(variables['PythonVersion'], '3.11'), eq('${{parameters.package}}', 'typespec-python'))
26+
# - script: npm run test -- --env=docs --flavor=${{ parameters.folderName }}
27+
# displayName: ApiView ${{ parameters.folderName }} - Python $(PythonVersion)
28+
# workingDirectory: $(Build.SourcesDirectory)/autorest.python/packages/${{parameters.package}}
29+
# condition: and(eq(variables['PythonVersion'], '3.11'), eq('${{parameters.package}}', 'typespec-python'))

‎eng/scripts/sync_from_typespec.py‎

Lines changed: 56 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -9,16 +9,8 @@
99
1010
The typespec repo is the source of truth for:
1111
1. regenerate-common.ts — shared regeneration logic
12-
2. requirements.txt — common test dependencies (delimited by marker comments)
13-
3. Test files — mock API tests and test data
14-
15-
Marker convention in requirements.txt:
16-
# === common azure dependencies across repos ===
17-
...
18-
# === end common azure dependencies across repos ===
19-
# === common test dependencies across repos ===
20-
...
21-
# === end common test dependencies across repos ===
12+
2. requirements — test dependency files (azure.txt, unbranded.txt under tests/requirements/)
13+
3. Test files — mock API tests under tests/mock_api/{shared,azure,unbranded}
2214
2315
Usage:
2416
python sync_from_typespec.py <local-typespec-repo-path>
@@ -36,39 +28,50 @@
3628
# --- Path configuration (relative to each repo root) ---
3729

3830
TYPESPEC_COMMON_TS = Path("packages/http-client-python/eng/scripts/ci/regenerate-common.ts")
39-
AUTOREST_COMMON_TS = Path("packages/typespec-python/scripts/eng/regenerate-common.ts")
31+
AUTOREST_COMMON_TS = Path("packages/typespec-python/eng/scripts/regenerate-common.ts")
4032

41-
TYPESPEC_TEST_DIR = Path("packages/http-client-python/generator/test")
42-
AUTOREST_TEST_DIR = Path("packages/typespec-python/test")
33+
TYPESPEC_TEST_DIR = Path("packages/http-client-python/tests")
34+
AUTOREST_TEST_DIR = Path("packages/typespec-python/tests")
4335

44-
# --- Marker patterns for requirements.txt sync ---
36+
# --- Marker patterns for requirements sync ---
37+
#
38+
# Convention in requirements files (e.g. azure.txt, unbranded.txt):
39+
# # === common azure dependencies across repos ===
40+
# azure-core>=1.37.0
41+
# ...
42+
# # === end common azure dependencies across repos ===
4543

4644
_MARKER_PATTERN = re.compile(r"^# === (common .+ across repos) ===$")
4745
_END_MARKER_PATTERN = re.compile(r"^# === end (common .+ across repos) ===$")
4846

49-
# --- Test file sync configuration ---
47+
_REQUIREMENTS_FILES = ["azure.txt", "unbranded.txt"]
48+
49+
50+
# ---------------------------------------------------------------------------
51+
# Test file sync
52+
# ---------------------------------------------------------------------------
5053

5154
_SKIP_DIRS: Set[str] = {"__pycache__", "generated", ".venv", "node_modules", ".tox"}
5255

5356
_TEST_SUBDIRS = [
54-
"generic_mock_api_tests",
55-
os.path.join("azure", "mock_api_tests"),
56-
os.path.join("unbranded", "mock_api_tests"),
57+
os.path.join("mock_api", "shared"),
58+
os.path.join("mock_api", "azure"),
59+
os.path.join("mock_api", "unbranded"),
5760
]
5861

59-
# Files that remain repo-specific (different relative paths between repo layouts)
62+
# Files that remain repo-specific (e.g. conftest.py differs between repos)
6063
_SKIP_FILES: Set[str] = {
61-
os.path.join("generic_mock_api_tests", "conftest.py"),
62-
os.path.join("azure", "mock_api_tests", "conftest.py"),
63-
os.path.join("unbranded", "mock_api_tests", "conftest.py"),
64+
os.path.join("mock_api", "shared", "conftest.py"),
65+
os.path.join("mock_api", "azure", "conftest.py"),
66+
os.path.join("mock_api", "unbranded", "conftest.py"),
6467
}
6568

6669
_SKIP_EXTENSIONS: Set[str] = {".pyc"}
6770
_SKIP_FILENAMES: Set[str] = {"tox.ini", "requirements.txt", "dev_requirements.txt"}
6871

6972

7073
# ---------------------------------------------------------------------------
71-
# Requirements.txt marker-based sync
74+
# Requirements sync
7275
# ---------------------------------------------------------------------------
7376

7477

@@ -125,14 +128,30 @@ def _replace_marker_sections(filepath: Path, source_sections: Dict[str, List[str
125128
filepath.write_text("\n".join(result) + "\n", encoding="utf-8", newline="\n")
126129

127130

128-
def sync_requirements(source: Path, target: Path) -> None:
129-
"""Sync common marker sections from source to target requirements.txt."""
130-
source_sections = _extract_marker_sections(source)
131-
if not source_sections:
132-
print(f" WARNING: no marker sections found in {source}, skipping")
133-
return
134-
_replace_marker_sections(target, source_sections)
135-
print(f" Synced requirements: {source.name} ({source.parent.name}/)")
131+
def sync_requirements(source_dir: Path, target_dir: Path) -> None:
132+
"""Copy requirements files from typespec to autorest.
133+
134+
If marker sections are present, only the marker-delimited sections are
135+
replaced in the target (preserving repo-specific dependencies outside
136+
markers). Otherwise the file is copied directly.
137+
"""
138+
for filename in _REQUIREMENTS_FILES:
139+
src = source_dir / filename
140+
dst = target_dir / filename
141+
if not src.is_file():
142+
print(f" WARNING: {src} not found, skipping")
143+
continue
144+
145+
source_sections = _extract_marker_sections(src)
146+
if source_sections and dst.is_file():
147+
_replace_marker_sections(dst, source_sections)
148+
print(f" Synced markers: requirements/{filename}")
149+
else:
150+
if dst.is_file() and src.read_bytes() == dst.read_bytes():
151+
continue
152+
dst.parent.mkdir(parents=True, exist_ok=True)
153+
shutil.copy2(src, dst)
154+
print(f" Copied: requirements/{filename}")
136155

137156

138157
# ---------------------------------------------------------------------------
@@ -220,14 +239,12 @@ def main() -> int:
220239
shutil.copy2(src_ts, dst_ts)
221240
print(f"Synced regenerate-common.ts")
222241

223-
# 2. Sync requirements.txt marker sections
224-
for flavor in ("azure", "unbranded"):
225-
src_req = typespec_repo / TYPESPEC_TEST_DIR / flavor / "requirements.txt"
226-
dst_req = autorest_repo / AUTOREST_TEST_DIR / flavor / "requirements.txt"
227-
if src_req.is_file() and dst_req.is_file():
228-
sync_requirements(src_req, dst_req)
229-
else:
230-
print(f" WARNING: requirements.txt not found for {flavor}, skipping")
242+
# 2. Sync requirements files
243+
print("Syncing requirements...")
244+
sync_requirements(
245+
typespec_repo / TYPESPEC_TEST_DIR / "requirements",
246+
autorest_repo / AUTOREST_TEST_DIR / "requirements",
247+
)
231248

232249
# 3. Sync test files
233250
print("Syncing test files...")

‎packages/typespec-python/docs/test_framework.md‎

Lines changed: 180 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,180 @@
1+
# Test Framework for typespec-python
2+
3+
This document describes the test framework used in the `typespec-python` package
4+
and how it relates to the upstream
5+
[`http-client-python`](https://github.com/microsoft/typespec/tree/main/packages/http-client-python)
6+
package in the typespec repository.
7+
8+
## Overview
9+
10+
The test framework is a **dual-flavor testing system** (Azure and Unbranded) built
11+
on **pytest** and **tox**. Tests run against a mock API server
12+
([tsp-spector](https://github.com/microsoft/typespec)) that serves TypeSpec-defined
13+
HTTP endpoints on `localhost:3000`.
14+
15+
## Folder Structure
16+
17+
```
18+
packages/typespec-python/
19+
└── tests/
20+
├── conftest.py # Root fixtures (server lifecycle, core_library, credentials, image data)
21+
├── install_packages.py # Installs generated SDK packages before test runs
22+
├── pytest.ini # Pytest config (asyncio_mode = auto)
23+
├── tox.ini # Test environments (test, lint, mypy, pyright, docs, ci)
24+
│
25+
├── data/ # Static test data (image.png, image.jpg)
26+
│
27+
├── requirements/ # Dependency files
28+
│ ├── base.txt # Common: pytest, pytest-asyncio, tox, coverage, etc.
29+
│ ├── azure.txt # Azure flavor: azure-core, azure-mgmt-core, geojson
30+
│ ├── unbranded.txt # Unbranded flavor: corehttp
31+
│ ├── lint.txt # Linting: pylint, black
32+
│ ├── typecheck.txt # Type checking: pyright, mypy
33+
│ └── docs.txt # Documentation: sphinx, myst_parser
34+
│
35+
├── generated/ # Auto-generated SDK packages from TypeSpec specs
36+
│ ├── azure/ # ~116 Azure-flavored packages
37+
│ └── unbranded/ # ~64 Unbranded packages
38+
│
39+
└── mock_api/ # Hand-written integration tests
40+
├── azure/ # Azure-specific tests
41+
│ ├── conftest.py # Azure fixtures (credentials, LRO polling, header validation)
42+
│ ├── asynctests/ # Async test variants
43+
│ ├── data/ # Test image data
44+
│ └── test_*.py # Sync test files
45+
├── shared/ # Tests that run for both flavors
46+
│ ├── conftest.py # Shared fixtures
47+
│ ├── asynctests/ # Async test variants
48+
│ ├── unittests/ # Unit tests (e.g. pyproject parsing)
49+
│ ├── data/ # Test image data
50+
│ └── test_*.py # Sync test files
51+
└── unbranded/ # Unbranded-specific tests
52+
├── conftest.py # Unbranded fixtures
53+
├── asynctests/ # Async test variants
54+
├── data/ # Test image data
55+
└── test_*.py # Sync test files
56+
```
57+
58+
## Test Flavors
59+
60+
| Flavor | Core library | Credential class | What it tests |
61+
|--------|-------------|-----------------|---------------|
62+
| **azure** | `azure.core` | `AzureKeyCredential` | Azure SDK conventions, ARM resources, LRO, paging |
63+
| **unbranded** | `corehttp` | `ServiceKeyCredential` | Non-Azure SDK generation without Azure branding |
64+
65+
When tests run:
66+
- **Azure**: `pytest mock_api/azure mock_api/shared` with `FLAVOR=azure`
67+
- **Unbranded**: `pytest mock_api/unbranded mock_api/shared` with `FLAVOR=unbranded`
68+
69+
The `shared/` tests run for **both** flavors. Root `conftest.py` uses `core_library()`
70+
to dynamically import the appropriate core library.
71+
72+
## Running Tests
73+
74+
```bash
75+
cd packages/typespec-python/tests
76+
77+
# Run Azure flavor tests
78+
tox -e test-azure
79+
80+
# Run Unbranded flavor tests
81+
tox -e test-unbranded
82+
83+
# Run all CI checks for a flavor (tests + lint + type checking)
84+
tox -e ci-azure
85+
tox -e ci-unbranded
86+
```
87+
88+
### Available tox Environments
89+
90+
| Environment | Description |
91+
|------------|-------------|
92+
| `test-azure` / `test-unbranded` | Run pytest integration tests |
93+
| `lint-azure` / `lint-unbranded` | Run pylint |
94+
| `mypy-azure` / `mypy-unbranded` | Run mypy type checking |
95+
| `pyright-azure` / `pyright-unbranded` | Run pyright type checking |
96+
| `docs-azure` / `docs-unbranded` | Build API docs with Sphinx |
97+
| `ci-azure` / `ci-unbranded` | All checks combined |
98+
99+
## Key Components
100+
101+
### Mock API Server
102+
103+
Tests rely on `tsp-spector` to serve TypeSpec-defined mock endpoints. The root
104+
`conftest.py` starts the server automatically at session start and tears it down
105+
after all tests complete. The server runs on `localhost:3000`.
106+
107+
### Generated Packages
108+
109+
Each test spec produces a generated SDK package under `tests/generated/{flavor}/`.
110+
Before tests run, `install_packages.py` installs all generated packages into the
111+
test environment using `uv pip install --no-deps`.
112+
113+
### Async Tests
114+
115+
Every `test_*.py` in the sync directory has a corresponding `test_*_async.py` in
116+
`asynctests/`. Async fixtures use `@pytest_asyncio.fixture` and `pytest.ini`
117+
configures `asyncio_mode = auto`.
118+
119+
## Folder Mapping to the typespec Repository
120+
121+
The test files are shared with the upstream
122+
[`http-client-python`](https://github.com/microsoft/typespec/tree/main/packages/http-client-python)
123+
package. The **typespec repo is the source of truth** for shared test files.
124+
125+
### Path Mapping
126+
127+
| typespec repo | autorest.python repo |
128+
|--------------|---------------------|
129+
| `packages/http-client-python/tests/mock_api/azure/` | `packages/typespec-python/tests/mock_api/azure/` |
130+
| `packages/http-client-python/tests/mock_api/shared/` | `packages/typespec-python/tests/mock_api/shared/` |
131+
| `packages/http-client-python/tests/mock_api/unbranded/` | `packages/typespec-python/tests/mock_api/unbranded/` |
132+
| `packages/http-client-python/tests/requirements/` | `packages/typespec-python/tests/requirements/` |
133+
| `packages/http-client-python/eng/scripts/ci/regenerate-common.ts` | `packages/typespec-python/eng/scripts/regenerate-common.ts` |
134+
135+
### What Is Synced
136+
137+
The script `eng/scripts/sync_from_typespec.py` copies from typespec → autorest.python:
138+
139+
1. **`regenerate-common.ts`** — shared regeneration logic
140+
2. **Requirements files** — `azure.txt` and `unbranded.txt` (marker-delimited
141+
common sections are synced; repo-specific deps like `geojson` are preserved)
142+
3. **Test files** — all files under `mock_api/{shared,azure,unbranded}` except:
143+
- `conftest.py` (each repo has its own)
144+
- `tox.ini`, `requirements.txt`, `dev_requirements.txt`
145+
- `.pyc` files
146+
147+
### What Is NOT Synced
148+
149+
| Item | Reason |
150+
|------|--------|
151+
| `conftest.py` files | Different server startup and fixture logic per repo |
152+
| `tests/mock_api/shared/unittests/` | Repo-specific unit tests |
153+
| `tests/generated/` | Regenerated independently in each repo |
154+
| `tests/unit/` (typespec only) | Internal to http-client-python |
155+
| `pytest.ini`, `tox.ini` | Different CI configurations per repo |
156+
157+
### Requirements Marker Convention
158+
159+
Requirements files (`azure.txt`, `unbranded.txt`) use markers to delimit the
160+
common section synced between repos:
161+
162+
```
163+
# === common azure dependencies across repos ===
164+
# Azure SDK dependencies
165+
-r base.txt
166+
azure-core>=1.37.0
167+
azure-mgmt-core==1.6.0
168+
# === end common azure dependencies across repos ===
169+
geojson>=3.0.0 # <-- autorest.python-only dependency, outside markers
170+
```
171+
172+
Dependencies outside the markers are preserved during sync.
173+
174+
### Sync Workflow
175+
176+
The sync is run automatically as part of [pipeline](https://dev.azure.com/azure-sdk/internal/_build?definitionId=7257), or manually:
177+
178+
```bash
179+
python eng/scripts/sync_from_typespec.py <path-to-typespec-repo>
180+
```

0 commit comments

Comments
 (0)