Skip to content

Commit e36f355

Browse files
weidongxu-microsoftweidongxu-microsoft
authored
doc, update developer documentation (#3052)
1 parent 5313117 commit e36f355

File tree

10 files changed

+225
-212
lines changed

10 files changed

+225
-212
lines changed

‎.gitignore

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -57,8 +57,6 @@ Thumbs.db
5757
# reduced pom files should not be included
5858
dependency-reduced-pom.xml
5959

60-
preprocessor/*.yaml
61-
fluentnamer/*.yaml
6260
fluentgen/*.yaml
6361
javagen/*.yaml
6462
/*.yaml

‎checkstyle-suppressions.xml

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,8 +27,6 @@
2727
<!-- class type -->
2828
<suppress checks="[ConstantName|MethodName]" files="com[/\\]azure[/\\]autorest[/\\]model[/\\]clientmodel[/\\]GenericType.java"/>
2929
<suppress checks="[ConstantName|MethodName]" files="com[/\\]azure[/\\]autorest[/\\]fluent[/\\]model[/\\]FluentType.java"/>
30-
<!-- androidgen -->
31-
<suppress checks="[a-zA-Z0-9]*" files="com[/\\]azure[/\\]autorest[/\\]android[/\\].*"/>
3230

3331
<!-- com.microsoft.typespec -->
3432
<suppress checks="[a-zA-Z0-9]*" files="com[/\\]microsoft[/\\]typespec[/\\].*"/>

‎docs/developer/code-generator-mode.md

Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,117 @@
1+
# Mode in Code Generator
2+
3+
## Vanilla
4+
5+
AutoRest with flag `--java`. This is used primary for lib that involves a handwritten (data-plane) client over generated code. For example, `azure-storage-blob`.
6+
7+
This mode is not recommended to be used for new lib.
8+
9+
### Client Method
10+
11+
```java
12+
Response<ResponseBody> operationWithResponse(String pathParam, EnumType requiredQueryParam, RequestBody body, EnumType optionalQueryParam, String optionalHeaderParam, Context context);
13+
14+
ResponseBody operation(String pathParam, EnumType requiredQueryParam, RequestBody body);
15+
```
16+
17+
### Client
18+
19+
By default, the client takes the form of
20+
21+
```java
22+
ServiceClient client = new ClientBuilder().buildServiceClient();
23+
24+
OperationGroup operationGroup = client.getOperationGroup();
25+
26+
operationGroup.operation();
27+
```
28+
29+
Lots of the flags can be used to tune the generated client/method in this mode.
30+
31+
## Data-plane from Swagger
32+
33+
AutoRest with flag `--java --data-plane`. This is used for data-plane lib generated from Swagger.
34+
35+
### Client Method
36+
37+
```java
38+
Response<BinaryData> operationWithResponse(String pathParam, String requiredQueryParam, BinaryData body, RequestOptions requestOptions);
39+
```
40+
41+
The request body and response body both take the type of `BinaryData`. And optional parameters are set via `RequestOptions`. This pattern of client method is called "Protocol API".
42+
43+
No model class is generated in this mode.
44+
45+
### Client
46+
47+
The client is accessed via the Builder pattern.
48+
49+
```java
50+
OperationsClient operationsClient = new ClientBuilder().buildOperationsClient();
51+
52+
OperationGroupClient operationGroupClient = new ClientBuilder().buildOperationGroupClient();
53+
54+
operationsClient.operation1();
55+
operationGroupClient.operation2();
56+
```
57+
58+
## Data-plane from TypeSpec
59+
60+
This is used for data-plane lib generated from TypeSpec source.
61+
62+
### Client Method
63+
64+
```java
65+
Response<BinaryData> operationWithResponse(String pathParam, String requiredQueryParam, BinaryData body, RequestOptions requestOptions);
66+
67+
ResponseBody operation(String pathParam, EnumType requiredQueryParam, RequestBody body);
68+
69+
ResponseBody operation(String pathParam, EnumType requiredQueryParam, RequestBody body, EnumType optionalQueryParam, String optionalHeaderParam);
70+
```
71+
72+
In additional to "Protocol API", it also generates the client method overloads with model class and optional parameters. This is called "Convenience API".
73+
74+
### Client
75+
76+
By default, The client is accessed via the Builder pattern, same as those in [Data-plane from Swagger](#data-plane-from-swagger).
77+
78+
`enable-subclient` emitter options can be used to switch the client structure to parent-child clients.
79+
80+
```java
81+
ServiceClient client = new ClientBuilder().buildClient();
82+
83+
ChildClient childClient = client.getChildClient(childClientParam);
84+
85+
GrandchildClient grandchildClient2 = childClient.getGrandchildClient(grandchildParam);
86+
87+
client.operation1();
88+
childClient.operation2();
89+
grandchildClient2.operation3();
90+
```
91+
92+
## Management-plane from Swagger/TypeSpec
93+
94+
This is used for management-plane lib generated from Swagger, with flag `--java --fluent=LITE`, or from TypeSpec.
95+
96+
The generated client method is similar to that generated from [Vanilla mode](#vanilla).
97+
98+
### Client
99+
100+
Model class would be modeled as `Resource`, `ProxyResource`.
101+
102+
Another Fluent code layer would be generated, modeling concept as "subscription", "resource group", "resource", "parent resource", "extension resource", and CRUD of the resource.
103+
104+
## Unbranded
105+
106+
This is used for 3rd-party lib that generated from TypeSpec.
107+
108+
### Client Method
109+
110+
The client method takes the form of
111+
```java
112+
Response<ResponseBody> operationWithResponse(String pathParam, String requiredQueryParam, BinaryData body, RequestOptions requestOptions);
113+
114+
ResponseBody operation(String pathParam, EnumType requiredQueryParam, RequestBody body);
115+
116+
ResponseBody operation(String pathParam, EnumType requiredQueryParam, RequestBody body, EnumType optionalQueryParam, String optionalHeaderParam);
117+
```

‎docs/developer/design.md

Lines changed: 29 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -2,21 +2,16 @@
22

33
## Modules
44

5-
`extension-base` is the shared module providing `CodeModel` represented in Java language, JSON RPC, and `NewPlugin` plugin entry.
5+
`javagen` is the shared module on models and operations. It contains the shared classes such as: `CodeModel` represented in Java language, JSON RPC, and `NewPlugin` plugin entry.
6+
It alone is the pipeline for data-plane SDK generator (includes "vanilla" -- the mode that requires a handwritten client).
7+
It uses Eclipse JDT Language Server to provide [Customizations](../../readme.md#customizations), in postprocess stage.
68

7-
`preprocessor`, `javagen`, `postprocessor` is the shared modules on models and operations.
8-
Also, they are the pipeline for data-plane SDK generator.
9-
`postprocessor` uses Eclipse JDT Language Server to provide customizations.
10-
11-
`fluentnamer`, `fluentgen` is activated when `--fluent` option is specified.
12-
They are the pipeline for management-plane SDK generator.
13-
14-
`javagen` with Android-style generation is activated when `--android` option is specified.
15-
Together with `preprocessor`, they are the pipeline for data-plane SDK generator for Android.
9+
`fluentgen` is activated when `--fluent` option is specified.
10+
Built with code in `javagen` module, it is the pipeline for management-plane SDK generator.
1611

1712
## Configure on Modeler Four
1813

19-
Modeler Four is configured on `readme.md` of `preprocessor` and `fluentnamer`.
14+
Modeler Four is configured on `readme.md` at repository root, as well as that of `javagen` and `fluentgen`. [The readme.md at repository root](../../readme.md#autorest-plugin-configuration) determined whether the entry point be `javagen` or `fluentgen`.
2015

2116
The most important configure is the version of Modeler Four.
2217
The important options are:
@@ -27,25 +22,18 @@ The important options are:
2722

2823
Historically, data-plane leaves all of these options as `true`, while management-plane try to turn them to `false`.
2924

30-
Usually `preprocessor` and `fluentnamer` would save the input from pipeline to `code-model.yaml` in module root.
25+
At preprocess stage, `javagen` or `fluentgen` would save the input from pipeline to `code-model.yaml` in a temporary folder (the environment variable `codegen.java.temp.directory` can be used to specify a path).
3126
The content of this file is helpful for inspecting Modeler Four output.
3227

3328
One can set `--output-artifact=code-model-v4-no-tags` option to let AutoRest write the code-model file to output folder.
3429

35-
## RPC within AutoRest Java
36-
37-
### Between `preprocessor` and `javagen`
38-
39-
The content of RPC still follows `CodeModel`, after transformation of `preprocessor`.
40-
However, it could look quite different from output of Modeler Four.
30+
## Details on Modules
4131

42-
### Between `java` and `postprocessor`
32+
### `javagen` Module
4333

44-
The content would be Java files.
34+
This module re-uses code from `com.microsoft.typespec:http-client-generator-core` module.
4535

46-
## Details on Modules
47-
48-
### `extension-base` Module
36+
#### Code Model
4937

5038
`NewPlugin` as entry for plugin, providing RPC for pipeline.
5139

@@ -55,22 +43,12 @@ Roughly speaking, `schemas` contain the models, while `operations` grouped into
5543

5644
`JavaSettings` provides basic configuration for AutoRest Java.
5745

58-
### `preprocessor` Module
46+
#### Pre-process
5947

60-
`Transformer` renames model/property/method/parameter to be consistent to Java naming. It also supplies the `next` operation for `x-ms-pageable` extension.
48+
`Preprocessor` with `Transformer` renames model/property/method/parameter to be consistent to Java naming. It also supplies the `next` operation for `x-ms-pageable` extension.
6149

6250
`CodeNamer` as utility for Java naming, including conflict handling e.g. from reserved word in Java.
6351

64-
### `fluentnamer` Module
65-
66-
In addition to what is done in `preprocessor` module, its `Transformer` handles normalization of resource type (e.g. subclass of `Resource`), error type (e.g. subclass of `ManagementException` and `ManagementError`), operation name (e.g. `getByResourceGroup` and `listByResourceGroup`).
67-
68-
It also handles naming conflict between operation groups and models (e.g. `FooClient` from operation group client and `FooClient` from model, maybe another `FooClient` of the service client), naming of anonymous model (e.g. property of anonymous schema, anonymous superclass), rename from options, cleaning up of unused schemas (which is necessary because of the normalization).
69-
70-
`FluentJavaSettings` provides additional configuration on fluentnamer and fluentgen.
71-
72-
### `javagen` Module
73-
7452
#### Client Model
7553

7654
Classes under `model.clientmodel` namespace is the client model, which is the representation of Java code to be generated in memory.
@@ -122,11 +100,27 @@ It writes Java code using classes under `model.javamodel` namespace.
122100

123101
Classes of mapper and template is constructed by abstract factory, which enables overriding the behavior in other modules.
124102

103+
#### Post-process
104+
105+
`Postprocessor` modifies generated Java code, with `PartialUpdateHandler` for [Partial Update](../generate/dataplane-customization.md), and with `Customization` for [Customizations via code](../../readme.md#customizations).
106+
107+
The formatting of generated Java code is also done in this stage, leverage [spotless Maven plugin](https://github.com/diffplug/spotless).
108+
125109
### `fluentgen` Module
126110

127111
Fluent Java SDK for management-plane has a different [API design](https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/resourcemanager/docs/DESIGN.md).
128112
Simply speaking, the Fluent interface wraps the vanilla Client and REST API.
129113

114+
This module re-uses code from `com.microsoft.typespec:http-client-generator-mgmt` module.
115+
116+
#### Pre-process
117+
118+
In addition to what is done in [Pre-process of javagen](#pre-process), its `Transformer` handles normalization of resource type (e.g. subclass of `Resource`), error type (e.g. subclass of `ManagementException` and `ManagementError`), operation name (e.g. `getByResourceGroup` and `listByResourceGroup`).
119+
120+
It also handles naming conflict between operation groups and models (e.g. `FooClient` from operation group client and `FooClient` from model, maybe another `FooClient` of the service client), naming of anonymous model (e.g. property of anonymous schema, anonymous superclass), rename from options, cleaning up of unused schemas (which is necessary because of the normalization).
121+
122+
`FluentJavaSettings` provides additional configuration on fluentnamer and fluentgen.
123+
130124
#### Client Model
131125

132126
`FluentClient` is the container of the client model objects, which includes `Client`.

‎docs/developer/readme.md

Lines changed: 18 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -4,21 +4,21 @@
44

55
AutoRest Java is the Java language generator for AutoRest.
66

7-
As [npm package](https://github.com/Azure/autorest/blob/main/docs/developer/writing-an-extension.md), it is defined by the [`package.json`](https://github.com/Azure/autorest.java/blob/main/package.json) located in project root, and the `package.json` in `preprocessor`, `javagen`, `postprocessor`, `fluentnamer`, `fluentgen`.
7+
As [NPM package](https://github.com/Azure/autorest/blob/main/docs/developer/writing-an-extension.md), it is defined by the [`package.json`](https://github.com/Azure/autorest.java/blob/main/package.json) located in project root, and the `package.json` in `javagen` and `fluentgen`.
88

99
As AutoRest plugin, it is defined by the [YAML section in `readme.md`](https://github.com/Azure/autorest.java/blob/main/javagen/readme.md), located in the same folders of `package.json`.
1010

11-
## Build and Unit Test
11+
## Build and Test
1212

1313
### Build
1414

1515
Build is configured via Maven.
1616

1717
There is several build profiles:
18-
- `local` profile. It is required to be enabled. It uses `maven-shade-plugin` to combine project output to a single jar.
19-
- `testVanilla` profile. It enables the integrated vanilla tests, which is the common ground for all modules.
20-
- `testAzure` profile. It enables the integrated Azure tests, which tests handling of some advanced [AutoRest Extensions for OpenAPI 2.0](https://github.com/Azure/autorest/blob/main/docs/extensions/readme.md).
21-
- `testFluent` profile. It enables the integrated Fluent tests, which tests generation of Fluent management SDKs for [ARM](https://docs.microsoft.com/azure/azure-resource-manager/management/overview).
18+
- **`local` profile**: It is required to be enabled. It uses `maven-shade-plugin` to combine project output to a single jar.
19+
- **`testVanilla` profile**: It enables the integrated vanilla tests, which is the common ground for all modules.
20+
- **testAzure profile**: It enables the integrated Azure tests, which tests handling of some advanced [AutoRest Extensions for OpenAPI 2.0](https://github.com/Azure/autorest/blob/main/docs/extensions/readme.md).
21+
- **`testFluent` profile**: It enables the integrated Fluent tests, which tests generation of Fluent management SDKs for [ARM](https://docs.microsoft.com/azure/azure-resource-manager/management/overview).
2222

2323
### Unit Test
2424

@@ -51,12 +51,20 @@ Use `--use` option of the AutoRest to load the local AutoRest Java build (instea
5151

5252
The `name` of `@autorest/java` in `package.json` makes sure that it is loaded when `--java` option is provided.
5353

54-
## Pre-release
54+
## Publish `@autorest/java` to NPM
5555

56-
1. Update `version` in root `package.json`.
57-
2. Run "Pre-release autorest.java" in [GitHub Actions](https://github.com/Azure/autorest.java/actions). Provide the same `release version`.
56+
1. Update `version` in root `package.json`, merge the PR.
57+
2. Run "autorest.java - publish" in (internal) DevOps.
5858
3. Update Release notes in [GitHub Releases](https://github.com/Azure/autorest.java/releases).
5959

60+
## Publish customization libraries to Maven
61+
62+
Modules in following folders are published to Maven
63+
- **`extension-base` module**: [azure-autorest-extension](https://central.sonatype.com/artifact/com.azure.tools/azure-autorest-extension)
64+
- **`customization-base` module**: [azure-autorest-customization](https://central.sonatype.com/artifact/com.azure.tools/azure-autorest-customization)
65+
66+
The publish task involves local build and manual operation via Partner release pipeline.
67+
6068
## Logging
6169

6270
**DO NOT** log to stdout. AutoRest uses stdin and stdout as [RPC channel for plugins](https://github.com/Azure/autorest/blob/main/docs/developer/writing-an-extension.md#rpc-channel).
@@ -105,14 +113,4 @@ suspend=n,address=*:5005".
105113

106114
After the remote debugger is configured, run autorest with this additional argument `--java.debugger`. The AutoRest
107115
process will block until the JVM debugger is attached, so, after the AutoRest process pauses, start the debugger
108-
which will connect to the remote debugger and this will hit the breakpoints set in the "Javagen" plugin. If you want
109-
to set up a breakpoint in a different plugin like preprocessor or postprocessor, you can change the argument to
110-
`--preprocessor.debugger` or `postprocessor.debugger` respectively.
111-
112-
113-
### Interactive UI to trace AutoRest pipeline
114-
115-
Running autorest with `--interactive` mode on will pop up a browser window to show an interactive UI of the AutoRest
116-
pipeline. This generally helps in identifying all the plugins that are executed, their order of execution, their
117-
inputs and outputs.
118-
116+
which will connect to the remote debugger and this will hit the breakpoints set in the "Javagen" plugin.

‎docs/developer/typespec/readme.md

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# Developer Guide for TypeSpec Emitter
2+
3+
## TypeSpec Java
4+
5+
As [NPM package](https://www.npmjs.com/package/@azure-tools/typespec-java), it is the Java client [emitter](https://typespec.io/docs/extending-typespec/emitters-basics/) for Azure SDKs.
6+
7+
## Build and Test
8+
9+
At present, almost all the code of the emitter is in [microsoft/typespec repository](https://github.com/microsoft/typespec/tree/main/packages/http-client-java).
10+
11+
### Build
12+
13+
NPM package is in `typespec-extension` folder.
14+
15+
The [`Build-TypeSpec.ps1` script](../../../Build-TypeSpec.ps1) builds the emitter.
16+
It first build the JAR using Maven, copy it into `typespec-extension` folder, then build and pack the NPM package to a `tgz` file.
17+
18+
### Integrated Test
19+
20+
End-to-end test resides in `typespec-tests` folder.
21+
22+
At present, the tests is copied from
23+
24+
The [`Generate.ps1` script](../../../typespec-tests/Generate.ps1) generates the test code from [http-specs](https://github.com/microsoft/typespec/tree/main/packages/http-specs) and [azure-http-specs](https://github.com/Azure/typespec-azure/tree/main/packages/azure-http-specs).
25+
26+
`npm run spector-serve` starts the mock server.
27+
28+
Then standard Maven Surefire would run the tests.
29+
30+
## Publish `@azure-tools/typespec-java` to NPM
31+
32+
1. Update `version` in `typespec-extension/package.json`.
33+
2. Update `typespec-java` `version` in `typespec-test/package.json`.
34+
3. Update `typespec-extensions/changelog.md`, merge the PR.
35+
4. Run "typespec-java - publish" in (internal) DevOps.
36+
5. Update Release notes in [GitHub Releases](https://github.com/Azure/autorest.java/releases).
37+
38+
## Debugging
39+
40+
### Debugging TypeScript Code
41+
42+
In project that runs TypeSpec compiler (e.g. `typespec-tests` folder), run
43+
```shell
44+
node --inspect-brk "node_modules/@typespec/compiler/dist/src/core/cli/cli.js" compile <tsp-file>
45+
```
46+
to run `tsp compile` in debug mode (as Node.js process).
47+
48+
Then, have the debugging client attach to port 9229 of the Node.js process.
49+
Here is a [Visual Studio Code configuration](../../../typespec-tests/.vscode/launch.json).
50+
51+
Add break point to the `emitter.js` or `code-model-builder.js` under `node_modules/@azure-tools/typespec-java/dist/src` to debug the emitter.
52+
53+
### Debugging Java Code
54+
55+
See [Debugging Java Code](../../../typespec-extension/readme.md#debugging-java-code).
56+
57+
Alternatively, since the communication from TypeScript to Java is via the `code-model.yaml` file (plus the `EmitterOptions`), one can modify the `DEFAULT_OUTPUT_DIR` in `Main.java` under `core/packages/http-client-java/generator/http-client-generator` and debug `Main.main()`.

0 commit comments

Comments
 (0)