$ref does not work inside schema.json

[fdrozdowski]

While trying to do the conversion for EBS 12.1 appsTier Provisioning, I am unable to build the artifact.json as it says me always:

Validating plugin entry point : plugin_runner:plugin
 Importing plugin module : plugin_runner
 Error: No module named display
0 Warning(s). 1 Error(s).
BUILD FAILED.
 Traceback (most recent call last):
  File "/home/mkapoor/delphix/vsdk/ENV/local/lib/python2.7/site-packages/dlpx/virtualization/_internal/cli.py", line 39, in command_error_handler
    yield
  File "/home/mkapoor/delphix/vsdk/ENV/local/lib/python2.7/site-packages/dlpx/virtualization/_internal/cli.py", line 174, in build
    build_internal.build(plugin_config, upload_artifact, generate_only)
  File "/home/mkapoor/delphix/vsdk/ENV/local/lib/python2.7/site-packages/dlpx/virtualization/_internal/commands/build.py", line 100, in build
    raise exceptions.BuildFailedError(err)
 BuildFailedError: Error: No module named display
0 Warning(s). 1 Error(s).
BUILD FAILED.

It seems that schema.json is not accepting a property defined as below:

 
"services": {
                "$ref": "#/definitions/services"
            },
            "display": {
                "$ref": "#/definitions/display"
            },

This is used at multiple places inside VirtualSourceDefinition and hence $ref
was used to maintain the clarity and readability of schema. Similar issue has been also seen while trying to convert Lua code to configure EBS appsTier on multi-node environment.

┆Issue is synchronized with this Jira Bug by Unito
┆Attachments: schema.json

[fdrozdowski]

➤ Mehak Kapoor commented:

Please find the schema.json attached which throws this issue (check the VirtualSourceDefinition) section for this issue and $ref usage.[^schema.json]

[fdrozdowski]

➤ Tom Walsh commented:

I think the format of the reference here is incorrect.

The attached schema file has this:

"services": {
"$ref": "#/definitions/services"
}That reference breaks down like this:

• The # means "Go to the root of the document".
• Next, find the value of the top-level key definitions
• Within that definitions object, find the value of the key services

(For more details, see https://json-schema.org/understanding-json-schema/structuring.html)

But, the attached schema does not have a top-level key called definitions, so the lookup at step 2 fails.

The schema does have a definitions key, but it is not at the top-level... Instead, it's within the virtualSourceDefinition object. So, I think there are two choices to make the format correct...

1. You could move the definitions section to the top-level. That is probably appropriate if these types are not specific to VDBs (and might be used in future by dSource or repositories, etc.)
2. You could change the references to look like "#/virtualSourceDefinition/definitions/services".

[fdrozdowski]

➤ Mehak Kapoor commented:

[~tom.walsh] I tried to modify my schema.json from 
"$ref": "#/definitions/services"
to
"$ref": "#/virtualSourceDefinition/definitions/services"
but still got the below error:

Validating plugin config file content : {'pluginType': 'DIRECT', 'name': 'ebs121-app', 'language': 'PYTHON27', 'manualDiscovery': True, 'hostTypes': ['UNIX'], 'version': '0.1.0', 'entryPoint': 'plugin_runner:plugin', 'srcDir': 'src', 'prettyName': 'ebs121-app', 'schemaFile': 'schema.json'}
Validating plugin entry point : plugin_runner:plugin
Importing plugin module : plugin_runner
Error: No module named virtual_source_definitiondefinitionsdisplay
0 Warning(s). 1 Error(s).
BUILD FAILED.
Traceback (most recent call last):
File "/home/mkapoor/delphix/vsdk/ENV/local/lib/python2.7/site-packages/dlpx/virtualization/_internal/cli.py", line 39, in command_error_handler
yield
File "/home/mkapoor/delphix/vsdk/ENV/local/lib/python2.7/site-packages/dlpx/virtualization/_internal/cli.py", line 174, in build
build_internal.build(plugin_config, upload_artifact, generate_only)
File "/home/mkapoor/delphix/vsdk/ENV/local/lib/python2.7/site-packages/dlpx/virtualization/_internal/commands/build.py", line 102, in build
raise exceptions.BuildFailedError(err)
BuildFailedError: Error: No module named virtual_source_definitiondefinitionsdisplay
0 Warning(s). 1 Error(s).
BUILD FAILED.As we discussed, for now, I am simply replacing the $ref section with the actual json schema intended for services as well as disply module and it's rendered fine on the UI. But, since, this needs to be done at multiple places and increases the schema by multiple lines - would definitely prefer if, the $ref could be used as this maintains the much needed clarity.

[fdrozdowski]

➤ Tom Walsh commented:

Looks like JSON schema references are maybe not supported (or not fully supported?) by our code generation tool.

We should research if this is a bug or a known limitation and if there are workarounds.

At the very least, we'd need to update our docs to let people know about this.

[fdrozdowski]

➤ Lindsey Nguyen commented:

Spent a little time working on this and it seems like there was a bug in the swagger code gen version we're currently on. They have a fix out but there will be sdk changes we have to do to make it work correctly. Given that for a plugin writer they'd just have to copy the reference instead, we've determined it's not a blocker for GA.

[~mehak.kapoor] To proceed with this for now I recommend just copying the references definitions to where the references are.

[fdrozdowski]

➤ Mehak Kapoor commented:

[~lindsey.nguyen] Any plans with this change? Or I should continue to use the workaround still?

[hemant-gairola]

Fix for this bug could be very helpful in such cases where we want to maintain the complex part of the schema in a separate subschema definition.
Any idea when are we planning a fix for this issue?