naming conflicts in generated schema files (xsd)

[FirefoxMetzger]

The file world.sdf includes both model.sdf and state.sdf

https://github.com/ignitionrobotics/sdformat/blob/722b92c92e2b19c408f3d4846ea7fcb5dfa7ed58/sdf/1.8/world.sdf#L62-L68

state.sdf in turn includes model_state.sdf

https://github.com/ignitionrobotics/sdformat/blob/722b92c92e2b19c408f3d4846ea7fcb5dfa7ed58/sdf/1.8/state.sdf#L37

Both model_state.sdf and model.sdf define a <model> tag:

https://github.com/ignitionrobotics/sdformat/blob/722b92c92e2b19c408f3d4846ea7fcb5dfa7ed58/sdf/1.8/model.sdf#L2

https://github.com/ignitionrobotics/sdformat/blob/722b92c92e2b19c408f3d4846ea7fcb5dfa7ed58/sdf/1.8/model_state.sdf#L2

which in itself shouldn't be a problem, because they exist in different scopes, one inside <world> and the other inside <state>.

This include chain is reflected in the generated .xsd files; however, the includes are all placed at the top of their respective document, which makes both <xsd:element name='model'> tags appear on the same level inside world.xsd, i.e., the nesting of model_state's model is not preserved. This - naturally - leads to a conflict and in the current version model_state's <model> overwrites model's <model>, which is (very) undesirable.

I'm not 100% sure how to fix this, since I am not familiar enough with the XMLSchema spec, but I assume the solution is something along the lines of renaming state tags, introducing namespaces, or ensuring that model_state's model is included inside the state element (preferred, but not sure if feasible).

[FirefoxMetzger]

This appears to affect all elements that have an accompanying *_state.xsd, so model, light, and link,

[FirefoxMetzger]

This also affects <pose> defined in /sdf/v1.X/schema/types.xsd and <pose> defined in /sdf/v1.X/pose.sdf. This is a slightly different path than the X_state.sdf vs X.sdf files, but appears to have the same root cause and a similar result.

In this case (<pose>) I guess the solution is to remove the entry in types.xsd, since pose.sdf is meant to supersede it?

[azeey]

Yes, we have been trying to solve this issue in #535, but we got stuck and put it in the backlog for a while. In that PR, we tried to paste the included xsd element instead of referencing it to avoid the name collision, but that doesn't work because we have recursive relationships in models, i.e., a model can contain a nested model.

[FirefoxMetzger]

Thanks for the response @azeey . In my local clone, I implemented unique namespaces for each file with respective referencing. This produces a valid schema that is very similar to sdf, but it no longer describes existing sdf files, because tags in it should be namespaced as well (which they aren't). This kind of rules out the namespace avenue.

Judging from this and my half-knowledge of XMLSchema I am beginning to fear that the current SDFormat is no longer valid XML. At the same time, it should be possible to write a correct schema, as aliased elements can never occur within the same tag, which means that a valid schema should exist. More thinking is needed here.

that doesn't work because we have recursive relationships in models, i.e., a model can contain a nested model

Does this mean it would be okay to just have a single SDFormatSchema.xsd in the end?

[azeey]

Judging from this and my half-knowledge of XMLSchema I am beginning to fear that the current SDFormat is no longer valid XML. At the same time, it should be possible to write a correct schema, as aliased elements can never occur within the same tag, which means that a valid schema should exist. More thinking is needed here.

I still think there should be a way to generate the .xsd files from .sdf description files, but it will require more time and effort.

Does this mean it would be okay to just have a single SDFormatSchema.xsd in the end?

Yeah, that's what we were thinking. I think the only benefit of having multiple *.xsd files was that they can reference each other making the root.xsd a small file.

\cc @jennuine

[FirefoxMetzger]

A night of sleep does indeed do wonders.

Here is a proposal for a solution: Transpile each file individually and introduce a unique namespace for each file. During transpiling generate a complexType and an Element for each tag in the respective SDFormat. Replace <include>s with a tag for that element (in the parent namespace) using as type the complexType within the respective file's namespace. I haven't tested this solution yet, but if it works it should work for the general case and make SDFormat quite flexible/extensible.

For my specific situation (generating python bindings) I have a second solution, which is to again use namespaces but to not introduce complexTypes. Instead <include> tags are replaced with a <xsd:element ref="..." /> tag. This generates correct bindings; however, expects SDF tags to use the correct namespace, e.g. <model> would become <model:model> assuming that this is how the namespace is referred to inside root.xsd. What I can do (which is dirty, but works) is to then simply remove all the namespace checks from the stubs, and voila, I can parse SDFormat 🤷 . (With the exception of <pose> inside of <include>, see #633 )

[FirefoxMetzger]

Here is a proposal for a solution [...]

An update: I tried this and it seems to work. I (re-)wrote the ruby generation script in python - I'm more familiar with that language and hence faster - and I can now generate xsd which validates and which allows me to generate bindings from it. Are there internal unit tests for the generated xsd I can check against? It also addresses some of the other issues I raised.

If there is interest in this solution, I can contribute this generator upstream.

[jennuine]

Here is an integration test.

The solution also needs to work with nested models (e.g., nested_canonical_link.sdf) as well as a saved world from Gazebo classic. You can run the following command to check your generated xsd with another sdf model/world:

xmllint --noout --schema /path/to/generated.xsd /path/to/model.sdf

[FirefoxMetzger]

@jennuine thanks for that. Ill give it a shot once I am back from my vacation on Wednesday.

I checked it with the simulation world I am using and I didnt find any issues yet. It doesnt have nested models iirc, but I dont see them as a big issue for xsd ... maybe I am missing something.

Edit: my worlds dont have nested models, the xsd should support nested models.

[jennuine]

@FirefoxMetzger sounds good, hope you have a great vacation! I've updated the branch from #535 to include the two other cases that the solution needs to work with (in test/integration/schema_test.cc). Feel free to reach out if you have any issues or questions 😃