Documentation and Traceability of Requirements

[floroks]

Summary / Discussion

In the CDA we experimented with AsciiDoc and openfasttrace for the documentation of requirements and architecture,
which can also serve as a general "book-type" documentation of the application.

Generally speaking, the idea was to document requirements & architecture with AsciiDoc, and to utilize the api
documentation of the code as detailed design/implementation. Traceability was ensured by using openfasttrace
to annotate and link the requirements with the architecture, the comments in the code, as well as tests.

For more complex designs, additional design documentation could also be used, which would then link to the
implementation.

This works - aside from the typical asciidoc include path shenanigans - pretty well, and offers a lot of flexibility.

Since one of the longer term project goals is to be integrated with SCORE, and SCORE utilizes sphinx-needs, I
evaluated the use of sphinx-needs for the same use-case, and came the these findings:

Good:

• reStructuredText with sphinx-needs works for the requirements/architecture & generation of "book-type" documentation
• overviews of items (requirements/architecture) are better with sphinx-needs, as is automatic linking between the items
• sphinx-needs is more popular than openfasttrace, and more tooling/integrations exists
• compatibility with score

Bad:

• book-type documentation is a bit harder to read with sphinx-needs, due to the clearer distinction between individual needs
• reStructuredText is a lot less powerful then AsciiDoc, and writing it requires more conventions and thinking about it
  • includes can't automatically increase the level offset of sections, sections always need underlines (and overlines sometimes, if you need many levels) with the correct ascii char

Ugly:

Traceability into code doesn't work out of the box for Rust, since doxygen and doxysphinx do not support Rust.

It is however possible to utilize rustdoc to generate json, and write a small script that expects some convention in the
documentation to generate need-items as an additional document, which can then be utilized by sphinx-needs.

Alternatively it'd be possible to use the external needs json mechanism, and to create a json file, but you'll lose
the ability to click on the items to navigate to them.

Rustdoc is also only usable in rust nightly builds, since it requires the -Z unstable-options switch. This seems to
be the case for multiple years now.

There are other possible solutions, like utlizing syn to parse the actual source files, and extract the comments
from there, but those require even more effort.

--

My opinion is, that we should utilize sphinx-needs with the custom script. This requires the use of nightly builds just
for rustdoc-json, but other options exist should this path not be usable in the future.

We'll also need to come up with a convention for the sphinx needs items in code documentation, and a proper small piece of code
that extracts them from the rustdoc-json to sphinx-needs convention (e.g. rst files).

Since as a project we should find & use a common solution, I'm asking for your opinions and a discussion on this matter.

I've included a generated full documentation from the proof-of-concept as an attachment here.

Proof of Concept

In the branch sphinx_needs_docs of the CDA, I've built a small proof-of-concept of using sphinx-needs with rustdoc to json output
and a converter to generate the rst files with the needs from the code comments. The existing documentation has been duplicated
in the docs_sn folder and can be generated with make. It's very rough llm generated code for the converter, and translated
documentation with many manual fixes. It also misses some bits and pieces, like a conversion from markdown to rst (or conversion
to html, and using that in the rst), and some section markers might be the wrong level.

To get it running yourself, you'll need to install all the dependencies (plantuml, python, sphinx-needs, sphinxcontrib-plantuml
and so on. I'd highly recommend to just look at the source files, and the attached result.

Note: the format for the needs items in rustdoc will probably need to change to make sure it's compatibly with clippy, and
happens in the least introsive way (maybe html commented out sections?)