Auto generate xsd images

[clagms]

This is a preview of the solution to #569.

Please have a look and give me feedback, so I can implement the full solution.

Changes and approach:

• I've modified the tool XSDDiagram to enable the automatic generation of the pictures from the schema (see the ./bin/ folder).
• Generated images are in ./docs/images/gen/ folder.
• Automatically marked attribute images with an "outdated" label (see limitations below).

Advantages:

• Automated generation of images (so they are kept consistent).

Limitations:

• Edit: I patched XSDDiagram to generate attributes (see Draw attributes along with elements in GDI Renderer dgis/xsddiagram#28 and Recursively add elements to be drawn and avoid elements that have no name dgis/xsddiagram#27).
• I haven't made sure it works on linux.

Note that the tools placed in ./bin/ are temporary and just so you can give me some feedback.

[t-sommer]

The images in docs/images/gen are not being copied into the distribution archive in CI (all broken).
Also, without the attributes, they're kind of useless since all the information is stored there.
I think we should go with the semi-automated approach proposed by @CSchulzeTLK.

[t-sommer]

See Limitations in the first comment.

[clagms]

Thank you @t-sommer !

The images in docs/images/gen are not being copied into the distribution archive in CI (all broken).

Ups, my bad. Working on correcting this.

Also, without the attributes, they're kind of useless since all the information is stored there.

I fixed the tool to produce the attributes (see dgis/xsddiagram#28)
It now produces something like this:

I think we should go with the semi-automated approach proposed by @CSchulzeTLK.

In the light of the fix above, it is now possible to plot the attributes. Are you sure that you want to go for the semi-automated approach? It's bound to lead to inconsistencies, and I don't want to be the one doing it manually :P
Just let me know so I don't spend more time on this!

[chrbertsch]

See Limitations in the first comment.

@clagms 👍 I like your approach.
Is it possible to influence e.g. the spacing ?

[andreas-junghanns]

I agree with Christian: if we can reduce spacing a bit to waste less space, this is a great solution because we drop even more redundancy - a process started by TorstenS a while back.
I also prefer the slightly cleaner look.

[clagms]

@clagms 👍 I like your approach.
Is it possible to influence e.g. the spacing ?

Yes, this option is part of the tool, but is not exposed in the command line. I can expose it.

With the compact option, the drawing goes from this:

To this:

[clagms]

Commit 78327ab generates compact diagrams.

[clagms]

The images in docs/images/gen are not being copied into the distribution archive in CI (all broken).

@t-sommer I might need a bit of help with this.
I'm running the following command on the master branch, which is essentially copied from ci config:

docker run -v ${FMIStandarDir}:/documents/ -w=/documents/ --name $BUILDERID myubuntu zip fmi-standard.zip README.adoc CHANGELOG.adoc LICENSE.txt docs/index.html docs/fmi-spec.css docs/images/* headers/*.h schema/*.xsd

And I'm still not getting the images in the zip file. So this is an already existing problem, and was not introduced by the current PR, right?
If that's the case, I won't try to solve this here, to avoid mixing changes.

EDIT: the following command seems to fix the problem:

zip -r fmi-standard.zip README.adoc CHANGELOG.adoc LICENSE.txt docs/index.html docs/fmi-spec.css docs/images/ headers/*.h schema/*.xsd

[beutlich]

clagms#1 removes the absolute path (being not available).

[chrbertsch]

Regular design meeting:

@clagms : I will continue on this, perhaps with some help of @t-sommer

Torsten: It we good enough to check in the images created with an open-source tool and document how to create them. We do not change the xsd. The people who change the xsd will know to do it.

[clagms]

@t-sommer, @beutlich, I think I'm done with the PR. Could you have a look?

[beutlich]

Nice to see that you forked xsddiagram and even prepared a release. What about downloading https://github.com/clagms/xsddiagram/releases/download/XSDDiagram_1.2_FMI/XSDDiagram_1.2_FMI.zip and extracting the binary in the powershell script for a fully automatic workflow (as I proposed earlier)?

[beutlich]

I do not understand what the motivation was to move the images from docs/images to docs/images/gen. If generated or obtained differently should not matter.

[clagms]

having binaries in the repository

Sure. The workflow proposed in https://github.com/clagms/fmi-standard/pull/1/files#diff-f892b8c44b29d86a02974bc00e9c24c7R18-R23 therefore is not about having binaries (EXE files) in the repository, but how to get and use them in an automated way for the easiest way of reproducing the PNGs. If you add these EXE files to the gitignore, you can be sure to not commit them by accident. So, after all, it is about extending the powershell script.

Ah, I think I understand what you mean now @beutlich . Do you want to change the script to automatically download the exe file? Can you make that change?

[clagms]

Based on the above, is it clear why there must be a separate folder for the generated images?

Actually, it is not. You simply can add a Readme file to doc/images, stating, that this directory will be used as output directory for xsddiagram, and that PNGs should not be edited manually. Then you simply can use doc/images directly as output directory and get rid of the intermediate step using doc/images/gen.

How would you change the script in order to avoid overwriting a generated png @beutlich ? If you can make that change, I'm fine with this! Beware that there are many other figures in the images folder, that are not generated. If you decide to create the README file, make sure that people will know which figures are generated, and make sure to keep that consistent as more generated figures are added.

[beutlich]

Do you want to change the script to automatically download the exe file? Can you make that change?

See clagms#2, where I added (and improved) it again.

[beutlich]

Regarding docs\images\gen. I have no strong opinion here. However, there are also drawio generated SVG files in doc\images along with their XML configuration. I had simply expected, that generate_figures.ps1 generates or regenerates all schema PNGs in doc\images and does not care if the file already exists. (We have version control for that reason.)

[clagms]

See clagms#2, where I added (and improved) it again.

Thank you so much @beutlich ! I've merged this!

Regarding docs\images\gen. I have no strong opinion here. However, there are also drawio generated SVG files in doc\images along with their XML configuration. I had simply expected, that generate_figures.ps1 generates or regenerates all schema PNGs in doc\images and does not care if the file already exists. (We have version control for that reason.)

It's a matter of personal opinion, but I'd rather be more defensive, and ensure that no override happens unintentionally. For example, if the following instructions are used:

Export-Schema Float64 3 -schema fmi3Type.xsd 
Export-Schema Float64 3

There will be an overwrite. I'm afraid the source control will not be of much help here, since it will only show that one element has been modified. If the person who added the second instruction above does not pay close attention, an override has been made, and the doc will have two pictures which are the same, but one of those references are wrong.
One requires visual inspection of the doc to later detect this error and that seems too much to ask.

An alternative is to rewrite the script into accepting an array of parameter records, and checks that no duplicates are generated.

However, there are also drawio generated SVG files in doc\images along with their XML configuration.

I was unaware of this. I recommend that those be put in a separate gen folder (not the same as the xdiagram one.. we can reorganize this). It's very easy to edit SVG using inkspace, so I wouldn't be surprised if at some point someone does some improvements on the generated figures.

[t-sommer]

Please move the generated PNGs to images/schema and remove the _schema suffix form the file names (as discussed at the Regular Design Meeting). The rest looks good!

[CSchulzeTLK]

I am searching for the information if an element is optional. Is this visible in the images?

[CSchulzeTLK]

... just found it