Add ExtraCode declarationFile and implementationFile directives

[m-fila]

BEGINRELEASENOTES

• Added ExtraCode declarationFile and implementationFile directives

ENDRELEASENOTES

Added 'ExtraCode' directives 'declarationFile' and 'implementationFile' used to include/embed files in code generated with jinja. This feature was planned but not implemented so far. Can be used to fix key4hep/EDM4hep#296

The paths to 'declarationFile' and 'implementationFile 'can be either absolute or relative to datamodel yaml. Should this be relative to working directory?

The files declared in 'declarationFile' and 'implementationFile' are not added as configuration dependency in cmake, so the changes to these files will not automatically trigger re-generating a data model. I don't know if there is an easy way to fix this

TODO:

• test mutable declaration and mutable implementations in datatype
• add component 'declarationFile'
• test declarationFile in component
• fix tests that copy the yml file

[tmadlener]

Thanks for taking this on. IIUC the injection happens in the jinja stage, right? This also means that the implementation files (or at least their contents) are not stored in the output files, right? (At least that is my explanation for having to carry them to the tests directory for the roundtrip tests).

Would it be possible to have the "parsing" of these implementation file at the datamodel reading stage? I.e. whenever we encounter such an implementation file, we just inject the contents directly into the existing ExtraCode bit? In that way we they should directly end up in the datamodel definition that is stored in the file (I think / hope). Additionally, I think we would not have to touch the generation step at all(?).

[m-fila]

Yes, during jinja step a python function is used to read a file and return its content to jinja, which just put it as is into generated file. The files itself are not copied anywhere.
In the metada datamodel definitions the 'declarationFile' and 'implementationFile' are put without any modifications, which was causing problems when the paths are relative, hence I copied the files to build directory

I think appending the contents of 'declatationFile' to 'declaration' during parsing should work better and give more transparent metadata. Thanks 😃

[tmadlener]

Very nice. Did you check whether this works in the end for EDM4hep and whether it really resolves the issue you set out to fix? edit: yes you did, I missed the corresponding PR. Sorry.

---

The paths to 'declarationFile' and 'implementationFile 'can be either absolute or relative to datamodel yaml. Should this be relative to working directory?

No I think absolute, or relative to the yaml file is fine. But we should document it.

The files declared in 'declarationFile' and 'implementationFile' are not added as configuration dependency in cmake, so the changes to these files will not automatically trigger re-generating a data model. I don't know if there is an easy way to fix this

One possibility (that would make it easy for us / podio again) would be to document this and tell people that they have to add this to their cmake configuration via, similar to what we do in the macro:

podio/cmake/podioMacros.cmake

Lines 201 to 211 in a18229b

	set_property(
	DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS
	${YAML_FILE}
	${PODIO_TEMPLATES}
	${podio_PYTHON_DIR}/podio_class_generator.py
	${podio_PYTHON_DIR}/podio_gen/generator_utils.py
	${podio_PYTHON_DIR}/podio_gen/podio_config_reader.py
	${podio_PYTHON_DIR}/podio_gen/generator_base.py
	${podio_PYTHON_DIR}/podio_gen/cpp_generator.py
	${podio_PYTHON_DIR}/podio_gen/julia_generator.py
	)

The only thing we could potentially offer is to make the macro take an additional list of the files that are used in the datamodel and add them there. But at that point the users do not gain too much, I think.

[hegner]

If builds run through, I will merge it