HDF5: Explicit control over chunking

[franzpoeschel]

Series("output.h5", Access::CREATE, R"(hdf5.dataset.chunks = "auto")");
// ...
Dataset dataset = Dataset(datatype, global_extent, R"(
[hdf5.dataset]
chunks = [10, 100]
)");
record_component.resetDataset(dataset);

TODO:

• Merge Initial support for HDF5 subfiling #1580 first
• Documentation
• Further testing
• Decide if the JSON helper should be exposed publicly

[guj]

I see there are two styles of json for series and dataset

• hdf5.dataset.chunks = "auto"
• [hdf5.dataset] chunks = [10, 100]

any chance to consolidate?

[franzpoeschel]

I see there are two styles of json for series and dataset

* hdf5.dataset.chunks = "auto"

* [hdf5.dataset] chunks = [10, 100]

any chance to consolidate?

That is TOML, and both forms are equivalent in TOML. The JSON form would be {"hdf5":{"dataset":{"chunks": "..."}}}.
TOML does not have a canonical form on purpose, you can write either of the above as the other as well.

[guj]

Oh, I see. Was wondering.

That is TOML, and both forms are equivalent in TOML. The JSON form would be {"hdf5":{"dataset":{"chunks": "..."}}}. TOML does not have a canonical form on purpose, you can write either of the above as the other as well.

[ax3l]

@franzpoeschel can this todo now be removed?

[franzpoeschel]

yep, this todo is now fixed

[ax3l]

Looks great!

Please see last inline comment and then feel free to merge :)

[franzpoeschel]

Looks great!

Please see last inline comment and then feel free to merge :)

Thanks! I removed the old todos