Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Initial support for HDF5 subfiling #1580

Merged
merged 4 commits into from
Jan 30, 2024
Merged

Initial support for HDF5 subfiling #1580

merged 4 commits into from
Jan 30, 2024

Conversation

franzpoeschel
Copy link
Contributor

@franzpoeschel franzpoeschel commented Dec 20, 2023 •
edited
Loading

I'm following the documentation in this presentation.

TODO:

  • parameterization
  • opt-in
  • documentation

@franzpoeschel franzpoeschel force-pushed the hdf5-subfiling branch 2 times, most recently from 94c4d5b to 5212931 Compare December 22, 2023 18:29
@franzpoeschel franzpoeschel requested a review from ax3l January 19, 2024 16:41
@franzpoeschel franzpoeschel mentioned this pull request Jan 24, 2024
Merged
4 tasks
@ax3l ax3l requested a review from jeanbez January 30, 2024 18:38
ax3l
ax3l approved these changes Jan 30, 2024
View reviewed changes
Copy link
Member

@ax3l ax3l left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thank you!

@ax3l ax3l merged commit 8faadde into openPMD:dev Jan 30, 2024
31 checks passed
@ax3l ax3l requested a review from guj January 30, 2024 18:39
@ax3l
Copy link
Member

ax3l commented Jan 30, 2024

cc @guj and @jeanbez : just FYI, sub-filing support for HDF5 in openPMD-api now :)

@ax3l
Copy link
Member

ax3l commented Jan 30, 2024

We should check if Conda-Forge (hdf5 and h5py), h5py (pip builds) and Spack enable this feature in the HDF5 package already (non-default).

@franzpoeschel
Copy link
Contributor Author

Note that this needs to be enabled explicitly when configuring HDF5 -DHDF5_ENABLE_SUBFILING_VFD=ON.

@guj
Copy link
Contributor

guj commented Jan 30, 2024

Is MPI_THREAD_MULTIPLE. still a requirement to use subfiles for HDF5?
If so, needs to state in the doc.

@franzpoeschel
Copy link
Contributor Author

Is MPI_THREAD_MULTIPLE. still a requirement to use subfiles for HDF5? If so, needs to state in the doc.

Yes, it is a requirement. It's mentioned in the docs (Note that the subfiling VFD needs to be enabled explicitly when configuring HDF5 and threaded MPI must be used.), though the specific level is not mentioned yet (MPI_THREAD_MULTIPLE).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ax3l ax3l ax3l approved these changes

@jeanbez jeanbez Awaiting requested review from jeanbez

@guj guj Awaiting requested review from guj

Assignees
No one assigned
Labels
backend: HDF5
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@franzpoeschel @ax3l @guj