From c469da5fac3df5c1a6e47bced6503994126adf09 Mon Sep 17 00:00:00 2001 From: derrick chambers Date: Thu, 26 Dec 2024 10:07:05 -0800 Subject: [PATCH] update docs --- docs/contributing/adding_test_data.qmd | 4 +++- docs/contributing/new_format.qmd | 2 +- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/contributing/adding_test_data.qmd b/docs/contributing/adding_test_data.qmd index b82570b3..3bb3300c 100644 --- a/docs/contributing/adding_test_data.qmd +++ b/docs/contributing/adding_test_data.qmd @@ -45,7 +45,7 @@ If, in the test code, the example patch or spool is used only once, just call th Of course, not all data can easily be generated in python. For example, testing [support for new file formats](./new_format.qmd) typically requires a test file. -If you have a small file that isn't already hosted on a permanent site, you can put it into [dasdae's data repo](https://github.com/DASDAE/test_data). Simply clone the repo, add you file format, and push back to master or open a PR on a separate branch and someone will merge it in. +If you have a small file that isn't already hosted on a permanent site, you can put it into [dasdae's data repo](https://github.com/DASDAE/test_data). Simply clone the repo, add you file format, and push back to master or open a PR on a separate branch and someone will merge it. Next, add your file to dascore's data registry (dascore/data_registry.txt). You will have to get the sha256 hash of your test file, for that you can simply use [Pooch's hash_file function](https://www.fatiando.org/pooch/latest/api/generated/pooch.file_hash.html), and you can create the proper download url using the other entries as examples. @@ -62,3 +62,5 @@ https://github.com/dasdae/test_data/raw/master/das/jingle_test_file.jgl from dascore.utils.downloader import fetch path = fetch("jingle_test_file.jgl") ``` + +If you need to create a smaller version of an existing hdf5 file you can use the `modify_h5_file.py` in DASCore's scripts directory. It will require some modifications, but shows how to copy and modify datasets and attributes. diff --git a/docs/contributing/new_format.qmd b/docs/contributing/new_format.qmd index 81cd6d83..4a100515 100644 --- a/docs/contributing/new_format.qmd +++ b/docs/contributing/new_format.qmd @@ -199,7 +199,7 @@ path.unlink() # cleanup test file ## Writing Tests -Next we need to write tests for the format (you weren't thinking of skipping this step were you!?). The hardest part of testing new file formats is finding a *small* (typically no more than 10 ish mb) file to include in the test suite. Once you have such a file, [Adding test data](adding_test_data.qmd) details how to add it to the registry. +Next we need to write tests for the format (you weren't thinking of skipping this step were you!?). The hardest part of testing new file formats is finding a *small* (typically no more than 10 ish mb) file to include in the test suite. The `modify_h5_file.py` script in DASCore's scripts folder can help downsize an existing hdf5 file. Once you have a small test file, [Adding test data](adding_test_data.qmd) details how to add it to DASCore's registry. Once the test file is added to the data registry, you can register the new format so a suite of tests run automatically. This is done by adding the format to the appropriate data structures in `tests/test_io/test_common_io.py`. The comments at the top of the file will guide you through this process.