Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Validation errors out with pynwb 3.0.0 #1582

Closed
t-b opened this issue Feb 28, 2025 · 10 comments · Fixed by #1584
Closed

Validation errors out with pynwb 3.0.0 #1582

t-b opened this issue Feb 28, 2025 · 10 comments · Fixed by #1584
Labels
released

Comments

@t-b
Copy link

t-b commented Feb 28, 2025

dandi version: 0.66.7
OS: Debian bookworm x64

I'm about to update pynwb and dandi for our MIES CI 1.

But I'm hitting an issue with the new pynwb 3.0.0 version:

pip install pynwb==3.0.0

breaks

$dandi validate --ignore "(NWBI|DANDI)" ../../Packages/HardwareBasic-NI_11-V2.nwb 
/home/thomas/.venvs/pynwb/lib/python3.11/site-packages/pynwb/core.py:56: UserWarning: Use of icephys_filtering is deprecated and will be removed in PyNWB 4.0. Use the IntracellularElectrode.filtering field instead
  warn(error_msg)
[pynwb.GENERIC] /home/thomas/devel/mies/Packages/HardwareBasic-NI_11-V2.nwb — not enough values to unpack (expected 2, got 0)
2025-02-28 19:42:41,566 [    INFO] Logs saved in /home/thomas/.local/state/dandi-cli/log/2025.02.28-18.42.40Z-19255.log

but if I do

pip install pynwb==2.8.3

it works with

$dandi validate --ignore "(NWBI|DANDI)" ../../Packages/HardwareBasic-NI_11-V2.nwb 
No errors found.
2025-02-28 19:44:20,995 [    INFO] Logs saved in /home/thomas/.local/state/dandi-cli/log/2025.02.28-18.44.18Z-20319.log

I'm attaching the example file in case it's relevant:
HardwareBasic-NI_11-V2.nwb.zip

This also reproduces in fresh environment in a docker image.
@t-b t-b mentioned this issue Feb 28, 2025
Merged
1 task
@kabilar
Copy link
Member

kabilar commented Feb 28, 2025

Hi @h-mayorquin, could you please look into this issue? Perhaps this might be an issue with pynwb or nwbinspector? Thank you.

cc @bendichter

@h-mayorquin
Copy link

h-mayorquin commented Feb 28, 2025 •
edited
Loading

Hi, can you share the files that break validation? Are they stored somewhere where we can access them? This is to reproduce the error on my side.

@kabilar
Copy link
Member

kabilar commented Feb 28, 2025

I'm attaching the example file in case it's relevant: HardwareBasic-NI_11-V2.nwb.zip

@h-mayorquin Please see above.

@h-mayorquin
Copy link

My bad, thanks, I will take a look.

@h-mayorquin
Copy link

Hi, @kabilar .

I opened the file with the latest version pynwb and inspected it with the last version of the inspector and both of them worked fine. No error thrown.

I can reproduce this locally:

dandi validate --ignore "(NWBI|DANDI)" HardwareBasic-NI_11-V2.nwb 
2025-02-28 15:39:31,579 [    INFO] NumExpr defaulting to 16 threads.
/home/heberto/miniconda3/envs/work/lib/python3.11/site-packages/pynwb/core.py:56: UserWarning: Use of icephys_filtering is deprecated and will be removed in PyNWB 4.0. Use the IntracellularElectrode.filtering field instead
  warn(error_msg)
[pynwb.GENERIC] /home/heberto/Downloads/HardwareBasic-NI_11-V2.nwb — not enough values to unpack (expected 2, got 0)
2025-02-28 15:39:32,517 [    INFO] Logs saved in /home/heberto/.local/state/dandi-cli/log/2025.02.28-21.39.30Z-96539.log

So I think th error is on dandi cli indeed but I can't track where that error is originated.

@h-mayorquin
Copy link

h-mayorquin commented Feb 28, 2025 •
edited
Loading

Perhaps you could follow the advice on the warning if you have control over the nwbfile creation?

Must be some hardcoded return expectation at the dandi cli python API.

For the record this is the output of the inspector:

  nwbinspector HardwareBasic-NI_11-V2.nwb 
  /home/heberto/miniconda3/envs/work/lib/python3.11/site-packages/pynwb/core.py:56: UserWarning: Use of icephys_filtering is deprecated and will be removed in PyNWB 4.0. Use the IntracellularElectrode.filtering field instead
    warn(error_msg)
    0%|                                                                              | 0/1 [00:00<?, ?it/s]/home/heberto/miniconda3/envs/work/lib/python3.11/site-packages/pynwb/core.py:56: UserWarning: Use of icephys_filtering is deprecated and will be removed in PyNWB 4.0. Use the IntracellularElectrode.filtering field instead
    warn(error_msg)
NWBInspector Report Summary 
                 
    Timestamp: 2025-02-28 15:40:55.745717-06:00
    Platform: Linux-6.8.0-54-generic-x86_64-with-glibc2.39
    NWBInspector version: 0.6.3
    
    Found 24 issues over 1 file:
         4 - BEST_PRACTICE_VIOLATION
        20 - BEST_PRACTICE_SUGGESTION
BEST PRACTICE VIOLATIONS (4) 
    0.0  HardwareBasic-NI_11-V2.nwb: check_intracellular_electrode_cell_id_exists - 'IntracellularElectrode' object with name 'electrode_1'
         Message: Please include a unique cell_id associated with this IntracellularElectrode.

0.1  HardwareBasic-NI_11-V2.nwb: check_intracellular_electrode_cell_id_exists - 'IntracellularElectrode' object with name 'electrode_0'
     Message: Please include a unique cell_id associated with this IntracellularElectrode.

0.2  HardwareBasic-NI_11-V2.nwb: check_subject_species_exists - 'Subject' object at location '/general/subject'
     Message: Subject species is missing.

0.3  HardwareBasic-NI_11-V2.nwb: check_time_interval_time_columns - 'TimeIntervals' object with name 'epochs'
     Message: ['start_time'] are time columns but the values are not in ascending order. All times should be in seconds with respect to the session start time.
</pre>
BEST PRACTICE SUGGESTIONS (20) 
    1.4  HardwareBasic-NI_11-V2.nwb: check_description - 'Subject' object at location '/general/subject'
         Message: Description is missing.

1.5  HardwareBasic-NI_11-V2.nwb: check_description - 'VoltageClampStimulusSeries' object at location '/stimulus/presentation/data_00002_DA1'
     Message: Description ('PLACEHOLDER') is a placeholder.

1.6  HardwareBasic-NI_11-V2.nwb: check_description - 'CurrentClampStimulusSeries' object at location '/stimulus/presentation/data_00002_DA0'
     Message: Description ('PLACEHOLDER') is a placeholder.

1.7  HardwareBasic-NI_11-V2.nwb: check_description - 'VoltageClampStimulusSeries' object at location '/stimulus/presentation/data_00001_DA1'
     Message: Description ('PLACEHOLDER') is a placeholder.

1.8  HardwareBasic-NI_11-V2.nwb: check_description - 'CurrentClampStimulusSeries' object at location '/stimulus/presentation/data_00001_DA0'
     Message: Description ('PLACEHOLDER') is a placeholder.

1.9  HardwareBasic-NI_11-V2.nwb: check_description - 'VoltageClampStimulusSeries' object at location '/stimulus/presentation/data_00000_DA1'
     Message: Description ('PLACEHOLDER') is a placeholder.

1.10  HardwareBasic-NI_11-V2.nwb: check_description - 'CurrentClampStimulusSeries' object at location '/stimulus/presentation/data_00000_DA0'
      Message: Description ('PLACEHOLDER') is a placeholder.

1.11  HardwareBasic-NI_11-V2.nwb: check_description - 'VoltageClampSeries' object at location '/acquisition/data_00002_AD1'
      Message: Description ('PLACEHOLDER') is a placeholder.

1.12  HardwareBasic-NI_11-V2.nwb: check_description - 'CurrentClampSeries' object at location '/acquisition/data_00002_AD0'
      Message: Description ('PLACEHOLDER') is a placeholder.

1.17  HardwareBasic-NI_11-V2.nwb: check_experimenter_exists - 'NWBFile' object at location '/'
      Message: Experimenter is missing.

1.18  HardwareBasic-NI_11-V2.nwb: check_experiment_description - 'NWBFile' object at location '/'
      Message: Experiment description is missing.

1.19  HardwareBasic-NI_11-V2.nwb: check_institution - 'NWBFile' object at location '/'
      Message: Metadata /general/institution is missing.

1.20  HardwareBasic-NI_11-V2.nwb: check_keywords - 'NWBFile' object at location '/'
      Message: Metadata /general/keywords is missing.

1.21  HardwareBasic-NI_11-V2.nwb: check_subject_age - 'Subject' object at location '/general/subject'
      Message: Subject is missing age and date_of_birth. Please specify at least one of these fields.

1.22  HardwareBasic-NI_11-V2.nwb: check_subject_id_exists - 'Subject' object at location '/general/subject'
      Message: subject_id is missing.

1.23  HardwareBasic-NI_11-V2.nwb: check_subject_sex - 'Subject' object at location '/general/subject'
      Message: Subject.sex is missing.
</pre>

@kabilar
Copy link
Member

kabilar commented Mar 1, 2025

Thanks for investigating, @h-mayorquin. That was very helpful.

Hi @jwodder, next week, could you please look into why the dandi client is throwing the error with pynwb==3.0.0 for the above file? Thank you.

@jwodder jwodder changed the title validation does error out with pynwb 3.0.0 Validation errors out with pynwb 3.0.0 Mar 3, 2025
@jwodder jwodder mentioned this issue Mar 3, 2025
Merged
yarikoptic added a commit that referenced this issue Mar 3, 2025
@yarikoptic
Merge pull request #1584 from dandi/gh-1582
633e141 
Fix support for pynwb v3 which has changed API slightly
@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 3, 2025

🚀 Issue was released in 0.67.0 🚀

@kabilar
Copy link
Member

kabilar commented Mar 3, 2025

Thank you @jwodder.

@t-b, please upgrade your dandi client to version 0.67.0 by running: pip install --upgrade dandi and retry the dandi validate ... command.

@t-b
Copy link
Author

t-b commented Mar 4, 2025

@kabilar Thanks for the quick fix. Works now like a charm!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
released
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix support for pynwb v3 which has changed API slightly dandi/dandi-cli
3 participants
@kabilar @t-b @h-mayorquin