Ways to help users navigate the issue of heterogenous metadata

[cfrick13]

Use Case

If I am looking for long (>24 hr) time lapse movies acquired with the 20x objective, there are heterogenous metadata/annotations to sort through.

To get the right objective lens, I filter for two different fields
Objective Filters

1. 'Objective'==20x
2. 'Microscopy Objective'==20x.

(NOTE: being able to use OR logic to combine the two filters would be very useful.

To get long time-lapses I could filter for:
Timelapse Filters

1. 'Total Time Duration'>24hr
2. 'Timelapse'==True AND 'Size' > 1 TB
3. 'Image Size T' > 100 AND 'Timelapse Interval' >5 min

The challenges are:

1. right now is that I cannot combine filters 1&2 in either case with OR logic.
2. It is hard to know when heterogeneous metadata is leading to missed files. For example, lets say I have 100 long timelapse files total and 25 of then use Microscopy Objective instead of Objective. If I filter for long timelapses, then filter for 'Objective'==20x I will get 75 files, and nothing in BFF tells me that I excluded an additional 25 files.

One thing I cannot do is ask: out of the 100 long time lapse files, how many have the Objective annotation. If I saw it was 75 only then I would know to look for another metadata field.

Suggest a Solution

1. the ability to use OR logic to combine filters
2. some way to handle metadata that means the same thing, e.g. Objective and Microscopy Objective. Maybe some sort of alias feature?
3. some way to ask how many files have the annotation field at all (not just a specific value within that annotation field).

[SeanLeRoy]

the ability to use OR logic to combine filters
This would be awesome and technically isn't too crazy hard, the only issue might be that the UX of it might be kind of hairy. I'll see what we can brainstorm about this.

some way to handle metadata that means the same thing, e.g. Objective and Microscopy Objective. Maybe some sort of alias feature?

If these mean the same thing why are they separate annotations?

some way to ask how many files have the annotation field at all (not just a specific value within that annotation field).

We could provide the count of files excluded (like how we show "X total files" which in your case would be "75 total files", we could show "75/100 total files")

[cfrick13]

If these mean the same thing why are they separate annotations?

The annotations were created by different groups during different upload efforts. Unfortunately this is likely to occur many many times. When we have an upload-first mentality, then annotations will be given before standardization is achieved.

Thanks for looking into solutions for these challenges, @SeanLeRoy !