-
Hello, I am working on trying to apply MP techniques to the research I am involved in. Basically there are events in the solar wind called small-scale flux rope (SFR) events. There is a paper (zhao et al. 2020) that outlines a method for detecting the data based on taking various properties from both magnetic field and proton data, running calculations and generating spectrograms. When a contoured region in all three spectrograms falls within their respective thresholds, it considers it to be an SFR event. I tried running MP on just the magnetic field data for now to see what happens. I am confused by the results, particularly the fact that the big spike in the magnetic field magnitude for the big interplanetary coronal mass ejection (ICME) event is basically the top motif at most window sizes. I would also appreciate any advice on how to use the multi dimensional data for anomaly detection to e.g. find candidates for SFR events without generating spectrograms, or to otherwise use MP to draw insights from the multi dimensional data. Here is the WIP Jupyter notebook attached. The code, especially at the end, is thrown together just for experimentation, and the comments are very minimal |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
Just to be clear about terminology: The red event contains a low distance at the start and a high distance at the end. Furthermore vanilla STOMP ignores the magnitude of the input data, because each window gets normalized to 0-mean and unit-std. You can transform the z-normalized euclidean distance to pearson-correlation. On another topic: Wavelet-Transform: A necesary criteria to find a Motif with STOMP is that the data contains another event with roughly the same length/frequency. Maybe you can replace STOMP with VALMOD to deal with the varying length. Maybe it is helpful to perform lowpass and highpass filtering on the dataset if you are looking for an event with a specific frequency. Maybe you can use the periodicity/auto-correlation as an Annotation-Vector for MatrixProfile, thereby penalizing time-spans with little periodicity/wavelet-similarity. |
Beta Was this translation helpful? Give feedback.
Just to be clear about terminology:
Top Motif -> Lowest Distance to Nearest Neighbor -> Smallest Value in MatrixProfile
Top Outlier -> Highest Distance to Nearest Neighbor -> Largest Value in MatrixProfile
The red event contains a low distance at the start and a high distance at the end.
Maybe the window-size is too long.
Furthermore vanilla STOMP ignores the magnitude of the input data, because each window gets normalized to 0-mean and unit-std.
Therefore the physical magnitude of the data is lost. Only the shape matters for motif-detection.
You can transform the z-normalized euclidean distance to pearson-correlation.
Pearson-R is easier for interpretation.
On another topic:
If the probl…