Discuss sktime Collaboration

[seanlaw]

There was a recent comment/discussion in the sktime Gitter channel

We do have however a matrix profile transformer here: https://github.com/alan-turing-institute/sktime/blob/v0.4.1/sktime/transformers/series_as_features/matrix_profile.py#L193-L240
There is also the stumpy package: https://github.com/TDAmeritrade/stumpy

We'd be very happy about interfacing the package in sktime if you end up using both libraries, that would be great contribution!

It would be useful to the time series community to openly discuss whether there is potential for collaboration here as there could be some nice synergies.

[seanlaw]

Hi, @mloning! I hope you and your team are doing well. I really appreciate all of the work that you and your group have done with sktime and, based on your recent comments in the sktime Gitter channel (see comment above), I was wondering if there is a potential opportunity to collaborate here. There is absolutely no obligation but I thought that it would at least be worth a discussion. I am curious to hear where you think STUMPY could fit into sktime. What do you think?

[mloning]

Hi @seanlaw, thanks for reaching out! It would be great to collaborate! One of our main motivations is to make the ecosystem more unified and interoperable.

At first place, the matrix profile seems to be a series-to-series transformation and hence would be a transformer. But we're happy to have a discussion to see how to best combine efforts and integrate our projects.

[seanlaw]

Yes, if I understand your terminology correctly, a "series-to-series" transformer sounds about right. Essentially, for matrix profiles, you'd take a 1-D NumPy array, T, of length n=T.shape[0] and then you choose a window size, m and the output would be a 4 column array of shape (n-m+1, 4) (note that this series is shorter than the original time series by m+1):

• Column 1: The matrix profile values (these are floating point distances)
• Column 2: The matrix profile indices (int)
• Column 3: The left matrix profile indices (int)
• Column 4: The right matrix profile indices (int)

From an ML modeling perspective, I think all 4 columns of data are potentially useful as features.

I have joined the Slack channel but, in case it matters, for official STUMPY business, I tend to prefer having discussions in the Github issues because:

1. It is more transparent
2. The discussion is searchable (via Google)

[mloning]

Yes, so STUMPY would be a series-to-series transformer that takes a univariate series as input and returns a multivariate series as output. We'd love to interface STUMPY in sktime so that people can use it with the other functionality. We're currently working on refactoring the transformer API and improving the support for multivariate series. So perhaps it's best to wait until we've made some progress with that.

Do you provide any additional tools for composition, pipelining and so on that I can take a look at?

I completely agree about having the discussion openly!

cc: @fkiraly @TonyBagnall

[seanlaw]

Yes, so STUMPY would be a series-to-series transformer that takes a univariate series as input and returns a multivariate series as output. We'd love to interface STUMPY in sktime so that people can use it with the other functionality. We're currently working on refactoring the transformer API and improving the support for multivariate series. So perhaps it's best to wait until we've made some progress with.

Sounds good. Just so you're aware, we've purposely designed STUMPY to be (mostly) flat and, more importantly, stateless. So, the majority of our library is just collection of independent functions (or a wrapper function calling another function). Classes are only present in some rare cases where keeping track of the state is important but your case should not be affected by this. So, for a single time series, one would do something like:

import stumpy
import numpy as np

time_series = np.random.rand(10_000)
window_size = 50

matrix_profile = stumpy.stump(time_series, window_size)

In case it matters, we also have support for multi-dimensional time series (see mstump) but be forewarned that:

1. this is not the same as analyzing the matrix profile for each dimension and then stacking the results together and
2. these usually take a long time to run even with the fastest code

What would you say is average length of the time series that your users will be transforming?

Also, note that we have GPU support for the 1-D case as well via:

matrix_profile = stumpy.gpu_stump(time_series, window_size)

Do you provide any additional tools for composition, pipelining and so on that I can take a look at?

No, since a unique input produces a unique matrix profile output (with 4 columns), we've purposely limited the scope of STUMPY to be solely focused on being fast, scalable, and efficient at computing the matrix profile and with minimal dependencies. It's important for STUMPY not to step too far outside of this scope as it can dramatically increase the things we'd need to support. It should essentially feel like calling a native NumPy function on your input array. Having said that, I'm sure that we can find an appropriate way to collaborate! Hopefully, our simple design makes it easy to interface with sktime.

[fkiraly]

@mloning, I believe we already had some custom implementation of matrix profile a while ago (implemented by Claudia Sanchez)?
https://www.sktime.org/en/latest/modules/auto_generated/sktime.transformers.series_as_features.matrix_profile.MatrixProfile.html#sktime.transformers.series_as_features.matrix_profile.MatrixProfile
https://github.com/alan-turing-institute/sktime/blob/master/sktime/transformers/series_as_features/matrix_profile.py

Not suggesting that it is better than stumpy, just pointing out that interface and scitype choices were already made and might facilitate the "interface" discussion?

[mloning]

Hi @fkiraly, yes I pointed that out in the Gitter chat! :-) The matrix profile in sktime is currently a series-as-features transformer, but it may be useful in other settings too. As discussed above, fundamentally it's a series-to-series transformer and doesn't require multiple series as input.

[TonyBagnall]

there are two forms of the matrix profile, single series transform, or multiple series join/distance measure so it can require multiple series as input

[mloning]

Yes, the multiple series version in STUMPY is mstump as far as I understood, so we could have both

• series-to-series transformer
• series-as-features transformer

[seanlaw]

If I may be honest, I would consider avoiding multi-dimensional cases as the published definition (and what is implemented faithfully in STUMPY) isn't what you think it might be and people will get really confused. Additionally, computing the matrix profile for multiple dimensions can be extremely costly, computationally speaking. I would recommend just sticking to the 1-D case.

Do you have any rough idea as to what the average time series length is that people are transforming? Or how long people are willing to wait?

[fkiraly]

Do you have any rough idea as to what the average time series length is that people are transforming? Or how long people are willing to wait?

That really depends - e.g., whether the use case is as a component in forecasting, or in time serires classification. Could be a length of 10, or 1000.

If I may be honest, I would consider avoiding multi-dimensional cases

Given the above, should we maybe start with the series-to-series transformer?

[seanlaw]

That really depends - e.g., whether the use case is as a component in forecasting, or in time serires classification. Could be a length of 10, or 1000.

For 1-D time series, the computation is no longer instantaneous (i.e., less than a few seconds) at around 50K data points (100K takes ~40s to process). Anything less than 10K data points should be fast!

Given the above, should we maybe start with the series-to-series transformer?

Yes, I think that would be wise and then I recommend letting the user submit a future request if they truly want to compute a multi-dimensional matrix profile. Again, this capability is available in STUMPY already but I am fairly certain that 99% of people who use it blindly will interpret the multi-dimensional matrix profile output incorrectly. I want to save us all time from having to support something due to user error/misunderstanding/ignorance if that makes sense?

[mloning]

Just a quick update: we're working on the transformer typing here (sktime/sktime#420), I'll ping you once it's done.

[seanlaw]

Just a quick update: we're working on the transformer typing here (sktime/sktime#420), I'll ping you once it's done.

Awesome! Thanks for the update

[mloning]

Hi @seanlaw, we merged the update and I created an issue here: sktime/sktime#450 to interface stumpy

[seanlaw]

Great! Thanks for letting me know and feel free to ping me with any questions

[mloning]

Hi @seanlaw - are you aware of a good default value (or procedure to infer a good value from the data) for the window length (m) in stumpy.stump? We'd like to set as many sensible defaults as possible, not sure if there's any literature or experiments on this?

cc @utsavcoding who's working on the implementation

[mloning]

So what are the things to keep in mind when choosing a window length? How do people usually go about it? Based on assumptions about the length of interesting temporal patterns?

[seanlaw]

I am borrowing some of my response from somebody else’s but have added my thoughts as well:

Similar to, say, a moving average, the choice of the window size is a judgment about the time scale on which data variations change from being merely "noise" to more meaningful indications of some underlying activity (i.e., conserved behavior or the presence of a motif). Often, "ball-park" estimates based on common sense will be adequate and choosing a window size that is larger than the motif size does not seem to matter (assuming that you aren’t off by an order of magnitude). In some cases, it will be worthwhile to experiment with different window sizes, as doing so may give you more insight into the existing patterns/motifs (or maybe there are none). Note that it is possible for a dataset to contain unrelated patterns that are of significantly different lengths and so one strategy is to choose a window size, compute the matrix profile, double the window size (or choose some multiple), and repeat again. This is essentially a “poor man’s” pan matrix profile but, perhaps for ML, these could all be feature columns (again, similar to moving average with different window sizes). The key goal is to choose a window size that you believe will be just big enough to capture your pattern of interest. Choose something too small and you’ll miss it. Choose something way too big and you’ve added too much noise to the window and you drown out your pattern. It doesn’t have to be perfect but one should err on being slightly over where possible. The existence of clear global minima in the matrix profile is a good sign.

[seanlaw]

@utsavcoding I was wondering how things were going? Is there anything I can do to help?

[mloning]

Yes we created a univariate matrix profile transformer for now: https://github.com/alan-turing-institute/sktime/blob/master/sktime/transformations/series/matrix_profile.py

[seanlaw]

Amazing!! Thank for letting me know. This is very exciting!