how to update egress attribute of stumpy ?

[sidali126]

Hi,

I'm using stumpy in real time (Event Driven Architecture) for anomlay detection.

I want to change the egressvalue from False to True after some max history size.

At init:
stream = stumpi(value_list, egress=False, m=subsequence_length)

After max length I create a new object with egress=True :

stream = stumpi.stumpi(stream._T, egress=True, m=subsequence_length)

Is there a simpler and more efficient way to do this ?

By the way I tried updating the private attribute _egress of stream object but I got some errors.

Thanks for your help.
Kind regards,
Ali

[seanlaw]

I want to change the egressvalue from False to True after some max history size.

As you've discovered, I don't think that is possible. Once the stumpi object has been instantiated with egress=False (or True) then there is no way to convert it to its egress counterpart.

Is there a simpler and more efficient way to do this ?

Off the top of my head, no, there is no other way to do this. Perhaps, if I understand you correctly, when you create new object with egress=True, you incur the cost of re-computing the matrix profile for this entire time series again, which may be undesirable. There is currently an open issue to add a feature that would allow the user to supply their pre-computed matrix profile (both global and left matrix profiles), which should help with this.

One other possibility might be to start with egress=True and to populate your stream with a time series that is filled with np.nan:

T = np.full(max_history_size, np.nan)
stream = stumpi.stumpi(T, egress=True, m=subsequence_length)
stream.update(t)

This may or may not suit your needs.

[NimaSarajpoor]

@seanlaw
I have an idea that might help with this. I implemented it locally. We can think more about this to make sure we are not missing anything:

Potential solution
We can add .switch() method to the class stumpi.

def switch(self):
        """
        switch from egress==False to egress==True, or vice versa.
        """
        self._egress = not self._egress
        if self._egress:
            self._QT_new = np.empty(self._QT.shape[0], dtype=np.float64)
            self._n_appended = 0

I think the if-block is necessary as this is the main difference between egress==True and egress==False when we instantiate an object in stumpi (see the last few lines in the __init__ method). So, by providing this block in switch, we can make sure that we can cover this part.

So, every time that we call switch, we use the latest time series self._T. we can clarify this matter in the docstring of switch method

We can think about ways to test this. What do you think?

---

Example:

seed = 0
np.random.seed(seed)

n = 1000
m = 50

T = np.random.rand(n)
t_vals = np.random.rand(5)

stream = stumpi(T, m)  # egress==True (default)
print(stream.P_.shape[0]) 
>>> 951


stream.update(t_vals[0])
print(stream.P_.shape[0]) 
>>> 951 
    
stream.switch()  # now: egress==False 
stream.update(t_vals[1])
print(stream.P_.shape[0])   
>>> 952

stream.switch() # now: egress==True
stream.update(t_vals[2])
print(stream.P_.shape[0])  
>>> 952

[NimaSarajpoor]

@seanlaw
Now that I think more about your solution, i.e. " ... start with egress=True and to populate your stream with a time series that is filled with np.nan", I think your approach is cleaner and better in terms of interpretability.

[sidali126]

Init with nan values worked fine so far !
Thanks all