Experiment with some clustering algorithms on top of the similarity metric

[marco-c]

No description provided.

[mansimarkaur]

I want to work on this. Could you please provide a description explaining this issue a little. Thanks!

[marco-c]

We have a way to evaluate the distance between two crash traces (WMD, but in the future we might experiment with other distance metrics #9), we can use this distance to cluster the stack traces in groups.

We should test with some clustering algorithms (http://scikit-learn.org/stable/modules/clustering.html) and see how they perform.

If the implemented algorithm turns out to be too slow (it's possible, as WMD is really slow), we can try two things:

• see if alternative distance metrics (Experiment with alternative distance metrics to WMD #9) are fast enough;
• create a clustering algorithm customized for our task. E.g. instead of clustering on the complete list of stack traces, we could implement a two-level clustering, where the first level is generated by the algorithm currently used on Socorro [signature] and the second level is used to split stack traces from a given signature in multiple groups. Or, where the first level is generated by a simper distance metric and the second level by WMD.

But let's not worry about this slowness problem for now. Let's try with the WMD distance and a well-known algorithm first.

[aditya-iitd]

With WMD, we can only use those clustering algorithms in which metric used is distance between points or others (eg-: Euclidean distance ) can also be used?

[marco-c]

With WMD, we can only use those clustering algorithms in which metric used is distance between points or others (eg-: Euclidean distance ) can also be used?

Can you rephrase this question? Euclidean distance is a distance between points too.

[mansimarkaur]

How exactly should we compare the various algorthims?
One metric would be speed which can be covered by speed benchmarks.
Another one would be accuracy, how do we go about testing this?

[marco-c]

Another one would be accuracy, how do we go about testing this?

A possible approach is #39.