Skip to content

Add clustering analysis #13

@danbraunai

Description

@danbraunai

Migrated from: goodfire-ai/spd-gf#51
Original author: @leesharkey


SPD produces a set of subcomponents which need to be clustered together in order to identify parameter components. Some of the models in our evals suite will have parameter components that must be clustered together from subcomponents. We will want to evaluate the results of this clustering process in order to test our analysis pipeline end to end. If either SPD or clustering are broken, then we will end up with poor results. So we need to evaluate our clustering method.

It will probably suffice to use the clustering methods on our toy models where we have ground truth. We can usually identify if SPD has succeeded without evaluating the subsequent clustering. But the clustering should still work! So we should test the clustering separately from our SPD metrics.

Metadata

Metadata

Assignees

Labels

Priority: InUImportant & not Urgentclusteringanything to do with clusteringfeatureNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions