How to stabilize h_estimate plz? Got very very large est_norm.

[CaptainCandy]

I'm using a dataset contains airplanes and cars (910 train+390 test) from ImageNet, and trained several fine-tuned models including vgg16_bn, mobilenetv2, alexnet, resnet50. I tried to use this code to diagnose models by influence functions, but I found that it's really hard to make h_estimate stabilize or converge, which makes the influences output of a same test point different every time and makes top N influential train points varies every time.
The picture following is the results of h_estimate when I tried to calculate influence on a test point with resnet50. The original paper suggested that we make r*depth = len(trainset), and I have tried different combinations of r and depth. When the depth is big, like 500 (less than len(trainset)), the est_norm goes larger and larger and finally ends up with inf. So I keep depth small like 50 or 100, and make r large like 10 or 20. However, the est_norm still doesn't seem to converge. I would like to ask that is there any tricks to stabilize h_estimate? I'm really stuck here...

Thank you very much!

[thongnt99]

Hi, I encountered this problem too. Have you fixed this problem yet?

[CaptainCandy]

Hi, I encountered this problem too. Have you fixed this problem yet?

Well, there are two things needed attention. First, depth is more important than r, try a big number of depth like 100, 300, 500, you can monitor the change of est_norm to choose the right number for you own model. Second, scale is the reason that est_norm went inf. Try a big scale like 10000, 50000 will sovle the problem. est_norm will stablize if you choose the right config. Following figure is the est_norm in my experiment.

[thongnt99]

@CaptainCandy Thanks. That helps me. Previously, I already tried with larger scale, however it still returned Inf after around 100 iterations. Now with your comment, I've tried to look back at the code and found out that in calc_img_wise method (which is called from my main code), the author forgot to pass damp and scale arguments from the config file when calculating s_test . This made any changes of damp and scale from the config file invalid. I will make a request to fix this.