-
Notifications
You must be signed in to change notification settings - Fork 777
Closed as not planned
Closed as not planned
Copy link
Labels
Description
Describe the bug
The F1AdaptiveThreshold
is designed to select the optimal threshold for separating normal and anomalous classes based on the F1 score. Ideally, this optimal threshold should correspond to the "maximum" predicted value when all images are normal. However, the current implementation incorrectly uses the "minimum" predicted value instead. This error leads to an abnormally low threshold when the model is trained exclusively with normal images, resulting in an excessively high false alarm rate.
Dataset
N/A
Model
N/A
Steps to reproduce the behavior
labels = torch.tensor([0, 0, 0, 0])
preds = torch.tensor([1.0, 2.0, 3.0, 4.0])
adaptive_threshold = F1AdaptiveThreshold(default_value=0.5)
threshold = adaptive_threshold(preds, labels)
# now threshold is tensor(1.0) but it should be tensor(4.0) because all images are normal
OS information
OS information:
- OS: macOS Sequoia 15.1.1
- Python version: 3.10.14
- Anomalib version: 2.0.0dev
- PyTorch version: 2.5.1
- CUDA/cuDNN version: N/A
- GPU models and configuration: N/A
- Any other relevant information: N/A
Expected behavior
When all images are normal, the optimized value of F1AdaptiveThreshold
should be the "maximum" predicted value.
In other words, the F1AdaptiveThreshold
should pass the following test case:
labels = torch.tensor([0, 0, 0, 0])
preds = torch.tensor([1.0, 2.0, 3.0, 4.0])
adaptive_threshold = F1AdaptiveThreshold(default_value=0.5)
adaptive_threshold.update(preds, labels)
threshold_value = adaptive_threshold.compute()
assert threshold_value == 4.0
Screenshots
No response
Pip/GitHub
GitHub
What version/branch did you use?
Configuration YAML
N/A
Logs
N/A
Code of Conduct
- I agree to follow this project's Code of Conduct