Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

c should compare resampling hashes when combining benchmark results or resample results #1252

Open
sebffischer opened this issue Jan 31, 2025 · 0 comments

Comments

@sebffischer
Copy link
Member

sebffischer commented Jan 31, 2025

What's wrong with the code below? Well, the resampling is not instantiated.
I think we should throw a warning when a user does this. This can be achieved by comparing the resampling hash (for resamplings instantiated on the same task).

library(mlr3verse)
#> Loading required package: mlr3

l1 = lrn("classif.ranger")
l2 = lrn("classif.rpart")

task = tsk("iris")

resampling = rsmp("holdout")
rr1 = resample(task, l1, resampling)
#> INFO  [06:59:59.544] [mlr3] Applying learner 'classif.ranger' on task 'iris' (iter 1/1)
rr2 = resample(task, l2, resampling)
#> INFO  [07:00:00.024] [mlr3] Applying learner 'classif.rpart' on task 'iris' (iter 1/1)

bmr = c(rr1, rr2)

Created on 2025-01-31 with reprex v2.1.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant