Skip to content

Graph updates #241

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 10 commits into from
Closed

Graph updates #241

wants to merge 10 commits into from

Conversation

topepo
Copy link
Member

@topepo topepo commented Feb 14, 2022

All of these plots have figure captions.

@topepo
Copy link
Member Author

topepo commented Feb 14, 2022

The hash for the SA results appears to have changed.

Locally, I get the same iteration results at the website:

#> Optimizing roc_auc
#> Initial best: 0.86627 
#>  1 ♥ new best           roc_auc=0.87293   (+/-0.007638) 
#>  2 ♥ new best           roc_auc=0.89656    (+/-0.006779) 
#>  3 ─ discard suboptimal roc_auc=0.88247    (+/-0.00731) 
#>  4 ─ discard suboptimal roc_auc=0.8812  (+/-0.006122) 
#>  5 ◯ accept suboptimal  roc_auc=0.88226    (+/-0.007343)

website:

#> Optimizing roc_auc
#> Initial best: 0.86627
#>  1 ♥ new best           roc_auc=0.87293  (+/-0.007638)
#>  2 ♥ new best           roc_auc=0.89656  (+/-0.006779)
#>  3 ─ discard suboptimal roc_auc=0.88247  (+/-0.00731)
#>  4 ─ discard suboptimal roc_auc=0.8812   (+/-0.006122)
#>  5 ◯ accept suboptimal  roc_auc=0.88226  (+/-0.007343)

Perhaps something else in the object has changed. @juliasilge Can you send your results?

@juliasilge
Copy link
Member

I do see the error if I run locally:

> verify_consistent_bo(svm_bo)
Error in `verify_consistent_bo()`:
! These Bayesian optimization results don't match the previous values.
Run `rlang::last_error()` to see where the error occurred.

I have the same results for the first five iterations:

Optimizing roc_auc
Initial best: 0.86627
 1 ♥ new best           roc_auc=0.87293	(+/-0.007638)
 2 ♥ new best           roc_auc=0.89907	(+/-0.007041)
 3 ─ discard suboptimal roc_auc=0.88247	(+/-0.00731)
 4 ─ discard suboptimal roc_auc=0.8812	(+/-0.006122)
 5 ◯ accept suboptimal  roc_auc=0.88226	(+/-0.007343)

But the last ten are different:

40 ◯ accept suboptimal  roc_auc=0.89	(+/-0.005141)
41 ◯ accept suboptimal  roc_auc=0.8877	(+/-0.007133)
42 + better suboptimal  roc_auc=0.89898	(+/-0.00658)
43 ◯ accept suboptimal  roc_auc=0.89895	(+/-0.006654)
44 ─ discard suboptimal roc_auc=0.87502	(+/-0.007454)
45 ─ discard suboptimal roc_auc=0.88489	(+/-0.007197)
46 ◯ accept suboptimal  roc_auc=0.88445	(+/-0.007064)
47 x restart from best  roc_auc=0.87697	(+/-0.007277)
48 ─ discard suboptimal roc_auc=0.84918	(+/-0.008598)
49 ─ discard suboptimal roc_auc=0.88926	(+/-0.005202)
50 ─ discard suboptimal roc_auc=0.87209	(+/-0.006013)

topepo added a commit that referenced this pull request Feb 16, 2022
@topepo
Copy link
Member Author

topepo commented Feb 16, 2022

Changes moved to #231 and 9a04f7e

@topepo topepo closed this Feb 16, 2022
@topepo topepo deleted the graph-updates branch February 16, 2022 22:00
@github-actions
Copy link

github-actions bot commented Mar 3, 2022

This pull request has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.

@github-actions github-actions bot locked and limited conversation to collaborators Mar 3, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants