Skip to content

fix: Raise an error when CrossValidationReport fails before at least one estimator is fitted #1574

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

fix: Raise an error when CrossValidationReport fails before at least one estimator is fitted #1574

wants to merge 4 commits into from

Conversation

YoussefMAllam
Copy link

Fixed #1561
I added an if condition at the end of the "crossvaldiationreport" funcrion to check if there are 0 estimators and raise an error.
I also added a test for it in the file "testcrossvalidation.py" where i created a broken estimator to test the case where no estimators are fitted

@YoussefMAllam
Fixed Issue fix: If cross-validation in CrossValidationReport fails b…
879bd51 
…efore at least one estimator is fitted, raise an error probabl-ai#1561
@thomass-dev thomass-dev changed the title Fixed Issue fix: If cross-validation in CrossValidationReport fails before at least one estimator is fitted, raise an error #1561 fix: Raise an error when CrossValidationReport fails before at least one estimator is fitted Apr 23, 2025
@github-actions GitHub Actions
Copy link
Contributor

Coverage

Coverage Report for backend
FileStmtsMissCoverMissing
venv/lib/python3.12/site-packages/skore
   __init__.py220100% 
   _config.py280100% 
   exceptions.py440%4–23
venv/lib/python3.12/site-packages/skore/persistence
   __init__.py00100% 
venv/lib/python3.12/site-packages/skore/persistence/item
   __init__.py55198%97
   altair_chart_item.py19191%14
   item.py22195%86
   matplotlib_figure_item.py36195%19
   media_item.py220100% 
   numpy_array_item.py27194%16
   pandas_dataframe_item.py29194%14
   pandas_series_item.py29194%14
   pickle_item.py220100% 
   pillow_image_item.py25193%15
   plotly_figure_item.py20192%14
   polars_dataframe_item.py27194%14
   polars_series_item.py22192%14
   primitive_item.py23291%13–15
   sklearn_base_estimator_item.py29194%15
venv/lib/python3.12/site-packages/skore/persistence/repository
   __init__.py20100% 
   item_repository.py59591%15–16, 202–203, 226
venv/lib/python3.12/site-packages/skore/persistence/storage
   __init__.py40100% 
   abstract_storage.py220100% 
   disk_cache_storage.py33195%44
   in_memory_storage.py200100% 
venv/lib/python3.12/site-packages/skore/project
   __init__.py20100% 
   project.py84298%282, 394
venv/lib/python3.12/site-packages/skore/sklearn
   __init__.py60100% 
   _base.py1711492%45, 58, 126, 129, 182–191, 203–>209, 224, 227–228
   find_ml_task.py61099%136–>145
   types.py130100% 
venv/lib/python3.12/site-packages/skore/sklearn/_comparison
   __init__.py50100% 
   metrics_accessor.py165297%163, 164–>166, 1278
   report.py67197%17, 249–>252
venv/lib/python3.12/site-packages/skore/sklearn/_cross_validation
   __init__.py50100% 
   metrics_accessor.py190099%153–>155, 155–>157
   report.py112198%23
venv/lib/python3.12/site-packages/skore/sklearn/_estimator
   __init__.py70100% 
   feature_importance_accessor.py133099%483–>489, 569–>578
   metrics_accessor.py3441096%174–183, 211–>220, 219, 249, 260–>262, 290, 317–321, 336, 371, 372–>374
   report.py145198%24, 253–>255
venv/lib/python3.12/site-packages/skore/sklearn/_plot
   __init__.py20100% 
   base.py60100% 
   style.py280100% 
   utils.py122595%51, 75–77, 81
venv/lib/python3.12/site-packages/skore/sklearn/_plot/metrics
   __init__.py40100% 
   precision_recall_curve.py173199%660
   prediction_error.py1640100% 
   roc_curve.py176199%649
venv/lib/python3.12/site-packages/skore/sklearn/train_test_split
   __init__.py00100% 
   train_test_split.py51196%16, 154–>158
venv/lib/python3.12/site-packages/skore/sklearn/train_test_split/warning
   __init__.py80100% 
   high_class_imbalance_too_few_examples_warning.py17190%79
   high_class_imbalance_warning.py180100% 
   random_state_unset_warning.py12188%15
   shuffle_true_warning.py10183%46
   stratify_is_set_warning.py12188%15
   time_based_column_warning.py23286%17, 73
   train_test_split_warning.py40100% 
venv/lib/python3.12/site-packages/skore/utils
   __init__.py60100% 
   _accessor.py46197%102
   _environment.py27097%30–>35
   _fixes.py80100% 
   _index.py50100% 
   _logger.py22485%15–19
   _measure_time.py100100% 
   _parallel.py38388%23–33, 124
   _patch.py13553%21–37
   _progress_bar.py360100% 
   _show_versions.py330100% 
TOTAL31858296% 

Tests Skipped Failures Errors Time
815 8 💤 0 ❌ 0 🔥 54.060s ⏱️

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Apr 23, 2025 •
edited
Loading

Documentation preview @ 2142146

@glemaitre glemaitre self-requested a review April 23, 2025 16:21
@github-actions GitHub Actions
Copy link
Contributor

Coverage

Coverage Report for backend
FileStmtsMissCoverMissing
venv/lib/python3.12/site-packages/skore
   __init__.py220100% 
   _config.py280100% 
   exceptions.py440%4–23
venv/lib/python3.12/site-packages/skore/persistence
   __init__.py00100% 
venv/lib/python3.12/site-packages/skore/persistence/item
   __init__.py55198%97
   altair_chart_item.py19191%14
   item.py22195%86
   matplotlib_figure_item.py36195%19
   media_item.py220100% 
   numpy_array_item.py27194%16
   pandas_dataframe_item.py29194%14
   pandas_series_item.py29194%14
   pickle_item.py220100% 
   pillow_image_item.py25193%15
   plotly_figure_item.py20192%14
   polars_dataframe_item.py27194%14
   polars_series_item.py22192%14
   primitive_item.py23291%13–15
   sklearn_base_estimator_item.py29194%15
venv/lib/python3.12/site-packages/skore/persistence/repository
   __init__.py20100% 
   item_repository.py59591%15–16, 202–203, 226
venv/lib/python3.12/site-packages/skore/persistence/storage
   __init__.py40100% 
   abstract_storage.py220100% 
   disk_cache_storage.py33195%44
   in_memory_storage.py200100% 
venv/lib/python3.12/site-packages/skore/project
   __init__.py20100% 
   project.py83298%280, 392
venv/lib/python3.12/site-packages/skore/sklearn
   __init__.py60100% 
   _base.py1711492%45, 58, 126, 129, 182–191, 203–>209, 224, 227–228
   find_ml_task.py61099%136–>145
   types.py130100% 
venv/lib/python3.12/site-packages/skore/sklearn/_comparison
   __init__.py50100% 
   metrics_accessor.py165297%163, 164–>166, 1278
   report.py67197%17, 249–>252
venv/lib/python3.12/site-packages/skore/sklearn/_cross_validation
   __init__.py50100% 
   metrics_accessor.py190099%153–>155, 155–>157
   report.py112198%23
venv/lib/python3.12/site-packages/skore/sklearn/_estimator
   __init__.py70100% 
   feature_importance_accessor.py133099%483–>489, 569–>578
   metrics_accessor.py3441096%174–183, 211–>220, 219, 249, 260–>262, 290, 317–321, 336, 371, 372–>374
   report.py148198%24, 253–>255
venv/lib/python3.12/site-packages/skore/sklearn/_plot
   __init__.py20100% 
   base.py60100% 
   style.py280100% 
   utils.py122595%51, 75–77, 81
venv/lib/python3.12/site-packages/skore/sklearn/_plot/metrics
   __init__.py40100% 
   precision_recall_curve.py173199%660
   prediction_error.py1640100% 
   roc_curve.py176199%649
venv/lib/python3.12/site-packages/skore/sklearn/train_test_split
   __init__.py00100% 
   train_test_split.py51196%16, 154–>158
venv/lib/python3.12/site-packages/skore/sklearn/train_test_split/warning
   __init__.py80100% 
   high_class_imbalance_too_few_examples_warning.py17190%79
   high_class_imbalance_warning.py180100% 
   random_state_unset_warning.py12188%15
   shuffle_true_warning.py10183%46
   stratify_is_set_warning.py12188%15
   time_based_column_warning.py23286%17, 73
   train_test_split_warning.py40100% 
venv/lib/python3.12/site-packages/skore/utils
   __init__.py60100% 
   _accessor.py46197%102
   _environment.py27097%30–>35
   _fixes.py80100% 
   _index.py50100% 
   _logger.py22485%15–19
   _measure_time.py100100% 
   _parallel.py38388%23–33, 124
   _patch.py13553%21–37
   _progress_bar.py360100% 
   _show_versions.py330100% 
TOTAL31878296% 

Tests Skipped Failures Errors Time
815 8 💤 0 ❌ 0 🔥 50.807s ⏱️

thomass-dev
thomass-dev requested changes May 2, 2025
View reviewed changes
Copy link
Collaborator

@thomass-dev thomass-dev left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a nitpick, otherwise LGTM. Thanks.

Edit: can you please fix the merge-conflict?

skore/src/skore/sklearn/_cross_validation/report.py
Comment on lines +216 to +220
if len(estimator_reports) == 0:
raise RuntimeError(
"Cross-validation failed: no estimators were successfully fitted. "
"Please check your data, estimator, or cross-validation setup."
) from e
Copy link
Collaborator

@thomass-dev thomass-dev May 2, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: can you move this block before the construction of the message (between 201 and 203)? It should be more readable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@glemaitre glemaitre glemaitre left review comments

@thomass-dev thomass-dev thomass-dev requested changes

Requested changes must be addressed to merge this pull request.

Assignees

@YoussefMAllam YoussefMAllam

Labels
None yet
Projects
None yet
Milestone
No milestone
3 participants
@YoussefMAllam @thomass-dev @glemaitre