[DRAFT] [Megathread] Removing the sklearn extensions from the openml-python code base

[LennartPurucker]

We are not planning to support the sklearn extensions in the future.

Currently, the code related to the sklearn extensions within openml-python provides a straightforward way to upload models and results from running sklearn models on OpenML.

Note:

• Even without these extensions, users cannot upload such data to OpenML.
• The sklearn extensions are in the code base of openml-python, unlike the code for other extensions.

Justification for Removing the Extensions from openml-python

As a result of a recent survey by us (to be published) and our interactions with OpenML and its users, we have obtained a strong signal that this feature is not widely used, if at all, by the broader community. Yet, at the same time, it takes up an enormous amount of overhead during maintenance of OpenML with many lines of code, many tests, and multiple reasons for not being able to upgrade dependencies stemming only from the sklearn extension. Moreover, the sklearn extension has not been maintained or upgraded for a long time.

Previously, a part of the mission statement of OpenML was to enable the reproduction of, for example, old runs via this extension. We do not believe this to be an essential mission statement of the openml-python code. More importantly, the extensions are holding us back and are requiring too many of our already limited resources.

The Way Forward

openml-python will remove all code related to the sklearn extension in a future version

• One will still be able to reproduce the old runs (to some extent) and use the sklearn extension with an older version of openml-python
• If there are users of the sklearn extension, or we see renewed need for this extension, a new project can be created -- similar to the other extensions. However, the current openml-python team does not commit to supporting this extension.

These issues will serve as a megathread to reference all code removed related to the sklearn extension and any other content that may enable future readers to recreate the sklearn extension in a separate repository.

[LennartPurucker]

A short survey on code we could change/remove as a result:

• examples/20_basic/introduction_tutorial.py
• examples/20_basic/simple_flows_and_runs_tutorial.py (likely replace with code from examples/30_extended/custom_flow_.py)
• examples/30_extended/fetch_runtimes_tutorial.py
• examples/30_extended/flow_id_tutorial.py
• examples/30_extended/flows_and_runs_tutorial.py
• examples/30_extended/run_setup_tutorial.py
• examples/30_extended/study_tutorial.py
• examples/30_extended/task_manual_iteration_tutorial.py (make this the default way to talk about using tasks, etc)
• examples/40paper (maybe we can remove all of these at the same time?) -> TBD
• openml/extensions/sklearn
• LICENSE
• pyproject.toml (dependencies)
• way too many tests (not only for the extension)
• CONTRIBUTING.md

[LennartPurucker]

Rel.: #1317

[PGijsbers]

Tests on openml-sklearn are also green as of now but it does require installing openml-python from GitHub since otherwise the sklearn extension is registered twice which leads to errors.