Skip to content

Align splitter payloads with schemas and fix split validation - #833

Open
Irozuku wants to merge 17 commits into
developfrom
feat/splitter-schema-forms
Open

Align splitter payloads with schemas and fix split validation#833
Irozuku wants to merge 17 commits into
developfrom
feat/splitter-schema-forms

Conversation

@Irozuku

@Irozuku Irozuku commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Summary

Splitter configuration forms in the model session wizard are now generated from the backend component SCHEMA, like models, converters and explainers already are. Previously the holdout section was hand coded JSX and the cross-validation section used the schema only to decide which fields to show.

Aligning the stored splits payload with the schema keys also fixes four bugs caused by the payload and the splitters disagreeing:

  • The seed box did nothing: the UI sent seed, splitters read random_state.
  • Manual and predefined holdout splits crashed training (unsupported operand type(s) for /: 'list' and 'list'), because index lists were sent under the keys the splitter reads as proportions.
  • StratifiedKFold and StratifiedGroupKFold crashed with shuffling off, since they passed random_state unconditionally and scikit-learn rejects that pair.
  • The Live Metrics validation tab was inverted: shown for cross-validation runs, which have no validation partition, hidden for holdout runs, which do.

Legacy payloads are translated at read time, so existing sessions keep working without a data migration.


Type of Change

  • Backend change
  • Frontend change
  • CI / Workflow change
  • Build / Packaging change
  • Bug fix
  • Documentation

Changes (by file)

Backend

  • splitters/splits_payload.py: new. Normalizes legacy payloads (missing splitter_name, seed instead of random_state, index lists under partition names) and collects schema placeholders.
  • splitters/fold_splitter.py: new sklearn_random_state() helper, forwarding a seed only when shuffling is on.
  • splitters/stratified_k_fold.py, stratified_group_k_fold.py: use it, the crash fix. k_fold.py, group_k_fold.py: use it instead of their inline equivalent.
  • splitters/holdout.py: proportions are plain floats instead of nullable, so they render as number inputs; validator requires them to sum to 1.
  • splitters/*_k_fold.py: n_splits capped at 20, n_repeats at 10, and the shuffle / random_state descriptions now note that the seed is ignored without shuffling (all five languages).
  • job/model_job.py: normalize the payload before building the splitter.
  • dataloaders/classes/dashai_dataset.py: prepare_for_model_session reads manual indexes from splitted_indexes and accepts random_state.
  • api/api_v1/endpoints/model_sessions.py: POST validates the payload against the splitter's SCHEMA, returning 422. No new route.

Frontend

  • utils/splitsPayload.js: new. Owns the payload contract (buildSplitsPayload, resolveSplitterName, hasPartition).
  • modelSession/SplitDatasetRows.jsx: hand coded parameter controls replaced by one FormSchema; the shell keeps only what a schema cannot express (strategy toggle, splitter picker, split type toggle, manual indexes, group_column, cross field checks).
  • modelSession/PrepareDatasetStep.jsx: builds splits via the new helper and no longer gates column validation on the splits being ready.
  • shared/FormSchemaRenderFields.jsx: adds excludeFields to the memo dependencies (the prop itself came from develop, see Notes).
  • models/CreateSessionSteps.jsx: picks metric sets via hasPartition.
  • models/SessionInfoContent.jsx: reads random_state with a seed fallback, plus a cross-validation summary.
  • models/LiveMetricsChart.jsx: one line fix for the inverted validation tab.

Tests

  • tests/back/splitters/: new package for the normalizer, holdout schema and index modes, fold caps, session prep, and the shuffle/seed guard.
  • tests/back/api/test_jobs.py, test_model_session_splits_validation.py: end to end training on new and legacy payloads, and 422 on invalid ones.

Testing

  • In the app: holdout manual and predefined modes, and Group K-Fold with nested CV (the configuration that hit the seed crash).

@Irozuku Irozuku added bug Something isn't working front Frontend work back Backend work labels Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

back Backend work bug Something isn't working front Frontend work

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant