-
Notifications
You must be signed in to change notification settings - Fork 8
Labels
bugSomething isn't workingSomething isn't workingenhancementNew feature or requestNew feature or request
Description
Problem:
-
When a seed is provided, the package resets the random number generator (RNG) states multiple times within a single campaign:
- The optimizer, model, designer, and DOE components each reset RNG states independently.
- During a normal workflow (without explicitly defining the optimizer/designer), random seed reset happens when
- the optimizer is initialized (torch and numpy)
- the designer is initialized (torch only)
- if DOE is used (numpy only)
- every iteration when each model is instantiated (torch and numpy)
-
When no seed is provided, there is no RNG control at all, resulting in no reproducibility.
Impact:
Repeated RNG resets cause correlated and repeated noise that biases results. In the model fitting stage, the model and scipy/Adam optimizer will always start from the same initial conditions. Additionally, global seed control is fragile, as unexpected random draws between steps break reproducibility unless perfectly replicated.
Proposed Fix:
- Centralize RNG state management to avoid multiple independent resets.
- Ensure the simulator noise RNG is initialized once per campaign and not reset every iteration.
- Provide consistent RNG control whether or not a seed is specified, enabling reproducibility.
- Refactor the code to use a single RNG state or RNG generator instance passed explicitly to components that require randomness.
- Add a package level toggle to switch back to current behavior for backward compatibility.
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't workingenhancementNew feature or requestNew feature or request