Make model_id and model_family in PipelineContext optional
#511
Comments
|
This is necessary as we work towards more flexible Pipelines, where each LLMBlock of a pipeline can call different models instead of requiring all LLMBlocks in a pipeline to only talk to a single model. |
bbrowning
added a commit
to bbrowning/instructlab-sdg
that referenced
this issue
Jan 28, 2025
This makes model_family, model_id, and num_instructions_to_generate optional on PipelineContext instead of required. This should be a backwards-compatible change, and includes tests that verify passing the previously-required parameters into PipelineContext work as expected. This is a move towards supporting and expecting Pipelines that have LLMBlocks that may use different model families or different model ids. If a Block's yaml config specifies a model_family or model_id and none is set on the PipelineContext, then the Block's values will get used. However, if these are set on the PipelineContext, they're treated as overrides and will override the values for all Blocks in this Pipeline. This behavior emulates the previous behavior, but we should steer users away from ever setting model_id or model_family on a PipelineContext if the Pipeline contains multiple LLMBlock entries. Fixes instructlab#511 Signed-off-by: Ben Browning <[email protected]>
bbrowning
added a commit
to bbrowning/instructlab-sdg
that referenced
this issue
Jan 29, 2025
This makes model_family, model_id, and num_instructions_to_generate optional on PipelineContext instead of required. This should be a backwards-compatible change, and includes tests that verify passing the previously-required parameters into PipelineContext work as expected. This is a move towards supporting and expecting Pipelines that have LLMBlocks that may use different model families or different model ids. If a Block's yaml config specifies a model_family or model_id and none is set on the PipelineContext, then the Block's values will get used. However, if these are set on the PipelineContext, they're treated as overrides and will override the values for all Blocks in this Pipeline. This behavior emulates the previous behavior, but we should steer users away from ever setting model_id or model_family on a PipelineContext if the Pipeline contains multiple LLMBlock entries. Fixes instructlab#511 Signed-off-by: Ben Browning <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The configuration of an
LLMBlockin a pipeline yaml file should be able to define its own model family and model id to use and it can differ perLLMBlockwithin a singlePipeline. Thus, we should not require one to be set for the entirePipelineContextand instead these should be treated as optional parameters that override the values found within the yaml config.Try to do this in a backwards-compatible way, where existing code still works but we just make these optional and shift users to doing this in their Block configuration instead of their PipelineContext.
The text was updated successfully, but these errors were encountered: