use GitHub alerts and fix LLMS_AVAILABLE token - #1153
Conversation
|
|
||
| > **Note:** `LLMS_AVAILABLE` is a curated subset of models intended for local development and testing — it is **not** the full set of available models, and the Model Proxy token itself is not restricted to these models. To see all available models, use `kaggle benchmarks tasks models`. To run a task against any model (including those not in `LLMS_AVAILABLE`), use `kaggle benchmarks tasks run`, which executes on Kaggle's infrastructure with access to the full model catalog. | ||
| > [!WARNING] | ||
| > `LLMS_AVAILABLE` is a curated subset of models intended for local development and testing — it is **not** the full set of available models, and the Model Proxy token minted for local development **is** restricted to these models. To see all available models, use `kaggle benchmarks tasks models`. To run a task against any model (including those not in `LLMS_AVAILABLE`), use `kaggle benchmarks tasks run`, which executes on Kaggle's infrastructure with access to the full model catalog. |
There was a problem hiding this comment.
Just a side note: Actually users can use whatever models they choose by manually injecting them into LLMS_AVAILABLE. I don't think we have any backend restrictions for this so it's purely conventional.
There was a problem hiding this comment.
FYI, the MP token we mint does restrict the ones they can use. The local development MP token is different from the one minted during Kaggle KKB sessions
There was a problem hiding this comment.
To illustrate, this is the MP token creation that we do when users make one via the API: https://github.com/Kaggle/kaggleazure/blob/ci/Kaggle.Services.Shared/ModelProxy/Utils/TokenUtil.cs#L61
Note that the LocalModelProxyGroup is tied to this specific subset of models that aligns with what we set for them in LLMS_AVAILABLE: https://source.corp.google.com/piper///depot/google3/cloud/kaggle/modelproxy/backends/config/config.gcl;l=2424-2440
There was a problem hiding this comment.
Oh that's good to know. I guess my knowledge was outdated or never accurate.
3 participants
https://chat.kaggle.net/kaggle/pl/f8gs75d1qfd8tkx5cs68zdhezo