Providing Agent With ML model skill in the context

[cainemerrick98]

I think it would be a cool feature if, as an analyst configuring an agent, I could provide a pickled ML model as part of the agent context. The analyst would train and evaluate the model outside of the NAO environment and upload it as an artifact.

In rules.md, the analyst could instruct the agent that it has access to an ML model skill (for example, predicting late payments). The rules would reference the file which describes the expected inputs and how and when the model should be used.

At runtime, the agent would run a query to retrieve the relevant rows, call the model tool using the query result (or query_id), and receive a table containing predictions that can then be reported back to the user.

I think for business users this would be really impactful as from their perspective it would feel like their Nao agent was making predictions on fly.

For example, I imagine the business user asking: "Which payments are going to be late this month?" and the UI responds with "Predicting late payments for payments due this month..." And finally, the agent responds with which payments are expected to be late and why (if using XGBoost the agent could leverage the shap values to answer why it thinks it will be late)

What do you think?

[Bl3f]

Hello @cainemerrick98, nice to meet you and thank you for opening this.

So actually this is something we almost have today, we lack just the last mile which is giving a library of Python code that can be executed by the agent (outside of what can be included in the skill). We had this idea from the beginning and I think we will work on this in April.

Because here you give an example of a ML model be actually it could be whatever Python (or even other languages) and the code could be either imported in a Python sandbox or via a CLI script that the script is exposing and described in the skill.

TL;DR — today we support skills and Python sandboxes so you can almost achieve it, we just lack the arbitrary Python exec from a user-defined library.

Great idea!

[cainemerrick98]

Hey, sounds cool I'll keep lookout for when this feature is released.