[Q&A] URML App Support for local "under 2GB memory" LLM

[slowrunner]

What are you trying to do?

I have successfully run qwen2.5:0.5b on my Raspberry Pi 4 GoPiGo3 robot with both type-to and a python API prompt interface using the ollama server. Additionally, I managed to use the Ollama MCP with the qwen2.5:0.5b LLM to add two basic "tools" (get_datetime_server and get_vbatt_server).

The Ollama context limit defaults to 4096, but is configurable. I don't know what a "urml tool" would look like and what the size of the urml validated prompts are to know if the qwen2.5:0.5b will handle the initial query prompt (with NLU->URML info), or if it will handle the urml tool output prompt.txt that will need to be fed back to qwen2.5:0.5b to actuate the robot tools.

Any chance the URML stack will support some "under 2GB memory usage" edge LLM?

"tiny LLM" -> prompt.en.txt-> urml tool -> prompt.txt -> "tiny LLM" -> robot tools

Profile

home

Exact command and inputs

Example command:  "When your battery drops to 10 volts, do a little dance once a minute for five minutes, then shutdown"

What happened

N/A

Validator / CLI version

0.2.0

[slowrunner]

URML app already supports other LLMs and it appears to work:

(.venv) ubuntu@u26llpi5:~/U26LLPi5/systests/urml/my-first-robot$ urml translate "Bring me the red mug from the kitchen."     --manifest manifest.yaml     --envelope envelope.yaml     --profile home     --provider openai --model "qwen2.5:0.5b"
urml: translation failed: validator rejected the LLM's emission in all 4 attempt(s).

  ERROR [argument.missing_required] behavior
    field: behavior
    Field required

The processing took just under 4 minutes on my Raspberry Pi 5

• 1m load of 3.7 which is 93% ,
• 100% CPU usage,
• approx 800MB memory
• Temp reached 75C (Pi Cooler - no throttling)

The error is consistent with the issue #508

but there is a known issue with qwen2.5:0.5b of "forgetting and halucinating" which was evident the next time I ran the same prompt:

(.venv) ubuntu@u26llpi5:~/U26LLPi5/systests/urml/my-first-robot$ urml translate "Bring me the red mug from the kitchen."     --manifest manifest.yaml     --envelope envelope.yaml     --profile home     --provider openai --model "qwen2.5:0.5b"
urml: translation failed: validator rejected the LLM's emission in all 4 attempt(s).

  ERROR [argument.missing_required] profile
    field: profile
    Field required

  ERROR [argument.missing_required] behavior
    field: behavior
    Field required

  ERROR [argument.unknown_field] code
    field: code
    Extra inputs are not permitted

  ERROR [argument.unknown_field] primitive
    field: primitive
    Extra inputs are not permitted

  ERROR [argument.unknown_field] path
    field: path
    Extra inputs are not permitted

  ERROR [argument.unknown_field] field
    field: field
    Extra inputs are not permitted

I have run larger models on my Mac and remotely accessed them from my robot, which is probably the better approach.

The process and configuration on the URML robot would be the same, just changing the OPENAI_BASE_URL environment variable.