Prepare containers to run with ai-lab-recipes

[rhatdan]

Add two new scripts llama-server.sh and whisper-server.sh which can handle environment variables from the ai-lab-recipes.

Summary by Sourcery

New Features:

• Added llama-server.sh and whisper-server.sh to support running the containers with ai-lab-recipes.

[sourcery-ai]

Reviewer's Guide by Sourcery

This pull request adds scripts to run llama and whisper servers, configurable via environment variables, and modifies the containerfiles to copy the scripts to /usr/bin and set the MODEL_PATH environment variable. This prepares the containers to be used with ai-lab-recipes.

No diagrams generated as the changes look simple and do not need a visual representation.

File-Level Changes

Change	Details	Files
Added scripts to run llama and whisper servers, configurable via environment variables.	Created llama-server.sh to run the llama server with configurable host, port, GPU layers, and model path via environment variables. Created whisper-run.sh to run the whisper server with configurable host, port, and model path via environment variables.	container-images/scripts/llama-server.sh container-images/scripts/whisper-run.sh
Modified containerfiles to copy scripts to /usr/bin and set the MODEL_PATH environment variable.	Updated ramalama/Containerfile to copy scripts to /usr/bin instead of /scripts. Updated ramalama/Containerfile to set the MODEL_PATH environment variable. Updated vulkan/Containerfile to copy scripts to /usr/bin instead of /scripts. Updated vulkan/Containerfile to set the MODEL_PATH environment variable. Updated cuda/Containerfile to copy scripts to /usr/bin instead of /scripts. Updated cuda/Containerfile to set the MODEL_PATH environment variable.	container-images/ramalama/Containerfile container-images/vulkan/Containerfile container-images/cuda/Containerfile

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!
• Generate a plan of action for an issue: Comment @sourcery-ai plan on
  an issue to generate a plan of action for it.

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[rhatdan]

@benoitf @bmahabirbu @ericcurtin PTAL

[sourcery-ai]

Hey @rhatdan - I've reviewed your changes - here's some feedback:

Overall Comments:

• Consider using COPY --from=builder to copy scripts from a build stage instead of the host.
• It looks like the RUN command in vulkan/Containerfile should execute vulkan/build_llama_and_whisper.sh.

Here's what I looked at during the review

• 🟡 General issues: 1 issue found
• 🟢 Security: all looks good
• 🟢 Testing: all looks good
• 🟢 Complexity: all looks good
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[sourcery-ai]

suggestion: Quote the variable in the if condition.

To avoid potential word-splitting or globbing issues when MODEL_PATH is empty or contains spaces, consider using quotes, e.g., if [ -n "${MODEL_PATH}" ]; then.

Suggested change

	if [ ${MODEL_PATH} ]; then
	if [ -n "${MODEL_PATH}" ]; then

[benoitf]

these scripts look ok but my question is:
who will call these scripts ?

because it does not seems that there is an entrypoint calling these scripts automatically

[rhatdan]

I considered doing the entrypoint, but I don't want to have separate scripts for lama-server and whisper-server, so the idea would be to change the quadlets used to launch the model server.

[benoitf]

I would expect to have a default mode as I'm using ramalama

$ podman run --rm -it quay.io/ramalama/ramalama
[root@f6ae4f8862d6 /]#

today it's defaulting to bash

so at elast I would expect the list of scripts I could call ?

but for example if there is a MODEL_CHAT_FORMAT it could just run llama-server script
if there is nothing it then just run /bin/bash as the parent image

or if I provide env variable RAMALA_something="serve" it's running the script to serve a model

[rhatdan]

Lets merge this and we can continue the conversation, since this is not a breaking change for RamaLama.