🖥️ Mac Mini AI Stack (GPU Accelerated)

This repository contains the deployment automation and orchestration files to run a high-performance local AI stack on macOS (specifically optimized for Apple Silicon Mac Mini with 16GB Unified Memory).

---

⚡ Architecture & Services

The stack is designed in a hybrid model:

1. Ollama (Native macOS Service): Runs directly on the host OS to gain full Apple Silicon Metal GPU acceleration, allowing models like qwen2.5-coder:7b to execute at peak speeds (~22-50 tokens/second).
2. LiteLLM, Postgres, Open WebUI, and ntfy (Docker Containers): Standard Docker services that act as the API gateway, model router, chat UI, and notification bus.

---

🚀 1-Command Installation

To deploy the entire stack on your Mac Mini, open the macOS Terminal and run:

curl -fsSL https://raw.githubusercontent.com/priyanshusaleshandy/mac-mini-ai-stack/main/setup-mac-llm.sh | bash

Manual Usage

Alternatively, clone the repository and run:

chmod +x setup-mac-llm.sh
./setup-mac-llm.sh

---

🔌 VS Code & Tabby Client Configuration

To connect this local GPU-accelerated LLM to your development environment (e.g. Tabby or another OpenAI-compatible client), use the following configuration schema:

name: My Config
version: 1.0.0
schema: v1
models:
  - name: IKI-MAC-27
    provider: openai
    model: qwen2.5-coder:7b
    apiBase: http://192.168.126.101:4000/v1
    apiKey: sk-ikigai-local-key

Parameters

• name: Identifies your Mac Mini endpoint model context (IKI-MAC-27).
• provider: Set to openai (since the LiteLLM gateway uses standardized OpenAI API format).
• model: Point to the loaded model qwen2.5-coder:7b.
• apiBase: Set to your Mac Mini LiteLLM proxy port: http://192.168.126.101:4000/v1.
• apiKey: Authorized key sk-ikigai-local-key to pass through the LiteLLM gateway security layer.

---

🔗 Endpoint Endpoints

• Open WebUI (Chat UI): http://192.168.126.101:3000
• LiteLLM Gateway (OpenAI Compatible API): http://192.168.126.101:4000/v1
• ntfy server (Notifications): http://192.168.126.101:8080