diff --git a/README.md b/README.md index 87ff2490..d3eb3b0d 100644 --- a/README.md +++ b/README.md @@ -168,6 +168,47 @@ run well on NVIDIA Jetson Orin Nano 8 GB hardware. Spam-like PRs, AI-generated issue churn, duplicate reports, unplanned bug-fix batches, or changes without real behavior proof will be closed immediately to protect review quality. +### Accepted contribution scope + +A PR is accepted **only** if it lands in one of these two buckets, with +reproducible on-device proof. Anything else will be closed. + +> πŸ’Ž **Performance PRs are rewarded.** Land a performance-improvement PR that +> meets the rules below β€” measurable Jetson win, reproducible beforeβ†’after proof β€” +> and you're eligible for a reward through [gittensor](https://gittensor.io/), +> the Bittensor subnet that pays out for merged open-source contributions. + +1. **Performance improvement** β€” measurable latency / throughput / memory wins + on Jetson Orin Nano 8 GB, with beforeβ†’after numbers. + - e.g. [genie-ai-runtime#85](https://github.com/GeniePod/genie-ai-runtime/pull/85) + β€” in-memory KV prefix cache, **~13Γ— faster prefill** (16s β†’ ~1s per + command); cut the BFCL eval from ~62 min to ~20 min. + +2. **Tool-dispatch / real-Home-Assistant correctness** β€” fixes to tool routing, + tool-call arguments, or home actuation, **measured** (BFCL) and/or + **reproduced against a real Home Assistant**. A runnable sample HA config is + provided at [`deploy/homeassistant/`](deploy/homeassistant/) so you can + reproduce the failure and prove the fix. + - **Accuracy, measured:** [#399](https://github.com/GeniePod/genie-claw/pull/399) + β€” ground the predict prompt in the home device catalog: raw BFCL strict + **20.19% β†’ 50.96%**, grounded **72.12% β†’ 82.69%** (Qwen3-4B @ 4096, same + model β€” deterministic device-state grounding, not scale); + [#390](https://github.com/GeniePod/genie-claw/pull/390) β€” action-synonym + canonicalization + wrong-room fidelity guard; + [#388](https://github.com/GeniePod/genie-claw/pull/388) β€” grounded + entity-argument metric. + - **Live-HA actuation:** [#400](https://github.com/GeniePod/genie-claw/pull/400) + β€” canonicalize `home_control` action synonyms. *Before:* the model emits + `"turn off"`, the runtime rejects it (*"action 'turn off' is invalid"*) and + the light stays on. *After:* `"turn off" β†’ "turn_off"`, and + `light.kitchen_lights` goes `off β†’ on`, confirmed via the HA API. Also + [#380](https://github.com/GeniePod/genie-claw/pull/380) β€” stop leaking + unparsed tool-call JSON to the user. + +Every such PR needs a **Real Behavior Proof**: what you ran, on what hardware, +and what changed β€” for HA fixes, live-HA before/after confirmed via the API. +No reproducible proof, or outside these two buckets β†’ closed. + ## Product Quality Bar PRs must improve the product behavior or make it easier to measure product diff --git a/deploy/homeassistant/README.md b/deploy/homeassistant/README.md new file mode 100644 index 00000000..23621ab3 --- /dev/null +++ b/deploy/homeassistant/README.md @@ -0,0 +1,71 @@ +# Sample Home Assistant β€” for reproducing tool-dispatch / actuation issues + +The **actual mock-home Home Assistant config from the GenieClaw Jetson +deployment**, packaged to run anywhere. Point GenieClaw at it to reproduce and +prove **real home-control** fixes β€” the standard the [contribution +gate](../../README.md#accepted-contribution-scope) holds tool-dispatch / +Home-Assistant PRs to. + +The `config/configuration.yaml` here is committed verbatim from the Jetson: the +**Demo** integration (a fully simulated house β€” lights, fans, covers, locks, +climate, media players) plus mock `input_boolean` / `input_number` helpers and +template sensors (front-door contact, hallway motion, outdoor temp, COβ‚‚). It +loads from YAML, so the simulated devices appear with **no UI integration step**. + +## 1. Bring up Home Assistant + +```bash +docker compose -f deploy/homeassistant/docker-compose.yml up -d +# HA serves on http://:8123 with the simulated house already loaded +``` + +This mirrors the Jetson's HA service in `/opt/geniepod/docker/docker-compose.yml`, +but bind-mounts the committed `./config` so the mock home is reproduced exactly. + +## 2. Onboard + create a long-lived token + +Open `http://:8123`, complete first-run onboarding (create a user), then: +Profile (bottom-left) β†’ **Security** β†’ **Long-Lived Access Tokens** β†’ **Create +Token** β†’ name it `GenieClaw` β†’ copy it (shown once). Headless alternative: see +the auth-flow steps in the main `README` / `GETTING_STARTED`. + +The mock entities are already present (loaded from `configuration.yaml`), e.g.: + +- `light.kitchen_lights`, `light.bed_light`, `light.ceiling_lights` +- `fan.living_room_fan`, `fan.ceiling_fan` +- `cover.kitchen_window`, `cover.garage_door` +- `lock.front_door`, `climate.hvac`, `switch.decorative_lights` +- mock sensors: `binary_sensor.front_door`, `binary_sensor.hallway_motion`, + `sensor.outdoor_temperature`, `sensor.indoor_co2_level` + +## 3. Wire it into GenieClaw + +```toml +# /etc/geniepod/geniepod.toml +[services.homeassistant] +url = "http://127.0.0.1:8123/" +systemd_unit = "homeassistant.service" +``` + +Token via env (preferred) or config: + +```bash +# genie-core.service: Environment=HA_TOKEN= (config reads HA_TOKEN when [core] ha_token is empty) +sudo systemctl daemon-reload && sudo systemctl restart genie-core +``` + +## 4. Reproduce + verify + +```bash +genie-ctl chat "what is the state of the kitchen lights?" # read path +genie-ctl chat "turn on the kitchen lights" # actuation path + +# confirm against HA itself +curl -s -H "Authorization: Bearer $HA_TOKEN" \ + http://127.0.0.1:8123/api/states/light.kitchen_lights | jq .state +``` + +A tool-dispatch / HA PR's **Real Behavior Proof** should show the entity state +changing (e.g. `off β†’ on`) confirmed via the HA API β€” exactly how +[#400](https://github.com/GeniePod/genie-claw/pull/400) (action-synonym +canonicalization) was validated against this mock home. diff --git a/deploy/homeassistant/config/automations.yaml b/deploy/homeassistant/config/automations.yaml new file mode 100644 index 00000000..fe51488c --- /dev/null +++ b/deploy/homeassistant/config/automations.yaml @@ -0,0 +1 @@ +[] diff --git a/deploy/homeassistant/config/configuration.yaml b/deploy/homeassistant/config/configuration.yaml new file mode 100644 index 00000000..91b6fcf5 --- /dev/null +++ b/deploy/homeassistant/config/configuration.yaml @@ -0,0 +1,85 @@ +# Loads default set of integrations. Do not remove. +default_config: + +# Load frontend themes from the themes folder +frontend: + themes: !include_dir_merge_named themes + +automation: !include automations.yaml +script: !include scripts.yaml +scene: !include scenes.yaml + +# ========================================== +# MOCK HARDWARE FOR GENIE-CLAW AI TESTING +# ========================================== + +# 1. The "Demo" Integration +# Instantly creates a fully simulated house with controllable lights, switches, +# media players, covers (blinds/garage), and climate entities. +# Your AI can immediately call services like light.turn_on, cover.open_cover, etc. +demo: + +# 2. Input Helpers (The "Under the Hood" Virtual Hardware) +# These act as the raw memory registers of your mock devices. +# You can change these in the HA UI to simulate physical events. +input_boolean: + mock_front_door_contact: + name: Front Door Contact (Mock) + initial: off + icon: mdi:door + mock_motion_in_hallway: + name: Hallway Motion (Mock) + initial: off + icon: mdi:motion-sensor + mock_is_night_mode: + name: Night Mode (Mock) + initial: off + icon: mdi:weather-night + +input_number: + mock_outdoor_temperature: + name: Outdoor Temperature (Mock) + initial: 15.0 + min: -10 + max: 40 + step: 0.5 + mock_indoor_co2_level: + name: Indoor CO2 Level (Mock) + initial: 450 + min: 400 + max: 2000 + step: 10 + mock_living_room_temp: # ← Creates entity_id: input_number.mock_living_room_temp + name: Living Room Temp (Mock) + initial: 22 + min: 10 + max: 30 + step: 0.5 + +# 3. Template Entities (What the AI Actually "Sees") +# We map the inputs above to proper HA entities with correct device classes. +# This ensures the AI understands context (e.g., knows CO2 is in ppm, door is a boolean). +template: + - binary_sensor: + - name: "Front Door" + unique_id: "genie_mock_front_door" + device_class: door + state: "{{ is_state('input_boolean.mock_front_door_contact', 'on') }}" + + - name: "Hallway Motion" + unique_id: "genie_mock_hallway_motion" + device_class: motion + state: "{{ is_state('input_boolean.mock_motion_in_hallway', 'on') }}" + + - sensor: + - name: "Outdoor Temperature" + unique_id: "genie_mock_outdoor_temp" + unit_of_measurement: "Β°C" + device_class: temperature + state: "{{ states('input_number.mock_outdoor_temperature') | float }}" + + - name: "Indoor CO2 Level" + unique_id: "genie_mock_co2" + unit_of_measurement: "ppm" + device_class: carbon_dioxide + state: "{{ states('input_number.mock_indoor_co2_level') | int }}" diff --git a/deploy/homeassistant/config/scenes.yaml b/deploy/homeassistant/config/scenes.yaml new file mode 100644 index 00000000..ebc4b869 --- /dev/null +++ b/deploy/homeassistant/config/scenes.yaml @@ -0,0 +1 @@ +# No scenes defined for the mock home. diff --git a/deploy/homeassistant/config/scripts.yaml b/deploy/homeassistant/config/scripts.yaml new file mode 100644 index 00000000..66203460 --- /dev/null +++ b/deploy/homeassistant/config/scripts.yaml @@ -0,0 +1 @@ +# No scripts defined for the mock home. diff --git a/deploy/homeassistant/config/themes/.gitkeep b/deploy/homeassistant/config/themes/.gitkeep new file mode 100644 index 00000000..e69de29b diff --git a/deploy/homeassistant/docker-compose.yml b/deploy/homeassistant/docker-compose.yml new file mode 100644 index 00000000..9d4d41ba --- /dev/null +++ b/deploy/homeassistant/docker-compose.yml @@ -0,0 +1,34 @@ +# Sample Home Assistant for reproducing GenieClaw tool-dispatch / home-actuation +# issues against a REAL Home Assistant. This mirrors the GenieClaw Jetson +# deployment's Home Assistant service (`/opt/geniepod/docker/docker-compose.yml`), +# but bind-mounts the committed ./config so the mock-home `configuration.yaml` +# (demo devices + input helpers) loads with no UI onboarding. +# +# docker compose -f deploy/homeassistant/docker-compose.yml up -d +# # HA serves on http://:8123 with the simulated house already loaded +# +services: + homeassistant: + container_name: homeassistant + image: ghcr.io/home-assistant/home-assistant:stable + restart: unless-stopped + # host networking so HA serves on http://:8123 β€” matches the + # `url = "http://127.0.0.1:8123/"` GenieClaw uses. + network_mode: host + volumes: + - ./config:/config # committed mock-home config (demo + mocks) + - /etc/localtime:/etc/localtime:ro + # - /run/dbus:/run/dbus:ro # Jetson only (local hardware via dbus) + environment: + - TZ=${TZ:-America/Denver} + # privileged: true # Jetson only (local hardware); not needed to test tool dispatch + deploy: + resources: + limits: + memory: 512M + healthcheck: + test: ["CMD", "curl", "-sf", "http://127.0.0.1:8123/"] + interval: 30s + timeout: 5s + retries: 3 + start_period: 60s