Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .tmux-attach.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/bash
# Auto-generated tmux attach script
tmux attach-session -t workmux:ushadow-purple
31 changes: 31 additions & 0 deletions .tmux.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# User-friendly tmux configuration for Ushadow environments

# Enable mouse support (scroll, select, resize panes)
set -g mouse on

# Increase scrollback buffer
set -g history-limit 50000

# Don't rename windows automatically
set -g allow-rename off

# Start window numbering at 1
set -g base-index 1

# Enable 256 colors
set -g default-terminal "screen-256color"

# Faster command sequences
set -s escape-time 0

# Status bar styling
set -g status-style bg=default,fg=white
set -g status-left-length 40
set -g status-right "#[fg=yellow]#S #[fg=white]%H:%M"

# Pane border colors
set -g pane-border-style fg=colour238
set -g pane-active-border-style fg=colour39

# Fix mouse scrolling in terminal applications
set -g terminal-overrides 'xterm*:smcup@:rmcup@'
212 changes: 212 additions & 0 deletions ARCHITECTURE_OVERVIEW.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,212 @@
# Architecture Overview: Services vs Instances vs Deployments

## Core Concepts

### 1. **Service** (Services Page)
**Model**: Managed by `DockerManager.MANAGEABLE_SERVICES` (dynamically built from compose registry)
**Location**: `ushadow/backend/src/services/docker_manager.py`

- Represents a **Docker Compose service** that can be started/stopped
- Lives in docker-compose files (e.g., `compose/openmemory-compose.yaml`)
- Discovered and registered automatically by `ComposeServiceRegistry`
- **Single instance per service** - only one copy can run at a time
- Start/stop controls the actual Docker container directly
- Has port conflict checking with user dialog for port overrides
- Port overrides saved to: `services.{service_name}.ports.{ENV_VAR}`

**UI**: Services Page (`ushadow/frontend/src/pages/ServicesPage.tsx`)
- Shows cards with Start/Stop buttons
- Port conflict flow: preflight check → dialog → port override → retry

**APIs**:
- `POST /api/services/{name}/start` - Start service
- `POST /api/services/{name}/stop` - Stop service
- `GET /api/services/{name}/preflight` - Check for port conflicts
- `POST /api/services/{name}/port-override` - Set port override

**Hook**: `useServiceStart` from `ushadow/frontend/src/hooks/useServiceStart.ts`

---

### 2. **Instance** (Instances Page)
**Model**: `Instance` in `ushadow/backend/src/models/instance.py`
**Manager**: `InstanceManager` in `ushadow/backend/src/services/instance_manager.py`

- Represents a **template + configuration + deployment target**
- Can have **multiple instances** of the same template (e.g., openai-1, openai-2)
- Has lifecycle: PENDING → DEPLOYING → RUNNING → STOPPED → ERROR
- Can be deployed to:
- Local Docker (deployment_target=None)
- Remote unode (deployment_target=hostname)
- Cloud provider (deployment_target="cloud", status="n/a")

**Deployment Types**:
- **Local Docker**: Uses `ServiceOrchestrator` (compose services) or direct Docker
- **Remote unode**: Creates a `Deployment` record, uses `DeploymentManager`
- **Cloud**: No actual deployment, just config storage

**Port Handling**:
- For LOCAL deployments via orchestrator: Has port conflict checking code in `instance_manager.py:551-563`
- For REMOTE deployments: No port conflict checking (just added in `deployment_manager.py:512-547`)
- **Problem**: No user dialog, just logs + auto-remap

**UI**: Instances Page (`ushadow/frontend/src/pages/InstancesPage.tsx`)
- Shows cards with Start/Stop buttons (similar to services)
- Start button calls `handleDeployInstance()`
- Stop button calls `handleUndeployInstance()`
- **No port conflict dialog** - just fails or auto-remaps silently

**APIs**:
- `POST /api/instances` - Create instance
- `POST /api/instances/{id}/deploy` - Deploy/start instance
- `POST /api/instances/{id}/undeploy` - Stop instance
- `DELETE /api/instances/{id}` - Delete instance

---

### 3. **Deployment** (Database Record)
**Model**: `Deployment` in `ushadow/backend/src/models/deployment.py`
**Manager**: `DeploymentManager` in `ushadow/backend/src/services/deployment_manager.py`

- Represents a **service deployed to a specific unode**
- Lower-level runtime record tracking container state
- Created when an instance is deployed to a remote unode
- Stores: container_id, container_name, status, access_url, exposed_port
- Has relationship to Instance: `Instance.deployment_id` → `Deployment.id`
- Also has `Deployment.instance_id` → `Instance.id` (bidirectional)

**Backends**:
- `DockerDeploymentBackend` - Deploys to Docker hosts
- `KubernetesDeploymentBackend` - Deploys to K8s clusters

**Not visible in UI** - only used internally for tracking remote deployments

---

## Current Problems

### 1. **Duplicate Port Conflict Logic**
- Services page: Full preflight + dialog + port override flow
- Instances page (local): Port conflict check + auto-remap to settings (no dialog)
- Instances page (remote): Port conflict check + auto-remap to resolved_service.ports (no dialog)
- **Different implementations** in 3 places!

### 2. **No User Confirmation for Instances**
Services ask user: "Port 8765 is in use, switch to 8766?"
Instances: Just auto-remap (or fail silently before my changes)

### 3. **Port Override Storage Inconsistency**
- Services: `services.{name}.ports.{ENV_VAR}` (service-level, shared)
- Instances (local): Also `services.{name}.ports.{ENV_VAR}` (conflicts with other instances!)
- Instances (remote): Only in `resolved_service.ports` (temporary, not persisted)

### 4. **Instance Config Not Used for Ports**
Instances have a `config` field but ports aren't stored there per-instance

---

## Proposed Unified Architecture

### Goal: Reuse `useServiceStart` Pattern for Instances

### Backend Changes

#### 1. Add Preflight Check for Instances
```python
# /api/instances/{id}/preflight
# Returns same format as services preflight
{
"can_start": false,
"port_conflicts": [
{
"port": 8765,
"env_var": "MEM0_PORT",
"used_by": "Docker: mem0-abc123",
"suggested_port": 8766
}
]
}
```

#### 2. Add Port Override for Instances
```python
# /api/instances/{id}/port-override
# Sets port in instance.config (per-instance, not service-level)
instance.config.values["MEM0_PORT"] = 8766
save_instances()
```

#### 3. Update Deploy Flow
```python
async def deploy_instance(instance_id):
# 1. Check ports using existing check_port_conflicts()
conflicts = docker_mgr.check_port_conflicts(service_name)

# 2. If conflicts, return 409 with conflict info
# (Let frontend handle it)

# 3. Apply instance.config port overrides to env vars
# before starting container
```

### Frontend Changes

#### 1. Create `useInstanceDeploy` Hook
Similar to `useServiceStart` but for instances:
```typescript
export function useInstanceDeploy(
onSuccess?: (instanceId: string) => void,
onError?: (instanceId: string) => void
) {
// Call preflight check
// Show port conflict dialog if needed
// Call port override API
// Retry deploy
}
```

#### 2. Update InstancesPage
```typescript
// Replace handleDeployInstance with:
const instanceDeploy = useInstanceDeploy(...)
onClick={() => instanceDeploy.startInstance(instance.id)}

// Render port conflict dialog
<PortConflictDialog
isOpen={instanceDeploy.portConflictDialog.isOpen}
conflicts={instanceDeploy.portConflictDialog.conflicts}
onResolve={instanceDeploy.resolvePortConflict}
onDismiss={instanceDeploy.dismissPortConflict}
/>
```

---

## Code Reuse Plan

### ✅ Already Shared
- `check_port_conflicts()` in `docker_manager.py`
- `PortConflictDialog` component (can reuse for instances)

### ❌ Currently Duplicated
- Port conflict checking logic (3 implementations)
- Preflight check flow (services only)
- Port override storage (inconsistent)

### 🎯 Should Be Shared
- Preflight check pattern (services + instances)
- Port conflict resolution dialog (same UI)
- Port override API pattern (adapt for instance config)

---

## Next Steps

1. **Add instance preflight endpoint** (`/api/instances/{id}/preflight`)
2. **Add instance port override endpoint** (`/api/instances/{id}/port-override`)
3. **Remove auto-remap logic** from deployment_manager.py (let frontend handle)
4. **Create useInstanceDeploy hook** (mirror useServiceStart)
5. **Add PortConflictDialog to InstancesPage**
6. **Store port overrides in instance.config** (not service-level settings)

This unifies the UX while respecting the difference that instances are per-config vs services are singleton.
105 changes: 105 additions & 0 deletions CHRONICLE_IPV6_RESOLUTION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
# Chronicle IPv6 Resolution - Final Status

**Date:** 2026-01-17

## TL;DR

✅ **Chronicle will work with IPv6** - No changes needed to your image or deployment.

## What Was Wrong

Our CoreDNS was blocking AAAA (IPv6) DNS records, preventing applications from resolving IPv6 addresses.

## What We Fixed

1. Removed AAAA blocking template from CoreDNS
2. Restarted CoreDNS deployment
3. Verified DNS now returns both A and AAAA records

## Why Our Tests Were Misleading

The test pods we created use **Alpine Linux (musl libc)**, which has a known bug with dual-stack DNS resolution.

Chronicle uses **Debian Bookworm (glibc)**, which does NOT have this issue.

## Verification with Chronicle's Exact Base Image

```bash
# Using python:3.12-slim-bookworm (Chronicle's base)
kubectl run test --image=python:3.12-slim-bookworm --rm -it -- python3 -c \
"import socket; print('Resolved:', len(socket.getaddrinfo('pypi.org', 443)), 'addresses')"
# Output: Resolved: 24 addresses ✅

kubectl run test --image=python:3.12-slim-bookworm --rm -it -- sh -c \
"apt-get update -qq && apt-get install -y -qq python3-pip && \
pip3 download --no-cache-dir --no-deps httpx"
# Output: Successfully downloaded httpx ✅
```

## Base Image Compatibility

### ✅ Works (glibc-based)
- `python:3.x` (includes Chronicle)
- `python:3.x-slim`
- `python:3.x-slim-bookworm` (Chronicle's exact base)
- `ubuntu:*`
- `debian:*`
- Most production Python images

### ❌ Broken (musl-based)
- `python:3.x-alpine`
- `alpine:*`
- `nicolaka/netshoot` (our test pods)
- Any Alpine-derived image

## When Chronicle Starts

Chronicle's startup command:
```bash
uv run --extra deepgram python src/advanced_omi_backend/main.py
```

Will execute as:
1. `uv` queries DNS for dependencies → gets both IPv4 and IPv6 ✅
2. `uv` downloads from PyPI over IPv6 or IPv4 ✅
3. Application starts successfully ✅

## If You're Still Seeing Failures

Check:
1. **Are you testing locally?** Docker Desktop DNS settings may differ
2. **Is Chronicle actually deployed?** The pod needs to exist in K8s cluster
3. **Check the actual error:** Run `kubectl logs <chronicle-pod>` to see the real failure

The CoreDNS fix we applied ONLY affects pods running in the Kubernetes cluster.

## Commands to Verify

```bash
# Check DNS returns AAAA records
kubectl run test --image=busybox --rm -it -- nslookup pypi.org
# Should show both A and AAAA records

# Test with Chronicle's base image
kubectl run test --image=python:3.12-slim-bookworm --rm -it -- \
python3 -c "import urllib.request; urllib.request.urlopen('https://pypi.org/simple/httpx/'); print('SUCCESS')"
```

## Files Changed

1. **CoreDNS ConfigMap** (`kube-system/coredns`)
- Before: Had AAAA blocking template
- After: Returns AAAA records normally
- Backup: `/tmp/coredns-before-aaaa-unblock.yaml`

2. **Documentation Updated**
- `docs/IPV6_DUALSTACK_CONFIGURATION.md` - Full details
- This file - Quick reference

3. **Test Scripts**
- `scripts/quick-ipv6-test.sh` - Tests Alpine pods (will show failures)
- `scripts/diagnose-ipv6.sh` - Full diagnostic

## Bottom Line

**Your Chronicle deployment will work.** The DNS issue is resolved, and we've verified your exact base image successfully downloads from PyPI with both IPv4 and IPv6 enabled.
Loading
Loading