Skip to content

Memory pressure from glasses camera stream causes instability #1

@DarlingtonDeveloper

Description

@DarlingtonDeveloper

Problem

The glasses camera stream at 24fps raw causes memory pressure on the phone, eventually leading to instability and app crashes. The DAT SDK Bluetooth buffer fills up when the app is busy with Gemini + OpenClaw calls simultaneously.

Root Causes

  1. 24fps is too aggressiveGlassesCameraManager streams at 24fps with videoCodec: .raw, but we only consume 1 frame/sec (via FrameThrottler). 23 out of 24 frames are allocated and immediately discarded every second.

  2. Raw codec — uncompressed UIImages in transit use significantly more memory than compressed formats.

  3. latestFrame retention@Published var latestFrame: UIImage? holds a reference to the last raw frame even when the preview isn't visible on screen. SwiftUI keeps it alive.

  4. No autoreleasepool — the onFrameCaptured callback creates UIImages on a background thread without an autoreleasepool, so deallocation is deferred.

Suggested Fixes

1. Reduce frame rate (high impact, 1 line)

// GlassesCameraManager.swift
let config = StreamSessionConfig(
    videoCodec: .raw,
    resolution: .low,
    frameRate: 5  // was 24
)

We only use 1fps via the throttler. 5fps gives headroom without flooding memory.

2. Use JPEG codec if DAT SDK supports it

videoCodec: .jpeg  // instead of .raw

Compressed frames use ~10x less memory in transit.

3. Nil out latestFrame when not visible

Clear the retained UIImage when the camera preview isn't on screen.

4. Autoreleasepool in frame callback

activeCamera.onFrameCaptured = { [weak self] image in
    autoreleasepool {
        Task { @MainActor in
            self?.latestFrame = image
        }
        self?.frameThrottler.submit(image)
    }
}

Priority

Fix #1 alone should resolve the issue. The others are defence-in-depth.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions