diff --git a/desktop/README.md b/desktop/README.md index 017601b..b0bfb8a 100644 --- a/desktop/README.md +++ b/desktop/README.md @@ -73,18 +73,12 @@ git clone https://github.com/JStone2934/LiveGalGame.git cd LiveGalGame/desktop pnpm install -# 为 Electron 原生依赖编译本机二进制 -pnpm exec electron-builder install-app-deps ``` -### 配置语音识别 (ASR) - -当前开箱即用路线:**Faster‑Whisper Medium**(WhisperLiveKit 主服务,失败时自动回退本地 faster‑whisper worker)。直接 `pnpm dev` 即可,无需额外环境变量。 - -> 说明:仓库保留了 FunASR / whisper.cpp 的安装脚本,方便后续扩展;目前运行时不会自动切到 FunASR,`WHISPER_IMPL` 环境变量暂未生效。 +### 配置语音识别 ```bash -# 可选预备:安装 FunASR(如需更好中文识别,未来接入可复用) +# 安装 FunASR(推荐,中文识别效果最好) npm run setup-funasr ``` @@ -129,6 +123,6 @@ pnpm dev - `src/renderer/` - React 前端界面 - `src/asr/` - 语音识别服务 - `src/db/` - 本地数据存储 -- `src/agent/` & `agent-dev.md` - Agent 占位实现与开发指南 欢迎提交 PR!有问题请加 QQ 群:**1074602400** + diff --git a/desktop/agent-dev.md b/desktop/agent-dev.md deleted file mode 100644 index d4293a8..0000000 --- a/desktop/agent-dev.md +++ /dev/null @@ -1,111 +0,0 @@ -# Agent 开发说明(占位版) - -> 目标:让协作同学在不踩坑的情况下快速接入 / 替换 Agent‑LLM 后端。当前实现为本地 Python mock,接口稳定,可在此基础上逐步替换为真实模型与记忆逻辑。 - -## 1. 范围与现状 -- 作用:接收对话上下文(含最新 ASR 文本),返回 AI 建议(可流式)。 -- 现状:内置 mock(`src/agent/agent_worker.py`),无真实推理/记忆,仅供前端联调。 -- 默认通路:Electron 主进程通过 stdin/stdout 与 Python worker 通信;渲染层通过 IPC (`agent-run` / `agent-run-stream`) 调用。 - -## 2. 架构与数据流 -``` -ASR sentence_complete - │ - ▼ - 主进程 (IPCManager) - │ IPC invoke / stream - ▼ - AgentService (Node) - │ spawn + JSONL over stdin/stdout - ▼ - agent_worker.py (Python) - │ emits partial/final JSON lines - ▼ - 主进程转发 → 渲染层 HUD/页面 -``` -- 通信方式:JSON 每行一条(JSONL),UTF‑8。 -- 流式:Python 侧可发送多条 `event=partial`,最后 `event=final`。 - -## 3. 接口契约 -### 3.1 请求(主进程 → Python) -```json -{ - "id": "uuid", - "type": "run", - "stream": true, - "payload": { - "latest_text": "string", - "history": [ { "role": "user|partner", "text": "..." } ], - "character": { "name": "...", "tags": ["..."] }, - "conversation_id": "uuid", - "lang": "zh|en|..." - } -} -``` - -### 3.2 响应事件(Python → 主进程) -- `partial` -```json -{ "id": "...", "event": "partial", "data": { "stage": "thinking|draft|...", "text": "..." } } -``` -- `final` -```json -{ - "id": "...", - "event": "final", - "data": { - "suggestions": [ - { "title": "string", "content": "string", "tags": ["..."], "affinity_delta": 3, "confidence": 0.54 } - ], - "rationale": "why these suggestions", - "safety_flags": [] - } -} -``` -- `error`(可选) -```json -{ "id": "...", "event": "error", "data": { "code": "internal_error", "message": "..." } } -``` - -### 3.3 渲染层 API(preload 暴露) -- 非流式:`window.electronAPI.agentRun(payload)` → Promise -- 流式:`window.electronAPI.agentRunStream(payload)` 返回 `{requestId}`;在 `onAgentStream(cb)` 里收到 `{requestId, event, data}`。 - -## 4. 运行与依赖 -- Python 3.8+,默认解释器:`python3`;可用 `AGENT_PYTHON_PATH` 指定。 -- 启动:主进程懒加载。独立测试: - ```bash - cd desktop - echo '{"type":"run","payload":{"latest_text":"你好"},"stream":true}' | python3 src/agent/agent_worker.py - ``` -- 日志:stdout 用于协议;stderr 前缀 `[AgentWorker]`。 - -## 5. 节流与超时建议 -- 触发:ASR sentence_complete 或用户显式点击;避免对每个 partial 调用。 -- 并发:同一会话串行,1 在跑 + 1 排队。 -- 超时:主进程侧 8~12s 超时,超时返回降级提示。 - -## 6. 回退/降级 -- Worker 崩溃:AgentService 清空 pending,下一次调用重启;调用方应提示“AI 暂不可用”。 -- 限流/超时:返回 `event=error`,主进程转换为可展示提示。 - -## 7. 记忆/向量检索(规划) -- 分层记忆:短窗 + 摘要 + 向量召回(bge/MiniLM),存 SQLite + 内存检索;尚未实现。 -- 召回打分:相似度 + 时效衰减;在输出中标注引用来源。 - -## 8. 安全与隐私 -- API Key 仅存主进程/DB,Python 通过环境变量传入。 -- 输出过滤:长度上限、危险词/PII 粗过滤(待实现)。 - -## 9. 常见问题 -- **没有 partial?** 确认 `stream=true` 且已监听 `onAgentStream`。 -- **worker 不启动?** 检查 `AGENT_PYTHON_PATH` 或 `.venv/bin/python`;看主进程控制台。 -- **如何关掉 Agent?** 暂无开关,可在主进程调用前加环境变量 `AGENT_DISABLED=1`(待实现)。 - -## 10. 变更流程(建议) -- 兼容:新增字段向后兼容;破坏性变更用 `api_version`。 -- 发布:先保持契约,在 worker 内替换逻辑;保留 `--mock` 开关便于回滚。 - ---- -Owner: @TODO -最后更新:2025-12-02 diff --git a/desktop/src/agent/agent-service.js b/desktop/src/agent/agent-service.js deleted file mode 100644 index 166e6b7..0000000 --- a/desktop/src/agent/agent-service.js +++ /dev/null @@ -1,159 +0,0 @@ -import { spawn } from 'child_process'; -import path from 'path'; -import fs from 'fs'; -import { randomUUID } from 'crypto'; -import { EventEmitter } from 'events'; -import { app } from 'electron'; - -const DEFAULT_PYTHON = 'python3'; - -function detectPythonPath() { - const envPython = process.env.AGENT_PYTHON_PATH; - if (envPython && fs.existsSync(envPython)) return envPython; - - const projectRoot = path.resolve(app.getAppPath(), app.isPackaged ? '../..' : '.'); - const venvPython = path.join(projectRoot, '.venv', 'bin', 'python'); - if (fs.existsSync(venvPython)) return venvPython; - - if (process.platform === 'win32') { - const venvPythonWin = path.join(projectRoot, '.venv', 'Scripts', 'python.exe'); - if (fs.existsSync(venvPythonWin)) return venvPythonWin; - } - return DEFAULT_PYTHON; -} - -export default class AgentService extends EventEmitter { - constructor() { - super(); - const projectRoot = path.resolve(app.getAppPath(), app.isPackaged ? '../..' : '.'); - this.scriptPath = path.join(projectRoot, 'src/agent/agent_worker.py'); - this.pythonPath = detectPythonPath(); - this.workerProcess = null; - this.pending = new Map(); - this.isStarting = false; - } - - async ensureWorker() { - if (this.workerProcess || this.isStarting) return; - this.isStarting = true; - - const env = { - ...process.env, - PYTHONUNBUFFERED: '1', - PYTHONIOENCODING: 'utf-8' - }; - - console.log(`[AgentService] Spawning worker: ${this.pythonPath} ${this.scriptPath}`); - this.workerProcess = spawn(this.pythonPath, [this.scriptPath], { env }); - - this.workerProcess.stdout.on('data', (data) => { - const lines = data.toString().split('\n'); - lines.forEach((line) => { - const trimmed = line.trim(); - if (!trimmed) return; - this.handleWorkerMessage(trimmed); - }); - }); - - this.workerProcess.stderr.on('data', (data) => { - console.log(`[AgentWorker] ${data.toString().trim()}`); - }); - - this.workerProcess.on('close', (code) => { - console.warn(`[AgentService] Worker exited with code ${code}`); - this.workerProcess = null; - this.isStarting = false; - // fail all pending - for (const [, entry] of this.pending) { - entry.reject(new Error('Agent worker exited')); - } - this.pending.clear(); - }); - - this.workerProcess.on('error', (err) => { - console.error('[AgentService] Worker error:', err); - }); - - this.isStarting = false; - } - - handleWorkerMessage(line) { - let msg; - try { - msg = JSON.parse(line); - } catch (err) { - console.error('[AgentService] Failed to parse worker message:', line); - return; - } - const { id, event, data } = msg; - if (!id || !this.pending.has(id)) { - console.warn('[AgentService] Unknown request id from worker:', id); - return; - } - const entry = this.pending.get(id); - if (event === 'partial') { - if (entry.onStream) entry.onStream({ event: 'partial', data }); - } else if (event === 'final') { - if (entry.onStream) entry.onStream({ event: 'final', data }); - entry.resolve({ id, ...data }); - this.pending.delete(id); - } else if (event === 'pong') { - entry.resolve({ pong: true }); - this.pending.delete(id); - } else { - console.warn('[AgentService] Unknown event from worker:', event); - } - } - - async run(payload, { stream = false, onStream, requestId } = {}) { - await this.ensureWorker(); - const id = requestId || randomUUID(); - - const message = { - id, - type: 'run', - payload, - stream - }; - - const promise = new Promise((resolve, reject) => { - this.pending.set(id, { resolve, reject, onStream }); - try { - this.workerProcess.stdin.write(JSON.stringify(message) + '\n'); - } catch (err) { - this.pending.delete(id); - reject(err); - } - }); - - return promise; - } - - async ping() { - await this.ensureWorker(); - const id = randomUUID(); - const message = { id, type: 'ping' }; - const promise = new Promise((resolve, reject) => { - this.pending.set(id, { resolve, reject }); - try { - this.workerProcess.stdin.write(JSON.stringify(message) + '\n'); - } catch (err) { - this.pending.delete(id); - reject(err); - } - }); - return promise; - } - - async destroy() { - if (this.workerProcess) { - try { - this.workerProcess.kill(); - } catch (err) { - console.error('[AgentService] Failed to kill worker:', err); - } - this.workerProcess = null; - } - this.pending.clear(); - } -} diff --git a/desktop/src/agent/agent_worker.py b/desktop/src/agent/agent_worker.py deleted file mode 100644 index eb67f59..0000000 --- a/desktop/src/agent/agent_worker.py +++ /dev/null @@ -1,95 +0,0 @@ -""" -Minimal JSONL-based Agent worker. - -读取 stdin 的 JSON 行,写入 stdout(每行一个 JSON)。 -事件: - - partial:流式中间结果 - - final:最终结构化建议 - -当前是占位实现,返回固定建议,便于前端联调。 -""" - -import json -import sys -import time -import uuid -from typing import Any, Dict - - -def log(message: str) -> None: - sys.stderr.write(f"[AgentWorker] {message}\n") - sys.stderr.flush() - - -def send(obj: Dict[str, Any]) -> None: - sys.stdout.write(json.dumps(obj, ensure_ascii=False) + "\n") - sys.stdout.flush() - - -def build_mock_final(request_id: str, payload: Dict[str, Any]) -> Dict[str, Any]: - latest = payload.get("latest_text", "") or "(空)" - suggestions = [ - { - "title": "共情回应", - "content": f"听起来你在说「{latest[:40]}...」,我理解你的感受,可以这样回应:‘嗯,我在听,你继续说~’", - "tags": ["共情", "倾听"], - "affinity_delta": 5, - "confidence": 0.62, - }, - { - "title": "推进话题", - "content": "顺着对方话题抛一个轻量问题:‘那件事后来怎么样了?’", - "tags": ["推进", "提问"], - "affinity_delta": 3, - "confidence": 0.54, - }, - ] - return { - "id": request_id, - "event": "final", - "data": { - "suggestions": suggestions, - "rationale": "基于最新转写内容给出两条可选建议(占位实现)。", - "safety_flags": [], - }, - } - - -def handle_run(msg: Dict[str, Any]) -> None: - request_id = msg.get("id") or str(uuid.uuid4()) - payload = msg.get("payload", {}) or {} - stream = bool(msg.get("stream")) - - if stream: - send({"id": request_id, "event": "partial", "data": {"stage": "thinking", "text": "分析上下文中..."}}) - time.sleep(0.1) - send({"id": request_id, "event": "partial", "data": {"stage": "draft", "text": "生成候选建议..."}}) - time.sleep(0.1) - - final_msg = build_mock_final(request_id, payload) - send(final_msg) - - -def main() -> None: - log("Agent worker started (mock mode).") - for line in sys.stdin: - line = line.strip() - if not line: - continue - try: - msg = json.loads(line) - except json.JSONDecodeError: - log(f"Invalid JSON: {line[:80]}") - continue - - msg_type = msg.get("type") - if msg_type == "run": - handle_run(msg) - elif msg_type == "ping": - send({"id": msg.get("id"), "event": "pong"}) - else: - log(f"Unknown message type: {msg_type}") - - -if __name__ == "__main__": - main() diff --git a/desktop/src/core/modules/ipc-handlers.js b/desktop/src/core/modules/ipc-handlers.js index e4e86a5..a1ebec8 100644 --- a/desktop/src/core/modules/ipc-handlers.js +++ b/desktop/src/core/modules/ipc-handlers.js @@ -1,9 +1,7 @@ import electron from 'electron'; -import { randomUUID } from 'crypto'; import DatabaseManager from '../../db/database.js'; import ASRManager from '../../asr/asr-manager.js'; import ASRModelManager from '../../asr/model-manager.js'; -import AgentService from '../../agent/agent-service.js'; const { ipcMain, systemPreferences } = electron; @@ -19,7 +17,6 @@ export class IPCManager { this.asrModelPreloading = false; this.asrModelPreloaded = false; this.asrServerCrashCallback = null; - this.agentService = null; } /** @@ -65,7 +62,6 @@ export class IPCManager { this.setupWindowHandlers(); this.setupDatabaseHandlers(); this.setupLLMHandlers(); - this.setupAgentHandlers(); this.setupASRModelHandlers(); this.setupASRAudioHandlers(); this.setupMediaHandlers(); @@ -432,46 +428,6 @@ export class IPCManager { console.log('LLM IPC handlers registered'); } - /** - * 设置 Agent / LLM 推理相关 IPC 处理器 - */ - setupAgentHandlers() { - ipcMain.handle('agent-run', async (event, payload = {}) => { - try { - const svc = this.getOrCreateAgentService(); - const result = await svc.run(payload, { stream: false }); - return result; - } catch (error) { - console.error('[Agent] run error:', error); - return { error: error.message || 'agent run failed' }; - } - }); - - ipcMain.handle('agent-run-stream', async (event, payload = {}) => { - const requestId = payload.requestId || randomUUID(); - try { - const svc = this.getOrCreateAgentService(); - await svc.run(payload, { - stream: true, - requestId, - onStream: (chunk) => { - try { - event.sender.send('agent-stream', { requestId, ...chunk }); - } catch (err) { - console.error('[Agent] failed to forward stream chunk:', err); - } - } - }); - return { requestId }; - } catch (error) { - console.error('[Agent] stream error:', error); - return { requestId, error: error.message || 'agent stream failed' }; - } - }); - - console.log('Agent IPC handlers registered'); - } - /** * 设置 ASR 模型管理相关 IPC 处理器 */ @@ -822,16 +778,6 @@ export class IPCManager { return this.asrManager; } - /** - * 获取或创建 Agent 服务 - */ - getOrCreateAgentService() { - if (!this.agentService) { - this.agentService = new AgentService(); - } - return this.agentService; - } - /** * 检查 ASR 模型是否就绪 */ @@ -926,13 +872,5 @@ export class IPCManager { console.error('Error destroying ASR manager:', error); } } - - if (this.agentService) { - try { - this.agentService.destroy(); - } catch (error) { - console.error('Error destroying Agent service:', error); - } - } } -} +} \ No newline at end of file diff --git a/desktop/src/preload.js b/desktop/src/preload.js index 0d1f466..599791d 100644 --- a/desktop/src/preload.js +++ b/desktop/src/preload.js @@ -78,15 +78,6 @@ contextBridge.exposeInMainWorld('electronAPI', { testLLMConnection: (configData) => ipcRenderer.invoke('llm-test-connection', configData), setDefaultLLMConfig: (id) => ipcRenderer.invoke('llm-set-default-config', id), - // Agent / LLM 推理 - agentRun: (payload) => ipcRenderer.invoke('agent-run', payload), - agentRunStream: (payload) => ipcRenderer.invoke('agent-run-stream', payload), - onAgentStream: (callback) => { - const listener = (event, data) => callback(data); - ipcRenderer.on('agent-stream', listener); - return () => ipcRenderer.removeListener('agent-stream', listener); - }, - // ASR(语音识别)API asrInitialize: (conversationId) => ipcRenderer.invoke('asr-initialize', conversationId), asrCheckReady: () => ipcRenderer.invoke('asr-check-ready'),