Expanding the lab - #5
Conversation
…ed `reviews` branch
- Implemented OpenAI backend in `openai.py` with session creation and API key resolution. - Developed vLLM backend in `vllm.py` supporting managed mode, process management, and model resolution. - Introduced `FileLogger` class in `log_file.py` for logging backend activities. - Created transport layer in `openai_http.py` for handling API requests and responses. - Added utility functions for URL normalization and message sanitization. - Enhanced error handling and logging throughout the backend processes.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aa510e29e1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if early_backend == "openai" and not pre_args.model_id: | ||
| # Attach mode should not inherit a local model path from config defaults. | ||
| # Let the OpenAI-compatible session resolve /v1/models unless the user | ||
| # explicitly provided a model id. | ||
| args.model_id = "" |
There was a problem hiding this comment.
For --backend openai runs started via --config, this branch unconditionally wipes the already-loaded model.id unless the user also passed a positional model id. That regresses the normal per-model OpenAI workflow (models/*/openai/config/default.toml): multi-model servers will silently fall back to resolve_model_once(), and providers that require an explicit model will start failing even though the config specified one.
Useful? React with 👍 / 👎.
| payload: dict[str, Any] = { | ||
| "session_id": self.session_id, | ||
| "turn_id": int(turn_id), | ||
| "started_at": _as_iso(timing.get("start")), | ||
| "ended_at": _as_iso(timing.get("end")), | ||
| "streaming_enabled": True, |
There was a problem hiding this comment.
streaming_enabled from the actual turn mode
This now hard-codes every turn_finished event as streaming_enabled: true, but the CLI still exposes --no-stream and the HF backend still has a separate non-stream path. Any benchmarking or dashboard logic built on the new telemetry will therefore misclassify non-stream turns as streamed runs, which makes backend comparisons inaccurate.
Useful? React with 👍 / 👎.
| runtime_device = str(info.get("runtime_device") or "") | ||
| try: | ||
| import torch # type: ignore | ||
|
|
||
| cuda_index = None | ||
| if runtime_device.startswith("cuda:"): | ||
| cuda_index = int(runtime_device.split(":", 1)[1]) | ||
| elif runtime_device == "cuda" and torch.cuda.is_available(): | ||
| cuda_index = int(torch.cuda.current_device()) | ||
| if cuda_index is not None and torch.cuda.is_available(): |
There was a problem hiding this comment.
When runtime sampling is enabled for managed vLLM, this code only collects GPU metrics if session.describe() exposes runtime_device. VLLMSession.describe() never sets that field, so every vLLM sample falls through to _finalize_gpu_group(... runtime_device='') and reports session_not_running_on_cuda with no VRAM numbers, even when the server is actually using the GPU. That makes the new telemetry/dashboard path systematically wrong for vLLM sessions.
Useful? React with 👍 / 👎.
- Replace pip-based virtualenv workflow with uv for faster, more reliable dependency resolution - Update README with new uv sync commands and backend group installation instructions - Add uv-specific dependency groups for optional backends (gguf, exl2) - Update all Python execution paths to use .venv/bin/python consistently - Remove outdated pip install instructions and replace with uv sync commands - Maintain compatibility with existing backend configurations while modernizing the build system
- Update bin/tui script to configure LD_LIBRARY_PATH for PyTorch and NVIDIA libraries - Add CUDA environment variables (CUDA_HOME, CUDA_PATH, CUDACXX, NVCC_CCBIN) when available - Improve model cache resolution in config_utils.py to handle full HF paths and deduplicate candidates - Fix EXL2 backend model path resolution to respect exl2_model_root - Add Angelic_Eclipse_12B_FP8 model configuration for vllm backend - Update uv.lock with CUDA 13.0 compatible PyTorch packages
-added tts to the tui -connect to custom kokoro fast api endpoints for tts -added slash commands in the tui
Introduces functionality to discover and list running agent sessions managed by `agent-session-srv`. This is achieved by scanning session directories, loading state files, and validating connectivity via HTTP requests. Also adds comprehensive command-line arguments for Speech-to-Text (STT) configuration, including enabling/disabling STT, setting base URLs, models, and languages.
1 participant
failed telemetry loop iteration.