[codex] Add CLI workflow process E2E#316
Closed
0xroylee wants to merge 514 commits into
Closed
Conversation
* Introduced a new polling status API endpoint at `/api/polling/status`. * Implemented a `PollingMonitorPanel` component to display polling health and operational events. * Enhanced the database schema to store polling status and events. * Added tests for the polling status client and database schema. * Updated the real-time client to handle polling events and invalidate queries accordingly. * Refactored existing components to integrate the new polling features.
…ling * Deleted the internal task polling and linear task polling implementations to streamline the codebase. * Removed associated types and tests to ensure consistency. * Updated task routes to handle daemon task-changed notifications. * Adjusted server initialization to remove polling scheduler setup. * Enhanced API documentation for clarity on browser realtime updates.
…cy task-changed route * Introduced `/daemon/events` WebSocket for task mutation notifications. * Updated client and server to handle task changes via WebSocket instead of HTTP POST. * Removed deprecated `/api/internal/daemon/task-changed` route. * Enhanced documentation to reflect new event handling architecture.
* Replaced `bun run ./packages/cli/src/index.ts` with `npx devos` in daemon command execution. * Introduced `buildCliCommandDaemonExecutorOptions` for better command configuration. * Updated tests to reflect changes in command execution. * Added `test-paths.ts` for improved path management in tests.
- Introduced `--poll-forever` and `--all-projects` options for the CLI daemon. - Refactored daemon command handling to support attached workflow polling. - Added progress emission for agent actions and workflow cycles. - Implemented validation for daemon port conflicts and improved error handling. - Enhanced tests to cover new polling features and daemon command behaviors.
…egration * Introduced a new module for managing workflow data, including task creation, updates, and polling. * Implemented WebSocket support for real-time communication regarding task events and polling status. * Added tests to ensure the functionality of the new workflow data features and WebSocket interactions. * Updated existing tests to accommodate changes in the database schema and ensure compatibility with new features.
* Moved `createWorkflowDataClient` import to the correct position in `board-task-creator.ts`. * Adjusted function signatures for better readability in multiple files. * Enhanced formatting in test files for consistency and clarity.
* Changed references from "Linear" to "task" in error messages and logging for consistency. * Improved clarity in comments and logging related to task management.
* Updated task titles for clarity and consistency. * Improved agent representation in task cards by replacing "none" with "-". * Enhanced user instructions and descriptions for better understanding. * Introduced a new skills tree feature with detailed skill descriptions and examples. * Added Telegram integration for real-time messaging and task updates. * Refactored code for better readability and maintainability.
* Introduced a new endpoint `/api/tasks/{id}/activity` to retrieve task activity timelines.
* Updated `fetchWork` method to accept options for including unprojected tasks.
* Enhanced tests to cover new functionality and ensure proper behavior of task lifecycle.
* Introduced a checklist for dependency updates and Bun checks. * Added notes on using Bun for strict CI and lockfile management.
* Added `bun-version: 1.3.8` to the Bun setup step in the CI workflow. * Ensures consistent environment setup for builds. feat(docs): ✨ Add operator-first roadmap documentation * Created `operator-first-roadmap.md` to outline the product direction and implementation checkpoints. * Provides clarity on the roadmap for stakeholders. feat(cli): ✨ Implement resolvePglitePackageEntry function * Added `resolvePglitePackageEntry` to resolve PGlite from the server database boundary. * Enhances the build process by ensuring correct asset resolution. test(cli): ✨ Add tests for PGlite resolution * Implemented tests for `resolvePglitePackageEntry` to verify correct behavior. * Ensures reliability of the CLI build script.
* Refactored the main CLI entry point to handle command parsing errors using `CommanderError`. * Enhanced the `parseArgs` function to throw specific errors for invalid command usage. * Introduced new tests for various commands including `run`, `skills`, and `task` to ensure proper argument parsing and error handling. * Added a new daemon readiness test suite to verify the behavior of the CLI daemon during startup and shutdown. * Updated the development script to include the `dev:daemon` command for easier local testing.
…ing process * Introduced `instance-config.ts` and `instance-config.types.ts` for managing instance configurations. * Updated onboarding steps to include writing `.devos/config/instance.config.json`. * Enhanced `writeSetupFiles` function to handle instance configuration file creation. * Updated `runSetupWizard` to reflect new instance configuration in output messages. * Added tests for rendering instance configuration and verifying its structure.
- Added `@clack/prompts` and `@clack/core` dependencies to enhance prompt handling. - Implemented a `PromptAdapter` to standardize prompt interactions across the CLI. - Created a new `setup-draft` feature to collect user input for project setup. - Refactored existing setup logic to utilize the new prompt system, improving user experience. - Added tests for the new prompt adapter to ensure reliability and correctness.
* Introduced a new test suite for database scripts, covering migration, seeding, and backup functionalities. * Refactored imports across the server package to utilize the new `devos-db` module instead of local db files. * Updated TypeScript configuration for the db package to ensure compatibility with ES2022 and Bun. * Adjusted package.json files to reflect the new structure and dependencies. * Enhanced test coverage for various components, ensuring that database interactions are thoroughly validated.
…ion validation * Introduced `collectConfigFileCheck` to validate the presence and accessibility of configuration files. * Implemented `collectInstanceSetupChecks` to verify instance configuration, including storage, database, and server port checks. * Added support for LLM provider checks to report configured backends. * Enhanced error handling for missing or malformed instance configurations. * Updated setup wizard to run doctor checks post-setup and report results clearly. * Improved test coverage for setup checks, ensuring all edge cases are handled.
…nment handling * Introduced a new configuration file for project setup. * Updated environment file handling to exclude sensitive information. * Refactored setup draft collection to streamline onboarding prompts. * Added tests to ensure proper loading of configuration values. * Removed deprecated status prompt functionality.
* Removed unnecessary repository path prompts. * Simplified `executionPath` resolution in `collectSetupDraft`. * Updated tests to reflect changes in path handling.
* feat(api): ✨ Enhance WorkspaceProjectRecord with additional fields * Added new fields: `repoOwner`, `repoName`, `baseBranch`, `localFolder`, `lead`, `category`, and `priority` to `WorkspaceProjectRecord`. * Introduced `readNullableNumber` utility function for handling nullable numbers in API responses. * Updated `parseWorkspaceProjectRecord` and `parseProject` functions to accommodate new fields. * feat(types): ✨ Add additional fields to project and board interfaces * Introduced new fields: `repoOwner`, `repoName`, `baseBranch`, `localFolder`, `lead`, `category`, and `priority` to various project and board-related interfaces. * Updated related functions and schemas to accommodate these new fields. * Enhanced tests to validate the new structure and ensure backward compatibility. * feat(database): ✨ Add project metadata fields and update migrations * Introduced new columns in `board_projects` for `repoOwner`, `repoName`, `baseBranch`, `localFolder`, `lead`, `category`, and `priority`. * Updated migration scripts to include the new metadata structure. * Modified `seed` and `schema` files to accommodate the new fields. * Adjusted tests to validate the presence of new fields in the database schema. * test: 🧪 refactor tests to use `captureWithRuntime` for CLI commands - Updated tests in `args-skills.test.ts` and `args-task.test.ts` to replace `parseArgs` with `captureWithRuntime` for better runtime command capturing. - Refactored error handling in `args-test-helpers.ts` to support the new runtime capturing method. - Adjusted assertions in various tests to validate the captured command calls and their payloads. - Enhanced test coverage for CLI commands including `skills`, `task`, and core commands like `status` and `daemon`. - Improved clarity and maintainability of test cases by aligning them with the new command execution model. * feat(codex): ✨ Add Codex agent adapter with configuration and runtime support * Introduced `CodexAdapter` for handling Codex-specific operations. * Added configuration documentation and constants for Codex. * Implemented runtime invocation logic for executing Codex commands in Docker. * Created utility functions for normalizing input and output data. * Added tests for agent creation and configuration validation. * Enhanced error handling for unknown backends and models.
* Add Cursor agent adapter * Keep Cursor adapter helpers separate * Address Cursor adapter review findings --------- Co-authored-by: John Ku <johnku@JK-2025-MacBook-Pro.local>
* feat(logger): 🎉 Replace `pino` with custom logger implementation - Introduced a new logging system using `picocolors` for colored output. - Removed dependencies on `pino` and `pino-pretty`. - Implemented a custom logger with methods for `info`, `warn`, `error`, and `fatal`. - Added support for context-aware logging, allowing additional fields to be included in log messages. - Created tests to validate logging functionality and ensure proper formatting of log messages. * feat(workflow): ✨ implement reliable workflow data client and execution logging * Introduced `createReliableWorkflowDataClient` to handle buffered workflow actions. * Added `drainProjectWorkflowDataOutbox` for processing buffered actions. * Implemented `WorkflowExecutionRecorder` for tracking task execution status and logs. * Enhanced workflow data types to include task execution actions and statuses. * Updated workflow data service to handle new task execution actions. * Added tests for workflow data outbox functionality and task execution logging. * feat(setup): ✨ Refactor setup draft and enhance project creation * Updated `SetupDraft` interface to replace project-related fields with workspace-specific fields. * Added tests to ensure proper handling of empty project lists and legacy configurations. * Implemented functionality to load web-created projects from the server database when config projects are empty. * Introduced a new `ProjectsPanel` component for managing projects in the UI. * Created utility functions for building project creation requests and managing form state. * Added comprehensive tests for project creation and mutation handling. This refactor aims to streamline the setup process and improve the user experience when managing projects. * feat(daemon): 🎉 Implement daemon startup messages and URL resolution * Added `renderProductionDaemonStartup` and `renderCliOnlyDaemonStartup` functions for improved CLI output during daemon startup. * Introduced `resolveServerBaseUrl`, `resolveWebUrl`, and WebSocket URL resolution functions for better URL management. * Enhanced `renderDevosBanner` with colorization for improved visual output. * Updated tests to verify new output formats and ensure proper functionality. * feat(config): 🎉 Enhance LoadedConfig with server configuration * Added `server` property to `LoadedConfig` interface. * Implemented `resolveRootServerConfig` and `resolveServerDatabasePathForRoot` functions for server configuration resolution. * Introduced `normalizeOptionalPath` utility for path normalization. * Updated related tests to validate server configuration handling. * chore(server): ✨ refactor logging and request handling * Removed the old `withRequestLogging` function and replaced it with a new `createExpressRequestLogger` that utilizes `pino-http` for improved logging. * Updated the logger implementation to use `pino` with pretty-printing for better readability. * Adjusted the `app.ts` and `express-server.ts` files to remove dependencies on the old logger and integrate the new request logger. * Enhanced test cases to validate logging behavior for successful and error responses. * Cleaned up unused imports and types related to the old logging system. * chore(logger): ✨ replace `pino` with `pino-http` for improved logging * Updated logger implementation to utilize `pino-http`. * Removed `pino` dependency from `package.json` and `bun.lock`. * Enhanced logging capabilities with custom timestamp formatting. * feat(daemon): 🎉 replace `npx devos run` with `bun run` for workflow polling * Introduced `buildWorkflowPollerInvocation` function to encapsulate command and arguments. * Updated `startAttachedWorkflowPoller` to use the new invocation structure. * Modified tests to reflect changes in command execution for the workflow-poller. * chore(docs): ✨ Update AGENTS.md files for improved clarity and structure * Enhanced documentation across multiple packages including `cli`, `server`, `db`, `landing`, and `web`. * Added new sections for `db` and `landing` packages to define ownership rules and testing guidelines. * Clarified existing rules and instructions to ensure better understanding of package responsibilities and boundaries. * chore(adapters): ✨ Refactor agent adapter imports and remove unused files * Updated imports to use the `adapters` workspace package. * Removed obsolete adapter files for `claude`, `codex`, and related integrations.
…figuration (#182) * Introduced `Server DB Readiness Recovery` section in `RELIABILITY.md` for handling PGlite startup failures. * Added `loadServerStartupConfig` function to load configuration without applying database project metadata. * Implemented `resolveDaemonWorkspaceEnv` to manage workspace paths for daemon processes. * Updated `buildDaemonCommands` and `buildAttachedPollerEnv` to utilize workspace environment. * Added tests for `loadServerStartupConfig` and `daemon workspace environment` to ensure proper functionality.
* feat(config): ✨ Add server DB readiness recovery and enhance load configuration * Introduced `Server DB Readiness Recovery` section in `RELIABILITY.md` for handling PGlite startup failures. * Added `loadServerStartupConfig` function to load configuration without applying database project metadata. * Implemented `resolveDaemonWorkspaceEnv` to manage workspace paths for daemon processes. * Updated `buildDaemonCommands` and `buildAttachedPollerEnv` to utilize workspace environment. * Added tests for `loadServerStartupConfig` and `daemon workspace environment` to ensure proper functionality. * chore(env): 🔧 Remove Linear and GitHub configuration variables * Cleaned up the `.env.example` file by removing unused Linear and GitHub settings. * This helps streamline the configuration for users and reduces clutter. * chore(env): 🔧 Remove unused workspace and execution path variables * Cleaned up the environment configuration by removing `PIV_WORKSPACE_PATH` and `PIV_EXECUTION_PATH` as they are not utilized.
…r config * Introduced `--ignore-user-config` flag in `buildCodexExecArgs` and `buildCodexResumeArgs` functions. * Added `normalizeCodexModel` function to handle deprecated models. * Updated default models to use `gpt-5.5` and removed references to deprecated models in tests. * Created new test suite for codex model compatibility to ensure legacy models are mapped correctly.
* [devos] WOR-63: Show unread state for updated unseen sessions * [devos] WOR-63: address review feedback * [devos] WOR-63: address review feedback
* docs: add chat session agent bubbles design * Refine chat session agent activity surfaces * Fix backend project service typecheck
…activities (#310) * [devos] WOR-72: Hide raw JSON command structures in task details and activities * [devos] WOR-72: address review feedback * [devos] WOR-72: address review feedback
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Changed
devos runentrypoint against a fake Codex binary and local workflow server.Why
plan -> implement -> testingcompletion.How to Test
bun run checkbun run typecheckbun testbun run test:e2eExpected outcome: all checks pass; the CLI-process E2E leaves the task
in_review, records workflow state asdone, and captures the dry-run PR URL.Additional Notes