diff --git a/test/feature/README.md b/test/feature/README.md new file mode 100644 index 00000000..267cdb68 --- /dev/null +++ b/test/feature/README.md @@ -0,0 +1,158 @@ +# Feature Testing Examples + +## πŸ“‹ Overview + +This folder contains integration test examples that focus on product features. These tests verify complete user scenarios rather than internal implementation details. + +## 🎯 Why Feature Tests? + +Feature tests deliver higher ROI: + +### Feature Tests vs Unit Tests + +| Characteristic | Unit Tests | Feature Tests | +|-----|---------|---------| +| **Scope** | Single function/component | Full user scenario | +| **Number of tests** | Many (one per function) | Few (one per feature) | +| **Maintenance cost** | High (heavy churn during refactors) | Low (stable as long as behavior stays) | +| **Execution speed** | Fast | Relatively slower | +| **Bug discovery** | Internal logic issues | Real user experience issues | +| **Refactor friendliness** | Low | High | + +### Example Comparison + +**Unit-test style** (needs multiple tests): +```typescript +// ❌ Requires dedicated tests per function +it('handleInputChange should update state', ...) +it('validateMessage should reject empty input', ...) +it('sendMessage should call the API', ...) +it('clearInput should reset the field', ...) +``` + +**Feature-test style** (one test covers the flow): +```typescript +// βœ… One test verifies the full flow +it('user can type and send a message', () => { + // User types text + // Clicks the send button + // Validates the message appears in the UI + // Confirms the input box is cleared +}) +``` + +## βœ… Testing Best Practices + +### 1. Assert user-facing behavior, not implementation details + +```typescript +// ❌ Wrong: assert internal state +expect(component.state.messages).toHaveLength(1) + +// βœ… Correct: assert what the user sees +expect(screen.getByText('Hello')).toBeInTheDocument() +``` + +### 2. Query only what a user can perceive + +```typescript +// ❌ Wrong: use test IDs +screen.getByTestId('message-list') + +// βœ… Correct: use roles or visible text +screen.getByRole('list') +screen.getByText('Messages') +``` + +### 3. Cover the entire workflow + +```typescript +// ❌ Wrong: test each handler separately +it('handleInput works', ...) +it('handleSubmit works', ...) + +// βœ… Correct: cover the whole user scenario +it('user can type and send a message', ...) +``` + +### 4. Use descriptive test names + +```typescript +// ❌ Wrong: vague names +it('test 1', ...) +it('works', ...) + +// βœ… Correct: describe expected behavior +it('disables send button when the input is empty', ...) +it('clears the input after sending a message', ...) +``` + +### 5. Avoid excessive mocking + +```typescript +// ❌ Wrong: mock everything +vi.mock('./MessageList') +vi.mock('./InputBox') +vi.mock('./SendButton') + +// βœ… Correct: mock only external dependencies +vi.mock('@/api/http') // API calls +vi.mock('electron') // Electron APIs +// Let other components run normally +``` + +## πŸ’‘ Test Strategy Guidance + +Consider this split: + +1. **Core features** (80% effort) – cover with feature tests + - User sign-in/sign-up + - Message sending + - File upload + - Task management + +2. **Utility helpers** (15% effort) – cover with unit tests + - Data formatting + - Validation helpers + - Calculation helpers + +3. **Edge cases** (5% effort) – add as needed + - Extreme inputs + - Concurrency scenarios + - Performance tests + +## ❓ FAQ + +### Q: How do I debug a failing feature test? + +A: +1. Inspect the test output to identify the failing assertion +2. Call `screen.debug()` to print the current DOM +3. Check whether you need `waitFor` for pending async work +4. Gradually simplify the test until you isolate the minimal failing scenario + +### Q: What if the tests run too slowly? + +A: +1. Use `npm run test:watch` to run only changed tests +2. Temporarily focus with `it.only` +3. Review any unnecessary `waitFor` timeouts +4. Consider splitting very large tests into smaller ones + +### Q: How do I test flows that require authentication? + +A: +1. Mock the logged-in state in `beforeEach` +2. Stub `authStore` to return an authenticated user +3. Or create a `setupLoggedInUser()` helper + +## πŸ“ Takeaway + +Keep this principle in mind: + +> **Tests should exercise your app the way a user would** +> +> If a test must understand internal implementation, it is likely over-specified. +> +> Focus on what users can see and do. + diff --git a/test/feature/SendFirstMessage.feature.test.tsx b/test/feature/SendFirstMessage.feature.test.tsx new file mode 100644 index 00000000..d793c743 --- /dev/null +++ b/test/feature/SendFirstMessage.feature.test.tsx @@ -0,0 +1,341 @@ +/** + * Feature Testing Sample Documentation + * + * ============================================================================ + * What is Integration/Feature Testing? + * ============================================================================ + * + * Feature tests focus on what users can see and do, rather than how the code is implemented. + * + * ## Feature Tests vs Unit Tests + * + * ### Unit Tests + * - Validate the internal logic of a single function or component + * - Example: ensure `calculateTotal(price, quantity)` returns the correct product + * - Pros: fast, isolated, precise failure signals + * - Cons: cannot guarantee the entire feature works correctly + * + * ### Feature Tests + * - Validate end-to-end user scenarios + * - Example: β€œuser enters price and quantity, clicks Calculate, and sees the total” + * - Pros: mirrors real usage, one test covers multiple code paths + * - Cons: comparatively slower, failures take longer to debug + * + * ## Why lean on feature tests? + * + * Feature tests deliver higher ROI: + * + * 1. **Fewer tests overall**: one feature test can replace several unit tests + * 2. **Refactor friendly**: internal changes rarely require test updates + * 3. **Higher confidence**: confirms real user journeys keep working + * 4. **Lower maintenance**: fewer tests means less upkeep + * + * ============================================================================ + * Below is a feature-test example + * ============================================================================ + */ + +import { describe, it, expect, beforeEach, vi, afterEach } from 'vitest' +import { render, screen, renderHook, waitFor, act } from '@testing-library/react' +import { BrowserRouter } from 'react-router-dom' +import ChatBox from '../../src/components/ChatBox' +// Import necessary mocks +import '../mocks/proxy.mock' +import '../mocks/authStore.mock' +import '../mocks/sse.mock' + +import { useProjectStore } from '../../src/store/projectStore' +import useChatStoreAdapter from '../../src/hooks/useChatStoreAdapter' +// Mock Electron IPC +(global as any).ipcRenderer = { + invoke: vi.fn((channel) => { + if (channel === 'get-system-language') return Promise.resolve('en') + if (channel === 'get-browser-port') return Promise.resolve(9222) + if (channel === 'get-env-path') return Promise.resolve('/path/to/env') + if (channel === 'mcp-list') return Promise.resolve({}) + if (channel === 'get-file-list') return Promise.resolve([]) + return Promise.resolve() + }), +} + +// Mock window.electronAPI +Object.defineProperty(window, 'electronAPI', { + value: { + uploadLog: vi.fn().mockResolvedValue(undefined), + selectFile: vi.fn().mockResolvedValue({ success: false }), + }, + writable: true, +}) + +const TestWrapper = ({ children }: { children: React.ReactNode }) => ( + {children} +) + +describe('Feature test example: chat experience', () => { + /** + * beforeEach runs before every spec + * Purpose: reset application state so each test starts clean + */ + beforeEach(() => { + vi.clearAllMocks() + + // Reset the project store + const projectStore = useProjectStore.getState() + projectStore.getAllProjects().forEach(project => { + projectStore.removeProject(project.id) + }) + + // Seed an initial project (mirrors the state when the app boots) + const projectId = projectStore.createProject( + 'Feature Test Project', + 'Testing user message flow' + ) + expect(projectId).toBeDefined() + }) + + afterEach(() => { + vi.clearAllMocks() + }) + + /** + * Test 1: verify the initial UI + * + * This spec ensures: + * - Users see the welcome copy on launch + * - The input field renders correctly + * + * Acts as a smoke test for the base layout. + */ + it('displays the welcome screen and input', async () => { + // 1. Render the component (akin to opening the app) + render( + + + + ) + + // 2. Assert the welcome text is visible + expect(screen.getByText(/layout.welcome-to-eigent/i)).toBeInTheDocument() + expect(screen.getByText(/layout.how-can-i-help-you/i)).toBeInTheDocument() + + // 3. Confirm the textarea exists so the user can type + const textarea = screen.getByPlaceholderText('chat.ask-placeholder') + expect(textarea).toBeInTheDocument() + }) + + /** + * Test 2: validate the send button state + * + * This spec asserts: + * - When the input is empty, the send button stays disabled to block empty submissions + * + * Captures a critical UX behavior. + */ + it('disables the send button when the input is empty', async () => { + // 1. Render the component + render( + + + + ) + + // 2. Find the send button (via its icon) + const buttons = screen.getAllByRole('button') + const sendButton = buttons.find(btn => + btn.querySelector('svg.lucide-arrow-right') + ) + + expect(sendButton).toBeInTheDocument() + + // 3. Assert the button is disabled + // Note: we do not care why it is disabled (privacy gate, empty input, etc.); + // we only care that the observable behavior matches expectations. + expect(sendButton).toBeDisabled() + }) + + /** + * Test 3: verify legal links + * + * This spec ensures: + * - Privacy Policy and Terms of Use links render + * - Each link points to the correct URL + * - Each link opens in a new tab + */ + it('shows Terms of Use and Privacy Policy links', async () => { + render( + + + + ) + + // Locate the anchor elements + const termsLink = screen.getByRole('link', { name: /layout.terms-of-use/i }) + const privacyLink = screen.getByRole('link', { name: /layout.privacy-policy/i }) + + // Verify link attributes + expect(termsLink).toBeInTheDocument() + expect(termsLink).toHaveAttribute('href', 'https://www.eigent.ai/terms-of-use') + expect(termsLink).toHaveAttribute('target', '_blank') + + expect(privacyLink).toBeInTheDocument() + expect(privacyLink).toHaveAttribute('href', 'https://www.eigent.ai/privacy-policy') + expect(privacyLink).toHaveAttribute('target', '_blank') + }) + + /** + * Test 4: TaskPlanning journey + * + * This spec validates the complete user workflow from sending a message to task breakdown: + * - User sends a message + * - System displays the user message + * - System splits the task into subtasks (task planning phase) + * - Task summary and subtasks are displayed to the user + * + * This test covers the core message-send and task-splitting workflow. + */ + it('processes TaskPlanning journey with task splitting and subtask display', async () => { + // 1. Get the chat store using the adapter hook + const { result, rerender: rerenderHook } = renderHook(() => useChatStoreAdapter()) + const { chatStore } = result.current + + if (!chatStore) { + throw new Error('ChatStore is null') + } + + const taskId = chatStore.activeTaskId + expect(taskId).toBeDefined() + + // 2. Render the component + render( + + + + ) + + // 3. Simulate user sending a message + const userMessage = 'Create a simple todo list application' + + await act(async () => { + chatStore.setHasMessages(taskId!, true) + chatStore.addMessages(taskId!, { + id: 'user-msg-1', + role: 'user', + content: userMessage, + attaches: [] + }) + rerenderHook() + }) + + // 4. Verify user message appears in the UI + await waitFor(() => { + expect(screen.getByText(userMessage)).toBeInTheDocument() + }) + + // 5. Simulate task splitting phase (to_sub_tasks SSE event) + await act(async () => { + chatStore.setSummaryTask(taskId!, 'Todo List Application|Create a simple todo list application') + chatStore.addMessages(taskId!, { + id: 'to-sub-tasks-msg', + role: 'assistant', + content: '', + step: 'to_sub_tasks', + data: { + summary_task: 'Todo List Application|Create a simple todo list application', + sub_tasks: [ + { + id: 'subtask-1', + content: 'Create HTML structure for todo list', + status: '', + subtasks: [] + }, + { + id: 'subtask-2', + content: 'Implement JavaScript functionality', + status: '', + subtasks: [] + }, + { + id: 'subtask-3', + content: 'Add CSS styling', + status: '', + subtasks: [] + } + ] + } + }) + chatStore.setTaskInfo(taskId!, [ + { id: 'subtask-1', content: 'Create HTML structure for todo list', status: '' }, + { id: 'subtask-2', content: 'Implement JavaScript functionality', status: '' }, + { id: 'subtask-3', content: 'Add CSS styling', status: '' } + ]) + rerenderHook() + }) + + // 6. Verify task summary and subtasks appear in the UI + await waitFor(() => { + expect(screen.getByText('Todo List Application')).toBeInTheDocument() + expect(screen.getByText('Create HTML structure for todo list')).toBeInTheDocument() + expect(screen.getByText('Implement JavaScript functionality')).toBeInTheDocument() + expect(screen.getByText('Add CSS styling')).toBeInTheDocument() + }) + + // 7. Verify chatStore state is correct by getting fresh state + const updatedChatStore = result.current.chatStore + expect(updatedChatStore!.tasks[taskId!].summaryTask).toContain('Todo List Application') + expect(updatedChatStore!.tasks[taskId!].taskInfo).toHaveLength(3) + }) +}) + +/** + * ============================================================================ + * Testing best-practice recap + * ============================================================================ + * + * 1. **Exercise user behavior, not implementation details** + * ❌ Wrong: expect(component.state.messages).toHaveLength(1) + * βœ… Correct: expect(screen.getByText('Hello')).toBeInTheDocument() + * + * 2. **Query elements the way users perceive them** + * ❌ Wrong: screen.getByTestId('message-list') + * βœ… Correct: screen.getByText('Messages') or screen.getByRole('list') + * + * 3. **Assert full user flows** + * ❌ Wrong: test handleInput, handleSubmit, addMessage separately + * βœ… Correct: test the flow β€œuser types a message and sends it” + * + * 4. **Pick descriptive test names** + * ❌ Wrong: it('test 1', ...) + * βœ… Correct: it('disables the send button when the input is empty', ...) + * + * 5. **Avoid over-mocking** + * - Mock only external dependencies (APIs, Electron APIs) + * - Keep application functions/components real + * - Let as much code run as possible + * + * ============================================================================ + * How to extend these tests + * ============================================================================ + * + * Suggested follow-up feature tests: + * + * 1. Full message-send journey + * - User types copy + * - Clicks the send button or presses Ctrl+Enter + * - Message appears in the chat history + * - Input resets to empty + * + * 2. File upload flow + * - User clicks the attachment button + * - Chooses a file + * - File appears in the attachment list + * + * 3. Error-handling path + * - Simulate an API error + * - Confirm the user-facing error surface renders + * + * 4. Task state transitions + * - Task moves from pending to running + * - UI reflects the correct state changes + * + * Remember: each spec should cover a complete user scenario. + */ diff --git a/test/feature/TODO.md b/test/feature/TODO.md new file mode 100644 index 00000000..44f62c3b --- /dev/null +++ b/test/feature/TODO.md @@ -0,0 +1,98 @@ +# Feature Test TODO + +## Overview + +This document tracks the feature tests needed to cover core user journeys. Feature tests validate complete user scenarios rather than implementation details, providing high ROI with fewer tests. + +--- + +## P0 - Critical User Flows + +Must be covered. These are the core paths every user takes. + +| # | Feature | User Journey | Key Files | +|---|---------|--------------|-----------| +| 1 | **Login Flow** | User enters email/password β†’ Validates credentials β†’ Redirects to home β†’ Displays user info | `Login.tsx`, `authStore.ts` | +| 2 | **Sign Up Flow** | User fills registration form β†’ Creates account β†’ Auto-login β†’ Redirects to home | `SignUp.tsx`, `authStore.ts` | +| 3 | **Send Message Flow** | User types message β†’ Clicks send β†’ User message appears β†’ AI response streams in β†’ Task completes | `ChatBox/`, `chatStore.ts` | +| 4 | **Task Confirmation Flow** | System splits task β†’ Shows subtasks β†’ User confirms/edits β†’ Execution begins | `ChatBox/`, `chatStore.ts` | + +**Status:** +- [ ] Login Flow +- [ ] Sign Up Flow +- [x] Send Message Flow (partial - `SendFirstMessage.feature.test.tsx`) +- [x] Task Confirmation Flow (partial - `SendFirstMessage.feature.test.tsx`) + +--- + +## P1 - Important Features + +Should be covered. These represent major user interactions. + +| # | Feature | User Journey | Key Files | +|---|---------|--------------|-----------| +| 5 | **Project Switching** | User creates new project β†’ Switches between projects β†’ Messages remain independent | `projectStore.ts`, `HistorySidebar/` | +| 6 | **History Replay** | User opens history β†’ Switches view mode β†’ Selects past project β†’ Replays conversation | `History.tsx`, `projectStore.ts` | +| 7 | **Model Switching** | User switches between Cloud/Custom API/Local modes β†’ Configuration takes effect | `Setting/Models.tsx`, `authStore.ts` | +| 8 | **API Key Configuration** | User enters API key β†’ Saves β†’ Key is validated and persisted | `Setting/API.tsx`, `authStore.ts` | +| 9 | **File Attachment** | User clicks attach β†’ Selects file β†’ File appears in input β†’ Sends with message | `ChatBox/BottomBox/` | + +**Status:** +- [ ] Project Switching +- [ ] History Replay +- [ ] Model Switching +- [ ] API Key Configuration +- [ ] File Attachment + +--- + +## P2 - Secondary Features + +Nice to have. These cover less frequent but still important scenarios. + +| # | Feature | User Journey | Key Files | +|---|---------|--------------|-----------| +| 10 | **Agent Q&A Interaction** | Agent asks question β†’ User replies β†’ Execution continues | `ChatBox/`, `chatStore.ts` | +| 11 | **Task Pause/Resume** | User pauses running task β†’ Status shows paused β†’ User resumes β†’ Execution continues | `ChatBox/`, `chatStore.ts` | +| 12 | **MCP Server Config** | User adds MCP server β†’ Server appears in list β†’ User can delete it | `Setting/MCP.tsx` | +| 13 | **Worker Management** | User views worker list β†’ Configures worker settings | `Dashboard/Worker.tsx` | +| 14 | **Language/Theme Toggle** | User switches language β†’ UI updates / User switches theme β†’ Styles update | `Setting/General.tsx` | + +**Status:** +- [ ] Agent Q&A Interaction +- [ ] Task Pause/Resume +- [ ] MCP Server Config +- [ ] Worker Management +- [ ] Language/Theme Toggle + +--- + +## P3 - Edge Cases + +Optional. Cover these as time permits. + +| # | Feature | User Journey | Key Files | +|---|---------|--------------|-----------| +| 15 | **Network Disconnect** | Network drops β†’ Error message shown β†’ Reconnects β†’ Resumes normally | Global | +| 16 | **Token Expiration** | Token expires β†’ Prompts re-login β†’ Redirects to login page | `authStore.ts` | +| 17 | **Budget Exhausted** | `budget_not_enough` event β†’ Shows warning β†’ Guides user to top up | `chatStore.ts` | +| 18 | **Browser Snapshot View** | Task generates snapshot β†’ User clicks to view β†’ Snapshot content displayed | `ChatBox/` | +| 19 | **Generated File Download** | Task generates files β†’ File list shown β†’ User clicks to download | `ChatBox/` | + +**Status:** +- [ ] Network Disconnect +- [ ] Token Expiration +- [ ] Budget Exhausted +- [ ] Browser Snapshot View +- [ ] Generated File Download + +--- + +## Minimum Viable Test Set + +If resources are extremely limited, prioritize these 4 tests for ~60% core path coverage: + +1. **Login Flow** - Entry point for all users +2. **Send Message Flow** - Core product value +3. **Task Confirmation Flow** - Key user interaction +4. **Model Switching** - Critical configuration \ No newline at end of file diff --git a/test/mocks/proxy.mock.ts b/test/mocks/proxy.mock.ts index 5a52e166..03950ee6 100644 --- a/test/mocks/proxy.mock.ts +++ b/test/mocks/proxy.mock.ts @@ -34,12 +34,13 @@ const mockImplementation = { if (url.includes('/api/providers')) { return Promise.resolve({ items: [] }) } - // Mock privacy settings + // Mock privacy settings - all required fields must be true if (url.includes('/api/user/privacy')) { return Promise.resolve({ - dataCollection: true, - analytics: true, - marketing: true + take_screenshot: true, + access_local_software: true, + access_your_address: true, + password_storage: true }) } // Mock configs