feat: Phase 0-1-2-3 — AI content tracking, approval gateway, multi-agent orchestration#3757
Open
haJ1t wants to merge 39 commits intohp-apr-13-create-risk-agentsfrom
Open
feat: Phase 0-1-2-3 — AI content tracking, approval gateway, multi-agent orchestration#3757haJ1t wants to merge 39 commits intohp-apr-13-create-risk-agentsfrom
haJ1t wants to merge 39 commits intohp-apr-13-create-risk-agentsfrom
Conversation
…s, agent foundation Phase 0 shared infrastructure for Evidence Agent, Control Assessment Agent, and AI Content Badge features (#3596): Database: - Migration 20260325161242: creates evidence_ai_analysis, control_readiness_scores, framework_readiness_scores, ai_content_metadata tables with indexes in verifywise schema Backend interfaces: - i.evidenceAi.ts: IEvidenceAiAnalysis, IQualityScore, ISuggestedControlLink - i.readiness.ts: IControlReadinessScore, IFrameworkReadinessScore, ReadinessLevel - i.aiContent.ts: IAIContentMetadata, BadgeType, ReviewAction Agent foundation: - agentRegistry.ts: registerAgent/getAgent/listAgents pattern - parsers/: PDF (pdf-parse) and DOCX (mammoth) text extraction with unified parseDocument interface Frontend types: - Mirror interfaces for evidenceAi, readiness, aiContent Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Full-stack implementation of the Evidence Agent that transforms the Evidence Hub from passive file storage into an active compliance intelligence system. Backend: - Evidence Agent definition with Vercel AI SDK agent registry - 4 AI tools: analyzeDocument, scoreQuality, matchControls, detectGaps - 6 REST endpoints under /api/evidence-ai/ with JWT auth - DB utils for analysis persistence, quality scores, gap detection - Document parsing via existing PDF/DOCX parsers - Integrated evidence AI tools into advisor controller Frontend: - EvidenceQualityBadge — color-coded score badge (0-100) - EvidenceAnalysisPanel — full analysis view with quality breakdown - EvidenceGapChart — coverage visualization with gap details - React Query hooks and Axios repository - Quality column integrated into Evidence Hub table Docs: - Phase 0 AI Implementation Plan with progress tracking Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…#3598) Full-stack implementation of the Control Assessment Agent that evaluates audit readiness per control and aggregates to framework level. Backend: - Readiness calculator with weighted scoring formula (5 dimensions) - Control Assessment Agent definition with Vercel AI SDK - 4 AI tools: evaluateEvidence, checkTaskCompletion, analyzeRiskStatus, generateRecommendations - 8 REST endpoints under /api/readiness/ with JWT auth - DB utils for upsert scores, aggregation queries, history - Migration for unique constraints (ON CONFLICT upsert support) Frontend: - ReadinessScoreCard — framework score with level badge and dimension breakdown - ReadinessHeatmap — controls colored by readiness level in grid view - ReadinessTrend — historical trend with bar visualization - WeakControlsList — prioritized weak controls with recommendations - ReadinessDashboard page at /readiness route - React Query hooks and Axios repository Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Full-stack implementation of the AI-Generated Content Badge system for EU AI Act Article 52 transparency compliance. Backend: - AI content tracker middleware for intercepting and recording AI tool outputs - trackAIContent() utility for programmatic badge creation - 4 REST endpoints under /api/ai-content/ (GET badges, PATCH review, GET unreviewed, GET stats) - DB utils for entity badges, review workflow, statistics aggregation - Route registration in index.ts Frontend: - AIContentBadge — 3 variants (inline, tooltip, card) with 4 badge types: Generated (purple), Assisted (blue), Reviewed (green), Suggested (amber) - AIContentReviewPanel — approve/modify/reject workflow with notes - AIContentStats — dashboard widget with review progress and type breakdown - React Query hooks and Axios repository Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove nonexistent control_id column reference from tasks queries (readiness.ctrl.ts, readinessFunctions.ts) — use org-wide task completion as proxy - Replace risk_level_after with final_risk_level in risk queries (readiness.ctrl.ts, readinessFunctions.ts) — match actual column name - Fix file_name to filename column reference in evidence queries (evidenceAi.utils.ts, evidenceAi.ctrl.ts) — match files table schema Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
[F1] Wire trackAIContent into evidence analysis and readiness calculation write paths so ai_content_metadata actually gets populated during normal use. [F2] Include project_id in unique constraints (COALESCE for NULL safety) so per-project readiness scores don't overwrite each other. Add readiness_history table for INSERT-only trend snapshots. History query now reads from this table. [F3] Scope task/risk queries per-control via file_entity_links instead of org-wide. Tasks and risks are found through shared file linkages with the control being evaluated, producing differentiated per-control scores. [F4] Fix evidence gap query to respect frameworkType parameter. Previously hard-coded eu_ai_act; now dynamically selects the correct struct table (control_category_eu_ai_act_struct or annex_category_struct_iso42001). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…kAIContent Bulgu 1 — Upgrade-safe migration strategy: - Restore 20260325183928 to its original form (org-only indexes) - Add new 20260325202908 migration that drops old indexes, creates project-aware COALESCE indexes, and readiness_history table - Works correctly whether old migration ran or not Bulgu 2 — project_id on all read paths: - All 6 read endpoints (scores, scores/:fw, controls/:fw, weakest, recommendations, history) now accept ?project_id query param - All utils queries filter by project_id (NULL = org-wide) - Frontend repository, hooks, and dashboard updated with project_id support - Dashboard key uses composite key to avoid duplicate React keys Bulgu 3 — calculateForFramework now calls trackAIContent: - Added trackAIContent call after framework score upsert, matching the existing pattern in calculateAll Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
#3598 #3599) Fixes: - Fix table names: control_category_eu_ai_act_struct → controls_struct_eu, annex_category_struct_iso42001 → annexcategories_struct_iso - Fix column names: control_title → title, control_description → description - Fix task status enum: 'done' → 'Completed' (matches DB enum) - Fix migration schema prefix: add verifywise. for DDL in consolidated migrations - Fix TypeScript errors: String() casting for req.params, unused var prefixes, pdf-parse require() import - Fix readiness.ctrl.ts unused import UI Integrations: - Evidence Hub: add AI Analyze button (Sparkles icon) per evidence row - AI Content Review page at /ai-content-review with stats, badge legend, and approve/modify/reject workflow - Route registration for AIContentReview page Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…analysis 1. Evidence Hub table: add qualityMap and triggerAnalysis to useMemo dependency array so the quality badge updates immediately after AI analysis completes (no page reload needed) 2. Evidence AI controller: auto-apply suggested control links to file_entity_links after analysis, so readiness scores reflect new evidence immediately on next calculation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ent Review Dashboard now has a tab bar (like Model Inventory) with three default tabs: - Overview: original dashboard content (non-removable) - Audit Readiness: readiness scoring dashboard (#3598) - AI Content Review: transparency badge review (#3599) Users can customize tabs via "+" button — add/remove any tab except Overview. Tab preferences persist in localStorage. The tab system is extensible for future dashboard pages. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… system Redesigned both pages to match the app's established visual language: - DashboardHeaderCard-style gradient stat cards (border, borderRadius, bg) - Red Hat Display heading font at 20px/600 - brand.primary (#13715B) button and tab indicator colors - DashboardCard pattern for heatmap/trend/controls containers - Removed inner borders from sub-components (now wrapped in Card) - Consistent spacing (16px gap, 16px margin-bottom) - Lucide icons matching app's icon system - Same tab styling (minHeight, columnGap, textTransform) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Checkbox color: brand.primary (#13715B) instead of MUI default - Icon color: brand.primary for active tabs, text.icon for inactive - Menu paper: rounded corners, subtle border, proper shadow - Font: Red Hat Display for header, accent color for subtitle - Hover/disabled states matching app patterns - Active tab label gets fontWeight 500 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…stem - DashboardCard gradient background + border pattern - Header row with entity name, timestamp, badge, separated by border-bottom - Metadata shown as icon+text chips (Cpu, Bot, Wrench, Target icons) - Prompt summary with brand.primary left-border accent - Approve: filled green button, Modified/Reject: outlined - Consistent 4px border-radius, Red Hat Display headings - Compact single-row review notes field Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Header padding: px 2→2.5, py 1.25→1.5 - Body padding: px 2→2.5, py 1.5→2 - Metadata chips: px 1→1.25, py 0.25→0.5, icons 10→12px, font 10→11px - Prompt summary: p 1→1.5, borderRadius 4→6px, font 11→12px - Buttons: px 1.5→2.5, py 0.5→0.75, font 11→12px, borderRadius 4→6px - Card gap: 12px→16px between review cards - TextField input padding increased for taller input Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Inner elements now have more breathing room: - Header: px 2.5→3, py 1.5→2 - Body: px 2.5→3, py 2→2.5 - Chip gap: 8→10px, chip internal spacing 0.5→0.75, padding px 1.25→1.5, py 0.5→0.75 - Chip row margin-bottom: 2→2.5 - Prompt summary: mb 2→2.5, p 1.5→2 - TextField: mb 1.5→2, input font 12→13px, py 1→1.25 - Button row: mt 0.5→1, font 12→13px, px 2.5→3, py 0.75→1 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ng, softer look Major visual improvements to review panel cards: - Metadata tags now use distinct accent colors (blue/indigo/teal/primary) - Entity type formatted as Title Case (readiness_score → Readiness Score) - Header has subtle accent background for visual separation - All border-radius bumped to 8px for softer corners - Body padding: px 3.5, py 3 — much more breathing room - Tag spacing: gap 2, row-gap 12px - Prompt summary: 13px font, tertiary color, 4px left-border accent - Approve button uses brand.primary, Modified is neutral outline - TextField expanded to 2 rows with 8px radius - Timestamp and date use muted color for less visual noise Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Component now has a fixed height (340px) instead of growing with data: - Header stays pinned at top - Trend entries scroll vertically inside the container - Subtle 4px scrollbar with hover state - Empty/loading states also respect the fixed height - Older calculations accessible by scrolling down Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Cells enlarged: 28→36px with 6px gap, borderRadius 6px - Hover: scale(1.12) + colored shadow + border accent - Legend: shield icons per level + count badges (Ready 0, Needs Work 10...) - Fixed height 340px with inner scroll (matches Readiness Trend) - Tooltip: two-line with bold control ID + score detail - Score font: 9→11px bold for readability - Border: 1.5px for more definition - Header: Red Hat Display font family Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
WeakControlsList redesign: - Priority icon box (AlertTriangle/ArrowDown/CheckCircle2) per severity - Framework name as subtitle under control ID - Score bar (LinearProgress) below header - Lightbulb icon for recommendations - Hover: border accent + background tint - Fixed height 340px + inner scroll (matches Heatmap/Trend) - Chip: outlined style with level-colored border Stat cards: - Added hover: border takes level color, background shifts to accent - Padding increased: 8px→12px top, 14px→18px horizontal Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ults Users can now choose whether AI analysis results are visible to the entire organization (public) or only to themselves (private). Database: - Migration adds visibility + created_by columns to all 5 AI tables - Updated unique indexes to support per-user private results - Visibility filter indexes for efficient querying - Existing data defaults to 'public' (backward compatible) Backend: - New visibility.utils.ts — shared SQL filter helper (buildVisibilityFilter) - All AI write paths accept visibility param from req.body - All AI read paths filter by visibility using userId from JWT - Evidence: analyzed_by used as owner field - Readiness: created_by + visibility on control/framework/history tables - AI Content: visibility on metadata, tracked through aiContentTracker Frontend: - New VisibilityToggle component (Public/Private toggle + All/Public/Private filter) - Repository layer: all POST/GET functions accept visibility param - Hooks: visibility in query keys for cache isolation - ReadinessDashboard: toggle before Calculate + filter above results - AIContentReview: filter in header area Query logic: - Public: visible to all org members - Private: visible only to creator (created_by = userId) - All (default filter): public + own private Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ce case, 8px spacing - Replace all hardcoded hex colors with palette theme tokens (accent.primary, brand, status) - Use custom Chip component instead of MUI Chip across all AI features - Convert all text to sentence case (only first letter capitalized) - Set 8px spacing between all logical blocks - Hide executive/operations toggle when not on overview tab - Merge VisibilityToggle + VisibilityFilter into single VisibilityChips component - Remove duplicate weakest controls card - Remove badge legend from AI content review header - Change "Remove tab" to "Hide tab" in dashboard tabs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…uler # Conflicts: # Clients/src/application/config/routes.tsx
…ue to 272 tools Implements Phase 1 AI features: - Human Confirmation Flow (#3700): All write tools (agent_* prefix) store a pending confirmation in Redis with 10-min TTL instead of executing directly. Frontend renders approve/reject UI with warning levels (info/warning/danger). Approval triggers actual execution via POST /api/ai-confirmation/approve/:id. Uses Redis WATCH/MULTI/EXEC for race-condition safety on concurrent approvals. - AI Tool Catalogue (#3701): Expands from 47 read-only tools to 272 total (165 read + 107 write) across 39 domains. All write tools use createWriteToolFn wrapper with proper tenant isolation (organizationId in all WHERE clauses). Edge cases handled: execution failure correctly rejects (not approves), missing executor auto-rejects, Redis store failure propagates errors, strict confirmation_required === true check in frontend. Closes #3700, Closes #3701 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add coverage for secrets, build outputs, IDE configs, Claude Code local files, K8s secrets, test coverage, and OS artifacts. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…e engine Phase 2 AI features: - XState Approval State Machine (#3713): Deterministic state machine governing every AI write operation lifecycle. States: idle → evaluate → auto_approve / pending_approval / auto_reject → executing → completed / failed. Database persistence in ai_action_approvals table with full state_history audit trail. New API: GET/POST /api/ai-approvals, GET /api/ai-approvals/stats. Timeout handler auto-rejects expired pending approvals. Backward compatible with Phase 1 /api/ai-confirmation/* endpoints. - Auto-Approve Rule Engine (#3714): json-rules-engine for declarative rule evaluation. 9 default rules (auditor write block, bulk delete requires admin, admin config, policy review, danger/warning require approval, read/info/ low-risk auto-approve). Tenant-customizable via ai_approval_rules table. CRUD API: GET/POST/PUT/DELETE /api/ai-approval-rules, POST /test. Frontend settings page at /settings/ai-approval-rules with rule list, create/edit dialog, active toggle, and condition editor. Closes #3713, Closes #3714 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…idge AI gateway Extends the approval workflow system from 2 entity types (use_case, file) to 15, covering all platform entities: risk, vendor, model_inventory, policy, incident, task, dataset, training, evidence, ai_action, automation, pmm_config, note. - Extend EntityType enum with 13 new values - Migration seeds default workflows per entity type for each organization - Add approval_request_id column to ai_action_approvals for workflow bridging - Extend LEFT JOINs in getApprovalRequestByIdQuery for new entity data loading - Bridge AI gateway pending_approval with approval_requests table + notifications - Add AI_ACTION to NotificationEntityType enum - Update frontend: entity type dropdown (15 options), entity type config for AI action/risk/vendor/policy/incident detail views, getEntityType mapping Closes #3715 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
EU AI Act Article 12 compliance — records every state transition from the XState approval machine. Includes: - ai_action_audit_log table with from_state, to_state, actor_type, metadata - Audit trail service logging all gateway transitions (auto-approve, pending, reject) - Analytics utils: actions by state/category, daily volume, top rules, avg wait time - API: GET /api/ai-audit/log, /log/:actionId, /analytics, /export (CSV) - Frontend dashboard at /ai-audit with Recharts (donut, bar, line), stat cards, paginated audit log table, action detail modal, CSV export - React Query hooks with 3min stale time Closes #3716 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…nd sandbox Phase 3 — Multi-Agent Orchestration: - #3728 Coordinator Agent & Mastra Agent Network: intent classification, multi-agent routing, agent registry, message protocol, parallel execution - #3729 Specialized Domain Agents: 6 agents (Risk, Compliance, Vendor, Policy, Incident, Model) with domain-specific tools, keywords, and prompts - #3730 Agent Memory System: three-tier memory (message history, working memory, semantic recall) with PostgreSQL backend, TTL cleanup, admin API - #3731 Langfuse Observability: trace/span management, cost tracking, generation logging, graceful no-op when unconfigured - #3732 Sandbox Isolation: E2B (Firecracker microVM) for untrusted code/files, Daytona (Docker) for parallel/long-running tasks, sandbox router with decision matrix Refs #3728, #3729, #3730, #3731, #3732 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
XState v5 getInitialSnapshot(undefined) throws "Cannot read properties of undefined (reading 'self')". Replaced with direct rule-engine-based decision routing — simpler, faster, and doesn't require actor lifecycle management for synchronous evaluate→route decisions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Vendor create tool was setting assignee=0 which violates the users foreign key constraint. Changed to use params.assignee or undefined to allow nullable vendor creation via AI advisor. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sequelize requires null (not undefined) for named replacements. undefined causes "no entry in the replacement map" error. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…VARCHAR The approval_workflows table had a CHECK constraint limiting entity_type to old enum values. Drop it before ALTER COLUMN to prevent violation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Approve and reject flows were not logging state transitions to the audit log table. Added logStateHistory calls after state updates in approveAction (success + failure paths) and rejectAction. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
createRiskQuery requires deadline and is_demo as named replacements. Without them Sequelize throws "no entry in the replacement map". Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
LLM may generate category values like "Bias & Fairness" not in the PostgreSQL enum. Added validateRiskCategory() with exact, case-insensitive, partial match, and fallback to "Operational risk". Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
LLM sends string names as risk_owner but the column is a FK to users.id. Now only accepts numeric IDs, defaults to null for non-numeric values. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Executors like policy create need _userId for author_id FK. Params are sanitized before storage but _userId must be re-injected at execution time in both approve and auto-approve paths. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…rompt LLM was generating past dates (2024) for review_date fields. Now system prompt includes today's date and rules: - Always use future dates - Default review date: 6 months from today - FK fields: only numeric IDs, leave empty for names Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Move /ai-audit into Dashboard tab system (remove duplicate header) - Rewrite AIAuditDashboard with palette tokens, cardSx, Chip, brand.primary - Convert chart/table wrappers to Card/CardContent pattern - Match stat card, tabs, header, period chips to reference pages - VisibilityToggle: switch from Chip to ToggleButtonGroup - My Submissions: fix empty detail via correct table names + entity_data fallback - ai_action entityTypeConfig with JSONB fallback - Backend: toolBridge strips LLM auto-fill defaults, incident/risk enum validation, re-inject _userId in approval executor paths Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Merges Phase 0–3 work (AI content tracking, human-in-the-loop approvals, multi-agent orchestration) into the risk-agents branch so both tracks land together.
Phase 0 — AI Content Tracking & Visibility
Phase 1 — Human Confirmation Flow (#3700, #3701)
ConfirmationToolUIReact component0FK values, empty strings)Phase 2 — Approval State Machine & Audit Trail (#3713, #3714, #3715, #3716)
idle → evaluate → auto_approve | pending | auto_reject → executing → completed | failedsubmit_for_approval,approve_action,reject_action)ai_action_audit_logBIGSERIAL trail with EU AI Act Art. 12 compliancePhase 3 — Multi-Agent Orchestration (#3728-#3732)
Key Fixes Included
approvalGateway: re-inject_userIdinto params before executor call (author_id FK)risk: validaterisk_categoryagainst enum, numericrisk_ownerwith null fallback,deadline/is_demodefaultsvendor: nullableassigneeto avoid FK violationincident: booleaninterim_report: false, case-insensitive filter mapping, enum values moved to descriptionprompts: inject current date + future-date rules (prevented LLM generating past dates)migration: drop CHECK constraint beforeALTER COLUMNfor entity_type extensionapprovalRequest.utils: fix LEFT JOIN table names (policy_manager,ai_incident_managements), use JSONBentity_data->>'ai_approval_id'for ai_action joinsTest plan
GET /api/ai-audit/logreturns paginated entries with filtersGET /api/ai-audit/analyticsreturns chart datacd Servers && npm run build,cd Clients && npm run build) pass with zero errors🤖 Generated with Claude Code