Skip to content

Commit 44975fc

Browse files
HomenShumHShuMclaude
authored
refactor: extract shared searchWithFallback into linkupClient (#368)
* fix(redesign): close all v2 prototype parity gaps for non-Home rails - Inbox: collapsible trace, 92% confidence label, related threads section - Me: collapsible traces in both guest and auth branches - AgentShell: expand/collapse toggle with useState + data-width attribute - Chat: graph SVG and report tab stats (prior session) - Reports: action cards, signal timeline, collapsible trace (prior session) - CSS: signal timeline, collapsible details/summary, confidence badge, related threads, graph SVG, rail expand styles Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(redesign): close remaining v2 prototype parity gaps from systematic audit Addresses 10 gaps found by side-by-side audit of home-v2.html vs shipped code: - Chat rail: expand/collapse toggle, 4th source, 3rd thread, new-note button - Inbox rail: tool tag spans, action card icons, "Defer to Thursday" label - Reports rail: tool tag spans, action card icons - Me rail (auth): session stats (threads/tools/cost/latency), permission colors (green/amber/red), action cards with icons, tool tag spans - Me rail (guest): tool tag spans - CSS: .rd-v2-msg-tools, .rd-v2-action-icon, .rd-v2-note-add Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(proto): add Reports v3 prototype with 5-view layout + agent inspector Standalone HTML prototype for the Reports redesign vision: - Gallery (3 cards/row, 820px center), Board (kanban), Table, Notebook, Canvas - Compact card v3: entity icon, status badge, thesis, signals, E/F/C scores - Agent Inspector right panel: pipeline state, quality scores, warnings, actions - Entity intelligence left nav: universes, entity types, operations - Dark/light theme, prefers-reduced-motion, print stylesheet, skip link Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * polish(proto): Reports v3 — board scores, full inspector data, icon + theme fixes Round 3 iterative polish of home-v3.html prototype: - Board mini-cards: quality score badges on review/verified/archived columns - Inspector: all 9 entities now update dynamically (added Bug0, Dario Amodei, Google DeepMind, Daily Brief, Cohere with pipeline, scores, warnings) - Left rail: replaced amateur Unicode icons (⚙→▶, !→⚑, ↷→◎) - Theme toggle: moon/sun icon swap with aria-label, title attribute - Board verified column: cleaner meta text ("12 sources · Updated 2h ago") Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * polish(proto): muted elegance pass — strip color to typography-only hierarchy Replace all semantic colors (green/amber/red/blue) with grayscale tokens across gallery, board, table, notebook, canvas, and inspector views. Only remaining color: 5px status badge dots (the whisper). Changes: - Score fills, quality numbers, mini-card scores → ink-faint/ink-mute - Quality ring JS + inspector dim fill JS → single ink-faint color - Notebook state badges, pipeline steps, warnings → muted grayscale - All inline-styled icons overridden via CSS !important → muted - Canvas nodes, table action links → neutral tones - Topbar nav, view tabs, status pills → ink-strong/muted (no accent) - All hover states → line-strong/muted (no accent) - Primary buttons → ink-strong ghost (no filled accent) - Focus-visible outline preserved for a11y Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * polish(proto): rebalance — restore semantic + brand color, keep icon chrome muted Over-corrected the muting pass. Restore: - Score numbers/rings/bars: green/amber/red (informational) - Table + inspector bar fills: colored (data visualization) - Pipeline steps: green done, amber warn (status) - Warnings: amber bg (attention signal) - Canvas nodes: typed colors (entity identity) - Notebook state badges: colored (ready/draft/stale) - Brand accent: logo, CTA, primary button, active score - Table action links: colored by urgency Keep muted (the actual noise reduction): - Card/table/left-rail icons: uniform gray (was loudest) - Card left borders: removed - Signal tags: muted - Nav/tab active states: muted (no accent tint) - Hover chrome: line-strong (no accent) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(proto): Ideaflow-inspired expandable graph notebook with entity context rail Add source citation popovers, wiki link expansion, mention expansion with agent deep dive simulation, backlinks with relation types, and entity context card in the right rail. All features are interactive, dismiss correctly across event boundaries, and coexist within a single notebook view. - Source popovers: click [src:N] badges for title/URL/excerpt/date - Mention expansion: click @Company/@person for inline entity blocks with claims, graph edges, and action buttons - Wiki expansion: click [[topic]] for inline definitions with tags - Agent expansion: 4-step timed simulation appending new claims - Entity context card: fills right rail dead space with metrics grid and connected entities - Right rail chat: entity-contextual agent thread - Backlinks section: 5 relation types (MENTIONS/COMPETES/CITES/FOUNDER_OF/RELATED) - Documentation: IDEAFLOW_FEATURES.md + EXPANDABLE_GRAPH_NOTEBOOK.md Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: replace Ideaflow references with Roam Research attribution Remove all Ideaflow branding from codebase to avoid attribution controversies. The expandable graph notebook patterns originate from Roam Research (bidirectional backlinks, block references, outliner-first editing). Updates 14 files across convex schema, architecture docs, CSS comments, test fixtures, agent palette, and prototype documentation. Exempt: chat-logs-unified (historical transcripts), docs/archive (superseded), scripts/tax-2025 (employer records). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(proto): unified entity card style + in-place chain navigation across all 3 surfaces Restyle all entity cards (Home mentions, report backlink panels, notebook inline expansions) to match the clean "Pricing Power" concept card design with kind-based color differentiation (company=terracotta, person=purple, concept=cyan). Add chain navigation via tag chips on all surfaces — clicking a tag dismisses the current card and opens the new entity's card at the same position without scrolling. Key changes: - Rewrite showBacklinkPanel() for clean card style with kind icons, paragraphs, tags - Rewrite buildRefBlock() for consistent inline expansion design - Extract wireRefBlock() helper for recursive in-place chain navigation - Fix notebook chain nav bug: tag clicks now replace card at anchor position instead of scrolling to find @mention elsewhere in text - Add buildHomeMentionCard() for Home surface chain navigation - CSS: kind-colored top borders, glass-card DNA, rounded tag chips Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: extract shared searchWithFallback into linkupClient module Both deepDiligence.ts and expandEntity.ts had near-identical searchWithFallback() functions. Extracted into a shared convex/domains/search/linkupClient.ts with SSRF validation, bounded response reading, and Gemini grounding fallback. deepDiligence gains SSRF + BOUND_READ protections it was missing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: hshum <hshum@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent d2f20a8 commit 44975fc

16 files changed

Lines changed: 1921 additions & 70 deletions
Lines changed: 268 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,268 @@
1+
/**
2+
* Validates and applies a graph patch proposed by the expansion agent.
3+
*
4+
* The agent NEVER writes directly to the graph — it proposes a structured
5+
* patch, and this mutation validates bounds, deduplicates, and applies.
6+
*
7+
* Pattern: scratchpad-first (agent proposes, mutation validates)
8+
* Prior art:
9+
* - Roam Research: structured backlink persistence after outline parsing
10+
* - Anthropic: Building Effective Agents (2024), tool output validation
11+
*
12+
* Invariants:
13+
* BOUND — max 50 claims, 100 edges, 50 backlinks per patch
14+
* HONEST_STATUS — returns actual counts, not claimed counts
15+
* DETERMINISTIC — dedup by (subject, predicate, object) triple
16+
* SSRF — sourceUrls validated before storing
17+
*
18+
* See: docs/architecture/EXPANDABLE_GRAPH_NOTEBOOK.md §3.2
19+
*/
20+
21+
import { v } from "convex/values";
22+
import { internalMutation } from "../../_generated/server";
23+
24+
// ── Constants ────────────────────────────────────────────────────────────
25+
26+
const MAX_CLAIMS = 50;
27+
const MAX_EDGES = 100;
28+
const MAX_BACKLINKS = 50;
29+
30+
// ── SSRF validation ──────────────────────────────────────────────────────
31+
32+
const SSRF_BLOCKED = [
33+
/^localhost$/i,
34+
/^127\./,
35+
/^10\./,
36+
/^172\.(1[6-9]|2\d|3[01])\./,
37+
/^192\.168\./,
38+
/^169\.254\./,
39+
/^0\.0\.0\.0$/,
40+
/^\[::1\]$/,
41+
/^metadata\.google\.internal$/i,
42+
];
43+
44+
function isUrlSafe(url: string): boolean {
45+
try {
46+
const parsed = new URL(url);
47+
if (!["http:", "https:"].includes(parsed.protocol)) return false;
48+
return !SSRF_BLOCKED.some((p) => p.test(parsed.hostname));
49+
} catch {
50+
return false;
51+
}
52+
}
53+
54+
// ── DETERMINISTIC: stable triple hash for dedup ──────────────────────────
55+
56+
function tripleKey(subject: string, predicate: string, object: string): string {
57+
return `${subject.toLowerCase().trim()}::${predicate.toLowerCase().trim()}::${object.toLowerCase().trim()}`;
58+
}
59+
60+
// ── Main patch mutation ──────────────────────────────────────────────────
61+
62+
export const applyPatch = internalMutation({
63+
args: {
64+
targetEntityId: v.id("entityProfiles"),
65+
agentRunId: v.string(),
66+
userId: v.optional(v.string()),
67+
68+
claims: v.array(
69+
v.object({
70+
subject: v.string(),
71+
predicate: v.string(),
72+
object: v.string(),
73+
claimText: v.string(),
74+
sourceUrls: v.array(v.string()),
75+
isHighConfidence: v.boolean(),
76+
}),
77+
),
78+
79+
backlinks: v.array(
80+
v.object({
81+
targetEntityName: v.string(),
82+
backlinkType: v.union(
83+
v.literal("relatedTo"),
84+
v.literal("causes"),
85+
v.literal("supports"),
86+
v.literal("derived"),
87+
),
88+
confidence: v.number(),
89+
sourceContext: v.optional(v.string()),
90+
}),
91+
),
92+
},
93+
94+
handler: async (ctx, args) => {
95+
let claimsCreated = 0;
96+
let edgesCreated = 0;
97+
let backlinksCreated = 0;
98+
99+
// ── 1. Find or create knowledge graph for this entity ───────────
100+
101+
// Find or create a knowledge graph scoped to this entity expansion.
102+
// knowledgeGraphs requires: name, sourceType, sourceId, userId, isOddOneOut,
103+
// claimCount, edgeCount, lastBuilt, createdAt, updatedAt.
104+
let graphDoc = await ctx.db
105+
.query("knowledgeGraphs")
106+
.filter((q) =>
107+
q.and(
108+
q.eq(q.field("sourceType"), "entity"),
109+
q.eq(q.field("sourceId"), args.targetEntityId),
110+
),
111+
)
112+
.first();
113+
114+
const now = Date.now();
115+
116+
if (!graphDoc) {
117+
const graphId = await ctx.db.insert("knowledgeGraphs", {
118+
name: `Expansion: ${args.targetEntityId}`,
119+
sourceType: "entity",
120+
sourceId: args.targetEntityId,
121+
// userId is required by knowledgeGraphs schema (v.id("users")).
122+
// Cast from string — the expansion action passes a Clerk user ID.
123+
userId: (args.userId ?? "system") as any,
124+
isOddOneOut: false,
125+
claimCount: 0,
126+
edgeCount: 0,
127+
lastBuilt: now,
128+
createdAt: now,
129+
updatedAt: now,
130+
});
131+
graphDoc = await ctx.db.get(graphId);
132+
}
133+
134+
if (!graphDoc) {
135+
return { claimsCreated: 0, edgesCreated: 0, backlinksCreated: 0 };
136+
}
137+
138+
const graphId = graphDoc._id;
139+
140+
// ── 2. Deduplicate + insert claims (BOUND: max 50) ─────────────
141+
142+
// Load existing claims for dedup
143+
const existingClaims = await ctx.db
144+
.query("graphClaims")
145+
.filter((q) => q.eq(q.field("graphId"), graphId))
146+
.take(500); // BOUND_READ
147+
148+
const existingTriples = new Set(
149+
existingClaims.map((c) => tripleKey(c.subject, c.predicate, c.object)),
150+
);
151+
152+
const newClaimIds: Array<{ id: any; subject: string }> = [];
153+
154+
for (const claim of args.claims.slice(0, MAX_CLAIMS)) {
155+
const key = tripleKey(claim.subject, claim.predicate, claim.object);
156+
if (existingTriples.has(key)) continue; // DETERMINISTIC: skip dupes
157+
158+
// SSRF: validate source URLs
159+
const safeUrls = claim.sourceUrls.filter(isUrlSafe);
160+
161+
const claimNow = Date.now();
162+
const claimId = await ctx.db.insert("graphClaims", {
163+
graphId,
164+
subject: claim.subject.slice(0, 200),
165+
predicate: claim.predicate.slice(0, 100),
166+
object: claim.object.slice(0, 500),
167+
claimText: claim.claimText.slice(0, 500),
168+
sourceDocIds: [], // expansion-sourced claims link via sourceUrls, not docs
169+
sourceSnippets: safeUrls.slice(0, 5), // Store source URLs as snippets for traceability
170+
extractedAt: claimNow,
171+
isHighConfidence: claim.isHighConfidence,
172+
createdAt: claimNow,
173+
});
174+
175+
newClaimIds.push({ id: claimId, subject: claim.subject });
176+
existingTriples.add(key);
177+
claimsCreated++;
178+
}
179+
180+
// ── 3. Create edges between claims (BOUND: max 100) ────────────
181+
182+
// For now, create "relatedTo" edges between claims about the same entity
183+
const claimsBySubject = new Map<string, any[]>();
184+
for (const nc of newClaimIds) {
185+
const key = nc.subject.toLowerCase();
186+
if (!claimsBySubject.has(key)) claimsBySubject.set(key, []);
187+
claimsBySubject.get(key)!.push(nc.id);
188+
}
189+
190+
let edgeCount = 0;
191+
for (const [, ids] of claimsBySubject) {
192+
if (ids.length < 2) continue;
193+
// Create chain edges between sequential claims
194+
for (let i = 0; i < ids.length - 1 && edgeCount < MAX_EDGES; i++) {
195+
await ctx.db.insert("graphEdges", {
196+
graphId,
197+
fromClaimId: ids[i],
198+
toClaimId: ids[i + 1],
199+
edgeType: "relatedTo",
200+
isStrong: false,
201+
createdAt: Date.now(),
202+
});
203+
edgeCount++;
204+
edgesCreated++;
205+
}
206+
}
207+
208+
// ── 4. Create backlinks (BOUND: max 50) ─────────────────────────
209+
210+
for (const bl of args.backlinks.slice(0, MAX_BACKLINKS)) {
211+
// Try to resolve the target entity by name
212+
const targetEntity = await ctx.db
213+
.query("entityProfiles")
214+
.filter((q) =>
215+
q.eq(
216+
q.field("canonicalName"),
217+
bl.targetEntityName,
218+
),
219+
)
220+
.first();
221+
222+
if (!targetEntity) continue; // Skip unresolvable entities
223+
224+
// Dedup: check if this exact backlink exists
225+
const existingBacklink = await ctx.db
226+
.query("backlinks")
227+
.withIndex("by_source", (q) =>
228+
q.eq("sourceType", "claim").eq("sourceId", args.targetEntityId),
229+
)
230+
.filter((q) =>
231+
q.and(
232+
q.eq(q.field("targetEntityId"), targetEntity._id),
233+
q.eq(q.field("backlinkType"), bl.backlinkType),
234+
),
235+
)
236+
.first();
237+
238+
if (existingBacklink) continue;
239+
240+
await ctx.db.insert("backlinks", {
241+
sourceType: "claim",
242+
sourceId: args.targetEntityId,
243+
targetEntityId: targetEntity._id,
244+
backlinkType: bl.backlinkType,
245+
confidence: Math.max(0, Math.min(1, bl.confidence)), // Clamp 0-1
246+
sourceContext: bl.sourceContext?.slice(0, 200),
247+
createdBy: "agent",
248+
agentRunId: args.agentRunId,
249+
createdAt: Date.now(),
250+
});
251+
252+
backlinksCreated++;
253+
}
254+
255+
// Update graph counters (honest counts)
256+
if (claimsCreated > 0 || edgesCreated > 0) {
257+
await ctx.db.patch(graphId, {
258+
claimCount: graphDoc.claimCount + claimsCreated,
259+
edgeCount: graphDoc.edgeCount + edgesCreated,
260+
lastBuilt: Date.now(),
261+
updatedAt: Date.now(),
262+
});
263+
}
264+
265+
// HONEST_STATUS: return actual persisted counts
266+
return { claimsCreated, edgesCreated, backlinksCreated };
267+
},
268+
});

0 commit comments

Comments
 (0)