Add elmo lab CLI commands for one-off AEO runs#362
Open
jrhizor wants to merge 3 commits into
Open
Conversation
Expose Elmo's core analysis as one-off, DB-free CLI commands that reuse the providers configured by `elmo init`: - `elmo lab brainstorm <website>` — generate tracking prompts + competitors - `elmo lab eval [prompts]` — run prompts across providers → responses, citations, mentions, share-of-voice, and query fan-out, plus a self-contained index.html report - `elmo lab plan <paths...>` — AEO recommendations from your content, optionally grounded in a prior eval Commands share a pipeable brand-pack artifact, `model:provider[:version][:online]` targeting (repeatable for multi-provider), and `--output`/`--format`/`--stdout`. Refactors: split deploy commands into deploy.ts with index.ts as thin wiring; extract the worker's duplicated analyzeMentions into @workspace/lib/mentions so tracking, reports, and the CLI share one implementation. Bundles @workspace/lib into the CLI (LLM SDKs externalized; no DB). Adds a docs guide and unit tests.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
…brand flags - eval clears its `responses/` tree before writing so re-runs (fewer runs/targets or an edited prompt's new slug) can't leave orphaned markdown that's inconsistent with the regenerated report - csvCell prefixes formula-injection-prone text cells (=, +, -, @, …) with `'`; numbers/booleans (incl. negatives) emitted verbatim - eval warns when --brand-domain/--alias are passed without a resolvable brand - content-gap input now explicitly excludes brand-less runs
Replace the regex-based HTML sanitizer (flagged by CodeQL as incomplete multi-character sanitization and incomplete URL scheme check) with a marked renderer that escapes raw HTML to inert text — so only marked's own element set is emitted — and an href allowlist that strips control chars and permits only http(s)/mailto/relative/anchor targets (javascript:, data:, vbscript:, … become `#`). No more bypassable regex stripping of `<script>` / `javascript:`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Expands the Elmo CLI from deployment management into a one-off AEO toolkit. New
elmo labcommand group exposes Elmo's core analysis as DB-free one-off runs that reuse the providers/keys configured byelmo init(~/.elmo/.env). Commands are ordered by the workflow you'd actually run:Highlights
brainstorm—analyzeBrand()→ suggested prompts + competitors, written as a pipeablebrand.json"brand pack".eval— runs each prompt × target × N replications viaprovider.run(). Captures responses (Markdown), citations, brand/competitor mentions, share-of-voice (reusingreport-metrics), and query fan-out (a byproduct of each web-search run — no separate command). Writes a directory tree plus a self-contained, offlineindex.htmlreport you can double-click to browse everything.plan— feeds your files/dirs (and optionally a prior eval) to a structured AEO prompt → prioritized recommendations + competitors, written back into an augmentedbrand.json.model:provider[:version][:online]targeting, repeatable-mfor multi-provider,-o/--format csv|jsonl/--stdout, default output to the current directory.Notable refactors
analyzeMentionsinto@workspace/lib/mentionsso day-to-day tracking, reports, and the CLI share one mention implementation.init/compose/edit) intodeploy.ts;index.tsis now thin wiring that also registers thelabgroup.@workspace/lib(LLM SDKs externalized). Verified the bundle has 0 drizzle/pg inlined — truly DB-free, never readsDATABASE_URL.Docs
New guide at
user-guide/cli-lab.mdxwalkinginit → brainstorm → eval → planwith copy-paste examples; validated via theapps/wwwbuild.Testing
check-typesgreen for cli/lib/worker; CLI bundle builds (DB-free verified).mentionstests (CSV/JSONL, slug, multi--mtarget resolution, brand-pack round-trip, mention detection, and a self-contained HTML-report test that also asserts active-content sanitization). Full lib suite (187) passes.