Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 11 additions & 12 deletions docs/INTERNALS.md
Original file line number Diff line number Diff line change
Expand Up @@ -254,7 +254,7 @@ Reason: [Concise reason why this score was given.]

### Refinement Trigger

Sections scoring below the threshold (default: 80) trigger additional research:
Sections scoring below the threshold (default: 70) trigger additional research:

```python
if score < GRADE_THRESHOLD:
Expand Down Expand Up @@ -339,29 +339,28 @@ Phase 1: Deep Research (Lead Researcher)
- Role: Gather facts, data, citations
- Output: ~12 page research dossier

Phase 2: Section Writing (Gemini 3 Flash)
- Model: gemini-3-flash-preview
Phase 2: Section Writing (Gemini 3.1 Pro)
- Model: gemini-3.1-pro-preview
- Role: Write each section with analytical depth
- Input: Dossier + previous sections in prompt
- Input: Dossier via previous_interaction_id + prior sections
- Output: 20 sections × ~1.5 pages = 30+ pages
```

This architecture treats Deep Research as the **researcher** and Gemini 3 Flash as the **writer**.
This architecture treats Deep Research as the **researcher** and Gemini 3.1 Pro as the **writer**.

### Why Not `previous_interaction_id`?
### Section Writing and `previous_interaction_id`

The Gemini docs mention `previous_interaction_id` for follow-up questions. However, testing showed that direct `generate_content()` calls with the dossier in the prompt produce better results:

- More reliable (no interaction state to manage)
- Faster (no interaction lookup overhead)
- More controllable (full prompt visible)
Section writing continues the Deep Research interaction through
`previous_interaction_id`, so each section builds statefully on the researched
dossier instead of re-sending the full dossier in every prompt. See
[ARCHITECTURE](ARCHITECTURE.md) for the current end-to-end pipeline diagram.

### Model Selection

| Component | Model | Rationale |
|-----------|-------|-----------|
| Research Dossier | `deep-research-preview-04-2026` | Autonomous web research |
| Section Writing | `gemini-3-flash-preview` | Fast, intelligent, cost-effective |
| Section Writing | `gemini-3.1-pro-preview` | Analytical depth, tiered pricing |
| Stage 1 Analysis | `gemini-3-flash-preview` | Quick section analysis |

### Adaptive Polling
Expand Down
95 changes: 51 additions & 44 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,61 +11,68 @@ Guides are grouped by what you are trying to do, following the
[Diataxis](https://diataxis.fr/) split: learning, doing, looking up, and
understanding.

> **Index currency:** reviewed against primr **1.37.1** on **2026-07-21**. The
> *Updated* column is each document's last substantive revision (git
> `last-commit` date). `tests/test_docs_index.py` fails CI if a doc under
> `docs/` is missing from this map or an index link does not resolve, so the
> map and the tree cannot silently drift apart.

## Getting started (tutorials)

| Document | Description |
|----------|-------------|
| [ZERO_COST](ZERO_COST.md) | Run keyless collection and finish a sourced dossier with an existing agent plan |
| [API_KEYS](API_KEYS.md) | API key setup, security, and troubleshooting |
| [CONFIG](CONFIG.md) | First-run configuration and the full settings reference |
| [RUN_MODES](RUN_MODES.md) | Run modes, costs, strategy selection, output locations, and zero-cost `primr render` (Markdown to DOCX/TXT) |
| [AZURE_QUICKSTART](AZURE_QUICKSTART.md) | Stand up the team/org Azure deployment end to end |
| Document | Description | Updated |
|----------|-------------|---------|
| [ZERO_COST](ZERO_COST.md) | Run keyless collection and finish a sourced dossier with an existing agent plan | 2026-07-18 |
| [API_KEYS](API_KEYS.md) | API key setup, validation (`primr keys test`), security, and troubleshooting | 2026-07-21 |
| [CONFIG](CONFIG.md) | First-run configuration and the full settings reference | 2026-07-21 |
| [RUN_MODES](RUN_MODES.md) | Run modes, costs, strategy selection, output locations, and zero-cost `primr render` (Markdown to DOCX/TXT) | 2026-07-21 |
| [AZURE_QUICKSTART](AZURE_QUICKSTART.md) | Stand up the team/org Azure deployment end to end | 2026-07-18 |

## How-to guides (task-oriented)

| Document | Description |
|----------|-------------|
| [BATCH](BATCH.md) | Run primr across many companies from a CSV |
| [RECOVERY](RECOVERY.md) | Resume after a crash, reboot, or interrupted run |
| [IMPROVE](IMPROVE.md) | Improve and refine an existing report (`primr improve` / `refine`) |
| [SKILL_PACK](SKILL_PACK.md) | `primr skills` end to end: planning, curation, artifacts, CLI/MCP |
| [EVAL](EVAL.md) | Evaluate and compare models with the eval harness |
| [MODEL_ONBOARDING](MODEL_ONBOARDING.md) | Register and validate a new model |
| [AGENT_INTEGRATION](AGENT_INTEGRATION.md) | Operate Primr from MCP, A2A, skills, and agent hosts |
| [OPENCLAW](OPENCLAW.md) | OpenClaw integration and governed workflows |
| [COPILOT_COWORK_GUIDE](COPILOT_COWORK_GUIDE.md) | Sideload a skill pack into Microsoft 365 Copilot Cowork |
| [COPILOT_STUDIO_GUIDE](COPILOT_STUDIO_GUIDE.md) | Use primr from Copilot Studio |
| [FOUNDRY_AGENT_GUIDE](FOUNDRY_AGENT_GUIDE.md) | Use primr from a Microsoft Foundry agent |
| [CLOUD_DEPLOYMENT](CLOUD_DEPLOYMENT.md) | Serverless deployment on AWS, Azure, and GCP |
| [SECURITY_OPS](SECURITY_OPS.md) | Key rotation, audit logs, operational security |
| Document | Description | Updated |
|----------|-------------|---------|
| [BATCH](BATCH.md) | Run primr across many companies from a CSV | 2026-07-18 |
| [RECOVERY](RECOVERY.md) | Resume after a crash, reboot, or interrupted run | 2026-07-17 |
| [IMPROVE](IMPROVE.md) | Improve and refine an existing report (`primr improve` / `refine`) | 2026-04-10 |
| [SKILL_PACK](SKILL_PACK.md) | `primr skills` end to end: planning, curation, artifacts, CLI/MCP | 2026-07-17 |
| [EVAL](EVAL.md) | Evaluate and compare models with the eval harness | 2026-07-17 |
| [MODEL_ONBOARDING](MODEL_ONBOARDING.md) | Register and validate a new model | 2026-06-29 |
| [AGENT_INTEGRATION](AGENT_INTEGRATION.md) | Operate Primr from MCP, A2A, skills, and agent hosts | 2026-07-21 |
| [OPENCLAW](OPENCLAW.md) | OpenClaw integration and governed workflows | 2026-07-17 |
| [COPILOT_COWORK_GUIDE](COPILOT_COWORK_GUIDE.md) | Sideload a skill pack into Microsoft 365 Copilot Cowork | 2026-07-21 |
| [COPILOT_STUDIO_GUIDE](COPILOT_STUDIO_GUIDE.md) | Use primr from Copilot Studio | 2026-07-21 |
| [FOUNDRY_AGENT_GUIDE](FOUNDRY_AGENT_GUIDE.md) | Use primr from a Microsoft Foundry agent | 2026-07-21 |
| [CLOUD_DEPLOYMENT](CLOUD_DEPLOYMENT.md) | Serverless deployment on AWS, Azure, and GCP | 2026-07-18 |
| [SECURITY_OPS](SECURITY_OPS.md) | Key rotation, audit logs, operational security | 2026-07-17 |

## Reference (look-up)

| Document | Description |
|----------|-------------|
| [API](API.md) | MCP server and A2A protocol, programmatic usage |
| [Job Status](JOB_STATUS.md) | Versioned CLI, MCP, A2A, and API lifecycle contract |
| [STRATEGY_PORTFOLIO](STRATEGY_PORTFOLIO.md) | Strategy document types and selection |
| [NEXT_STEPS](NEXT_STEPS.md) | What to build next, why it comes next, and what not to do yet |
| [CHANGELOG](CHANGELOG.md) | Version history |
| [MIGRATION](MIGRATION.md) | Error-hierarchy migration notes |
| [ROADMAP](https://github.com/blisspixel/primr/blob/main/ROADMAP.md) | Ordered development queue and version plan |
| Document | Description | Updated |
|----------|-------------|---------|
| [API](API.md) | MCP server and A2A protocol, programmatic usage | 2026-07-18 |
| [Job Status](JOB_STATUS.md) | Versioned CLI, MCP, A2A, and API lifecycle contract | 2026-07-10 |
| [STRATEGY_PORTFOLIO](STRATEGY_PORTFOLIO.md) | Strategy document types and selection | 2026-07-21 |
| [NEXT_STEPS](NEXT_STEPS.md) | What to build next, why it comes next, and what not to do yet | 2026-07-21 |
| [CHANGELOG](CHANGELOG.md) | Version history | 2026-07-21 |
| [MIGRATION](MIGRATION.md) | Error-hierarchy migration notes | 2026-02-02 |
| [EVAL_V1_24_0](EVAL_V1_24_0.md) | Historical decision record: the v1.24.0 cross-provider eval plan | 2026-06-26 |
| [ROADMAP](https://github.com/blisspixel/primr/blob/main/ROADMAP.md) | Ordered development queue and version plan | — |

## Explanation (understanding why)

| Document | Description |
|----------|-------------|
| [ARCHITECTURE](ARCHITECTURE.md) | System design and the 9-tier scraping engine |
| [ARTIFACTS](ARTIFACTS.md) | The research-vs-shipping artifact pipeline and ship-time gates |
| [INTERNALS](INTERNALS.md) | Core algorithms and prompt strategy |
| [STATE_MACHINES](STATE_MACHINES.md) | Tier escalation and job lifecycle |
| [CONCURRENCY](CONCURRENCY.md) | Threading and concurrency model |
| [SECURITY](SECURITY.md) | Security policy and the scoped AI/agent threat model |
| [design/](design/README.md) | Per-workstream design docs and decision audits (incl. the v1.24.0 cross-provider eval) |
| Document | Description | Updated |
|----------|-------------|---------|
| [ARCHITECTURE](ARCHITECTURE.md) | System design and the 9-tier scraping engine | 2026-07-21 |
| [ARTIFACTS](ARTIFACTS.md) | The research-vs-shipping artifact pipeline and ship-time gates | 2026-07-21 |
| [INTERNALS](INTERNALS.md) | Core algorithms and prompt strategy | 2026-07-21 |
| [STATE_MACHINES](STATE_MACHINES.md) | Tier escalation and job lifecycle | 2026-02-02 |
| [CONCURRENCY](CONCURRENCY.md) | Threading and concurrency model | 2026-07-12 |
| [SECURITY](SECURITY.md) | Security policy and the scoped AI/agent threat model | 2026-07-21 |
| [design/](design/README.md) | Per-workstream design docs and decision audits | — |

## Contributing

| Document | Description |
|----------|-------------|
| [CONTRIBUTING](CONTRIBUTING.md) | Dev environment setup and the contribution workflow |
| [CLAUDE.md](https://github.com/blisspixel/primr/blob/main/CLAUDE.md) | The development contract: seams, constraints, verification gates |
| Document | Description | Updated |
|----------|-------------|---------|
| [CONTRIBUTING](CONTRIBUTING.md) | Dev environment setup and the contribution workflow | 2026-07-09 |
| [CLAUDE.md](https://github.com/blisspixel/primr/blob/main/CLAUDE.md) | The development contract: seams, constraints, verification gates | — |
48 changes: 48 additions & 0 deletions tests/test_docs_index.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
"""The docs index (`docs/README.md`) must stay in lockstep with the doc tree.

Two failure modes rot a documentation map: a new doc lands but never gets
listed (readers cannot find it), or an entry outlives the file it points to
(a dead link). Both are caught here without needing git history, so the check
is safe under a shallow CI checkout.

The `Updated` column dates in the index are informational (git last-commit
dates, refreshed on review) and are deliberately not asserted here -- pinning
exact dates would be brittle across rebases and squash merges. What must never
drift is *coverage* and *link validity*, which is what this test guards.
"""

from __future__ import annotations

import re
from pathlib import Path

REPO_ROOT = Path(__file__).resolve().parent.parent
DOCS_DIR = REPO_ROOT / "docs"
INDEX = DOCS_DIR / "README.md"

_LINK = re.compile(r"\]\(([^)]+)\)")


def _index_link_targets() -> list[str]:
return _LINK.findall(INDEX.read_text(encoding="utf-8"))


def test_every_top_level_doc_is_listed_in_the_index() -> None:
targets = {Path(t).name for t in _index_link_targets()}
missing = sorted(
path.name
for path in DOCS_DIR.glob("*.md")
if path.name != "README.md" and path.name not in targets
)
assert not missing, f"docs/README.md must link every doc under docs/. Missing: {missing}"


def test_every_relative_index_link_resolves() -> None:
broken: list[str] = []
for target in _index_link_targets():
if target.startswith(("http://", "https://", "#", "mailto:")):
continue
resolved = (DOCS_DIR / target.split("#", 1)[0]).resolve()
if not resolved.exists():
broken.append(target)
assert not broken, f"docs/README.md has links that do not resolve: {broken}"