Skip to content

api.md

Latest commit

 

History

History
333 lines (235 loc) · 6.09 KB

api.md

File metadata and controls

333 lines (235 loc) · 6.09 KB
Raw

AutoLinks API Guide

This guide explains how to run the API locally, what each endpoint expects, and what shape to expect back.

Base URL

Local development:

http://127.0.0.1:8000

Versioned API prefix:

/api/v1

Start The API

From the backend directory:

go run ./cmd/server

Environment variables are loaded from the project root .env file.

If you want to avoid external API calls during development:

DRY_RUN=true go run ./cmd/server

The server starts an HTTP server on port 8000 and an in-process goroutine worker pool for background sitemap ingestion jobs.

Response Conventions

Successful recommendation responses return:

  • status
  • latency_ms
  • recommendations

Validation errors usually return HTTP 400.

Application errors usually return HTTP 500 with a detail field.

POST /api/v1/recommend

Analyze draft text and return internal link recommendations.

Request Body

{
  "text": "CUDA optimization can dramatically speed up model training.",
  "alpha": 0.7,
  "min_similarity": 0.65
}

Fields

Field Type Required Description
text string yes Draft text to analyze
alpha float no Similarity weight in the equity-aware rerank formula
min_similarity float no Minimum vector similarity score required before a candidate is surfaced

Example cURL

curl -X POST "http://127.0.0.1:8000/api/v1/recommend" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Wait But Why is organizing a global meetup this weekend.",
    "alpha": 0.7,
    "min_similarity": 0.65
  }'

Example Response

{
  "status": "success",
  "latency_ms": 720,
  "recommendations": [
    {
      "exact_phrase": "Wait But Why",
      "context_snippet": "A post about the Wait But Why community and meetup planning.",
      "suggested_url": "https://example.com/wait-but-why-meetup",
      "similarity_score": 0.82,
      "equity_need_score": 0.5,
      "final_score": 0.724,
      "inbound_link_count": 1
    }
  ]
}

Notes

  • Short noisy entities are filtered before search.
  • Abbreviation duplicates such as WBW and Wait But Why are deduplicated before query generation.
  • Low-similarity candidates below min_similarity are not returned.

POST /api/v1/ingest

Ingest a single article into Qdrant.

Request Body

{
  "url": "https://example.com/blog/cuda-optimization",
  "content": "CUDA optimization improves GPU throughput by reducing memory bottlenecks..."
}

Example cURL

curl -X POST "http://127.0.0.1:8000/api/v1/ingest" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/blog/cuda-optimization",
    "content": "CUDA optimization improves GPU throughput by reducing memory bottlenecks..."
  }'

Example Response

{
  "status": "success",
  "chunks_ingested": 1
}

Notes

  • The API chunks article text before embedding and upserting to Qdrant.
  • Stored payload includes the source URL and chunk text.

POST /api/v1/ingest/sitemap

Crawl a sitemap, extract article content, build the internal link graph, and ingest the articles.

Request Body

{
  "sitemap_url": "https://example.com/post-sitemap.xml",
  "max_concurrent": 5
}

Fields

Field Type Required Description
sitemap_url string yes Sitemap to crawl
max_concurrent integer no Max concurrent fetches during crawl

Example cURL

curl -X POST "http://127.0.0.1:8000/api/v1/ingest/sitemap" \
  -H "Content-Type: application/json" \
  -d '{
    "sitemap_url": "https://example.com/post-sitemap.xml",
    "max_concurrent": 5
  }'

Example Response

{
  "job_id": "550e8400-e29b-41d4-a716-446655440000",
  "status": "queued",
  "estimated_articles": 150
}

Notes

  • The crawler extracts clean text with trafilatura.
  • The same crawl also extracts internal <a href> links from page HTML.
  • Those links are inverted into inbound link counts and used by the equity-aware reranker.

GET /api/v1/ingest/status/{jobID}

Check the status of an async sitemap ingestion job.

Example cURL

curl "http://127.0.0.1:8000/api/v1/ingest/status/550e8400-e29b-41d4-a716-446655440000"

Example Response

{
  "status": "processing",
  "progress_pct": 45.0,
  "articles_done": 67,
  "total": 150,
  "errors": []
}

GET /api/v1/ingest/result/{jobID}

Retrieve the final result of a completed sitemap ingestion job.

Example cURL

curl "http://127.0.0.1:8000/api/v1/ingest/result/550e8400-e29b-41d4-a716-446655440000"

Example Response

{
  "status": "done",
  "chunks_ingested": 150,
  "duration_seconds": 82.0,
  "errors": []
}

POST /api/v1/ingest/retry-dead

Re-enqueue permanently failed ingestion jobs from the dead letter queue.

Example cURL

curl -X POST "http://127.0.0.1:8000/api/v1/ingest/retry-dead"

Example Response

{
  "retried_count": 3,
  "job_ids": ["abc-123", "def-456", "ghi-789"]
}

GET /api/v1/link-graph

Returns the current inbound link graph used by the equity-aware reranker.

Example cURL

curl "http://127.0.0.1:8000/api/v1/link-graph"

Example Response

{
  "status": "success",
  "url_count": 150,
  "link_graph": {
    "https://example.com/page1": 5,
    "https://example.com/page2": 0,
    "https://example.com/page3": 12
  }
}

Authentication

All endpoints except /api/v1/health require a Clerk JWT Bearer token in the Authorization header when CLERK_SECRET_KEY is configured.

GET /api/v1/health

Simple health check endpoint.

Example cURL

curl "http://127.0.0.1:8000/api/v1/health"

Example Response

{
  "status": "ok",
  "model_loaded": true
}

Common Workflow

  1. Start Qdrant.
  2. Start the Go server.
  3. Ingest existing content with /api/v1/ingest or /api/v1/ingest/sitemap.
  4. Send draft text to /api/v1/recommend.
  5. Render the returned recommendations in your frontend or CMS integration.