We tested linkup against top search APIs on the finance retrieval benchmark from the FinSearchComp paper (ground truth from SEC filings).
- T2 = simple historical lookups (119 questions)
- T3 = complex, multi-step historical investigations (84 questions)
Linkup research is 1st on both tiers — T2 82.4% and T3 58.3%.
Linkup is a web search & research API for AI applications. It returns answers grounded in live web content with citations back to the original sources. On FinSearchComp, Linkup's research endpoint reads primary sources — SEC filings, exchange data, financial reports — and returns a sourced answer per question.
FinSearchComp is a financial-search benchmark of 635 questions written by 70+ professional financial analysts. Every question has a single objective gold answer verified against primary sources, and answers are graded strictly right/wrong.
This comparison covers the two historical tiers of the Global market, run identically across every system: fin_search_comp_t2_global (119 questions) and fin_search_comp_t3_global
(84 questions).
T2 (Simple Historical Lookup) asks for a single fixed figure from a company's past filings — a line item such as revenue, R&D expense, or inventory for a specific fiscal year. There is one correct number per question, taken from SEC filings.
Example questions (full answers in results/linkup_t2.json):
- What were the research and development expenses of Uber for the full year 2019? →
$4,836 million - What were the inventories of Walmart in the 2018 fiscal year? →
$43,783 million - What was the cost of sales of Walmart in the 2018 fiscal year? →
$373,396 million
| # | System (best research/deep config) | Accuracy | p50 latency |
|---|---|---|---|
| 1 | Linkup research | 82.4% (98/119) | 64 s |
| 2 | Parallel ultra |
73.11% | 861.3 |
| 3 | Perplexity finance_historical_lookup |
72.27% | 32.2 |
| 4 | Perplexity sonar_deep_research_high |
53.78% | 92.6 |
| 5 | Exa research_pro |
42.02% | 366.8 |
| 6 | Tavily research_pro |
40.34% | 104.5 |
| 7 | Parallel pro |
34.45% | 317.0 |
T3 (Complex Historical Investigation) requires multi-step work: pulling figures from several sources or time periods and computing a result — a ranking, a ratio, a maximum, or a derived difference. It is graded the same right/wrong way, but reaching the answer takes aggregation and calculation rather than a single lookup.
Example questions (full answers in results/linkup_t3.json):
- For each fiscal year from 2021 to 2024, what is Apple's inventory turnover rate as a multiple of Tesla's?
- For fiscal years 2021–2024, what multiple was Wells Fargo's net income of Nvidia's net income?
- Which is greater, the full-year increase of the Nasdaq Index in 2024 or the S&P 500 in 2024, and by what difference? →
Nasdaq, by 5.33%
| # | System (best research/deep config) | Accuracy | p50 latency |
|---|---|---|---|
| 1 | Linkup research | 58.3% (49/84) | ~295 s |
| 2 | Perplexity finance_multi_step_research |
55.95% | 31.2 |
| 3 | Parallel ultra |
46.43% | 996.5 |
| 4 | Tavily research_pro |
36.90% | 283.3 |
| 4 | Parallel pro |
36.90% | 339.0 |
| 6 | Exa research_pro |
26.19% | 268.1 |
| 7 | Perplexity sonar_deep_research_high |
21.43% | 137.2 |
Set the API keys you want to test (any subset), plus OPENAI_API_KEY for grading, then run:
pip install -r requirements.txt
export OPENAI_API_KEY=... # grades answers against gold
export LINKUP_API_KEY=... # add any of the providers below
export PERPLEXITY_API_KEY=...
export EXA_API_KEY=...
export TAVILY_API_KEY=...
export PARALLEL_API_KEY=...
python run_benchmark.py # set LIMIT=10 for a quick checkEach provider runs only if its key is set. Scores print per provider and per tier, and answers
are written to results/<provider>_<tier>.json — the same schema and filenames as the committed
results/linkup_t2.json / results/linkup_t3.json.
On the committed Linkup results: they are archived outputs from Linkup's original runs, stored in the exact shape this runner produces. Linkup's research endpoint is non-deterministic, so a fresh run reproduces the format and method — not the identical answers or score.
The Linkup integration is what produced the numbers above. The other providers are wired up
best-effort from their published configs and haven't been re-run here, so double-check each
before relying on its output.
run_benchmark.py— the runner (one function per provider + an LLM grader)make_charts.py— regenerates the charts above from the numbers in this READMEdata/— the FinSearchComp T2 and T3 Global question sets with gold answersresults/— Linkup's stored answers and grades (linkup_t2.json, linkup_t3.json)assets/— the charts above