Skip to content

Add evaluation topic benchmark dataset#39

Open
tine1117 wants to merge 1 commit into
federicodeponte:masterfrom
tine1117:add-eval-topic-dataset
Open

Add evaluation topic benchmark dataset#39
tine1117 wants to merge 1 commit into
federicodeponte:masterfrom
tine1117:add-eval-topic-dataset

Conversation

@tine1117

Copy link
Copy Markdown

Summary

This adds the fixed benchmark topic set requested in #37:

  • data/eval_topics.json with 20 topics across CS, medicine, economics, social science, physics, biology, meta-science, and environmental science
  • canonical sources for each topic, including DOI or arXiv IDs where available
  • expected terms for lightweight coverage checks
  • a short schema note in EVALUATION.md

The structure is meant to be easy for the planned evaluation scripts to consume while still being readable for manual review.

Closes #37.

Test

python -m json.tool data\eval_topics.json

Also checked the dataset contains 20 entries and every entry has the required keys.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[good first issue] Create evaluation topic benchmark dataset

1 participant