Request for Dastaset

[wahaha0112]

Dear Dr. Bhardwaj,

I hope you are well.

I am currently reading your SkillFortify paper and trying to reproduce the benchmark setup. I wanted to ask whether the SkillFortifyBench dataset described in the paper, especially the 540-skill corpus, is publicly available.

I checked the paper, Zenodo page, GitHub repository, and PyPI package, but I could not find a downloadable release of the benchmark dataset. If there is an official link or an access procedure, could you please share it with me?

Thank you very much for your help.

Best regards.

[varun369]

Thank you for your interest in reproducing the benchmark.

The SkillFortifyBench dataset is now publicly available:

Standalone repository (MIT): https://github.com/qualixar/skillfortifybench
Main repository (benchmarks/ subtree): https://github.com/qualixar/skillfortify/tree/main/benchmarks

The repository contains all 540 pre-generated skills ready to download — no generation step needed:

• 270 malicious skills across 13 attack types (A1–A13)
• 270 benign skills across 5 categories
• Three formats: Claude (.md), MCP (.json), OpenClaw (.yaml)
• manifest.json with SHA-256 per skill
• attack_taxonomy.json
• Evaluation results in RESULTS.md

To regenerate from source (produces byte-identical output):

pip install skillfortify
PYTHONHASHSEED=0 python -m benchmarks.generator --output ./skills --seed 42

Please feel free to open further issues if you have questions during reproduction.