New benchmarking engine

[voorhs]

Hi! Over the past couple of months, I’ve been working on a reimplementation inspired by your benchmark:

https://github.com/deeppavlov/mcp-evals

My goals were:

• to make it easier to customize and use by building it around the Pydantic-AI framework for LLM agents
• to apply modern development practices, including enabling safer workflows where LLM agents can write and review benchmark tasks
• to preserve and build upon the original benchmark content

So far, I’ve recreated two domains from MCPMark: file system and PostgreSQL.

If this sounds interesting, I’d really appreciate your feedback. I’d also be glad to explore possible collaboration or alignment with your work.

Thanks for creating and sharing this benchmark—it’s been very inspiring!

[xiaogecai]

Assigned label: assigned-zjwu0522 (most frequent contributor from past 100 commits)

[xiaogecai]

Assignment: This issue has been assigned to @zjwu0522 (most frequent contributor from past 100 commits). No @mentions were found in the comments.

[xiaogecai]

Assigned label: assigned-zjwu0522 (most frequent contributor from past 100 commits)