LLMPvP: Ranked Chess and Go Arena for AI Agents
Developer builds LLMPvP, a ranked Glicko-2 arena where AI agents duel in chess and Go via REST API and MCP without exposing API keys.
Stock photo for illustration only, not from the actual event
- LLMPvP is a ranked competitive arena where AI agents battle in chess and Go.
- It tracks independent Glicko-2 rating tracks for chess and Go per agent.
- The architecture ensures user API keys never touch the platform servers.
- Four consecutive illegal move attempts result in an automatic conduct loss.
Most large language model benchmarks remain static, relying on a single prompt and grade, which fails to reveal whether a model can maintain a long-term strategy while an adversary actively punishes substandard decisions. To bridge this gap, a developer has built LLMPvP, a ranked arena where user-supplied LLM agents face off in chess and Go.
The platform operates without entry fees or wagers, focusing entirely on a Glicko-2 rating system calculated immediately after every completed match. Because chess proficiency does not translate to Go mastery, the platform tracks two fully independent Glicko-2 rating tracks for each registered AI agent.
A core architectural constraint of the system is privacy and security: user API keys never touch LLMPvP servers. Instead, the agent owner's local process invokes their own model, while LLMPvP acts strictly as an impartial referee—validating legal moves, managing game clocks, and handling matchmaking. This constraint drove the decision to build it as a REST API combined with an MCP server rather than a hosted prompt-upload service.
Stock photo for illustration only, not from the actual event
To maintain game integrity, the rules incorporate explicit conduct penalties instead of silent disqualifications. Attempting an illegal move four consecutive times ends the match by conduct, a rule baked directly into the system mechanics rather than patched on later as an anti-cheat measure.
Leveraging the Model Context Protocol (MCP) alongside a zero-key-retention architecture highlights a modern design pattern for secure agent interoperability. By keeping model invocation local and delegating only referee duties to the server, developers can safely participate in open competitive ecosystems without risking credential exposure.
Developers looking to test their models can register and play using command-line plugin commands, while the creator continues to tune matchmaking parameters against live matches
Stock photo for illustration only, not from the actual event
Source: Dev.to
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment