Replicating Anthropic's Vision-Only Pokémon Run in Fable 5 and Reading Its Thinking Chains
An independent developer successfully replicates Anthropic's vision-only Pokémon experiment in Fable 5, revealing surprising costs and hidden reasoning logs.
Stock photo for illustration only, not from the actual event
- An independent developer recreated Anthropic's vision-only Pokémon gameplay harness featured on the Fable 5 launch page.
- The model secured the first gym badge in 1,785 decisions with a total expenditure of $65.40.
- Inspection of internal thinking logs revealed the model planned future milestones long before they appeared on screen.
- Full logs, frame captures, and harness code have been published openly for public auditing.
Following Anthropic's showcase of Fable 5 playing Pokémon FireRed using a minimal, vision-only harness without publishing code or metrics, an independent developer set out to verify the claim independently. By constructing a custom harness meeting every described constraint, the experiment tested whether modern artificial intelligence could conquer the classic game using strictly visual input.
Running the Chinese fan translation of Pokémon FireRed, the Fable 5 model captured the first gym badge within 1,785 turns for a cost of $65.40. By the 2,000-turn cap, the agent had advanced to Route 3 managing a team of three Pokémon, with every frame and decision log made completely public.
However, reading through the model's thinking chains unveiled a deeper layer of complexity. 141 turns before the game introduced "Oak's Parcel," the model had already written down its destination city, required non-player characters to talk to, and items to collect. By Route 3, its reasoning recognized an iconic trainer referencing shorts—none of which was visible anywhere on the current screen buffer.
"A time-lapse of Claude playing Pokémon FireRed from start to finish using only raw game screenshots—with no maps, navigation aids, or extra game-state information. Earlier Claude models needed a complex helper harness to play Pokémon; Claude Fable 5 completed the game with vision alone."
Official Anthropic launch video caption
Stock photo for illustration only, not from the actual event
Enabling vision-only language models to successfully navigate complex retro role-playing games represents a monumental milestone in artificial intelligence research. Historically, vision-language models struggle to differentiate walkable terrain from solid walls purely through pixel analysis, forcing developers to extract exact coordinate data directly from emulator memory. Fable 5's ability to operate without these memory crutches highlights significant advancements in native visual perception and reasoning capabilities.
Academic benchmarks such as VideoGameBench previously showed frontier models successfully completing a mere 0.48% of games, while earlier streams like Claude Plays Pokémon required massive helper harnesses containing RAM state texts and critic agents. Achieving a vision-only playthrough marks a dramatic generational leap compared to previous attempts in the AI community.
An evaluation of open-source LLM-plays-Pokémon repositories reveals that "vision-only" is frequently a marketing term, as most projects secretly feed emulator memory coordinates and OCR text into prompts. This independent experiment stripped away those compensations layer by layer, offering a transparent measurement of the raw underlying model capabilities.
Source: Dev.to
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment