Why I looked.
An outside model router called Jev (typesafe/jev-router) promises to cut cost by picking a model for each request. I ran it against copies of the site’s real requests. For two request types it picked the same model every time: DeepSeek V4.1 Flash, served through OpenRouter. The two were Wargamer turns where you paste in your game’s rules, and the RPG step that condenses an uploaded rulebook.
A router’s pick proves nothing about quality, so I tested Flash on its own.
How I tested it.
Each round sent the production request bodies unchanged except for the model. The final round ran 7 test cases per request type. A judge model from a third company, Kimi K3, compared each Flash answer with Sonnet 5’s answer to the same request without knowing which model wrote which.
I brought in an outside judge because round 2 went wrong. Its judge was DeepSeek V4.1 Flash, one of the contestants, and it favored its own answers. Researchers call this self-preference, and I walked into it anyway. I threw out round 2’s wins and reran the final round with Kimi judging.
Jev went through the same comparison. It beat Sonnet 4 times out of 7, and it adds its own cost and delay on top of the model it picks. The site now calls Flash directly for the two cleared request types and leaves Jev out.
What won.
Rules turns: Flash beat Sonnet 5 in all 7 cases. Cost per turn fell from $0.0205 to $0.0011, and a full turn on the live site now comes back in about 9 seconds instead of about 19.
Rulebook condensing: Flash won 4, lost 2 and tied 1. Cost fell from $0.0260 to $0.0016. That result is weaker, and Flash still beat Sonnet twice as often as it lost.
Every other mode stays on Sonnet 5. I moved only the two that won a test.
What you get.
Prices stay the same. The Wargamer pasted-rules limit goes from 30,000 to 120,000 characters, roughly 10 to 12 rulebook pages up to 40 or 50. The RPG rulebook upload limit goes from 60,000 to 240,000 characters.
A player recently hit the old Wargamer limit and had to feed their rules in pieces. The new limit gives four times the room.
One caveat. My test cases used shorter rules than the new limits allow, so I haven’t yet seen Flash handle a 40-page paste. I’m watching those turns. If a long-rules answer or a condensed rulebook gets worse, tell me: your reports cover ground my test cases didn’t.