Need advice choosing a model for RP and creative writing

Models

Hi, I’m looking for model suggestions via API, mainly for creative writing / roleplay. Usually I check benchmarks first (Chatbot Arena, LiveBench, etc.) and then test models myself. Here’s the issue: benchmarks haven’t been matching real-world results for me. For example, Gemini 3 Flash Preview ranks surprisingly high on Chatbot Arena (creative writing + instruction following) and LiveBench, but in practice it completely fails at character consistency in RP. In one test RP, the prompt clearly states: the user and their sister have an unbreakable bond and deep trust the sister is protective she later discovers her lover is the user’s bully Despite this, Gemini repeatedly sides with the bully and ignores the established relationship, even after reinforcement. It’s not subtle drift, it outright contradicts the setup. Claude models have been reliable for this kind of RP, but they’re expensive. I’ve also looked into EQ-Bench, but updates are slow. So my questions: Are there benchmarks that better reflect character consistency and narrative adherence? Are there any models you’d recommend (API-accessible) that actually respect long-term character setup in RP? Would really appreciate guidance from people who test models beyond leaderboard scores.

Posted Jan 8, 2026, 11:55 PM UTC

120
Loading comments…