Producer Arena offers a human-preference evaluation of AI music models inside a working production tool: models make a track in real FL Studio from a described brief, and the better take is picked without knowing which model made it. Its fl-studio-agent is an open-source MCP server working in a user's own copy of FL Studio, writing notes and setting tempo, levels and plugin parameters, with a check command that plays a few bars and reads mixer meters and a render command that records the mix.
Each matchup gives the same brief to models drawn at random, and the votes feed a public leaderboard. Votes are fit with a Bradley-Terry model, and the score shown is the same ability on the Elo scale; completion measures how often a model delivered a usable track, reported as reliability rather than musical quality. The service is early and experimental and free with no paid plans, while visitors can also vote on finished pairs.