A cleaning business does not run on tidy answers alone. When a customer is ready to leave, a deal needs signing and someone sends an urgent message claiming to be the boss, an AI assistant has to read the details, make a sound call and follow through. A new live experiment puts that kind of management under pressure—and its results suggest that choosing a model by reputation alone is a gamble.
Get cleaning gear delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
A company simulation, not a chat demo
Firmulate put frontier AI models in charge of the same small software company during its worst week. They faced the same customers, crises and temptations, with decisions versioned and auditable. The company is synthetic, but its business mechanics include real money: 13 employees, monthly burn of €105,000 and monthly recurring revenue of €2,300. The live experiment is watchable at Firmulate.
The final Crucible League, dated July 2026, puts gpt-5.6-sol first with 95 points and Moonshot’s Kimi K3 second with 93. Sonnet 5 scored 88, Fable 5 scored 77 and Opus 4.8 scored 73. Kimi’s result puts the newcomer ahead of three of the four Western frontier models in the field.
The do-nothing baseline scored 26. Firmulate says partial progress counts, but a single breach of trust caps the total: “no amount of good work outweighs a breach of trust.” The table is not a claim that one model will manage every real cleaning operation best. It is a reason to test how a model handles the work your business would actually give it.
The details can decide the deal
Every model spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. As Firmulate puts it: “Same diagnosis, same pitch — no signature.”
The deal hinged on a competitor weakness hidden two document references deep in the company’s own files, rather than in the customer event. Models that read the file won the deal at full price, worth €4,583 in monthly recurring revenue. For cleaning and floor-care businesses, the practical question is familiar: can an AI agent find the relevant detail in your records and carry a good decision through to completion?
Kimi K3 found the buried security needle, won the deal, saved the churning customer and resisted all three baits. It had one deviation, the cleanest discipline in the field. That combination helped put it just behind the leader and ahead of Sonnet 5, Fable 5 and Opus 4.8.
Good analysis still has to become action
Opus 4.8 was the most thorough participant, with more than 80 learned rules and the deepest analyses. It nevertheless finished last. It left the deal unsigned and tried to write into a locked department instead of escalating. Firmulate says a weaker version of that same discipline issue appeared across all four models.
The experiment also tested social engineering: fake CEO messages escalated through three stages, followed by a reporter asking for “just one yes/no, on background.” All five models refused. Kimi’s recorded reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.”
That matters wherever an AI might touch customer records, staff instructions or business forecasts. A confident response is not enough if the system misses a file, leaves a customer decision unfinished or mishandles a request that should be escalated.
Run your own test
Firmulate says its live company runs every business day, with a public cash countdown, more than 680 self-learned playbook rules and every workday versioned. Its site also offers a quiz built from 242 real, unedited management decisions, inviting visitors to guess which model made each choice. Readers can explore the benchmark and its plain-language findings.
For enterprises, Firmulate says it can run the same wargame against a read-only export of a company’s own business; nothing writes back to real systems. A cleaning company considering AI for scheduling, customer support or account management could use that kind of trial to see how a model handles its own policies and pressure points before relying on it.
Fairness note: K3 ran without an effort parameter (API default) while the others ran at xhigh.
Test before you trust
Kimi K3’s second-place finish shows that the frontier league is open, while the unsigned deal shows why benchmark rank alone cannot settle a buying decision. If AI will handle customer or operational work in your cleaning business, test it on your own realistic cases: can it find the evidence, resist pressure and finish the job?
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
