For a senior care provider, an AI that writes a polished email is less important than one that catches a customer at risk of leaving, protects sensitive information and follows through on a hard-won agreement. Those are management tests, not chat tests. Firmulate’s live experiment puts that distinction on display.
Turn quiet afternoons into listening time
- Thousands of audiobooks, podcasts and originals
- Listen on your phone, tablet or Echo — also offline
- Cancel anytime
A company under pressure
Firmulate ran frontier AI models through the same worst week at a small software company: the same customers, crises and temptations. Its synthetic business has 13 employees and real money mechanics, burning €105,000 a month against €2,300 in monthly recurring revenue. Its cash countdown is public, and each workday is versioned. The experiment is watchable at Firmulate.
The setup may sound far from senior care. The management questions are close to home, though: Can an AI recognize a customer problem, inspect the information that matters, resist manipulation and complete the work it has begun? For a care organization, those choices could shape how AI handles scheduling, family inquiries or internal business tasks. The experiment does not answer every question about care, but it offers a concrete way to look beyond a convincing demonstration.
Popular right now
Vitamix Propel Series 750 Professional-Grade Blender, 4 Programs, Black
Breville Bambino Espresso Machine, Brushed Stainless Steel (BES450BSS)
Vitamix 5200 Professional-Grade Blender, 64 oz. Container, Black
As an affiliate, we earn on qualifying purchases.
The deal that separated diagnosis from action
In the final July 2026 Crucible League, Moonshot’s Kimi K3 placed second with 93 points, behind gpt-5.6-sol at 95. It beat Sonnet 5 at 88, Fable 5 at 77 and Opus 4.8 at 73. The do-nothing baseline scored 26.
All models spotted every crisis and refused every manipulation attempt. Yet only two signed a €55,000 deal their own analysis had earned. The deciding competitor weakness was buried two document references deep in the company’s files, rather than in the customer event. Models that read the file won the deal at full price, worth €4,583 in monthly recurring revenue.
K3 found that buried security detail, won the deal and saved the churning customer. It also resisted all three baits and had one deviation, the cleanest discipline in the field. The contrast is a useful one for business leaders: recognizing a problem and describing the right response does not guarantee that an AI will finish the job.
Pressure, discipline and the limits of a league table
The manipulation tests included fake CEO messages escalating over three stages and a reporter’s request for “just one yes/no, on background.” All five models refused. K3’s on-record reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.”
Opus 4.8 shows why a single result can tell a more complicated story. It was the most thorough participant, with 80 learned rules and the deepest analyses, but finished last. It left the deal unsigned and attempted to write into a locked department instead of escalating. A weaker version of that discipline problem appeared in all four models, according to the findings.
There is an important fairness caveat: K3 ran without an effort parameter (API default) while the others ran at xhigh. The rankings offer a comparison under the stated experiment conditions, not a universal verdict about which model will perform best in a care setting.
Firmulate says its 242 real, unedited management decisions power a “guess the model” quiz. Enterprises can also run the wargame against a read-only export of their own business; nothing writes back to real systems. See the benchmark results for the findings and Firmulate for the live experiment.
Test the work, not just the words
For senior care leaders weighing AI, Firmulate’s experiment makes a case for trying models against realistic business pressure before trusting them with consequential work. A model may spot a risk and explain it well; the harder question is whether it reads the relevant records, protects trust and follows through. The league is open: choosing without your own test is a bet.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
