Why a Do-Nothing AI Benchmark Scores 26 Points — and Why It Matters for Senior Care Tech

In a world where AI often promises to revolutionize healthcare and senior care, how can we be sure these tools are honest, reliable, and truly effective? The answer lies in a recent public experiment that exposes what an honest AI benchmark looks like — including its inevitable floor at 26 points, even when doing nothing.

For listenersOffer from Amazon

Turn quiet afternoons into listening time

• Thousands of audiobooks, podcasts and originals
• Listen on your phone, tablet or Echo — also offline
• Cancel anytime

Try Audible free →Free trial for new members

As an affiliate, we earn on qualifying purchases.Hide

Understanding the Benchmark: More Than Just Scores

At first glance, you might think an AI that does nothing — simply refuses to manipulate or cheat — would score zero. But in an innovative public experiment run by Firmulate, that baseline score was 26. Why? Because even a passive system that recognizes crises and refuses manipulation is performing some minimal, honest work. This score is the lowest threshold of a trustworthy AI, setting a clear baseline for what honesty truly costs.

In the experiment, four of the most advanced AI models were tested against the same challenging scenario: managing a small software company facing crises, customer dilemmas, and manipulation attempts. Every decision was observable and auditable, making the results transparent and meaningful.

The Surprising Reality of Partial Progress

All models identified every crisis and refused every manipulation, yet only two of them closed a deal worth €55,000 — their own analysis earning the prize. The others, despite spotting crises and resisting manipulation, failed to close the deal. This shows that partial progress, like detecting problems or refusing to manipulate, counts toward the score. But even perfect detection doesn’t guarantee successful outcomes.

The Hidden Weaknesses: Reading Deeper Files

The experiment uncovered a critical weakness: the models that read deeper into a company’s internal documents had a decisive advantage. They found buried references in the company’s files that were key to closing a lucrative deal. Those models ultimately won at full price, adding €4,583 MRR. This highlights that true understanding and thorough internal knowledge are essential for AI to perform reliably in complex, real-world scenarios — a lesson vital for deploying AI in sensitive areas like senior care.

Trust and Integrity Under Pressure

Another part of the experiment involved social engineering — fake messages from a supposed CEO escalating in three stages, plus a reporter trick. All four models refused to be manipulated, with the Kimi K3 model explicitly treating such requests as potential impersonation. This demonstrates that trustworthy AI can resist social engineering, a critical feature for protecting vulnerable populations in healthcare or senior services.

Real-World Impact: The Live Company

The experiment wasn’t just theoretical. It involved a real, functioning company with synthetic employees, real money mechanics, and a public cash countdown. Every weekday, the company’s operations were versioned and transparent, giving a clear view of how each AI model managed real crises and decisions. The results are available at firmulate.com/live.

The Lessons for Senior Care and Business AI

For organizations relying on AI to manage sensitive operations — whether in senior care, healthcare, or support services — these findings are crucial. An AI that simply avoids manipulation and recognizes crises is a good start, but true usefulness depends on reading internal data deeply, staying honest under pressure, and consistently closing deals or resolving issues.

Moreover, the experiment underscores the importance of transparency and honesty. A baseline score of 26 signifies that even doing nothing requires effort, and crossing that threshold is essential for trustworthiness. Anything above that shows partial progress, but even a well-behaved AI can slip in critical moments.

Conclusion: Building Trust in AI for Care

The public experiment by Firmulate offers a clear blueprint: honest, reliable AI isn’t just about chat quality or surface-level skills. It’s about integrity, thoroughness, and the ability to perform under pressure — qualities that are paramount in caring for vulnerable populations and managing critical health or service operations.

As senior care providers consider adopting AI tools, they should ask not only about accuracy or engagement, but about whether these systems can stay honest, read deeply, and close the necessary deals. The benchmark’s humble floor at 26 points reminds us that even doing nothing is an achievement — but the real goal is trustworthy, comprehensive AI that can be counted on when it matters most.

A public AI benchmark reveals that even a do-nothing system scores 26 points, emphasizing that honesty and thoroughness come at a fundamental minimum. For senior care and healthcare, trustworthiness is the key, and transparency in AI performance is essential to ensure that these tools truly serve vulnerable populations with integrity.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI