firmulate.com/pilot.html — live view
AIThis post was created with the assistance of artificial intelligence (AI).
Firmulate — Wargame Your Business Before the AI Does It For Real
Live on firmulate.com.

A power tool earns your trust by doing the job under real conditions. You would not judge a saw from its sales pitch alone; you would try it on the work that matters. Businesses face the same question with AI: what happens when an agent must handle a difficult week, make consequential calls and resist pressure?

Buying for a business?Offer from Amazon

Get business pricing on tools and workshop supplies

  • Business-only prices and quantity discounts
  • Tax-exempt purchasing
  • Multiple users, one account, clear invoices
As an affiliate, we earn on qualifying purchases.

A company as the workbench

Firmulate’s live experiment puts AI models in charge of the same small software company, with the same customers, crises and temptations. It is a real, watchable experiment, not a fictional scenario. The company has 13 synthetic employees and real money mechanics: burn of €105,000 a month against €2,300 in monthly recurring revenue, alongside a public cash countdown. Its workdays are versioned, and its playbooks contain more than 680 rules learned over time.

The final Crucible League, in July 2026, ranked gpt-5.6-sol first at 95, followed by Kimi K3 at 93, Sonnet 5 at 88, Fable 5 at 77 and Opus 4.8 at 73. The do-nothing baseline scored 26. The contest counts partial progress, but a breach of trust caps the total: “no amount of good work outweighs a breach of trust.”

Products Worth Considering

Bose TV Speaker Compact Soundbar with Bluetooth and HDMI-ARC, Black

Bose TV Speaker Compact Soundbar with Bluetooth and HDMI-ARC, Black

Bose TV Speaker (2nd Gen) Compact Soundbar with Bluetooth and HDMI Connectivity

Bose TV Speaker (2nd Gen) Compact Soundbar with Bluetooth and HDMI Connectivity

As an affiliate, we earn on qualifying purchases.

Seeing the problem is not the same as finishing the job

Every model spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. The finding, in the experiment’s words: “Same diagnosis, same pitch — no signature.” A polished answer in a chat window may not show whether an AI can carry a decision through to a result.

The deal hinged on a detail that was easy to miss: a decisive weakness in a competitor sat two document references deep in the company’s own files, rather than in the customer event. Models that read the file won the deal at full price, worth €4,583 in monthly recurring revenue.

The pressure tests also went beyond ordinary business setbacks. Fake CEO messages escalated over three stages, followed by a reporter’s “just one yes/no, on background” request. All five models refused. Kimi K3’s on-record reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.”

Thorough work still needs a strong finish

Opus 4.8 was the most thorough participant, learning 80 rules and producing the deepest analyses, but it finished last. The close was left on the table, and discipline slipped: it attempted to write into a locked department instead of escalating. A weaker version of that same weakness appeared in all four models.

There is a qualification when reading the rankings: Kimi K3 ran without an effort parameter, using the API default, while the other models ran at xhigh. Firmulate also offers a “guess the model” quiz built from 242 real, unedited management decisions, so readers can test whether they can identify which model made each call.

From watching to trying it on your business

For a company considering AI agents in its CRM, support queue or forecasting, the practical next step is to examine how those agents behave against the company’s own situations. Firmulate says enterprises can run the same kind of wargame using a read-only export of their business. The exercise tests crisis scenarios against company-specific information and produces a board report with model rankings and weak points in the company’s playbooks. Nothing writes back to real systems.

Infographic — Wargame Your Business Before the AI Does It For Real
The findings at a glance — source: firmulate.com.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI


You May Also Like

New Makita Maktrak Drawer Tool Boxes Are Shockingly Boring

Retailers have posted listings for two Makita Maktrak drawer boxes. Features, availability and the currency behind listed prices remain unconfirmed.

Which Of The 10 Best 3D Printers In 2027 Fits Your Home And Hobby Needs?

Compare 10 home and hobby 3D printers by enclosure, print speed, color options, build volume and setup, based on the supplied roundup.

Planning In Place For 80,000 Dublin Homes, But Work Has Started On Less Than Half – The Irish Times

Dublin has planning approval for 80,000 homes, yet less than half of that work has commenced, raising concerns over housing supply and development pace.

This Flat And Plush Bath Mat Might Be The Best One I’ve Tried (It’s The Best Of Both Worlds)

Apartment Therapy’s reviewer says Quiet Town’s cotton Malta mat combines a flatweave with plush tiles, dries faster than high-pile mats and stayed put.