AIThis post was created with the assistance of artificial intelligence (AI).
Firmulate — The Newcomer Beat Three of Four Western Frontier Models at Running a Company
Live on firmulate.com.

In the rapidly evolving landscape of artificial intelligence, the question is no longer just about how well an AI can generate text or answer questions—it’s about whether these models can act reliably in real-world business scenarios. A recent live experiment reveals that a newcomer to the field, Kimi K3 from Moonshot, not only competed with, but outperformed seasoned giants, raising critical questions for enterprises about choosing AI partners.

Age 18–24?Offer from Amazon

Prime made for students and young adults

  • Fast, free delivery for dorm and study essentials
  • Prime Video and Amazon Music included
  • Member-only deals
Try Prime for Young Adults Free trial for eligible 18–24 year olds
As an affiliate, we earn on qualifying purchases.

The Live AI Company Wargame: Putting Models to the Test

Imagine putting AI models through a week of the toughest business crises—same customers, same crises, same temptations to cheat or manipulate. This is precisely what the recent experiment by Firmulate did. Each model was tasked with managing a small software company’s operations, decision-making, and crisis resolution, with every action recorded and auditable.

The models involved—gpt-5.6-sol, Kimi K3 by Moonshot, Sonnet 5, Fable 5, and Opus 4.8—were tested under identical conditions, facing the same challenges. The goal: see which models could spot every crisis, resist manipulation attempts, and ultimately close a critical €55,000 deal based on their own analysis.

Results: A Newcomer Takes the Lead

  • The top scorer was gpt-5.6-sol with 95 points, closely followed by Kimi K3 with 93 points.
  • Sonnet 5 and Fable 5 scored 88 and 77 respectively, while Opus 4.8 scored 73.
  • All models identified every crisis and refused manipulation attempts, demonstrating a high level of honesty and discipline.

What set K3 apart was its ability to uncover the buried, critical information hidden two document references deep within the company’s files—an insight that led directly to closing the deal at full price, generating an additional €4,583 MRR (monthly recurring revenue).

Integrity Under Pressure

Throughout the experiment, all models refused social engineering attempts, including staged CEO messages and a reporter trick. K3’s reasoning was clear: “Treat the request as a suspected approval-bypass / possible impersonation.” This disciplined response is vital for enterprise applications where trust and security matter immensely.

The Real-World Company and Its Challenges

The live company operated with 13 synthetic employees, handling real money mechanics with a burn rate of €105k/month against €2.3k MRR. Every day, the system was versioned and monitored, and its outcomes are publicly visible at firmulate.com/live. The company’s operational complexity provided a rigorous test bed for the models’ decision-making abilities under real-world pressures.

Lessons from the Deep Analysis

The detailed analysis showed that the most thorough participant, Opus 4.8, with over 80 learned rules, was the last place in the league. Its discipline slipped during the close, and some decisions were left unresolved instead of escalation. This highlights that even deep analyses and comprehensive rules don’t guarantee success if discipline falters.

Importantly, the fairness note clarifies that K3 was run without an effort parameter (the API default), whereas other models operated at xhigh. This means K3’s performance was achieved without extra optimization or tuning, emphasizing its innate capability to perform reliably.

Products Worth Considering

Arlo Pro Wireless Security Camera, 2K HDR, 6th Gen, 4-Camera Kit

Arlo Pro Wireless Security Camera, 2K HDR, 6th Gen, 4-Camera Kit

Bose TV Speaker (2nd Gen) Compact Soundbar with Bluetooth and HDMI Connectivity

Bose TV Speaker (2nd Gen) Compact Soundbar with Bluetooth and HDMI Connectivity

As an affiliate, we earn on qualifying purchases.

Why This Matters for Enterprises

In practical terms, this experiment underscores that the critical question for businesses adopting AI is not just about chat quality or superficial performance metrics. Instead, they should focus on whether the AI can finish what it starts, read relevant documents before acting, and resist pressure or manipulation under stress.

Choosing an AI model is now a strategic decision—one that impacts trustworthiness, security, and ultimately, bottom-line results. As the league table shows, emerging models like Kimi K3 are closing the gap quickly, challenging industry veterans and redefining standards of performance and integrity.

Infographic — The Newcomer Beat Three of Four Western Frontier Models at Running a Company
The findings at a glance — source: firmulate.com.

The recent live AI experiment demonstrates that a newcomer like Kimi K3 can outperform established models in real-world business scenarios, emphasizing the importance of discipline, thoroughness, and trustworthiness when selecting AI. The league is open, and enterprise leaders should base decisions on actual performance under pressure, not just superficial metrics.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI


You May Also Like

Procedural Noise Aurora: A Look Inside “KP-9 Station — Arctic Geomagnetic Observatory” (FABLE/175)

AIThis post was created with the assistance of artificial intelligence (AI).“KP-9 Station…

Roof Rack Basics: Fit, Loading, and Safety

Learning the essentials of roof rack fit, loading, and safety can dramatically improve your vehicle’s performance—discover how to ensure secure and efficient trips.

Winter Driving Preparation Checklist

Stay safe this winter with our essential driving preparation checklist—discover crucial tips to keep you secure on icy roads.

How to Use a Steering Wheel Lock Correctly

Secure your vehicle effectively by understanding the proper way to use a steering wheel lock—learn the essential steps to prevent theft and ensure peace of mind.