From the #100WorkDays100Articles series - Day no. 30.
Last week, a former colleague called me about their company's "AI transformation strategy."
Forty-seven slides. Eighteen buzzwords. Zero understanding of what intelligence actually is.
Their presentation had everything: massive data lakes, scaled compute infrastructure, enterprise-wide AI deployment. It also had the unmistakable smell of expensive failure brewing.
You see, while this executive was busy planning to throw more money at bigger models, Karl Friston—the world's most cited neuroscientist—was explaining something that would make most Silicon Valley VCs uncomfortable: Intelligence doesn't scale the way you think it does.
The Goldilocks Problem Nobody Talks About
Friston just dropped a truth bomb that should terrify every CEO chasing "AI at scale": True intelligence has a sweet spot. Go too small, you get dumb automation. Go too big, you get diffuse nonsense that can't make coherent decisions.
He calls it the Goldilocks Principle, and it explains why your AI initiatives keep producing mediocre results despite astronomical budgets.
Think about it. Your current AI is probably doing one of two things:
Being too stupid: Processing transactions, flagging emails, recommending products. Reactive. Mindless. No different from a very expensive calculator.
Being too sprawling: Trying to optimize everything at once, creating systems so complex they can't actually understand what they're optimizing for.
Neither approach creates what Friston calls "recursive agency"—the ability to know that you're the one making decisions.
And here's the kicker: without that self-awareness, you don't have intelligence. You have automation pretending to be smart.
What Real Intelligence Actually Looks Like
Real intelligence isn't about processing more data. It's about what he calls "strange loops"—recursive self-modeling that lets a system understand it's the cause of its own experiences.
Your brain doesn't just see a coffee cup. It knows that it's doing the seeing, predicts what will happen if it reaches for the cup, and models how its actions will change the world.
Most business AI can't even pass that basic test. It processes inputs and generates outputs without any understanding that it's the one making choices.
Here's a simple question that will expose the gap: Ask your AI system to explain not just what decision it made, but why it believes it was the right entity to make that decision in the first place.
Watch it break.
The Mortal Computation Revolution
But here's where Friston gets really interesting for business leaders. He argues that current computer architecture—the von Neumann systems running your enterprise—literally cannot support real machine consciousness.
Why? Because processing and memory are separated. In biological intelligence, computation and physical structure are inseparable. Your neurons don't just process information; they are the information, embodied in living tissue that changes based on experience.
Friston calls this "mortal computation"—intelligence that's embedded in physical reality, not abstracted from it.
For business, this means something radical: Your AI needs to be embedded in your actual operations, not sitting in a separate system making recommendations that humans then implement.
The companies that figure this out first won't just have better AI—they'll have intelligent organizations that think, adapt, and evolve like living systems.
Why Your Competitors Are Climbing the Wrong Mountain
While everyone else obsesses over scaling models and collecting more data, they're missing the fundamental insight: consciousness isn't about size, it's about structure.
I've watched companies spend millions building AI systems that can process everything but understand nothing. They optimize for metrics without grasping context. They scale compute without developing wisdom.
It's like building a bigger and bigger library without training anyone to read.
The real opportunity isn't in scaling dumb systems. It's in finding your organization's Goldilocks Zone—the optimal level of complexity where genuine intelligence emerges.
Too simple: Your AI just follows rules without understanding consequences.
Too complex: Your AI gets lost in its own complexity and can't make coherent decisions.
Just right: Your AI develops recursive self-awareness and can genuinely collaborate with human consciousness.
The Three Questions That Will Change Everything
Here's how to find your Goldilocks Zone. Ask these three questions about every AI system you're considering:
1. Does it know that it's making decisions? Not just "can it choose between options," but "does it understand that it's the agent doing the choosing?" If your AI can't model its own role in decision-making, you're building expensive automation, not intelligence.
2. Can it question its own assumptions? Real intelligence adapts when reality doesn't match predictions. If your AI just doubles down on its training when faced with surprising results, you've built a very sophisticated form of stupidity.
3. Is it embedded in actual business operations? If your AI sits in a separate system making recommendations that humans implement, you're missing the mortal computation revolution. Intelligence needs to be embodied in the work itself.
Most enterprise AI fails all three tests. Which explains why most enterprise AI initiatives deliver disappointing results despite massive investments.
The Consciousness Edge
But here's what gets me excited about this moment. While your competitors chase bigger models and more data, you could be building something entirely different: organizations that think.
Not just organizations that process information, but organizations that develop genuine understanding, adapt intelligently to changing conditions, and evolve their approaches based on deep learning from experience.
This isn't science fiction. It's the natural next step for businesses that understand consciousness isn't just a human trait—it's an organizational capability.
The companies that master this won't just have better AI. They'll have better decision-making, better adaptation, better stakeholder relationships, and better long-term thinking.
They'll become what I call Conscious Organizations—businesses that operate with genuine intelligence rather than just sophisticated automation.
Your Goldilocks Moment
So here's my challenge to you: Stop asking "How can we scale our AI?" and start asking "How can we make our organization genuinely intelligent?"
The answer isn't in buying bigger models or collecting more data. It's in finding that sweet spot where your business develops recursive self-awareness—where it genuinely understands its impact on stakeholders and can adapt its approach based on that understanding.
The companies that find their Goldilocks Zone first will define the next era of business. The ones that don't will keep throwing money at systems that can process everything but understand nothing.
Which side of that divide do you want to be on?
Dan Thompson spent 25 years in corporate IT before discovering that most business AI is unconscious automation pretending to be intelligent. He helps conscious leaders build organizations that actually think. This is part of his #100WorkDays100Articles series documenting the journey from corporate executive to consciousness evangelist.
Sources:
Karl Friston – Why Intelligence Can't Get Too Large (Goldilocks Principle)
Personal observations from 25 years of watching companies confuse activity with intelligence
Way too many boardroom presentations about "AI transformation"
