← Back to list

The Cognitive Monoculture: Why We Need to Understand What We’re Building with AI

I was pregnant with my youngest son in 2008. I had no idea what a subprime mortgage was. I just knew something was very, very broken and…

The Sarah · 2026-05-07 17:58 · 3 claps · 6.4 min read
#ai #2008-financial-crisis #foundation-models #ai-governance #critical-thinking
Open on Medium ↗
Wiki topics: AI · AI · General PFI · Personal Finance ECO · Economy · General CUL · Culture & Media HUM · Humanities · General

The Cognitive Monoculture: Why We Need to Understand What We’re Building with AI

I was pregnant with my youngest son in 2008. I had no idea what a subprime mortgage was. I just knew something was very, very broken and thousands of people were affected.

What stays with me isn’t the crisis itself. It’s what happened after. When you finally understand the architecture; how everyone had the same broken model, the same assumptions, the same risk on their books. It wasn’t conspiracy. It was structure. And structure is harder to see than greed.

That’s what I think about now. Because we’re building something similar. Except this time, most people won’t see it until it’s too late. And that’s actually the point worth understanding.

What’s Actually Happening

So here’s the thing. Across the US and Europe, every major bank is making the same choice: use GPT, or Claude, or Gemini. Sometimes all three, but for different things.

One major US bank’s AI system handled 245 million customer interactions last year. A European bank — big one — put ChatGPT Enterprise in front of 3,000 employees. Just a pilot, right? Five months later, 11,000 employees. Because it worked.

Across Europe, banks are collaborating on fraud detection. Smart move, they keep their data private but train together. Except they’re training on the same models, the same logic, the same assumptions about what fraud looks like.

Different countries. Different regulators. Different logos on the buildings. Same three thinking engines underneath.

Is this a problem? Not really. Not yet. These models ARE better. They’re cheaper. Every bank that chooses them is making the right decision for themselves. That’s the thing nobody talks about: sometimes doing the right thing individually creates a different problem at the system level.

We’re building what I call a cognitive monoculture. Thousands of institutions thinking through the same three reasoning engines. And we need to understand what that actually means.

The Two Layers (and why you probably don’t see the second one)

Most conversations about AI in banking stop at the surface. But there are two layers here, and they work differently.

The first layer is visible.

ChatGPT, Claude, Gemini, they’re running productivity tools, risk reports, customer chat. When OpenAI or Anthropic updates their model, that change hits thousands of organisations on the same day. Everyone’s productivity tools update. Everyone’s risk systems recalibrate. In isolation, fine. All at once, across an entire industry? That’s a different story.

Most boards don’t even notice this is happening. They think AI updates are just… IT stuff.

The second layer is the interesting one. The structural one.

The models that actually make decisions likecredit decisions, fraud detection, trading signals, they’re trained on the same foundations. Banks keep their data separate, sure. Your fraud data stays in your vault. But you’re training it on the same underlying architecture. The same way of thinking about what fraud even is.

So you can have different data. But you have the same reasoning.

That matters. A lot.

Because when the model encounters something it hasn’t learned to recognize, something that breaks the assumptions it was built on all the banks miss it the same way. Not because they have bad teams. Because they’re all running variations of the same logic.

What This Actually Creates

Let me give you some concrete examples of how this plays out:

First: You all miss the same things.

Imagine a fraud pattern that exploits an assumption shared across all three foundation models. Org A’s fraud detection misses it. Org B misses it. Org C, D, E all miss it. The fraud gets through everywhere simultaneously. By the time anyone figures out what happened, it’s already a pattern across the system.

This isn’t theoretical. Similar things happen all the time with shared infrastructure. When everyone uses the same third-party service and it breaks, everyone breaks together.

Second: During volatility, you all make the same trading decisions.

Market shock hits. AI systems across Wall Street and Frankfurt interpret the data the same way. Trading signals fire simultaneously. Everyone starts selling at once. Your competitors are selling. Their competitors are selling. You’ve created a liquidity problem that no single bank can stop, and that central banks find out about too late.

Again, not coordination. Just structure. Everyone making the individually rational choice at the same moment.

Third: A model update can changes how you think about risk.

A foundation model gets updated. Supposed to improve efficiency. Instead, it subtly shifts how risk gets weighted. Maybe underweights certain types of credit risk. That shift spreads through every bank’s decision-making simultaneously. Silent. Invisible until it shows up in your rates.

These aren’t disasters. They’re design characteristics of consolidation. And we need to understand them.

The Conversation That Needs to Happen

If you can’t answer these questions clearly, your governance is missing something important:

How concentrated are you? Which foundation models power your critical decisions? What percentage of your credit decisions run on each one? If one provider has a problem, what actually breaks?

When you diversified your fraud data, did you diversify your thinking? Or are you running different data through the same model? These are completely different things. One is security through diversity. The other is… security theater.

Who decides when you update your models? When OpenAI pushes a change, when do you deploy it? Someone should be testing what that change does to your decision-making before it touches your production systems. Is that happening?

Do your stress tests accounts for this? You test what happens if your bank fails. Do you test what happens if your bank AND your top five competitors’ AI systems all reach the same wrong conclusion at the same time? Because that’s a different kind of stress.

If your main model provider breaks, what do you actually do? Not what you’d like to do. Actually do. How fast can you switch? Can you even?

These aren’t alarmist questions. They’re the questions you’d ask about any critical infrastructure.

What Actually Needs to Change

The good news (I think): This isn’t inevitable. You’re still at a point where you can design better.

For your org: Map where AI makes decisions. Actually map it. Which models? Which reasoning engines? Where would consolidation hurt you most?

For your critical decisions — credit, fraud, trading — think about whether you want different reasoning engines involved. Not everything needs diversity. But the decisions where correlated failure has systemic impact? Those ones do.

When a model updates, someone should understand what that means for your decision-making. Make that a governance conversation, not just an IT conversation.

Test what happens if you’re wrong together with your competitors. Not the catastrophe version. Just — what if your AI and their AI both misread the market the same way? What does that look like? Can you handle it?

For the model providers:

Be honest about what happens if things break. Not the marketing version. What actually happens if your API goes down? How fast can banks switch away? Publish real contingency plans, not nice words.

When you update a model, tell your customers what behavior changes. Not “improved efficiency.” What actually changes about how decisions get made? Customers need that information to govern well.

For regulators:

Stop thinking about AI risk the way you thought about mortgage risk. This isn’t about individual bad decisions. This is about structure. Build stress tests that actually model what happens when reasoning engines are correlated across thousands of institutions.

If a model provider is critical infrastructure for thousands of banks, they should have standards. Resilience standards. Transparency standards. Same way you would for any critical infrastructure.

Why this matters

Here’s what I think about during this sabbatical: We’re at a moment where the architecture is still being formed. Banks are still making their deployment choices. You can still build diversity and resilience into your systems by design.

In a couple of years, this will be locked in. Institutions will have invested so much, trained so many people, built so many systems on top. The cost of changing will be huge. But the window to do it deliberately? That’s closing.

The orgs that understand this now, the ones asking these questions, building for resilience, thinking about correlated risk, they’ll have an advantage. Not just in risk management. In confidence. In knowing they can actually operate at scale without hidden vulnerabilities.

The ones that treat AI like a vendor management problem? They’ll be playing catch-up when regulators finally catch up and standards get written.

The Real Point

I lived through 2008. I didn’t understand what was happening at the time. I just knew people were losing their houses and the system was broken. What I learned later is that it wasn’t complexity that made it hard to see. It was architecture. Everyone had the same broken model, and breaking it together looked invisible until it suddenly wasn’t.

We’re making a similar architectural choice right now. Consolidating reasoning around three foundation models. It’s rational. It creates real benefits. It also creates a specific kind of vulnerability that traditional risk frameworks don’t even have words for yet.

The professionals responsible for governance right now, you get to decide whether we understand this deliberately or whether we figure it out the hard way.

Understanding what we’re building. Asking good questions about how it fails. Designing for resilience. That’s the real work in AI right now.

Not moving faster. But thinking harder.

I’m curious what you think. Are these the questions your governance framework is asking? What am I missing? This conversation needs to happen everywhere, and I’d rather we have it now than learn these lessons the expensive way.

If you want to understand the financial crisis of 2008, how and the why: watch The Big Short.


메타데이터
post_id
a824672c20bb
slug
the-cognitive-monoculture-why-we-need-to-understand-what-were-building-with-ai-a824672c20bb
url
https://medium.com/@SarahAs/the-cognitive-monoculture-why-we-need-to-understand-what-were-building-with-ai-a824672c20bb
canonical_url
https://medium.com/@SarahAs/the-cognitive-monoculture-why-we-need-to-understand-what-were-building-with-ai-a824672c20bb
author_url
https://medium.com/@SarahAs
status
ok
fetched_at
2026-06-09 15:37:30