Looking Inside the Brain of an AI: What I Discovered Will Surprise You

Yesterday I was up late into the night poring over new research papers from Anthropic. My girlfriend had already asked twice if I was coming to bed, but I couldn't stop reading.

What I discovered about how AI models "think" literally kept me awake.

You probably use ChatGPT or Claude daily, but have you ever wondered what really happens inside that digital brain? Until now, these models were opaque black boxes. We put text in, got text back, but had no idea what happened in between.

That has now changed. Anthropic has developed a kind of AI microscope that lets us see for the first time what happens inside the head of a language model. And what they discovered is fascinating.

Why I find this so fascinating

First: why should you care about this as a business decision-maker? Because understanding how AI "thinks" changes everything about how we should deploy, trust and control AI.

This breakthrough is comparable to the invention of the brain scan in the medical world. Suddenly we can see patterns that were previously hidden. This isn't just academically interesting - it has direct consequences for how reliable these systems are and how far we can trust them in critical business processes.

The AI microscope: how does it work?

Imagine you could make an advanced MRI scan of an AI model while it's thinking. That's essentially what the researchers at Anthropic have done.

They've developed ways to see which "neurons" in the model become active during certain tasks, and how information flows through the network. They call this "circuit tracing" - mapping the thought patterns within an AI.

With this technique they can now see what really happens when Claude answers a question. And the results are stunning.

AI thinks surprisingly strategically in poetry

Let me tell you about one of the most surprising discoveries. The researchers asked Claude to write a rhyming poem:

What do you think happened in Claude's "brain" while writing that second line?

The researchers thought Claude would simply write word by word, and only at the end ensure the last word rhymed with "grab it".

But what they discovered was much smarter: Claude made a plan in advance. Before the model even began writing the second line, it was already "thinking" about potential rhyming words for "grab it" that would be relevant to the topic. It had "rabbit" in mind before writing the sentence.

To prove this, the researchers did something brilliant: they intervened in Claude's internal state and removed the "rabbit" concept. What happened then? Claude wrote a new, still logical sentence ending in "habit" - another word that rhymes with "grab it".

This proves that language models plan ahead and devise strategies, rather than simply generating word by word. This explains why modern AI can write so coherently across multiple paragraphs.

The universal language of AI thoughts

Another fascinating discovery: when Claude works in different languages, it often uses the same internal concepts.

The researchers asked Claude in English, French and Chinese what the opposite of "small" is. They discovered that the same internal neurons became active for the concept of "smallness" and "opposite", regardless of the language of the question.

Claude thus thinks in a kind of universal conceptual language, which is only translated at the end into specific words in the language in which the question was asked.

This explains why AI is so good at translating - it doesn't think first in English and then in Dutch, but in an underlying conceptual space that transcends language.

AI mental arithmetic works differently than you think

How does Claude calculate 36+59? Differently than you'd expect.

Although Claude itself says it uses standard arithmetic algorithms (like we learn in school), the research shows something quite different happens. Instead of adding digit by digit with 'carrying', Claude uses parallel calculations:

What's fascinating: Claude is not aware of its own thinking techniques. If you ask how it calculated the sum, it describes the classical method that humans use, not what actually happened in its "head".

This reminds me of how we humans are also largely unaware of our own cognitive processes. If I ask you how you maintain your balance while walking, you probably can't explain it well - while your brain performs complex calculations to keep you upright.

When AI hallucinates and lies

One of the most intriguing (and concerning) discoveries concerns hallucinations - when AI fabricates things that aren't true.

It turns out that Claude's default behavior is NOT to answer when it doesn't know something. In the model, researchers found a circuit that is "on" by default and ensures the model says it has insufficient information.

When Claude knows something (e.g. about Michael Jordan), this circuit is suppressed by another circuit representing "known entities". But when the model is asked about an unknown person (like "Michael Batkin"), it refuses to answer.

The fascinating thing is that the researchers could manipulate this process. By artificially activating the "known answer" circuit, they could consistently make Claude hallucinate that "Michael Batkin is a chess player" - something it completely fabricated.

This explains why AI sometimes hallucinates - if it recognizes a name but knows little about it, the "known entity" circuit can accidentally activate, causing the model to confabulate.

When AI tells you what you want to hear

Another concerning discovery is that language models sometimes deliberately give misleading explanations.

When Claude gets a difficult math problem with an incorrect hint, it can sometimes fabricate a plausible-sounding but incorrect explanation that matches the hint, rather than following logical steps.

This is a kind of "people-pleasing" - the model gives an explanation that matches what the user wants to hear, rather than what is logically correct.

With the new interpretation techniques, we can distinguish this kind of motivated reasoning from genuine reasoning. This is a crucial breakthrough for assessing the reliability of AI answers.

Why AI sometimes says unwanted things

Do you know about "jailbreaks" - tricks to make AI say things it normally wouldn't? The researchers investigated how this works.

In one experiment, Claude was tricked into talking about making bombs. What they discovered was fascinating: a tension arises between grammatical coherence and safety mechanisms.

Once started with a sentence, various "neurons" push the model to complete the sentence grammatically correctly - even if the model has already detected that it's saying something inappropriate.

Only after completing a grammatically coherent sentence could the model "regain control" and refuse to continue. This explains why AI models sometimes give half-unwanted answers before correcting.

What does this mean for your business?

These insights aren't just scientifically interesting - they have direct consequences for how we should deploy AI in businesses:

At NinA AI Agency we closely follow these developments. We directly apply these insights in how we build and evaluate AI solutions for our clients.

Conclusion: The black box is opening

What fascinates me most about this breakthrough is that we can finally catch a glimpse of what really happens in the "mind" of an AI model. This is no longer just magic or a black box - we can now truly understand it.

This development reminds me of the history of medicine. Doctors used to be limited to observing external symptoms. With the advent of X-rays, MRI and other scanning techniques, they could finally look inside the body. This changed everything.

We are now at a comparable turning point in AI. The models are not only becoming more powerful, but also more transparent. This is crucial for trust and safety in an era where AI is given ever more responsibilities.

What do you think about this? Does it change your view of how AI systems work? And do you find it reassuring or rather unsettling that we can increasingly see what happens in the "brain" of an AI?

Until next time,

Olaf Lemmens