Max van IJsselmuiden

home

blog

The dangerous comfort of AI answers

August 11, 2025 by

Max van IJsselmuiden

Every day, millions of people open a generative AI chatbot instead of a search engine. They're using it wrong, and that's partially our fault for not understanding what these tools actually are.

A knowledgeable friend recently argued that people are often using Large Language Models (LLM) to replace Google for reasoning and answering questions, but these models were never good at this. Their strengths, she said, is to extract meaning from text and build upon it, making them primarily useful for writing.

She's both right and wrong, and understanding why explains both LLM popularity and limitations.

Image of a crystal ball generated with GPT-5 (Dall-E 3).

ChatGPT is a household name now. Most people don't know GPT stands for "Generative Pre-trained Transformer" and honestly, why would they? To users, it's a box where you ask anything and get an answer with reasoning. Incredible. A technological breakthrough.

The name tells you everything.

Generative: the model creates text (or images, videos, audio, software code).

Pre-trained: it learned from existing data, not real-time information.

Transformer: a type of neural network architecture that processes patterns in sequences.

Nothing in that name says "truth engine" or "fact database". It says "pattern-based text generator."

At its core, artificial intelligence processes data, identifies patterns, and follows instructions. It cannot think, empathize, or reason. Instead, it replicates how people use knowledge without actually understanding it.

The disconnect between what the technology is (a transformer generating text based on training) and how we use it (as an oracle for answers) is where, in my eyes, danger lies. It’s when we trust these confident-sounding responses for important scenarios such as medical advice, legal guidance, or financial decisions.

The Google replacement that isn’t

When you ask Google a complex question, you get links (at least, before the AI Overview feature). We were used to scanning links, picking sources, and doing the thinking ourselves. Now, when you ask an LLM, you get an answer that feels like understanding, because they're better at understanding intent and context. Google requires translating problems into keywords.

LLMs let you explain problems like you would to a colleague. That's powerful, even when the response is hallucinated nonsense, such as an example I mentioned in an earlier blog: advising people to eat rocks.

What LLMs think they are

I asked several leading generative AI chatbots to define the strength of an LLM in a single sentence. Yes, I'm aware of the irony.

“The strength of an LLM is fundamentally defined by its ability to generalize learned patterns from training data to perform well on novel tasks and reasoning challenges it has never explicitly seen before.”

Claude Opus 4.1 (new chat, extended thinking)

Notice the focus on patterns and generalization.

“An LLM's strength is its proven ability to reliably produce correct, well-reasoned, context-aware outputs across tasks at acceptable latency and cost while adhering to instructions and safety constraints.”

GPT-5 Thinking (new chat)

'Correct', 'well-reasoned' and 'context-aware', but what is 'correct'?

“The strength of an LLM is best defined by its ability to generate coherent, relevant, and contextually appropriate text across a wide range of tasks and domains.”

Gemini 2.5 Flash (new chat)

Very much to the point: generating text.

Every model emphasizes different aspects, but they're all describing sophisticated pattern matching. The responses do not mention fact-based information.

The missing pieces

Current limitations are fundamental to how these systems work:

  • No true understandingLLMs generate statistically likely responses based on patterns.
  • No verificationLLMs are equally confident when right or completely wrong.
  • No real-time awarenessTraining data is static.

The paradox of utility

My friend was right: LLMs aren't built for facts or reasoning. But she was wrong about them being 'just' for writing. They're transformation engines that excel at any language-based task: summarizing, translating ideas across domains, adjusting tone or finding patterns.

Despite limitations, LLMs have become indispensable. Why? Most daily tasks don't require perfect accuracy, they require good-enough understanding and helpful transformation. Writing emails, summarizing documents, brainstorming ideas, these play to LLM strengths. You're asking for language work, not facts.

Problems arise when we treat LLMs as crystal balls rather than language tools.

Moving forward

We should learn how to use these new tools appropriately.

Use LLMs for drafting, not fact-checking.

For synthesis, not authoritative answers.

For creative connections, not systemic proofs.

The technology is transformative when we understand its true nature: a powerful pattern recognition system that works with language. The real breakthrough isn't having tools that can answer anything, it's having tools that can help us think and communicate better. That's transformative enough when we use it right.

Older

July 23, 2025

Artificial, not intelligent

The tech industry's rush to cram AI into every product is making software worse and more expensive, forcing users to pay for features they don't want or need. From AI buttons on mice to mandatory chatbots in browsers, these integrations prioritize revenue extraction over user experience. Good software should respect users' time and intelligence, not exploit them through forced feature bloat.

Want me to write about a topic? Something interesting to share? Just let me know.

Max van IJsselmuiden

home

blog

The dangerous comfort of AI answers

August 11, 2025 by

Max van IJsselmuiden

Every day, millions of people open a generative AI chatbot instead of a search engine. They're using it wrong, and that's partially our fault for not understanding what these tools actually are.

A knowledgeable friend recently argued that people are often using Large Language Models (LLM) to replace Google for reasoning and answering questions, but these models were never good at this. Their strengths, she said, is to extract meaning from text and build upon it, making them primarily useful for writing.

She's both right and wrong, and understanding why explains both LLM popularity and limitations.

Image of a crystal ball generated with GPT-5 (Dall-E 3).

ChatGPT is a household name now. Most people don't know GPT stands for "Generative Pre-trained Transformer" and honestly, why would they? To users, it's a box where you ask anything and get an answer with reasoning. Incredible. A technological breakthrough.

The name tells you everything.

Generative: the model creates text (or images, videos, audio, software code).

Pre-trained: it learned from existing data, not real-time information.

Transformer: a type of neural network architecture that processes patterns in sequences.

Nothing in that name says "truth engine" or "fact database". It says "pattern-based text generator."

At its core, artificial intelligence processes data, identifies patterns, and follows instructions. It cannot think, empathize, or reason. Instead, it replicates how people use knowledge without actually understanding it.

The disconnect between what the technology is (a transformer generating text based on training) and how we use it (as an oracle for answers) is where, in my eyes, danger lies. It’s when we trust these confident-sounding responses for important scenarios such as medical advice, legal guidance, or financial decisions.

The Google replacement that isn’t

When you ask Google a complex question, you get links (at least, before the AI Overview feature). We were used to scanning links, picking sources, and doing the thinking ourselves. Now, when you ask an LLM, you get an answer that feels like understanding, because they're better at understanding intent and context. Google requires translating problems into keywords.

LLMs let you explain problems like you would to a colleague. That's powerful, even when the response is hallucinated nonsense, such as an example I mentioned in an earlier blog: advising people to eat rocks.

What LLMs think they are

I asked several leading generative AI chatbots to define the strength of an LLM in a single sentence. Yes, I'm aware of the irony.

“The strength of an LLM is fundamentally defined by its ability to generalize learned patterns from training data to perform well on novel tasks and reasoning challenges it has never explicitly seen before.”

Claude Opus 4.1 (new chat, extended thinking)

Notice the focus on patterns and generalization.

“An LLM's strength is its proven ability to reliably produce correct, well-reasoned, context-aware outputs across tasks at acceptable latency and cost while adhering to instructions and safety constraints.”

GPT-5 Thinking (new chat)

'Correct', 'well-reasoned' and 'context-aware', but what is 'correct'?

“The strength of an LLM is best defined by its ability to generate coherent, relevant, and contextually appropriate text across a wide range of tasks and domains.”

Gemini 2.5 Flash (new chat)

Very much to the point: generating text.

Every model emphasizes different aspects, but they're all describing sophisticated pattern matching. The responses do not mention fact-based information.

The missing pieces

Current limitations are fundamental to how these systems work:

  • No true understandingLLMs generate statistically likely responses based on patterns.
  • No verificationLLMs are equally confident when right or completely wrong.
  • No real-time awarenessTraining data is static.

The paradox of utility

My friend was right: LLMs aren't built for facts or reasoning. But she was wrong about them being 'just' for writing. They're transformation engines that excel at any language-based task: summarizing, translating ideas across domains, adjusting tone or finding patterns.

Despite limitations, LLMs have become indispensable. Why? Most daily tasks don't require perfect accuracy, they require good-enough understanding and helpful transformation. Writing emails, summarizing documents, brainstorming ideas, these play to LLM strengths. You're asking for language work, not facts.

Problems arise when we treat LLMs as crystal balls rather than language tools.

Moving forward

We should learn how to use these new tools appropriately.

Use LLMs for drafting, not fact-checking.

For synthesis, not authoritative answers.

For creative connections, not systemic proofs.

The technology is transformative when we understand its true nature: a powerful pattern recognition system that works with language. The real breakthrough isn't having tools that can answer anything, it's having tools that can help us think and communicate better. That's transformative enough when we use it right.

Older

July 23, 2025

Artificial, not intelligent

The tech industry's rush to cram AI into every product is making software worse and more expensive, forcing users to pay for features they don't want or need. From AI buttons on mice to mandatory chatbots in browsers, these integrations prioritize revenue extraction over user experience. Good software should respect users' time and intelligence, not exploit them through forced feature bloat.

Want me to write about a topic? Something interesting to share? Just let me know.