The Power and Boundary of Language Models
Before going any further, we need to be honest about what large language models already are.
They are extraordinary.
It has become fashionable in some circles to dismiss them as “just autocomplete.” That phrase is technically useful in one narrow sense, but intellectually lazy in almost every other. A bird is “just” a biological machine. A human is “just” atoms. A brain is “just” electrical and chemical signaling. The word “just” does a lot of hiding.
Yes, large language models predict tokens. That is the basic mechanism. OpenAI’s own GPT-4 technical report describes GPT-4 as a Transformer-based model pre-trained to predict the next token in a document. Yet that same report also noted that GPT-4 showed human-level performance on a wide range of professional and academic benchmarks, including a simulated bar exam score around the top 10 percent of test takers.
That should make us pause.
A system trained to predict language can write essays, explain code, summarize legal documents, translate between languages, tutor students, generate business strategies, debug software, imitate style, answer technical questions, and reason through problems that would have seemed impossible for machines only a short time ago.
That is not nothing.
In fact, it is one of the most important technological breakthroughs in human history.
But this is exactly where we must be careful. The danger is not that people underestimate LLMs. Many still do. The deeper danger is that people misunderstand what kind of intelligence they represent.
An LLM is powerful inside the world of language. It can move through text the way a fish moves through water. It has absorbed more written knowledge than any human could read in a thousand lifetimes. It can recognize patterns across law, medicine, literature, software, science, politics, and business. It can produce answers that feel thoughtful because it has learned the shape of human thought as expressed in words.
But that last phrase matters.
As expressed in words.
That is the boundary.
An LLM does not learn the world directly. It learns what humans have said about the world. It learns our descriptions, explanations, arguments, metaphors, papers, manuals, stories, equations, and conversations. That is an astonishingly rich source of knowledge. But it is still a mediated source of knowledge.
It is not the same as contact with reality.
This distinction is easy to miss because language is so deeply woven into human intelligence. We think in words. We argue in words. We explain ourselves in words. Civilization itself is stored in words. So it is tempting to assume that if a machine masters language, it has mastered thought.
But perhaps language is not thought itself.
Perhaps language is the packaging.
A model that masters the packaging may become incredibly useful. It may even become superhuman at many professional tasks. It may become the greatest research assistant ever created. The greatest summarizer. The greatest coding partner. The greatest analyst of existing human knowledge.
But that is not the same as becoming Newton.
It is not the same as becoming Einstein.
It is not the same as a mind that invents a new frame of reality.
That is the question this article is really asking. Not whether LLMs are impressive. They plainly are. Not whether they will transform the economy. They almost certainly will. Not whether they can replace or augment millions of human workers. They already are beginning to.
The question is harder than that.
Can a system trained primarily on language ever produce the kind of embodied, cross-domain, world-modeling creativity we associate with the deepest human genius?
Or will it remain something different?
A genius in the library.
Brilliant, fast, fluent, tireless.
But still inside the library.
Language May Package Thought, But It May Not Be Thought Itself
There is a tempting assumption at the center of the modern AI debate.
Humans think in language. Large language models master language. Therefore, if we make the language model large enough, we may eventually get a machine that thinks like a human.
It is a clean argument.
It is also probably incomplete.
Language is so close to thought in our daily lives that it can feel almost impossible to separate the two. When we explain an idea, we use language. When we argue with ourselves, we often use language. When we remember an event, plan a business, write a book, defend a belief, or teach a child, language is usually nearby.
So it is understandable why people look at a system that can manipulate language at superhuman scale and conclude that the system must be approaching general intelligence.
But there is another possibility.
Language may be one way humans package thought. It may not be the source of thought itself.
In 2024, Evelina Fedorenko, Steven Piantadosi, and Edward Gibson published a paper in Nature arguing that language is primarily a tool for communication rather than thought. Their argument cuts directly into one of the hidden assumptions behind the LLM-to-AGI story. They review evidence from neuroscience and related fields and argue that, in modern humans, language appears to be optimized for communicating ideas, not for generating the full machinery of thought itself.
That distinction matters.
If language is mainly a communication system, then a machine that masters language may be mastering the output channel of thought rather than the engine of thought. It may become brilliant at handling the final form of ideas after humans have already compressed them into words. But that does not automatically mean it has acquired the deeper structures that produced those ideas in the first place.
This is where the debate becomes more serious.
A physics textbook is not physics. It is a description of physics. A medical chart is not the body. It is a description of the body. A recipe is not taste, heat, timing, texture, hunger, or the quiet knowledge of a cook who knows when something is “right” before the timer goes off.
Language is powerful because it lets us move these things from one mind to another. But the thing being moved is not always born inside language.
Before a child can explain gravity, he can drop a spoon from a high chair and learn something about the world. Before a musician can describe rhythm, she can feel it. Before an engineer can write a clean technical explanation, he can sense that a design is wrong because the parts do not fit, the load is uneven, or the machine will fail under stress.
In each case, language comes later.
It labels. It compresses. It transmits. It preserves.
But it may not be where the original intelligence begins.
This does not make language small. Quite the opposite. Language may be the most important tool humans ever developed. It lets one mind inherit the discoveries of another. It turns private experience into shared culture. It allows knowledge to survive death.
But a tool for sharing intelligence is not necessarily the same thing as intelligence itself.
That is the problem for LLMs.
They are trained on the shared record. They absorb the compressed residue of human perception, action, experiment, failure, analogy, and discovery. They learn the words left behind after human beings have already encountered the world and tried to explain what they found.
That is still an immense achievement. But it is not the same as encountering the world.
So when we ask whether LLMs can become AGI, we should be precise about the question. We are not merely asking whether language models can produce better answers, write better code, or pass harder exams. They almost certainly can.
We are asking whether the mastery of language can create the deeper kind of intelligence that language may only report.
That is a very different question.
Jimmy Carr has one of the better comic summaries of this problem. Talking about AI, he says, “Right brain things are about the gestalt, the whole,” and then adds that AI is “a very good covers band.” Crude, funny, and mostly right. AI can create a new version of what it knows, but can it create something completely new?
The Library Is Not the Laboratory
This is where the distinction becomes more concrete.
A language model can learn a great deal about fire. It can explain combustion. It can describe the chemical process. It can summarize the history of controlled fire in human civilization. It can write a safety manual, compare fuel sources, explain why oxygen matters, and produce a poem about a flame in the dark.
But it has never felt heat.
It can learn a great deal about gravity. It can describe Newton’s laws, Einstein’s field equations, orbital mechanics, acceleration, mass, and spacetime. It can solve physics problems that would defeat most people.
But it has never fallen.
It can learn a great deal about music. It can analyze rhythm, harmony, notation, performance history, instrument design, and the emotional structure of a symphony. It can explain why a violin is hard to play. It can describe the pressure of the bow, the placement of the fingers, the discipline of timing, and the relationship between tension and sound.
But it has never drawn a bow across a string and heard the note fail.
That failure matters.
The laboratory teaches differently than the library. It does not merely provide information. It provides resistance. It pushes back. It humiliates bad theories. It exposes the gap between what sounds right and what is right. A person can read ten books about woodworking and still split the board. A person can study negotiation and still lose the room. A person can understand nutrition and still fail to cook a decent meal.
Reality has a way of correcting language.
This is one of the great weaknesses of a system trained primarily on text. It does not meet reality directly. It meets reality after human beings have already translated it into symbols. It learns the report, not the event. It learns the conclusion, not the contact. It learns the language of experience, not experience itself.
That does not make its knowledge worthless. Far from it. Human civilization depends on secondhand knowledge. None of us personally rediscovered calculus, germ theory, electricity, agriculture, aviation, or semiconductor physics. We inherit the compressed discoveries of others, and language is one of the main ways we do it.
But there is a difference between inheriting knowledge and generating the kind of intelligence that first discovers it.
That difference is the heart of the AGI debate.
When a scientist makes a breakthrough, he is not only rearranging sentences from the scientific literature. He is often wrestling with stubborn reality. The experiment fails. The numbers do not fit. The instrument behaves strangely. The expected result refuses to appear. Somewhere inside that tension, a new idea begins to form.
A large language model can read every paper describing that process. It can summarize every failure, every theory, every argument, and every published result. But the model itself has not stood inside the loop of prediction, action, error, revision, and physical consequence.
That loop may be essential.
The same is true outside science. A founder does not learn business only from business books. He learns from payroll pressure, angry customers, broken software, hiring mistakes, bad contracts, missed deadlines, and the quiet terror of having to make a decision before all the information is available. A doctor does not learn medicine only from medical journals. A musician does not learn music only from theory.
The world teaches through friction.
This is why the library metaphor matters. The LLM lives inside the greatest library ever assembled. That library contains a stunning amount of human knowledge. In some ways, it contains more practical wisdom than any individual person could ever hope to hold.
But the library is not the laboratory.
This may be the line between language intelligence and world intelligence. LLMs are learning from humanity’s descriptions of reality. But AGI may require a system that can build its own models from reality itself.
Creativity Is Prediction Through Analogy
In On Intelligence, Jeff Hawkins argues that intelligence is built on prediction. The brain is constantly asking, in one form or another, “What happens next?” It uses memory to recognize patterns, compare the present against the past, and anticipate what the world is likely to do.
That may also be the best way to understand creativity.
Creativity is not magic. It is prediction through analogy. It is the ability to recognize a pattern in one domain and apply it somewhere else. A creative mind sees that an old structure may predict a new solution.
This is why the greatest breakthroughs often come from outside the official boundaries of a field. The new idea does not always arrive from deeper specialization. Sometimes it arrives because someone brings in a pattern from somewhere else.
Darwin is a useful example. His theory of natural selection did not come from biology alone. One of the key influences was Thomas Malthus, who wrote about population pressure and scarcity in human societies. Darwin took that pattern and applied it to nature. If more organisms are born than can survive, then small advantages matter. Over time, those advantages accumulate.
A pattern from population economics helped unlock a theory of life.
That is creativity.
Not merely knowing more facts. Not merely memorizing the literature. Not merely repeating the accepted language of the field. Creativity often happens when the mind imports a structure from one area and tests whether it explains another.
This is where language-only intelligence begins to look incomplete.
A large language model can read about biology, economics, scarcity, evolution, competition, adaptation, and survival. It can describe how Darwin made the connection. It can summarize the analogy after the fact. But the deeper question is whether it has access to enough kinds of patterns to create those leaps on its own.
Text is an enormous source of patterns. No serious person should deny that. Human beings have poured an astonishing amount of experience into language. Books, papers, manuals, arguments, stories, records, experiments, and observations all live there.
But language is still a compression of experience.
It is not the full experience.
A text-only model does not have the vast pattern library that comes from moving through the world, touching tools, testing materials, watching systems fail, hearing timing, feeling pressure, navigating space, or acting inside reality with consequences. It receives the translation of those things into words.
That translation is powerful, but it is incomplete.
If creativity is prediction through analogy, then the range of possible creativity depends on the range of available patterns. A mind with richer experience has more places to steal from. It has more structures to compare. It has more ways to notice that one thing secretly resembles another.
This may be the real limit of LLMs.
They may become extraordinary at recombining the patterns that humans have already expressed in language. They may become brilliant inside the library. They may even discover connections that humans missed inside the written record.
But if their world remains mostly text, then their creativity is drawing from a narrow slice of reality.
A very large slice of human language.
But still a narrow slice of the world.
That is why more text may not be enough. The issue is not simply whether the model has enough parameters, enough compute, or enough training data. The issue is whether it has access to the right kind of data.
Because if intelligence is prediction through analogy, then AGI may require far more than words.
It may require the world.
Jaron Lanier and the Pattern Library of the Body
Jaron Lanier, who is widely recognized as the “father of virtual reality,” is useful here because he is not only a computer scientist. He is also a musician and collector of rare instruments. His official biography describes him as a specialist in unusual and historical musical instruments, with one of the largest collections of actively played rare instruments in the world. (jaronlanier.com)
That matters.
An instrument is an interface. It teaches touch, timing, pressure, resistance, rhythm, and feedback. These are not abstract lessons. They are embodied patterns. They come from the loop between intention and response.
Lanier has spent much of his career thinking about human-computer interaction. So his work is not shaped only by code, screens, and software theory. It is also shaped by instruments. By gesture. By the physical relationship between a person and a tool.
That is creativity through analogy.
He can take patterns from music and apply them to computers because he has lived inside both worlds. That is the larger point. The most interesting ideas often come from crossing domains, not from going deeper into one narrow channel.
A text-only LLM may know more facts about instruments than any human ever could. But it does not have the pattern library that comes from playing them. It knows the descriptions. It does not know the loop.
And if intelligence is prediction through analogy, that gap matters.
Jeff Hawkins and the Thousand Models of the World
In A Thousand Brains, Hawkins argues that intelligence is not built from one central model of the world. It is built from many models working in parallel. The neocortex, in his view, is made of cortical columns that each learn models of objects and concepts through reference frames, movement, and sensory input. Numenta summarizes the theory this way: cortical columns do not merely detect features; they learn complete models of things by observing them from different locations and perspectives. (numenta.com)
That is a very different picture from a language model.
An LLM has a vast statistical model of text. It knows how words, facts, arguments, styles, and concepts tend to relate to one another. That is powerful. But Hawkins is pointing toward something richer: a mind built from many grounded models of reality itself.
A coffee cup is not understood only as the word “cup.” It is understood through shape, weight, orientation, touch, location, use, expectation, and interaction. The brain does not simply label the object. It predicts the object. It knows what should happen as you rotate it, lift it, look inside it, or reach for the handle.
That kind of intelligence is not merely verbal. It is spatial. It is sensory. It is active.
This may be the missing piece.
If Hawkins is even partly right, then intelligence is not just prediction over language. It is prediction inside models of the world. Those models are built through contact, movement, and feedback. Language may sit on top of them, but it may not replace them.
That distinction cuts directly into the LLM debate. A language model can describe a cup better than most humans. But does it possess the kind of internal model that comes from interacting with cups, rooms, hands, tools, gravity, and space?
Maybe not.
And if AGI requires those grounded models, then bigger language models alone may not get us there. They may become brilliant at manipulating the symbolic record of human experience while still missing the deeper machinery that human intelligence uses to understand reality.
Hawkins’ theory does not prove that LLMs can never become AGI. But it does suggest that something essential may be missing from the language-only path.
Not more words.
More world.
The Next Training Corpus May Be the Physical World
If language is not enough, then the next step is not simply a bigger language model.
It is a broader kind of experience.
This is already beginning in robotics. The Open X-Embodiment project assembled more than one million real robot trajectories across 22 different robot embodiments, from robot arms to bi-manual systems and quadrupeds. Google DeepMind described the dataset as covering more than 500 skills and 150,000 tasks across more than one million episodes.
That is important because it points toward a different path.
Instead of training AI only on what humans have written, we can train systems on what agents actually do. Grasping objects. Opening drawers. Moving through rooms. Failing at tasks. Correcting motion. Learning from the mismatch between prediction and result.
That kind of data is closer to the world.
It is still primitive compared to human experience. A robot arm moving blocks is not building with Legos. A camera feed is not watching a game of basketball. A million robot trajectories are not the full richness of human experience.
But the direction matters.
The next great training corpus may not be the internet. It may be reality itself, captured through video, robotics, scientific instruments, simulations, satellites, laboratories, factories, hospitals, vehicles, and machines acting in the physical world.
This is where AGI may need to go.
Not away from language. Language will still matter. It may remain the interface, the reasoning layer, and the way machines communicate with us. But language may need to be grounded in systems that can see, move, test, manipulate, and update their models from direct feedback.
That would be a very different kind of intelligence.
An LLM learns from the record of what humans have already said.
A more complete AI would learn from the world that made humans say it.
The “Right Wall” May Really Be a Reality Wall
There has been a growing debate over whether LLMs are beginning to hit a wall.
Maybe they are. Maybe they are not. The honest answer is that we do not know yet.
Models continue to improve. They are getting faster, more capable, more multimodal, and more useful in real work. At the same time, the easy gains from simply making everything bigger may be getting harder to find. More data, more compute, and more parameters may still help, but each leap appears to demand more cost, more energy, and more engineering.
So perhaps the wall is not simply a scaling wall.
Perhaps it is a grounding wall.
If the model is trained mostly on language, then it is trapped inside the limits of language. It can become better at reading the map, comparing maps, summarizing maps, and drawing new maps from old ones. But at some point, the missing ingredient may be the territory itself.
That would explain why the path forward seems to be moving beyond text. The serious AI labs are not only building chatbots anymore. They are pushing into video, robotics, agents, simulations, multimodal systems, and tools that can act in the world. They may not describe it this way, but the direction is obvious.
The model needs more than words.
This does not mean LLMs are a dead end. That would be too simple. They may become one of the central organs of artificial intelligence. Language is how we instruct machines, receive explanations, preserve knowledge, and coordinate action. Any future AGI will almost certainly need language.
If Hawkins is right that intelligence depends on many grounded models of the world, and if creativity depends on moving patterns across domains, then scaling text prediction may eventually run into the boundary of its own input. It may become incredibly powerful inside the library while still lacking the pattern library that comes from contact with reality.
That is the wall I am worried about.
Not a wall of intelligence.
A wall of experience.
The Architecture May Be the Problem
The deeper problem may not be that LLMs need more data.
It may be that they store data in the wrong shape.
The issue is not storage size. It is storage shape. Current LLMs do not appear to organize knowledge around persistent objects, changing states, physical actions, reference frames, or failed predictions. They organize it around patterns in representation. That works beautifully for language. But AGI may require a memory system that treats reality itself as the thing being modeled, not merely the record left behind after humans describe it in language.
But the world itself is not made of language.
The world is made of objects, motion, pressure, time, space, cause, resistance, and change. A thing is not merely what can be said about it. It is what remains stable across transformation. It is what changes when acted upon. It is what produces surprise when our prediction fails.
This is where the current architecture starts to look suspect.
If an AGI is going to understand reality, it may need to store reality more like the brain appears to store it: not as one giant soup of associations, but as many overlapping models grounded in reference frames, movement, and prediction. The neocortex builds many models of objects in parallel. These models are tied to location, sensory input, and movement, not just labels.
That is a radically different kind of memory.
A true world model would not merely know the word for a thing. It would store the thing as a structure of expectations. How it appears from different angles. How its parts relate. What happens when it moves. What happens when it is acted upon. What remains constant through change. What usually happens next.
That last part matters.
Temporal memory may be as important as object memory. Intelligence does not only ask, “What is this?” It asks, “What is about to happen?” The brain is constantly predicting sequences: motion, sound, social response, danger, opportunity, failure.
This is not how most people talk about AI.
They talk about more parameters. More GPUs. More tokens. More training data. More synthetic examples. But if the internal representation is wrong, more may not solve the problem. More may only give us a larger library with the same missing door.
The question is not whether the model has seen enough words about reality.
The question is whether it has a memory structure capable of holding reality.
That may require object-based memory, spatial memory, temporal memory, action memory, causal memory, and analogical memory. It may require representations where similarity is not just a statistical accident, but a structural feature of the system. Sparse distributed representations are one attempt in that direction, because overlap between representations can encode meaningful similarity and support generalization.
This is where AGI may demand a break from the current path.
Language models may remain essential. They may become the communication layer, the reasoning layer, maybe even the conscious-seeming layer. But underneath them, AGI may need a different backbone entirely.
Not just bigger weights.
Not just longer context.
Not just more training text.
A different way to store the world.
The Company That Models the World Wins
The next great AI breakthrough may not come from the company that builds the biggest chatbot.
It may come from the company that figures out how to model the world in silicon.
That is the real frontier. Not simply a model that can talk about physics, biology, tools, markets, or human behavior. A model that can represent those things in structures closer to how they actually exist: as objects, systems, sequences, forces, actions, constraints, failures, and transformations.
This is a much harder problem than scaling language. It requires a different view of what intelligence is supposed to store. The model cannot merely preserve associations. It has to preserve reality’s shape. It has to know what remains stable, what changes, what causes the change, and what prediction should be revised when the world pushes back.
That may be where the 800-pound gorilla is hiding.
The next unicorn in AI may not be another wrapper around an LLM. It may not be another company selling prompts, agents, or chat interfaces. It may be the company that creates the first practical architecture for storing world models: spatial memory, temporal memory, causal memory, action memory, object memory, and analogical memory working together.
In other words, a machine with a place to put the world.
Language would still matter. It may remain the way we talk to the system. But language would no longer be asked to carry the full burden of intelligence. It would become one layer in a deeper architecture.
That is the rethinking we may need.
For the last few years, the race has been dominated by scale: more compute, more data, more parameters, more GPUs, more tokens. That race may still produce extraordinary tools. It may reshape the economy. It may automate vast amounts of knowledge work.
But if the goal is AGI, the winning question may not be, “Who has the largest model?”
It may be, “Who has the best representation of reality?”
That is a different race.
And if this is even directionally right, it may be the race that matters most. The future of AI may belong to the company that stops treating intelligence as a larger pile of symbols and starts building systems that can model the world itself.
The library was the first miracle.
The world model may be the next one.
Ready to See What AI Looks Like for Your Business?
At Intellic Labs, we don’t believe in generic demos or one-size-fits-all AI solutions. The same way this article asked you to meet your employees where they are, we meet your business where it is — inside your actual workflows, your real tools, and the specific friction points that are costing you time and revenue right now.
That’s why we created something a little different.
We’ll build you a free, custom AI walkthrough video — made specifically for your business.
Not a slide deck. Not a product overview. A short video built around your systems, your team, and your goals — so you can see exactly where AI can reduce friction, drive revenue, and create a better experience for both your customers and your people.
In your custom video, you’ll see:
✅ Where AI plugs into your existing stack — CRM, support, email, docs, and operations tools
✅ Which workflows can be fully automated end-to-end, not just “assisted”
✅ How we measure real ROI — cycle time, cost-to-serve, conversion, and resolution speed
✅ How we avoid the vendor lock-in and static platforms that leave most businesses stuck
No sales pressure. No buzzwords. No generic AI overview that could apply to any company on the planet. Just an honest look at what’s actually possible inside your operation — before you spend a dollar or a month in pilot purgatory.
We produce seven of these each week. Spots move quickly.
👉 Claim your free custom AI walkthrough video
Your team has more potential than any tool can unlock on its own. Let Intellic Labs help you show them what’s possible.
