NewZhiyuan Report
Editors: Hao Kun, Tao Zi
[NewZhiyuan Guide] The architecture destined to终结 Transformer is about to be born! In his latest interview, Altman boldly claims that the next-generation AI architecture will completely颠覆 Transformer, and the fate of LSTM may replay itself.
The biggest beneficiary of Transformer has personally sentenced it to death!
In recent days, Sam Altman returned to Stanford and dropped a bombshell on a group of sophomore juniors—
A completely new underlying architecture will definitely emerge in the future, with a performance leap no less than the dimensional reduction strike Transformer once dealt to LSTM.
Keep in mind, the GPT empire was built on Transformer.
ChatGPT, GPT-4, o1, Codex—they are all fruits of this architecture.
And now, the person picking the fruits says aloud: this tree's lifespan is nearing its end.
Even more, Altman stated bluntly that the AGI we pursue might just be a "warm up"!
The breakthrough of the next-generation new architecture is already on the way—existing high-level LLMs possess sufficient cognitive ability to act as a lever for human intelligence, personally pushing open the door to another technological paradigm.
Using AI to Find the Next Transformer
People say brute force works miracles, but brute force itself has limits.
Transformer has a natural compute black hole: if text length increases 10x, computation increases 100x.
This is why running GPT-5.4 level models today burns money at astronomical speeds.
Altman obviously sees this wall. But he doesn't think there's no way out; on the contrary, he believes the tool to tear down this wall is already in hand.
There was an extremely key sentence in the interview: Current models are finally smart enough to assist humans in conducting this level of scientific research.
This means the task of finding the next-generation architecture itself can now be aided by AI.
The logic chain for using current AI to discover the new architecture that replaces it is clear:
Stronger models → Higher research efficiency → Greater probability of discovering new architectures → New architectures make models even stronger.
A self-accelerating flywheel has thus formed.
Altman's confidence in making this judgment stems from his unique嗅觉 for paradigm shifts along his journey.
During his freshman summer, he went to work in Stanford's AI lab, concluding that "these things have no chance," and then went off to start other ventures.
However, his attention to AI never ceased. In Altman's own words, it's a habit of "looking at the big picture," avoiding tunnel vision.
In 2012, when AlexNet emerged, he, like most people, thought it was "cool" but didn't take it to heart.
In the following years, as deep learning models grew larger and stronger, Altman watched from the sidelines. Until a certain tipping point where the feeling completely changed—this thing was like an approaching asteroid, extremely crazy, yet hardly anyone in the world seemed to care.
Thus, OpenAI was founded in 2015. There was only one core belief: push the scale of deep learning to the limit and see what happens.
But back then, saying they wanted to build an AGI lab made industry veterans think they were crazy, even calling them frauds directly.
However, everyone has seen the results.
GPT-2 let Altman see a computer do something unprecedented for the first time, GPT-3 stunned the world, and GPT-4 went a step further. When you stick to a correct paradigm, the returns are exponential.
Now, the same intuition is projected onto the next paradigm.
Transformer is not the end, just as LSTM was not the end.
Altman even gave specific advice:
If I were a researcher now, I would obsess over this direction, looking for "where a nuclear-level breakthrough can be dug out," and I would heavily rely on large models as research assistants.
The Whiteboard in Greg's Apartment: A Night That Changed the World
The most fascinating part of this interview is Altman's recollection of OpenAI's early days.
On the first day at OpenAI, everyone gathered in co-founder Greg Brockman's apartment.
By 9:30 or 10:00 AM, eight or nine people had arrived one after another, sitting on the sofa, looking at each other.
Then someone spoke up: "Okay, what do we do?"
Someone suggested writing a few papers. Another said they needed a whiteboard first. Then someone immediately placed an order on Amazon for expedited delivery.
Altman said he felt a moment of panic internally: This won't work. This didn't look like a serious startup, nor any organization capable of achieving great things.
But he immediately followed with a very Altman-esque remark: In such moments, you just have to take a deep breath and believe that if the best people are gathered around you, things will work themselves out.
He bet right.
In that very first week, most of the ideas that would become OpenAI's core philosophy for the next four years were written on that whiteboard, even though they themselves felt these ideas were unreliable at the time.
They hadn't even thought about making products initially.
Altman repeatedly emphasized that they thought they were just a pure research lab, meant to publish papers and that was it.
But later, two things became increasingly clear:
- First, the economic value inherent in this path far exceeded imagination.
- Second, the capital required wasn't billions, but hundreds of billions.
The true turning point that built Altman's faith was GPT-2.
He said he couldn't remember the exact date GPT-2 was released, but he would forever remember the night he first conversed with that model.
It did things I had never seen a computer do before.
In that moment, he felt, "It's settled, this is it."
As for why GPT-2's release was delayed? Altman admitted that in hindsight, they were perhaps overly cautious, but he felt that facing every new capability step of AI, leaning slightly towards caution does no harm.
Of course, one cannot be too cowardly. If enterprises do not embrace AI fast enough, they will be wiped out by fully autonomous AI companies, and that would be the real disaster.
Stanford Interview Panorama: Altman's 10 Judgments
Beyond architectural prophecies and startup stories, Altman outputted a dense array of viewpoints in this interview, almost every one worth discussing separately.
1. AGI will arrive within two years.
Altman told the sophomores directly:
By the time you graduate, you will step into a world that already has AGI.
Of course, fundamental human drivers won't change; you still have to move houses, find jobs, and consider starting families.
But scientific research will be highly automated, and the significance of founding startups or working for big tech will be completely rewritten.
2. Programming Agents are the next ChatGPT moment.
What is the next explosion point? Altman didn't hesitate: Programming AI Agents.
Following closely behind, though not yet fully exploded, is the equal capability of AI to execute tasks in all knowledge work.
However, that day is not far off.
3. One person can do the work of a medium-sized company.
In the future, a large number of micro-startups with one person or six partners will emerge, with influence and revenue that can rival today's large and medium enterprises.
Altman said the iPhone launch was the last opportunity of this magnitude; this time is even more intense.
Not only can we do things previously unthinkable, but we can also build products and companies extremely fast with minimal manpower.
4. AI CEO? Not impossible.
Discussing AI's impact on society, Altman made a thought-provoking remark:
He would never deceive himself into thinking that an AI CEO more suitable to lead OpenAI than himself won't appear in the not-so-distant future.
If some companies or countries embrace AI while others don't, the competitiveness gap will be crushing.
He admitted that the political, economic, and social shocks behind this are not yet fully figured out even by himself.
5. But don't panic; human adaptability is severely underestimated.
Altman is not an AI doomsayer.
He repeatedly emphasized a point: AGI sounds like it will completely颠覆 society, but the feeling of being in it won't be as thrilling as it sounds, maybe just a bit confusing for the first few days.
Humans' desire to be valuable to each other, to compete, to create, and to express—these underlying drivers will not disappear.
Perhaps careers 100 years from now will bear no resemblance to today's, but there will always be things for people to do, and people will always care about connections with others.
6. Don't be afraid to compete with OpenAI.
Someone asked, what if OpenAI becomes the ultimate giant?
Altman's answer was surprisingly honest: Back then, everyone said it was impossible to fight Google, but we fought our way out.
Someday, a company bigger and more successful than OpenAI will be born, and they absolutely will not walk the exact same path.
He even said that if Google hadn't been so "lousy" back then, OpenAI would never have had a chance to stand out.
Big companies have big company diseases.
7. Burning money fast, but not panicking.
Facing the sharp question about "OpenAI's terrifying burn rate," Altman was calm: Yes, the burn rate is fast, but if spending 1 billion this year earns 3 billion next year, there are plenty of capitals lining up worldwide to make this deal.
8. Self-developed chips are serious; building data centers is out of the question.
OpenAI has a massive custom chip plan and is extremely excited about its own inference chips.
As for building their own data centers, in Altman's own words: "Really, I'd rather not do this hard labor even if you pay me 10,000 times."
They will do it if forced to that step, but preferably they will design server racks to the extreme and let others do the dirty work.
9. Social products are about to be torn open.
Altman feels AI opportunities go far beyond just "stuffing AI" into existing software.
He cited social products as an example: Imagine a bunch of AI agents representing their respective users, chatting and exchanging information autonomously in virtual space; this is the disruption of the underlying logic.
10. Knowing is easy; doing is harder.
This is a sentence written in Altman's very first blog post.
Does it still hold in the AI era? He says it holds more than ever.
Acquiring knowledge is getting easier, and getting things done is also becoming easier, but that applies to everyone—you have to compete with the whole world.
He said the top experts he knows who play with AI tools the best all feel their work has never been harder than it is now.
The tools are ridiculously powerful, but using them well to maintain top-tier competitiveness is also unprecedentedly difficult.
Sam, Are You Really Happy?
The last unexpected moment of the interview was a soul-searching question from a student.
Keep in mind, this is a CEO whose life goes completely out of control after 8 AM every day.
Work for a few hours, spend an hour with the kids, then go to the company, and from then on it's pure chaos.
In his words, no company runs as fast as OpenAI, is so internally chaotic, and yet stays right in the line of fire for everyone.
But Altman says he is currently one of the happiest people he knows.
He shared a cognitive shift that changed his life.
Most people think the opposite of a bad experience is a good experience, so they suffer when bad things happen. But he reframed the problem: the opposite of a bad experience is actually the complete loss of the ability to experience.
Someday you won't even have the qualification to experience, and by then you might even miss those days when you were being tortured.
Post-Transformer Race: The Revolution Has Begun
Altman's prophecy is not castles in the air.
The race for "Post-Transformer" started long ago, progressing much faster than most imagine.
The most high-profile challenger is Mamba.
Proposed by Albert Gu and Tri Dao in late 2023, this architecture completely bypasses the "attention mechanism," using State Space Models (SSM) to process sequences instead.
Simply put, while Transformer requires every word in a passage to "look at" every other word, Mamba maintains a fixed-size memory state, handling it in linear time, making inference throughput 5 times faster. By early 2026, Mamba evolved to its third generation, with papers accepted by ICLR 2026.
Industry moves tell an even clearer story.
NVIDIA released the Nemotron-H series in 2025, replacing 92% of attention layers with Mamba layers, tripling inference speed while improving accuracy.
By the end of 2025, all of NVIDIA's new models (Nemotron 3 Nano/Super/Ultra) switched to a Mamba-Transformer hybrid architecture.
AI21 Labs' Jamba, IBM's Bamba, Microsoft's Phi-4-mini-flash-reasoning, and xLSTM personally crafted by the father of LSTM, Sepp Hochreiter, have also joined the hybrid camp.
There are wilder directions too: Liquid AI's Liquid Neural Networks, inspired by a nematode with only 302 neurons.
It uses differential equations to drive neurons, continuing to learn during inference and adapting to environmental changes in real-time; 19 neurons can control autonomous driving. The LFM2.5 model released in January 2026 achieved amazing performance with far fewer parameters than Transformer.
The Next Great Migration: Who Will Be Born?
Looking back at history, every architectural migration from LSTM to Transformer released more than an order of magnitude growth in capability, simultaneously birthing great companies that defined an era.
The last migration birthed OpenAI. What about the next?
Altman himself said: Someday a company bigger and more successful than OpenAI will appear.
Perhaps right now, that future founder is sitting in some dormitory, facing an Amazon expedited-delivery whiteboard, writing down the first unreliable idea.
And in their hand is an unprecedented weapon—AI itself.
References:
https://x.com/rohanpaul_ai/status/2033117083127644536?s=20
https://www.youtube.com/watch?v=FjlymGBt-vY