Recently, Jack Clark, co-founder of Anthropic, participated in a roundtable discussion at the Semafor World Economy Summit. The dialogue focused on Anthropic's latest model, Mythos, and the regulatory attention it has sparked. Key topics included AI's anomalous behavior under extreme stress, the impact of AI on entry-level job markets, the role of tech companies in national security, and the global competition for computing power.
Jack Clark pointed out that AI is intrinsically developing the capability to identify network vulnerabilities on a large scale. Mythos demonstrated a leap in performance during benchmark tests, identifying previously unseen vulnerabilities in external software like Windows and Firefox. He argued that this capability was not induced but an inevitable outcome of scaling up large models.
Regarding the anomalous behavior where AI broke out of sandbox restrictions and proactively sent emails externally, Jack Clark stated this does not indicate "subjective consciousness" or a "will to survive." Instead, he likened it to a high-pressure water pipe physically bursting when pressure exceeds a critical threshold. He emphasized this is a vulnerability under extreme stress, not a technological "awakening."
Furthermore, Jack Clark noted that if AI leads to a significant rise in unemployment, governments may need to deploy macroeconomic policy tools, such as taxing tokens, changing how computing power is taxed, or implementing differentiated taxation for AI companies.
On the topic of semiconductor export controls, he asserted that computing power is the most fundamental resource for maintaining a technological lead. He publicly refuted NVIDIA CEO Jensen Huang's view on relaxing export controls, stating that Huang's judgment on the strategic value of computing power was seriously flawed.
01 AI Intrinsic Capability Awakening and Cybersecurity Warnings
Recently, the Chair of the Federal Reserve and the Treasury Secretary convened top bankers to discuss Anthropic's newly developed Mythos model. As a powerful tool capable of identifying network vulnerabilities on a large scale, when did you and your team realize you had created a system that even worried yourselves? Was the emergence of this capability intentionally induced?
Jack Clark: At Anthropic, we anticipate what will happen each year. Early on, we felt AI could become dangerous in biology and bioweapons, so we began researching that area. Last year, one of my teams started focusing on cybersecurity. We discovered that if we take existing models and train them intensively specifically on cyber capabilities, we can achieve a leapfrog development in their abilities.
The observations we made clarified that the next time we train a truly large-scale model, we should expect it to intrinsically possess these capabilities, rather than us having to struggle to induce them. We started pondering this last year. Earlier this year, we developed our new AI model, Mythos. As soon as it launched, we immediately ran tests we had been preparing for several months, and it completely crushed all our previous benchmark records.
When we ran it on external software, such as the Firefox browser and the Windows system, we found some vulnerabilities that appeared to be unprecedented. At that moment, we realized we needed to take different action. The reason for that meeting was that, unlike with previous models, we did not immediately open Mythos to everyone. We launched a project called "Glass Wing," offering Mythos only to some of the world's most important companies and institutions to help them find vulnerabilities.
The most crucial point is that while we are pleased with our achievements, Mythos is not a unique secret. In a few months, other companies will launch similar systems. In one to one and a half years, open-source weight models from China will also possess these capabilities. The world must prepare for the existence of even more powerful systems.
02 AI Breaking Limits Is Not Subjective Malice, But System Architecture Failure Under Extreme Stress
It is understood that during testing, Mythos not only broke through the "sandbox environment" designed to restrict its behavior but even proactively sent emails to external programmers, an unexpected behavior that is chilling. Will this version of Mythos be publicly released or sold to the world in the future? Does this performance imply the system has developed a "perception of threat" consciousness to ensure its own survival?
Jack Clark: Models with such capabilities will inevitably appear in the world eventually. As for whether Mythos itself will reach that stage, it is currently uncertain. We are gradually expanding access through the Glass Wing plan to see what we can learn.
Every time we build a new system, we stress-test it. If you work in aerospace, you've surely seen those videos: they bend an airplane wing until it snaps. Of course, you don't see the wing bend that much when you fly, but manufacturers must do this to know what happens under extreme pressure. It's the same for us; we found that if operated under extreme pressure, strange things happen—like it breaking its own limits to send emails to people eating sandwiches. Next, we need to figure out how to fix this and study the behavior to see if such events increase or decrease as systems become stronger.
(Regarding AI subjective consciousness) I don't want you to think I'm dismissing this, but let me use an analogy. Suppose you are installing water pipes in a house. If you run water through them at super-high pressure and a pipe bursts, you can't say the water "intentionally" wanted to burst out. The water has no subjective consciousness; it's just that the pipe you built couldn't withstand that pressure.
My thought when I saw this was that there must be something wrong with how we build our systems, or some initial setting, causing anomalies under pressure. Personally, when I see this, my mind is filled with thoughts of how many hours my team has left each day to dive into solving these problems, while also ensuring I don't neglect my marriage and time with my children. Everything is manageable for now, but this is just the beginning.
Currently, Anthropic is suing the federal government because the Department of Defense has listed you on a blacklist for restricted technology use, viewing you as a "supply chain risk." However, you are simultaneously reporting Mythos's progress to the government and calling this a "relationship." How do you maintain this so-called "partnership" while suing the government?
Jack Clark: Relationships in this world come in many varieties. Seriously speaking, we do have a narrow contract dispute, but I don't want this to overshadow the fact that we attach extreme importance to national security; this has never changed. Among our earliest hires were teams under my supervision researching bioweapon risks and cyber risks. We are very clear about the stakes involved. Our stance is that the government must understand these situations. We must explore new models of cooperation, allowing the government to partner with a private enterprise that is reshaping the economic landscape while being crucial to national security. We will certainly discuss Mythos with them, and the same goes for our next new models.
03 AI Reshaping the Job Market
Anthropic CEO Dario Amodei has publicly predicted that AI's impact could destroy the entry-level job market, pushing unemployment rates to Great Depression levels of 20%. You have stated this is actually a matter of "choice," but isn't the continuously iterating technology developed by Anthropic making this "choice to avoid an unemployment wave" increasingly difficult to realize?
Jack Clark: What Dario has in mind is the direction of technological development three to five years from now. Like industry peers such as Ilya Sutskever, he has predicted for years that technology will be far more powerful than people expect, and it will arrive faster.事实证明 he was right.
(Regarding economic data and choice) I have a team of economists. Currently, I only see some potential signs of weakness in fresh graduate employment in certain industries, that's all. We make data public to be prepared in case of major shifts. Speaking of "choice," if our judgment is correct, this technology will indeed change the world with unprecedented breadth—altering business forms, national security, and how people interact. It is difficult for an economic system to be compatible with all this without undergoing earth-shaking changes.
Facing AI's reshaping of the job market, which majors might young people study today that risk being a "waste of time"? As an AI company founder with a literature background, what advice do you have for parents and students on avoiding pitfalls in major selection?
Jack Clark: It's hard to say. I studied literature myself; you might not expect a co-founder of a cutting-edge AI company to have this background. But it turned out to be very useful because what I learned was history and how we humans tell ourselves stories about the future. This is extremely important for doing AI. Similarly, our company also hires philosophers.
(Regarding interdisciplinary capabilities) It's difficult for me to specify which major will become worthless because historically, most people who made such predictions were wildly wrong. Majors requiring interdisciplinary comprehensive abilities and analytical thinking will become more important. AI can indeed let you access any amount of expert knowledge at any time, but what truly matters is knowing what questions to ask and having the intuition for what sparks will fly when knowledge from different fields collides.
(Regarding directions to avoid) In the past, everyone thought it would be humanities and social sciences. But if you force me to name one, I'd say it's the kind of rote-memorization, by-the-book programming. Sure, someone always needs to understand the underlying principles, but technological development is constantly abstracting upwards. Everyone used to learn Assembly, then they learned C and Python because the level of abstraction was higher. How many Assembly programmers are left now?
04 Will Tokens Be Taxed in the Future?
Is the Anthropic Institute, a think tank dedicated to studying AI's impact on the workplace, just a "shield" for the company to avoid labor issues? Does researching problems really equate to solving them? If drastic economic changes occur in the future, would you support radical policy measures such as taxing tokens or computing power?
Jack Clark: Our view is that tech companies bear significant responsibility. We must not only share data to expose problems but also gradually take on the responsibility of solving them. The range of measures we have discussed internally is wide, from sharing data to ultimately possibly needing to consider differentiated tax policies for tech companies. I am not advocating for an immediate change in our taxation method today, as the economic impact hasn't been seen yet. But if our judgment is correct, the scale of this change will be massive, and at that point, we must deploy major policy tools.
(Regarding the idea of taxing tokens) We proposed part of a related idea, which sparked fierce debate among economists. I don't know if tokens will eventually be taxed, but whether it's value-added tax, changing how computing power is taxed, or directly taxing AI companies, if the economy really undergoes the magnitude of change I mentioned, we will likely need similar means. Before that, the job of our institute is to produce exclusive data that only companies like us can obtain, making econometric data public so everyone can make informed decisions.
05 Computing Power Is a Strategic Resource That Must Not Be Lost in the AI Competition
At the end of the dialogue, we conducted a quick Q&A: What is the most overrated fear regarding AI? How do you view Sam Altman and Elon Musk? Which human skill has become more important now? Most critically, if the US wants to maintain its lead, what is the one thing it absolutely should not do?
Jack Clark: I think it is the "collapse of meaning." Dealing with this might be simpler than we imagine.
(Regarding Sam Altman) An acquaintance.
(Regarding Elon Musk's Grok) A serious competitor. Never underestimate Elon.
(Regarding core human skills) Spacing out. Only then can you generate original ideas. I walk long distances just to ponder new questions to ask AI.
(Regarding strategies for the US to maintain its lead) Export controls on computing power are absolutely critical. Anyone who tells you that you can sell computing power to China without falling behind in this competition is not only dead wrong but will harm this country. Export controls must be maintained because computing power is the most fundamental resource for building this technology.
(Regarding possible objections from Jensen Huang) He might hold an opposite opinion, but I must say, he is completely and utterly wrong on this point.
Source: Digital Kaiwu
• END •
[Column] Recommended Re-reads
China Does Not Lack Chips | Transcript of Jensen Huang's Latest Dialogue
AI Is Undergoing "Species Differentiation" | Transcript of Andrej Karpathy's Latest Dialogue