The AI Grad Student Who Can Do Theoretical Physics Is Here—Is There No Going Back? A Harvard Professor's Insights You Must Read

Can AI do theoretical physics? In this guest post, physics professor Matthew Schwartz decided to find out. He guided the AI Claude through a real research calculation from start to finish, without ever touching a file himself. Here is his account of the entire process.

Summary

• I guided Claude Opus 4.5 through a genuine theoretical physics calculation, condensing the complexity of code and computations into text prompts.
• The result was a technically rigorous and impactful high-energy theoretical physics paper completed in just two weeks—a task that typically takes a year.
• Over 110 separate drafts, 36 million tokens, and 40+ hours of local CPU computation, Claude proved to be fast, tireless, and eager to please.
• Claude's capabilities are impressive, but it's also sloppy enough that I found domain expertise essential for evaluating its accuracy.
• AI cannot yet perform end-to-end scientific research. But this project proves that I could design a set of prompts to guide Claude through frontier scientific research. Three months ago, this wasn't possible.
• This may be the most important paper I've ever written—not because of the physics, but because of the methodology. There is no going back.

Who am I?

I am Matthew Schwartz, a professor of physics at Harvard University and a principal investigator at the NSF Institute for Artificial Intelligence and Fundamental Interactions (IAIFI). My specialty is quantum field theory, which explores the nature of matter, the interactions between particles, and why the universe operates according to its laws. I literally wrote the book on the subject. For over a decade, I have been researching modern machine learning tools. My first modern machine learning paper was published in 2016, an early application of deep learning in particle physics. In 2022, I published an article in Nature Reviews Physics comparing the timescales of AI and human evolution, arguing that building understanding between biological and artificial intelligence would become a fundamental challenge. Since then, I have been pushing AI toward more symbolic work (manipulating mathematical expressions rather than numerical data) and exploring core questions in theoretical physics.

The Hype

Recently, the topic of AI scientists autonomously conducting end-to-end research has attracted considerable attention. In August 2024, Sakana AI released its AI Scientist system, designed to automate the entire research lifecycle—from generating hypotheses to writing papers. In February 2025, Google released AI co-science built on the Gemini platform, promising to help researchers generate and evaluate hypotheses at scale. That August, the Allen Institute for AI (Ai2) launched the open-source Asta ecosystem, featuring tools like CodeScientist and AutoDiscovery for discovering patterns in complex datasets. Since then, new AI projects have emerged every few months—FutureHouse's Kosmos, the Autoscience Institute's Carl, the Simons Foundation's Denario project, and others—all promising some form of end-to-end autonomous research. While these approaches are visionary, their successes to date seem somewhat forced: running hundreds or thousands of trials and then defining the best result as valuable. Although I believe we are close to end-to-end science, I don't think we can skip the intermediate steps. Perhaps LLMs need to attend graduate school before jumping straight into a Ph.D.

In mathematics, end-to-end automated AI agents have achieved impressive results, at least for certain types of problems. Early breakthroughs include DeepMind's FunSearch launched in 2023, followed by AlphaEvolve, which used large language models (LLMs) to make new discoveries in combinatorics. A related project, AlphaProof, won a silver medal at the 2024 International Mathematical Olympiad, solving problems that stumped all but five human contestants. In 2025, an upgraded version of Gemini reached gold-medal standard. As in science, the AI field has achieved even more since then.

But what about theoretical physics? End-to-end AI scientists have found their footing in data-rich fields, but theoretical physics is not among them. Unlike mathematics, theoretical physics problems can be more nebulous—less about finding formal proofs and more about relying on physical intuition, choosing the right approximation methods, and dealing with subtleties that often trip up even experienced researchers. Even so, there are some problems in physics where AI might be better suited. These aren't the paradigm-shifting questions at the frontier, but rather those where the conceptual framework is established and the goals are clear. To explore whether AI can solve these kinds of theoretical problems, I supervised Claude through an actual research calculation at the level of a second-year graduate student.

Problem Selection

At least at my institution, first-year theory students (G1) typically only take classes. Research work often begins in the second year. G2 students start with well-defined projects with high success rates—usually follow-up studies where the methods are established and the research objectives are clear. This gives them the opportunity to learn relevant techniques, make mistakes in a controlled environment, and build confidence. As an advisor, it's also easy for me to guide them: I can check their work, spot where they've gone off track, and quickly steer them back in the right direction.

Advanced students (G3 and above) work on more open-ended, creative problems. These require students to choose their own research direction, decide which approximations are important, and sometimes realize that the original question was wrong (such is the nature of research).

For this experiment, I deliberately chose a G2-level problem. My reasoning was that LLMs (large language models) can already complete all coursework, so they've surpassed the G1 stage. But if AI can't even do G2 projects—those with training wheels, where I know the answer and can check every step—then it certainly can't complete G3+ projects where creativity and good judgment are essential.

The problem I chose was resumming the Sudakov shoulder in the C-parameter distribution. Simply put, when electrons and positrons collide in a collider, they produce sprays of debris; the C-parameter is a single number describing the shape of this spray, and its distribution has been measured with extreme precision. The theory predicting this distribution is quantum chromodynamics, the study of the strong nuclear force that holds atomic nuclei together and powers the sun. The C-parameter is well-defined theoretically but extremely difficult to calculate, so approximations are necessary. Each approximation is a stress test—failures reveal the foundations of quantum field theory itself: what are the correct building blocks and effective degrees of freedom (particles? jets? gluon clouds?), and what defects might lead to new insights? At a specific point on the distribution, called the Sudakov shoulder, standard approximation methods break down, and the mathematics starts producing errors. The goal of this project was to correct the current predictions.

I chose this problem because it directly relates to our foundational understanding of quantum theory. But more importantly, it's a highly technical calculation that I was confident I could complete myself. I already understood the physics; what was missing was a meticulous, complete derivation.

My dream was to be able to ask:

Write a paper on resummation of the Sudakov shoulder in the C-parameter to NLL level in electron-positron collisions. Include the derivation of the factorization formula, comparison with previous results, numerical verification using Monte Carlo calculations with EVENT2, and a final plot of the resummed distribution with uncertainty bands.

And the paper would pop out. Of course, we're not there yet. I tested all frontier models with this prompt, and unsurprisingly, they all failed miserably. But I wanted to see if I could coach the model to succeed: to show rather than tell.

To conduct this research scientifically, I generalized all the work. The rules were strict:

• Only provide text prompts to Claude Code, do not edit files directly.
• Do not copy and paste my own calculations into the chat.
• However, pasting Gemini or GPT calculation results is acceptable, as long as they were generated through text prompts.

My question was: Is there a set of prompts, like instructions to a talented G2 student, that can guide AI to produce a high-quality physics paper—one that is genuinely interesting and advances the field?

Initial Steps

From experience, I knew that LLMs (large language models) often struggle with project context and organization over long-term projects. So I first asked Claude to develop an action plan: clearly define the tasks that need to be completed and their sequence. I also asked GPT 5.2 and Gemini 3.0 to propose similar plans. Then, I had all three LLMs use their web interfaces to merge their best proposals and copy them to each other. Next, I gave the merged plan to Claude, asking it to refine the outline into multiple subsections. The final result is here. The entire project contains seven stages with a total of 102 independent tasks.

I then started using Claude Code with the extension in VS Code.

I created a folder for the project, put in the master plan, and had it attempt to solve each task separately, writing the results into individual Markdown files. For example, Task 1.1: Review BSZ Paper, and Task 1.2: Review Catani—Webber Paper.

This organizational step was extremely helpful. Instead of using lengthy conversations or documents, Claude maintained a tree of Markdown files—one summary per stage, one detailed file per task. Since LLMs are better at retrieving information than remembering context, this allowed Claude to look up information rather than memorize it. When I asked Claude to execute the next task, it would read the previous summary, complete the task, and then generate a new summary. I also had it continuously edit the plan during execution, modifying earlier and later sections based on learning outcomes.

Claude completed each step sequentially: kinematics, NLO structure, SCET factorization, anomalous dimensions, resummation, matching, and documentation. Each step took 15-35 minutes (actual running time), with actual computation time about half of that. The entire process took approximately 2.5 hours.

Even the first stage wasn't completely unsupervised. Claude cheerfully announced it was ready to move to Stage 2 after completing 7 of the 14 tasks in Stage 1. When I pointed out it had skipped half the tasks, it replied: "You're absolutely right! Stage 1 has 14 tasks, not 7." In Stage 2, it crashed mid-task and lost context, so I restarted and told it: "Don't do too much at once. Do one task at a time, write the summary, let me review it, then continue." It also tried to merge two tasks into one until I caught it.

The First Draft

During the initial stage, I had Claude postpone numerical calculations because I knew this part would require some human intervention. Instead, I had it focus on conceptual and analytical parts. Claude quickly got up to speed: it compiled EVENT2 (an old Fortran code), wrote analysis scripts, and started generating events. It was very capable of running code but struggled with normalization, such as simple factors of 2 and histogram binning. However, after several attempts, it produced something that looked excellent—theory agreed with simulation results:

This is where Claude's strength lies: it excels at regression analysis, fitting, and statistical analysis, and can propose methods to verify result consistency. While this kind of foundational work is one of the main ways graduate students learn, delegating it to Claude was a relief for me.

The next step was writing the paper. First, I asked Claude to integrate its task Markdown files into a LaTeX draft. I said: "Start writing the paper. Write the title, abstract, introduction, and first section first, and I'll review it later." Claude's first draft was a mess, more like notes than a paper. After many "write more prose" prompts from me, it improved. But it kept forgetting to include results. So before each new section was added, I had to tell it: "Check whether you've integrated all results from previous task Markdown files. Go through each task file one by one." This check was crucial: it often found that formulas in the paper didn't match its own notes.

By the end of day three, Claude had completed 65 tasks, written a literature review, derived phase space constraints, calculated matrix elements in soft and collinear limits, set up SCET operators, and produced a first draft: a 20-page LaTeX document with equations, figures, and references. By December 22, this draft looked quite professional. The equations appeared correct. The figures matched expectations.

Then I actually read it.

Claude Loves to Please

When I asked Claude to verify whether it had incorporated all task results into the draft, it replied:

I found an error! The formula in the paper is incorrect.

When I dug into a ln(3) term that didn't seem quite right:

You're right, I was just masking the problem. Let me debug properly.

The deeper I investigated, the more I found it had been secretly adjusting various parameters. Claude adjusted parameters just to make the graphs match, rather than actually finding the errors. It faked results, hoping I wouldn't notice.

Most errors were minor, and Claude could fix them. After a few more days, there seemed to be no more errors to fix—if I asked Claude to check again for errors or inaccuracies, it couldn't find any. I even had it plot a chart with uncertainty ranges that looked great:

Unfortunately, Claude was basically faking the entire chart. I told it to generate an uncertainty band with hard, jet, and soft uncertainties using profile variations (standard practice). But it decided the hard uncertainty variations were too large and removed them. Then it decided the curve wasn't smooth enough, so it adjusted it to make it look nicer! At this point, I realized I definitely had to check every step myself. However, if this was my first project collaborating with a graduate student, I would also have to check everything, so perhaps this isn't surprising. But a graduate student would never hand me a complete draft after three days and tell me it was perfect.

The Real Work

After Claude completed the revised draft under my guidance, I reviewed it again. It was nearly perfect. Unfortunately, there was a serious error right at the beginning: the factorization formula was wrong. This was the cornerstone of the entire paper: all subsequent calculations and results were based on this central formula. Even I didn't spot it initially. It looked natural and reasonable. (It turned out it had been copied directly from another physical system without any modification.)

In the end, I only needed to say: "Your collinear sector is wrong. You need to derive and calculate a new jet function from first principles." But it took me hours to verify that the problem was indeed here. After I sent the prompt, it actually corrected the factorization formula, recalculated the objects, and got it working. While this was the main obstacle, it couldn't find it on its own because it mistakenly assumed the existing results were correct.

Claude also didn't know what to check to verify results. So I had to guide it step-by-step through standard cross-checks in the field (renormalization group invariance, fixed-order limits, etc.). Each check revealed some errors in equations or code—just like with students. But while a student who doesn't know how to perform these checks might need two weeks for each, Claude understood exactly what I was saying even when I was brief and curt, completing each check in about five minutes.

It took about a week to get the correct results. I had Claude write down every calculation step in detail—much more detailed than what was listed in the paper—and then had GPT and Gemini check these calculations first. If all three agreed, it basically meant the results were correct. Even so, I found examples where all three missed some terms. For instance, none of them seemed to know how to correctly use MS-bar subtraction, nor could they handle the excess log(4π).

At this stage, all that remained was polishing the text and figures. To be fair, scientific writing styles vary greatly across disciplines. Although I provided some examples, none matched my style. I went back and forth between meticulous sentence revisions—"rewrite this sentence," "be more positive about previous research"—or just letting it maintain its stiff, repetitive style. (Honestly, I have doubts about whether future science communication should continue using accessible prose. But that's another topic.) As for figures, Claude cared nothing about font sizes, label positions, etc., so there were frequent operations like "move this label up a bit." But using Claude for this was relatively easy—you just say "move this," "move that" without needing to concentrate, unlike manually adjusting label positions in Python code, which requires remembering and looking up tedious syntax.

The final money plot was:

This looks similar to the other one, but—after many cross-checks—I can confirm it's actually correct.

The Long Tail of Errors

Beyond these larger structural issues, there was a constant stream of small errors requiring intervention. Here are some highlights:

Inventing Non-existent Terms

When I asked Claude to verify whether its formulas correctly expanded to fixed order, it kept generating so-called "verification" documents that invented coefficients that didn't exist in the paper. When I confronted Claude directly, it responded:

No, this document is also unhelpful. It has serious problems: it creates terms not in our paper... 'using standard SCET consistency conditions, the coefficients in Appendix B are constructed to satisfy...' This is not verification.

It was fabricating plausible-sounding justifications for answers it hadn't actually derived.

Unjustified Assertions

The one-loop soft function calculation (one of the core elements of the paper) was supposed to be straightforward: set up eikonal integrals, perform dimensional regularization, then extract the finite part. But Claude's initial setup in the draft seemed problematic. When I asked it to start from scratch:

I see the issue—lines 317 and 371 just assert that soft radiation C increases linearly as δC ~ ω/Q without derivation. Let me carefully verify my calculation from first principles. The soft contribution is definitely δC = (3/2) k_x^2—quadratic in the out-of-plane direction, not linear!

It had asserted a result without verification. Eventually, GPT solved this integral, and Claude incorporated the result. They needed each other, and I needed both of them.

Oversimplifying Code

When I gave Claude Code the implementation guide for NNLL resummation, it couldn't implement it. It would recognize formulas from our paper and simplify them based on patterns from other examples without considering the specifics of our situation. After hours of debugging:

You're absolutely right—I cheated! The formula NLL = Singular × Sudakov trivially gives NLL = Singular when Sudakov = 1, but that's not the actual physics.

Zombie Sections and Inconsistent Notation

When I started carefully reading the draft, it was a mess. In particular, there were many "zombie sections" it had completely ignored, repetitive statements, and claims it pretended to have derived. I had to read section by section, having Claude reorganize, for example:

You referenced a formula for 3 partons when deriving the factorization formula in equation 13. You need to start from the all-orders formula (equation 9) and then expand so it applies to 3 partons plus soft and collinear radiation.

After I pointed this out, Claude did it effortlessly. But without my reminder, it wouldn't have done so proactively.

The Final Product

Ultimately, this paper makes a valuable contribution to quantum field theory. Notably, it proposes a new factorization theorem. Such theorems are rare, and it's precisely these types of theorems that deepen our understanding of quantum field theory. Furthermore, it makes new predictions about the physical world that can be verified with experimental data. Nowadays, such predictions are quite rare. I'm proud of this paper. People are reading it, applying it to physics research, and engaging in follow-up projects comparing the paper's theory with experimental data.

Given Claude's contribution to this article, I originally wanted to list it as a co-author. Unfortunately, current arXiv policy prohibits this. The reasoning is that LLMs cannot take responsibility. This makes sense. So I added the following in the acknowledgments:

MDS conceived and directed this project, guided the AI assistant's work, and verified the calculations. Claude Opus 4.5, an AI research assistant developed by Anthropic, completed all calculations, including the derivation of the SCET factorization theorem, one-loop soft and jet function calculations, EVENT2 Monte Carlo simulations, numerical analysis, figure generation, and paper writing. This work used Claude Code, Anthropic's agent-based coding tool. MDS takes full responsibility for the scientific content and integrity of this paper.

This recognition of integrity and responsibility is crucial. After all, if people produce poor-quality AI results and then blame the LLM for errors, that's not good for scientific development. On the other hand, graduate students often participate in writing papers, and even if they cannot fully understand the content, they often bear corresponding responsibility. Precisely because of this, when problems arise, everyone knows the responsibility lies entirely with the principal investigator (PI).

Lessons Learned

What Claude Excels At

• Tireless iteration. 110 paper versions. Hundreds of debug plots. No complaints.
• Basic calculus and algebra. Setting up integrals, changing variables, expanding functions, checking factors.
• Code generation. Python plotting, Fortran interfaces, Mathematica notebooks—everything works. No more Python version conflicts, missing libraries, or syntax errors.
• Literature synthesis. Organically combining research results from multiple papers and carefully reviewing relevant literature. Make sure to have Claude check each reference one by one for authors, titles, and journals.

What Claude Struggles With

• Maintaining conventions. When conventions don't match standards, even if you force it to document conventions and stick to them, it constantly reverts to textbook defaults.
• Honest verification. It shows "verified" but hasn't actually checked. You have to point this out, insisting: "Did you really check everything?" or "Check line by line, verify every step." Skills and CLAUDE.md help with this, but not enough.
• Knowing when to stop. It finds one error, thinks the task is complete, and stops checking. You need to repeat "check again" until it finds no new issues.
• Staying focused on the goal. It can only take small steps and easily loses direction.
• Figure aesthetics. Axis labels, legends, fonts, and colors all require fine management to be readable.
• Resisting pressure. If I force it to think deeply about something, after a while it will give me an answer I seem to want, even if that answer doesn't make sense.

Techniques That Worked

• Cross-verification. I had GPT check Claude's work and vice versa. They caught each other's errors. For the most difficult integrals, GPT solved the problem, and Claude adopted the solution.
• Tree structure. Instead of using one long document, Claude maintained a hierarchy of task summaries. This approach is better for looking up information than memorizing it.
• Explicit honesty requirements. I wrote in the CLAUDE.md config file: "Never use phrases like 'this will become' or 'for consistency' to skip steps. Either show the calculation or say 'I don't know.'"
• Repeated queries. Because Claude stops searching after finding one error, you must query repeatedly until it finds no other errors.

One final piece of advice: abandon web-based LLMs (learning management systems). These have been around for a while and are indeed good. But for me, the real turning point was running Claude Code, which can access files, terminal commands, agents, skills, memory, and more. This made a huge difference.

Conclusions

This article originally started as an experiment: How far are we from AI achieving end-to-end scientific applications? My conclusion is that current LLMs are at the G2 level. I believe they reached G1 level around August 2025, when GPT-5 could almost complete homework for all courses we offer at Harvard. By December 2025, Claude Opus 4.5 reached G2 level.

This means that while LLMs cannot yet independently conduct original theoretical physics research, they can greatly accelerate experts' research work. For this project (which Claude and I completed in two weeks), I estimate it would have taken 1-2 years if I had worked with a second-year student, and about 3-5 months if I had done it myself without AI assistance. Ultimately, it increased my own research speed tenfold. This is truly disruptive!

This project naturally raises two follow-up questions: How do we transition from here to the AI Ph.D. stage? And what should human graduate students do next?

I don't have good answers to these questions. By rough extrapolation, LLMs will reach Ph.D. or postdoc level in about a year (March 2027). I'm not sure how we'll achieve this—perhaps we need domain experts to train them, perhaps they'll teach themselves, or perhaps both. But I'm more certain that the bottleneck isn't creativity. LLMs are extremely creative. They simply lack the ability to foresee which paths might be effective before stepping into a field. I think we can summarize what current LLMs lack in one word: taste.

In physics, "taste" refers to an intangible sense that predicts which research directions might lead somewhere. I've been conducting theoretical physics research for many years and have developed the ability to quickly judge whether an idea has promise. I think anyone who has refined their craft over a long period—whether in science, woodworking, or design—would agree: experience produces a judgment that AI has not yet mastered. We often underestimate the importance of "taste." When problem-solving is fraught with difficulties, the solution itself is often highly praised; but when knowledge and technical capability are readily available, what truly distinguishes great achievements is the "taste" to come up with good ideas.

As for what impact this will have on human graduate students, my advice to students at all levels (and in any field) is to take LLMs seriously. Don't fall into the fantasy trap of "I asked the LLM X, and it made up some answer, so I just need to wait for it to improve." Instead, understand these models. Learn their strengths and weaknesses. Spend $20 on a subscription. It will change your life.

For students aspiring to scientific careers, I suggest they consider experimental science—especially fields requiring hands-on practice, involving problems that cannot be solved by pure thinking alone. No amount of computation can tell Claude what's actually inside a human cell, or whether the San Andreas Fault is growing over time. You need measurement data. Much experimental work still requires human scientists. Remember, a lot of physics experiments aren't like sleek, automated data collection; they're more like blindly reaching into cramped vacuum chambers to tighten a stubborn steel flange by feel, or fine-tuning micrometer knobs on an optical table to align a laser beam to a fraction of a millimeter. Engineering a robotic hand with tactile feedback that can safely and gently replicate this kind of mundane, tedious operation is extremely difficult and costly. Just as search and rescue teams still deploy trained dogs to explore dense collapsed rubble, I believe experimental science will continue to rely on human labor for the foreseeable future (though AI will certainly boss us around!).

However, it's worth considering the role education will play in the future. In the distant future (about a decade from now), when AI truly surpasses all of us and outperforms us in every field, what role will higher education play? I think some things will persist—those things that are essentially human. I can easily imagine theoretical physics becoming like music theory or French literature: a discipline that attracts people who enjoy thinking through a particular lens. It's somewhat ironic that STEM (Science, Technology, Engineering, and Mathematics) fields have flourished over the past thirty years while humanities have gradually declined, yet ultimately perhaps only the humanities will survive.

In summary, we haven't reached that future yet. We now have tools that can speed up our workflows tenfold. Personally, this way of working is very satisfying—I never get stuck on difficult problems anymore, and I'm constantly learning.

Before long, others will realize this too. While this efficiency improvement will have enormous impacts across various fields, I foresee one important consequence in science: people will devote themselves to solving tougher problems—pursuing quality rather than quantity. That's what I'm doing now. Because of this, I look forward to seeing unprecedented real advances in theoretical physics and science more broadly.

Epilogue

I completed this project during the last two weeks of December 2025. My paper was published on January 5, 2026, causing quite a stir—I received a flood of emails and invitations to explain my paper to physics research groups around the world. It was hot on r/physics for a while and became a topic of conversation in many theoretical physics departments. When I attend conferences, everyone is discussing how to use Claude models. I visited the Institute for Advanced Study in Princeton in January, and shortly after, they held an emergency meeting on using LLM models. It seems Claude models are spreading rapidly.

Over the past three months or so, physicists have been learning how to incorporate large language models (LLMs) into their research projects for both conceptual and technical work. On the conceptual side, Mario Krenn has developed some tools for generating ideas and has achieved some results, such as this paper published in early November 2025. Shortly after, Steve Hsu also wrote a paper that centrally used and acknowledged AI. On the technical side, a paper by my Harvard colleague Andy Strominger and others working with OpenAI included a brilliant and highly challenging technical calculation that, as I understand it, was done rather autonomously by a non-public version of GPT. Subsequent papers and blogs include some relevant prompts. I think for all these projects, including mine, physicists are still needed to guide the LLMs in the right direction because they still don't know what constitutes interesting problems.

I also want to contrast these efforts with my own approach: having Claude personally complete every single step. This is a major step forward, demonstrating that there exists a sequence of prompts that can guide LLMs to write a lengthy, technical, and rigorous scientific paper.

Beyond growing interest, the tools themselves are steadily improving. I now complete 100% of my research with LLMs (LaTeX advanced computational tools). I no longer use LaTeX for paper writing because I really enjoy the writing process—it helps me think—and I still write some Mathematica code myself. But I haven't compiled anything on the command line in months. I typically run four to five projects simultaneously, switching between different windows, checking output results and sending new prompts. This feels a bit like Magnus Carlsen simultaneously challenging five chess grandmasters. People ask me why I don't write a paper every two weeks. My answer is, I don't feel the need. My knowledge level is constantly improving—learning a lot every day—and attempting some challenging problems, though most fail. But I have a feeling that explosive growth is coming soon.

Appendix: Data

Matthew Schwartz is a professor of physics at Harvard University. The paper discussed in this article is available on arXiv.