Datawhale Insights
Author: Google Research Team
What makes an Agent stronger and stronger over time? Some rely on bigger models, others on more data. Just the day before yesterday, Google Research revealed a completely different answer with its latest release, WikiSkill: build a three-layer knowledge architecture for the Agent, letting the skills evolve themselves.
Paper link: https://arxiv.org/abs/2608.27454
The core of this architecture is separating "experience" from "knowledge." Previous skill evolution methods (EvoSkill, Trace2Skill, SkillOpt) directly modified skills after analyzing execution traces, using each experience once and never again. WikiSkill adds a persistent knowledge base in the middle, allowing experience to settle first and then be reused.
1. Three-Layer Architecture: Separating Experience, Knowledge, and Skills
Layer One — Raw Layer: Preserving original traces to provide a factual basis for evolution
The Raw Layer stores the execution traces from each iteration, including the Agent's complete reasoning process, tool calls, and output results. This layer is immutable, with the goal of preserving the original record of "what actually happened."
Why store this separately? Because both subsequent layers need to trace back to the original traces to analyze problems. The Wiki Maintainer extracts success and failure patterns from them, and the Skill Proposer consults them on demand to see how specific tasks were executed. If the raw data were overwritten or lost, the entire evolution process would lose its factual foundation.
Layer Two — Wiki Layer: Turning one-off experience into reusable knowledge
The Wiki Layer is the heart of the architecture. It compiles raw traces into structured knowledge that accumulates continuously across iterations. It consists of three parts:
- patterns/: One markdown file per pattern, recording specific failure causes or successful strategies, along with actionable fixes
- logs.md: An evolution log, recording by iteration what was discovered and what was changed
- skill-impact.md: Which skill changes were accepted and which were rejected, with complete diffs
This layer has two key design decisions.
First, the Wiki never rolls back. When a skill is rejected, the Skills Layer reverts to its previous version, but the Wiki retains all accumulated knowledge. In the next round, the Proposer can see "why this change was rejected last time" and avoid repeating the same mistake.
Second, Knowledge and Skill are separated. Knowledge answers "what do we know," while Skill answers "what should we do." Previous methods mixed the two together, losing the reasoning context behind changes when skills were modified. WikiSkill allows knowledge to accumulate continuously, with skills growing out of that knowledge.
Layer Three — Skills Layer: Executable skills with provenance
The Skills Layer is the set of currently active skills. Each skill directory contains two files:
- SKILL.md: The skill content that the Agent reads directly when executing tasks
- PURPOSE.md: Records which pattern in the Wiki this skill was created to address
PURPOSE.md answers the question of "why does this skill exist." When a skill needs modification, the Proposer can trace back through PURPOSE.md to the corresponding knowledge pattern, understand the original design intent, and avoid blindly patching things up.
2. The Evolution Loop: From Execution Traces to Skill Updates
Each iteration runs four steps:
1. Inference Agent: Uses the current skills to run rollouts on the training set, producing traces into the Raw Layer. During training, it cannot access the Wiki—otherwise the Agent would simply look up the answer, rendering the traces useless for reference.
2. Wiki Maintainer: Analyzes the sampled success and failure traces, performs root cause analysis, and updates the Wiki's pattern directory and logs.
3. Skill Proposer: In a ReAct fashion, reads the Wiki index, consults the skill-impact history, reads specific pattern pages and traces as needed, and then proposes one skill creation or patch.
4. Gating: Evaluates the candidate skill on a validation set. If the score improves, it is accepted; otherwise, it is rolled back. The Wiki is unaffected.
The key to this loop is the coordination between steps two and three. The Wiki Maintainer compiles scattered traces into structured knowledge, and the Skill Proposer generates skill updates from that structured knowledge. Without the Wiki layer, the Proposer would be analyzing raw traces from scratch every time.
3. Experimental Results: A 9B Model with Skills Beats a 27B Bare Model
Experimental results across five benchmarks and five models:
Skill evolution and model scale are complementary. Within the Qwen family, the gains from WikiSkill increase with model size: 4B gains +12.3 points, 9B gains +17.5 points, and 27B gains +23.9 points. Stronger models benefit more from skill evolution.
Skills can compensate for scale gaps. Qwen-3.5-9B with WikiSkill averages 47.4%, surpassing Qwen-3.6-27B without skills at 39.4%. A 9B model with skills beats a 27B bare model.
Skills transfer across model families. Qwen-3.5-9B using skills evolved by Qwen-3.6-27B reaches 70.2%, compared to only 63.4% with its own skills. This shows that "discovering strategies" and "executing strategies" are two different capabilities that can be split across models.
Ablation studies confirm the value of the Wiki. After removing Wiki access, the average score drops from 63.7% to 48.7%—a decline of about 15 percentage points. The cross-iteration knowledge accumulated by the Wiki Maintainer is the prerequisite for the Skill Proposer to resolve complex failure patterns.
Final Thoughts: An Agent Grows Stronger Not Just Because of Bigger Models
WikiSkill's contribution is not a new algorithm—it is an architectural design: separating experience, knowledge, and skills into three layers, letting knowledge accumulate continuously in the middle, and allowing skills to grow out of that knowledge.
For those building Agents, this means stop focusing solely on model size and prompt tuning. Build a three-layer knowledge architecture where experience settles into knowledge and knowledge guides evolution—that is the fundamental logic that allows an Agent to keep getting stronger.
Like and share! ↓