Author: Xiaoke, Datawhale Member
I. A Comprehensive Comparison Between Kimi K3 and GPT-5.6 Sol
Last week, the AI community welcomed a new hardcore open-source player: Kimi K3, which immediately topped the Frontend Code Arena leaderboard upon release. We at Datawhale conducted a head-to-head comparison test against another top-tier model at the earliest opportunity. We kept wondering: how could we give all our readers an intuitive feel for the differences between the two models? Finally, we came up with an idea: have them recreate the games we used to play as kids.
Kimi K3 is Moonshot's latest open-source large-parameter model, scaling up to a massive 2.8 trillion parameters. According to the official release, its overall performance still trails top closed-source models like Claude Fable 5 and GPT-5.6 Sol. For this comparison, we only chose GPT-5.6 Sol as the horizontal benchmark and didn't test Claude Fable 5 for two reasons: it's too expensive, and our account got banned, making stable re-testing impossible. Our primary focus here is evaluating its capability in 3D game development. Many models are already quite good at standard frontend web development, but 3D games involve rendering, collision, state management, and real-time interaction, making them much harder scenarios that more easily reveal a model's capability boundaries.
Therefore, we had them recreate 5 classic childhood games, using identical prompts for both Kimi K3 and GPT-5.6 Sol. We recorded videos, audited the code, and compared the finished products. Ultimately, we found: some games were immediately playable, some were close to finished products, and some still fell short.
II. Evaluating a Model: From "It Runs" to "It's Playable"
The common practice for evaluating a model's coding ability is to have it generate a single-file frontend webpage. This approach is low-cost, but it doesn't reveal true competence in complex engineering tasks.
3D game development is a better touchstone: it involves rendering, collision, state management, and real-time interaction. The model must not only write code but also iteratively debug it based on visual page feedback. Thus, we had both models use completely identical prompts and visual constraints to independently recreate 5 classic childhood games, to see who could deliver a more playable finished product.
But games can only compare "who is stronger"; they can't answer "whether K3 can independently deliver real engineering on its own." Therefore, we assigned K3 an extra non-game full-stack backend project: building a database from scratch, writing REST APIs, and getting frontend rendering to pass, entirely without human intervention.
Kimi K3 ran via Kimi Code + WebBridge, while GPT-5.6 Sol ran via Codex. The only variable was the model itself and its corresponding agent toolchain.
III. Kimi K3 vs. GPT-5.6 Sol: A Game-by-Game Comparison of 5 Childhood Classics
1. Plants vs. Zombies
When you open Kimi K3's version, you can see the green lawn, the advancing zombies, and potato mine explosions. During gameplay, you'll notice it only has endless waves—there is no level selection, no pause function, no star rating, and no win statistics. GPT-5.6 Sol's version, on top of core combat, added a level selection interface, star ratings, and win/loss statistics.
Looking at the code structure, Kimi K3 inlines all systems within a single file, directly reading and writing saves. GPT-5.6 Sol splits this into multiple modules and adds default save logic and data validation.
Kimi K3 can be played as soon as it opens, but after a few rounds, it lacks a sense of purpose and progress feedback. GPT-5.6 Sol has a longer flow, with clear levels and an evaluation system.
Kimi K3 - Plants vs. Zombies
GPT-5.6 Sol - Plants vs. Zombies
2. Metal Slug
Kimi K3's version features custom character rendering, but it has two fewer enemy types than GPT-5.6 Sol. The most noticeable difference during gameplay is that dying instantly ends the game with no option to continue. GPT-5.6 Sol implemented checkpoints, allowing you to resume from where you left off after dying.
From a code structure perspective, Kimi K3 uses circular collision detection and a more compact file organization. GPT-5.6 Sol uses rectangular collision boxes and a structure that more closely resembles a standard game architecture.
Kimi K3's visuals have their own style, but the fault tolerance is low, and the penalty for dying is high. GPT-5.6 Sol's enemies are more varied, the fault tolerance is higher, and it feels much closer to the original arcade experience.
Kimi K3 - Metal Slug
GPT-5.6 Sol - Metal Slug
3. The King of Fighters 97
The K3 version includes character and difficulty selection, with dense special-move feedback, and the character rendering details are better than GPT-5.6 Sol's blocky figures. However, it lacks stage selection, rules adjustments (like HP, time, or damage multipliers), and the settings are not persisted.
GPT-5.6 Sol implemented all these options, and the rules can be saved, making it much closer to a full-fledged fighting game you can tweak and play repeatedly.
Kimi K3 - The King of Fighters 97
GPT-5.6 Sol - The King of Fighters 97
4. Contra
The K3 version is playable out of the box, and shooting and movement work fine, but it lacks a title screen, level selection, pause, and continue features; the floating platforms are purely visual and characters cannot stand on them.
GPT-5.6 Sol built a full arcade flow, with functioning platform collisions, making it much closer to a complete arcade experience.
Kimi K3 - Contra
GPT-5.6 Sol - Contra
5. Battle City
The K3 version includes all core mechanics like base protection, upgrades, and landmines, but lacks menus, pause, level returns, and overall level management.
GPT-5.6 Sol has a full menu system, a pause menu, a settlement screen, and level config validation, making it feel much more like a game you would keep open long-term.
Kimi K3 - Battle City
GPT-5.6 Sol - Battle City
IV. Examining Kimi K3 Across 5 Games
The evaluation criteria are exactly five: whether it's immediately playable, visual completeness, control feel, rule completeness, and intelligence level.
K3's advantages: All 5 games can be opened and played immediately, and the core mechanics work; Plants vs. Zombies and The King of Fighters 97 have strong visual atmospheres and intense highlight feedback; it can proactively add content outside the prompted boundaries, like star upgrades and weapon switching; and in certain specific areas (like character rendering), it even outperforms GPT-5.6 Sol.
K3's shortcomings: Rule completeness is generally insufficient. Most games lack peripheral flows like menus, pause functions, level selection, and save validation; death penalties in some games are overly harsh, resulting in low fault tolerance; and the UI polish varies noticeably from game to game.
GPT-5.6 Sol has a stronger finished-product feel because it spends its budget on the things players won't praise but will break immersion if absent: menu hierarchies, rule persistence, boundary validation, and screen effects.
There is also a gap in efficiency. GPT-5.6 Sagittarius uses sub-agents reasonably well concurrently within Codex, with a completion time of about 25% of K3's, and is more economical in token consumption. However, GPT-5.6 Sol's API pricing is higher, and coupled with cross-border access and payment hurdles, the effective usage cost for domestic individual developers is actually higher. While K3 consumes more tokens per task, it is directly accessible within China, lowering the barrier to entry.
V. Beyond the Head-to-Head: A Pure K3 Full-Stack Project with a Database
Game comparisons intuitively showcase the differences between the two models, but they have one limitation: the scenario is fairly singular, focusing primarily on frontend expressiveness. Furthermore, regardless of whether these 5 games had a backend or not, we didn't focus on the database parts in our presentation. We also wanted to know if, when the task shifts from "writing a playable page" to a "real, non-game full-stack engineering project" requiring database design, REST APIs, and frontend integration all running correctly together, Kimi K3 could complete it independently.
So, we assigned K3 a standalone full-stack project based on the open-source project https://github.com/li-xiu-qi/paper-graph-manager (a paper citation relationship graph management system). We had it build the backend from scratch, fill in the APIs, and verify frontend rendering completely on its own. The entire process was done without manual intervention, entirely by K3 itself.
1. Environment Setup and Dependency Fixes
K3's first step was to install dependencies and start the service. It correctly identified and installed all dependencies required for the FastAPI backend, and then ran uvicorn main:app --host 0.0.0.0 --port 8001 to start the service.
During startup, it encountered three consecutive ModuleNotFoundError exceptions for dotenv, pyvis, and kimi_agent_sdk. K3 correctly assessed these as environment dependency issues rather than code bugs, and after installing them one by one, the service ran successfully on http://localhost:8001, returning a healthy status check.
2. Backend Feature Implementation and API Verification
The core requirement was to add full backend capabilities for the paper citation relationships. K3 modified three files:
- database.py: Added the paper_references table, including composite primary keys, foreign keys, and indexes.
- graph.py: Added build_reference_graph() and export_reference_html() functions.
- main.py: Added 4 citation endpoints and 2 graph endpoints.
To verify delivery quality, K3 ran an 8-step end-to-end test on a new port, 8002, all of which passed: adding papers to the database, adding citations, querying outgoing/incoming references, fetching the graph, exporting HTML, deleting citations, and verifying emptiness after deletion. 36 newly added tests all passed; in the full suite, 134 passed and 2 failed. K3 used git stash to roll back its changes, and these 2 failures persisted, proving they were pre-existing legacy issues in the project, unrelated to this new feature.
3. Frontend Integration and Real Page Rendering
With the backend running, K3 continued to process the frontend. Upon starting Vite, it encountered a path alias error: Failed to resolve import "@/lib/utils" from "src/components/ui/dialog.tsx". This is a typical engineering problem: the frontend can run, but code contains incorrectly configured alias references left over from the original project.
K3 didn't get stuck; instead, it used Kimi Code's WebBridge to directly control the browser and verify the page state: it injected JS to probe rootLen, confirming that the page grew from 0 to 30081, proving the frontend had indeed rendered.
Ultimately, the page rendered successfully. The Paper Graph Manager dashboard homepage displays the total number of papers, annotated count, note coverage, a list of recently added papers (such as Mistral 7B and Verifiable Fully Homomorphic Encryption), as well as entry points for modules like paper management, knowledge graph, smart chat, and note management.
VI. Concluding Thoughts
Kimi K3 demonstrates excellent performance in output looks, game logic, and operational smoothness. As an open-source model, achieving this level of completion is a rare, top-tier feat, marking a milestone for open-source models in China.
However, as of yesterday, K3 has closed purchasing channels entirely due to insufficient computing power, showing that Kimi very much cares about the existing user experience. Interested folks can wait until they have sufficient computing power back online before purchasing and trying it out.
But raising the standard to "something you can directly hand to a player to open repeatedly," K3 still has clear gaps in system completeness, rule rigour, and the packaging layer. It is more suited for rapidly producing playable prototypes rather than serving as an out-of-the-box finished product.
Beyond the head-to-head comparison, the full-stack project added another dimension of judgment: K3 can read existing project structures, distinguish between environment and code issues, independently complete database design, REST APIs, and frontend integration, and verify real rendering via browser automation. This proves its capability boundaries extend beyond just frontend pages; it can handle real backend engineering tasks end-to-end.
If you are considering using K3: it is already playable for core gameplay mechanics, can independently run real backend engineering, and has advantages in specific areas like character rendering and visual presentation. But if what you want is an out-of-the-box complete product experience, it still has room to catch up in system completeness and engineering standards. Evaluating purely from the game case completion perspective, K3 has reached over 90% of the level of top closed-source models. Did you manage to grab a Kimi membership?~