Yue zhi Anmian has released the model weights and technical report for Kimi K3, and has open-sourced MoonEP, FlashKDA, and AgentEnv—the key Infra technologies underpinning Kimi K3 model training—making them freely available for anyone to download and deploy.
Source: Moonshot Kimi
Today is Kimi K3 Open Day. We are releasing the model weights and technical report for Kimi K3, and open-sourcing key infrastructure technologies underpinning Kimi K3’s training: MoonEP, FlashKDA, and AgentEnv. We hope this will accelerate the deployment and adoption of cutting-edge intelligence and advance AGI research.
Kimi K3 Model Weights Released
Kimi K3 is our most capable model to date: a 2.8-trillion-parameter Mixture-of-Experts (MoE) model with native visual understanding capabilities and support for a 1-million-token context window.
Kimi K3 has approximately three times the parameter count of Kimi K2.5. However, scaling is not merely about stacking parameters. Under constrained computational resources, we achieved a 2.5x improvement in scaling efficiency—meaning that, under optimal compute allocation, each unit of compute now yields roughly 2.5 times more intelligence—thanks to a series of innovations including Kimi Delta Attention, Attention Residuals, and MoonEP.
Now, anyone can download and deploy the Kimi K3 model—for internal R&D or integration into end-user products—freely and without restriction (for other usage scenarios, please refer to the Kimi K3 license).
Kimi K3 Technical Report Published
Alongside the release of Kimi K3 model weights today, we are also publishing our model training methodology—the Kimi K3 technical report is now available.
From the technical report, you can learn about these technical details:
KDA + AttnRes: Efficient long-context modeling is achieved by mixing KDA and Gated MLA in a 3:1 ratio, while block-level attention residuals enhance cross-layer information flow.
Stable LatentMoE: For each token, 16 experts are activated from a pool of 896 routed experts, maintaining extremely high sparsity while ensuring training stability through SiTU-GLU and Quantile Balancing.
MoonViT-V2: The vision encoder is trained from scratch using next-token prediction without contrastive pretraining, achieving performance on par with the SigLIP initialization baseline while enabling a more stable optimization process.
Post-training and evaluation: Large-scale synthetic tasks across three domains—general reasoning, general agents, and programming agents—supported by reinforcement learning infrastructure handling million-token contexts, along with comprehensive evaluation results from nearly 20 internal benchmark datasets.
The points above are only an overview; detailed discussions and ablation studies are fully elaborated in the technical report.
Key Infra Technologies Behind Kimi K3 Open-Sourced
Behind the model’s capabilities lies the stable support of its training system—the infrastructure (Infra) layer. Today, we introduce three key Infra technologies that power Kimi K3 training: MoonEP, FlashKDA, and AgentEnv, covering critical components from high-performance communication and operators to distributed RL environments. FlashKDA was previously open-sourced, while MoonEP and AgentEnv are now officially open-sourced with this release.
MoonEP: MoonEP is our high-performance communication library designed for extremely large, fine-grained MoE models, enabling expert-parallel communication to achieve peak efficiency even under load imbalance.
FlashKDA: FlashKDA is our high-performance kernel implementation of Kimi Delta Attention. On NVIDIA H20 hardware, it delivers a 1.72–2.22× speedup in prefill latency compared to the flash-linear-attention baseline and can serve as a direct drop-in replacement backend for flash-linear-attention.
AgentEnv: AgentEnv is a sandbox system co-developed with KVCache.ai for large-scale agent environment execution. It provides high-fidelity, strongly isolated sandboxes for Kimi K3’s post-training phase, flexibly supporting rapid snapshotting, restoration, and forking to handle massively parallel agent workflows and training tasks.
These three technologies are pivotal to the training efficiency and stability of Kimi K3. By open-sourcing them today, we hope they can also empower your development of next-generation models.
Why We Choose Openness
We firmly believe in the value of open-weight models. They lower the barrier to accessing intelligence, drive innovation, and empower users with greater control over their data, enhanced privacy protection, and data ownership. Over the past few weeks, we have been encouraged by the strong support we’ve received from the AI community and technology leaders who share our vision.
We believe that for a transformative technology like AGI, a broad and open ecosystem is the most suitable foundation. To this end, we will continue to contribute our efforts.
Editor/Stephen