Model overview · Kimi
Kimi Linear 48B-A3B
Generated from Architecture IR 0.1.0 · revision 69e2a71feb
· report a correction
Overview
Fact sheet
- Layers
- 27
- Hidden size
- 2304
- Attention heads
- 32 · KV 32
- Vocab
- 163,840
- Context
- 1M
- Params
- 48B total / 3B active
Architecture Genome
One column per layer — top: attention, bottom: FFN/MoE.
Layer
attention (GQA/MLA) linear / recurrent attention MoE dense FFN
Evidence
Every visible number resolves to one of these claims. Inferred or unknown values are never shown as fact.
| Claim | Value | Status | Source |
|---|---|---|---|
facts.num_hidden_layers | 27 | • reported | hf_config · $.text_config.num_hidden_layers |
facts.hidden_size | 2304 | • reported | hf_config · $.text_config.hidden_size |
facts.num_attention_heads | 32 | • reported | hf_config · $.text_config.num_attention_heads |
facts.num_key_value_heads | 32 | • reported | hf_config · $.text_config.num_key_value_heads |
facts.head_dim | 72 | • reported | hf_config · $.text_config.head_dim |
facts.vocab_size | 163840 | • reported | hf_config · $.text_config.vocab_size |
facts.context_tokens | 1048576 | • reported | model_card · context window |
facts.total_params | 48000000000 | • reported | model_card · parameter count |
facts.active_params | 3000000000 | = derived | derived · activated experts + attention params |
topology.attention_groups[0] | "20 KDA layers (3 of every 4, shortconv kernel 4)" | • reported | hf_config · $.text_config.linear_attn_config.kda_layers |
topology.attention_groups[1] | "7 MLA layers (1 of every 4, kv_lora_rank 512)" | • reported | hf_config · $.text_config.linear_attn_config.full_attn_layers |
topology.experts.routed_total | 256 | • reported | hf_config · $.text_config.num_experts |
topology.experts.active_routed | 8 | • reported | hf_config · $.text_config.num_experts_per_token |
topology.experts.shared | null | • reported | hf_config · $.text_config.n_shared_experts |
Compare
Compare this model with any other catalog model (see catalog).