Built for demanding work. A 262 144-token context window, adjustable reasoning effort, tool calling, and text, image and video understanding.

# Architecture

Agnes-3.0-Flash is a hybrid-attention decoder: three of every four layers run a gated delta rule (recurrent, with per-layer state independent of sequence length), and the fourth runs standard global attention. Only 18 of the 72 layers therefore hold a KV cache that grows with context.

|||
|:-|:-|
|Context length|262 144 tokens|
|Decoder layers|72 = 54 delta-rule recurrent + 18 global attention, alternating 3 : 1|
|Hidden size|5120|
|Global attention|24 query heads / 4 KV heads (6 : 1 GQA), head dim 256; RMS-norm on q and k, sigmoid-gated output|
|Delta-rule layers|16 key heads / 48 value heads, head dim 128; causal conv (kernel 4) in front, gated RMS-norm; recurrent state in fp32|
|Feed-forward|SwiGLU, intermediate size 17408; plus a parallel SwiGLU 2048 branch in every layer|
|Positions|3-axis rotary (text / height / width), interleaved mrope sections 11 : 11 : 10, base 1e7, applied to the first 25 % of each head dim (64 dims)|
|Vocabulary|248 320|
|Vision tower|27 layers, hidden 1152, patch 16, 2 × 2 spatial merge, projected to 5120Architecture	Agnes-3.0-Flash is a hybrid-attention decoder: three of every four layers run a gated delta rule (recurrent, with per-layer state independent of sequence length), and the fourth runs standard global attention. Only 18 of the 72 layers therefore hold a KV cache that grows with context.		Context length	262 144 tokensDecoder layers	72 = 54 delta-rule recurrent + 18 global attention, alternating 3 : 1Hidden size	5120Global attention	24 query heads / 4 KV heads (6 : 1 GQA), head dim 256; RMS-norm on q and k, sigmoid-gated outputDelta-rule layers	16 key heads / 48 value heads, head dim 128; causal conv (kernel 4) in front, gated RMS-norm; recurrent state in fp32Feed-forward	SwiGLU, intermediate size 17408; plus a parallel SwiGLU 2048 branch in every layerPositions	3-axis rotary (text / height / width), interleaved mrope sections 11 : 11 : 10, base 1e7, applied to the first 25 % of each head dim (64 dims)Vocabulary	248 320Vision tower	27 layers, hidden 1152, patch 16, 2 × 2 spatial merge, projected to 5120|

AA shows its 'Proprietary model' but its open: [https://artificialanalysis.ai/models/agnes-3-0-flash](https://artificialanalysis.ai/models/agnes-3-0-flash)

Agnes-AI/Agnes-3.0-Flash 33B Multimodal, AA score: 36