Olmo Hybrid
ストックにはログインが必要です
7B open model mixing transformers and linear RNNs
Artificial Intelligence
Open Source
Olmo Hybrid is a fully open 7B model that combines transformer attention with linear RNN layers. Utilizing a 3:1 pattern of Gated DeltaNet to attention, it matches the accuracy of Olmo 3 on MMLU while using 49% fewer tokens.
投票数: 77