TransformerMamba

Jamba: AI21 Labs’ New Hybrid Transformer-Mamba Language Model

Language models have witnessed rapid developments, with Transformer-based architectures leading the way in natural language processing. However, as models grow…