Also known as transformer model, transformer architecture, transformers
machine-learning model architecture first developed by Google Brain
via Wikidata · CC0
~40 min read
A standard transformer architecture. Many modern diagrams show the pre-layer normalization (pre-LN) convention, while the original 2017 paper used post-layer normalization (post-LN).
In deep learning, the transformer is a family of artificial neural network architectures built around the attention mechanism. Transformers were introduced to model sequential data without recurrence and without convolutions, allowing much more parallel computation during training. They are now a dominant architecture for natural language processing, computer vision, speech processing, multimodal learning, robotics, and many other sequence-modelling tasks.
via Wikidata sitelinks · CC0
Discovered by embedding cosine similarity (sentence-transformers MiniLM, 384-dim).