Computing, IT & AI
The Transformer (Attention Is All You Need)
2017 CEAshish Vaswani, Niki Parmar and six Google co-authorsAttention Is All You Need (NeurIPS 2017)Diaspora (work abroad)
2017
The neural network architecture, built entirely on attention, behind today's large language models.
- Dated
- 2017
- As scholars put it
- 2017 CE
- Originator
- Ashish Vaswani, Niki Parmar and six Google co-authors
- Source text
- Attention Is All You Need (NeurIPS 2017)
- Region
- Diaspora (work abroad)
- Discipline
- Computing, IT & AI
- Evidence
- Documented
- Confidence
- High
- Verification
- Verified-web
Background
Eight researchers at Google, listed first among them Ashish Vaswani, together with Niki Parmar, proposed replacing recurrent networks for machine translation with layers of self-attention, which process all words in parallel. All eight are credited as equal contributors.
Significance
The Transformer became the main architecture of modern AI: BERT, GPT, Gemini, Claude and image and speech models are all built on it.
Fair to say
A team result; the attention mechanism itself came from Bahdanau and colleagues (2014).
Path to modern innovation
Every large language model, including India's own, is a Transformer.
2017The Transformer (Attention Is All You Need)Global impact
- Large language models and generative AI.
- Better machine translation, including for Indian languages.
Primary sources
Further reading
Related
Corrections and additional sources: edit IND-406 on GitHub.