GPT-2
Sign in to saveGenerative Pre-trained Transformer 2 (GPT-2) is a large language model by OpenAI and the second in their foundational series of GPT models. GPT-2 was pre-trained on a dataset of 8 million web pages. It was partially released in February 2019, followed by full release of the 1.5-billion-parameter model on November 5, 2019.
Key facts
- Software.name
- Generative Pre-trained Transformer 2 (GPT-2)
- Software.screenshot
- File:GPT2-talks-about-GPT2.png
- Software.caption
- GPT-2 completion using the Hugging Face Write With Transformer website, prompted with text from this article (All highlighted text after the initial prompt is machine-generated from the first suggested completion, without further editing.)
- Software.author
- OpenAI
- Software.repo
- https://github.com/openai/gpt-2
- Software.replaces
- GPT-1
- Software.replaced_by
- GPT-3
- Software.license
- MIT
via Wikipedia infobox
Article
11 sectionsContents
- Training
- Release
- Restrictions and partial release
- 774M release
- Full 1.5B release
- Small and Medium Releases
- Limitations
- Application and subsequent research
- Performance and evaluation
- See also
- References
Generative Pre-trained Transformer 2 (GPT-2) is a large language model by OpenAI and the second in their foundational series of GPT models. GPT-2 was pre-trained on a dataset of 8 million web pages. It was partially released in February 2019, followed by full release of the 1.5-billion-parameter model on November 5, 2019.
GPT-2 was created as a "direct scale-up" of GPT-1 with a ten-fold increase in both its parameter count and the size of its training dataset. It is a general-purpose learner and its ability to perform the various tasks was a consequence of its general ability to accurately predict the next item in a sequence, which enabled it to translate texts, answer questions about a topic from a text, summarize passages from a larger text, and generate text output on a level sometimes indistinguishable from that of humans; however, it could become repetitive or nonsensical when generating long passages. It was superseded by the GPT-3 and GPT-4 models, which are no longer open source.