GloVe
Sign in to saveAlso known as Global Vectors for Word Representation
GloVe, coined from Global Vectors, is a model for distributed word representation. The model is an unsupervised learning algorithm for obtaining vector representations of words. This is achieved by mapping words into a meaningful space where the distance between words is related to semantic similarity. Training is performed on aggregated global word-word co-occurrence statistics from a corpus, and the resulting representations showcase interesting linear substructures of the word vector space. As log-bilinear regression model for unsupervised learning of word representations, it combines the f
Source code
We provide an implementation of the GloVe model for learning word representations, and describe how to download web-dataset vectors or train your own. See the project page or the paper for more information on glove vectors. For documentation and analysis of 2024 vectors, please see the report Download pre-trained word vectors NEW 2024 VECTORS The links below contain word vectors obtained from the respective corpora. If you want word vectors trained on massive web datasets, you need only download one of these text files! Pre-trained word vectors are made available under the Public Domain Dedication and License . If the web datasets above don't match the semantics of your end use case, you can train word vectors on your own corpus. The demo.sh script downloads a small corpus, consisting of the first 100M characters of Wikipedia. It collects unigram counts, constructs and shuffles cooccurrence data, and trains a simple version of the GloVe model. It also runs a word analogy evaluation script in python to verify word vector quality. More details about training on your own corpus can be found by reading demo.sh or the src/README.md 2024 Vector Documentation The training scripts and data preprocessing pipeline used for training the 2024 vectors can be found in the Training README.md License All work contained in this package is licensed under the Apache License, Version 2.0. See the include LICENSE file.
Excerpt from the source-code README · 5,668 chars · not written by Vinony
Wikidata facts
- Official website
- nlp.stanford.edu/projects/glove
Show 2 more facts
- source code repository URL
- github.com/stanfordnlp/GloVe
- software version identifier
- 1.0
Sources (3)
via Wikidata · CC0
~7 min read
Article
9 sectionsContents
- Definition
- Word counting
- Probabilistic modelling
- Logistic regression
- Use
- Applications
- See also
- References
- External links
GloVe, coined from Global Vectors, is a model for distributed word representation. The model is an unsupervised learning algorithm for obtaining vector representations of words. This is achieved by mapping words into a meaningful space where the distance between words is related to semantic similarity. Training is performed on aggregated global word-word co-occurrence statistics from a corpus, and the resulting representations showcase interesting linear substructures of the word vector space. As log-bilinear regression model for unsupervised learning of word representations, it combines the features of two model families, namely the global matrix factorization and local context window methods.
It was developed as an open-source project at Stanford University and launched in 2014. It was designed as a competitor to word2vec, and the original paper noted multiple improvements of GloVe over word2vec. , both approaches are outdated, and Transformer-based models, such as BERT, which add multiple neural-network attention layers on top of a word embedding model similar to Word2vec, have come to be regarded as the state of the art in natural language processing (NLP).