PSDVec: a Toolbox for Incremental and Scalable Word Embedding

• Published in: arXiv.org
• Main Authors: Li, Shaohua, Zhu, Jun, Miao, Chunyan
• Format: Paper Journal Article
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 10.06.2016
• Subjects: Algorithms; Computer Science - Computation and Language; Distance learning; Embedding; Machine learning; Mapping; Natural language; Natural language processing; Perl
• ISSN: 2331-8422
• DOI: 10.48550/arxiv.1606.03192

PSDVec is a Python/Perl toolbox that learns word embeddings, i.e. the mapping of words in a natural language to continuous vectors which encode the semantic/syntactic regularities between the words. PSDVec implements a word embedding learning method based on a weighted low-rank positive semidefinite approximation. To scale up the learning process, we implement a blockwise online learning algorithm to learn the embeddings incrementally. This strategy greatly reduces the learning time of word embeddings on a large vocabulary, and can learn the embeddings of new words without re-learning the whole vocabulary. On 9 word similarity/analogy benchmark sets and 2 Natural Language Processing (NLP) tasks, PSDVec produces embeddings that has the best average performance among popular word embedding tools. PSDVec provides a new option for NLP practitioners.