Semi-Supervised Multi-Task Word Embeddings

Summary

This approach involves reconstructing word meta-embeddings while simultaneously using a Siamese Network to also learn word similarity where both processes share a hidden layer, and finds that performance is improved for all word similarity datasets when compared to unsupervised learning methods.

Keywords

Word (group theory), Task (project management), Natural language processing, Computer science, Artificial intelligence

References

The analogical mind : perspectives from cognitive scienceD. Gentner, K. Holyoak, B. Kokinov — 2001
Word Embeddings through Hellinger PCARémi Lebret, Ronan Collobert — 2013
Efficient Estimation of Word Representations in Vector SpaceTomáš Mikolov, Kai Chen, Greg S. Corrado — 2013
WordRep: A Benchmark for Research on Learning Word RepresentationsBin Gao, Jiang Bian, Tie‐Yan Liu — 2014
SimLex-999: Evaluating Semantic Models With (Genuine) Similarity EstimationFelix Hill, Roi Reichart, Anna Korhonen — 2014
Short Text Similarity with Word EmbeddingsTom Kenter, Maarten de Rijke — 2015
Placing search in context: the concept revisitedLev Finkelstein, Evgeniy Gabrilovich, Yossi Matias — 2002
Features of SimilarityAmos Tversky — 1977
Computational models of analogy.Dedre Gentner, Kenneth D. Forbus — 2011
VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITYGlenn W. Brier — 1950
Contextual correlates of synonymyHerbert Rubenstein, John B. Goodenough — 1965
SemEval-2012 Task 2: Measuring Degrees of Relational SimilarityDavid Jurgens, Saif M. Mohammad, Peter D. Turney — 2012
Learning Word Vectors for Sentiment AnalysisAndrew L. Maas, Raymond E. Daly, Peter T. Pham — 2011
Addressing the Rare Word Problem in Neural Machine TranslationThang Luong, Ilya Sutskever, Quoc V. Le — 2014
AutoExtend: Extending Word Embeddings to Embeddings for Synsets and LexemesSascha Rothe, Hinrich Schütze — 2015
A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled DataRie Kubota Ando, Tong Zhang — 2005
Distributional Semantics in TechnicolorElia Bruni, Gemma Boleda, Marco Baroni — 2012
Linguistic Regularities in Continuous Space Word RepresentationsTomáš Mikolov, Wen-tau Yih, Geoffrey Zweig — 2013
Large-scale learning of word relatedness with constraintsGuy Halawi, Gideon Dror, Evgeniy Gabrilovich — 2012
Distributed Representations of Words and Phrases and their CompositionalityTomáš Mikolov, Ilya Sutskever, Kai Chen — 2013

Cited by

Computational Linguistics: 16th International Conference of the Pacific Association for Computational Linguistics, PACLING 2019, Hanoi, Vietnam, October 11–13, 2019, Revised Selected PapersLe-Minh Nguyen, Xuan-Hieu Phan, Kôiti Hasida — 2019
A semantic relation preserved word embedding reuse methodDe‐Chuan Zhan, Xinchun LI — 2020
Learning to Evaluate Neural Language ModelsDanushka Bollegala, James O'Neill — 2019

Related papers

Meta-Embedding as Auxiliary Task RegularizationJames O’ Neill, Danushka Bollegala — 2018
Evaluating Word Similarity Measure of Embeddings Through Binary ClassificationA. Aziz Altowayan, Lixin Tao — 2019
A Comparative Study on Word Embeddings in Deep Learning for Text ClassificationCongcong Wang, Paul Nulty, David Lillis — 2020
Learning Word Embeddings for Data Sparse and Sentiment Rich Data SetsPrathusha K Sarma — 2018
Word Embeddings for Multi-label Document ClassificationLadislav Lenc, Pavel Král — 2017
Sentence Modeling via Multiple Word Embeddings and Multi-level Comparison for Semantic Textual SimilarityTien, Huy Nguyen, Minh Le, Tomohiro Yamasaki — 2018
TF-CR: Weighting Embeddings for Text ClassificationArkaitz Zubiaga — 2020