Task-Oriented Intrinsic Evaluation of Semantic Textual Similarity
Explore this paper's citation graph
Summary
This work defines how the validity of an intrinsic evaluation can be assessed and compares different intrinsic evaluation methods and proposes a framework for conducting the intrinsic evaluation which takes the properties of the targeted task into account.
- Type
- article
- Published
- 2016-12-01
- Cited by
- 76
- References
- 18
- OpenAlex
- https://openalex.org/W2572185161
- Semantic Scholar
- https://api.semanticscholar.org/CorpusID:18283203
Keywords
Computer science, Task (project management), Pearson product-moment correlation coefficient, Semantic similarity, Similarity (geometry)
References
- Using Lexical Chains for Text Summarization
- Study of semantic relatedness of words using collaboratively constructed semantic resources
- Developing a corpus of plagiarised short answers
- IR evaluation methods for retrieving highly relevant documents
- Binary and graded relevance in IR evaluations--Comparison of the effects on ranking of IR systems
- Discovery of inference rules for question-answering
- SemEval-2014 Task 10: Multilingual Semantic Textual Similarity
- Very Simple Classification Rules Perform Well on Most Commonly Used Datasets
- SemEval-2015 Task 2: Semantic Textual Similarity, English, Spanish and Pilot on Interpretability
- *SEM 2013 shared task: Semantic Textual Similarity
- AUTOMATED ESSAY SCORING WITH E‐RATER® V.2.0
- SemEval-2012 Task 6: A Pilot on Semantic Textual Similarity
- SemEval-2016 Task 1: Semantic Textual Similarity, Monolingual and Cross-Lingual Evaluation
- Overview of the 1st international competition on plagiarism detection
- Proposal for a STS Evaluation Framework for STS based Applications
- Resources
- IR evaluation methods for retrieving highly relevant documents
- Overview of the 2nd International Competition on Plagiarism Detection
- Link Detection with Wikipedia
Cited by
- SemEval-2017 Task 1: Semantic Textual Similarity Multilingual and Crosslingual Focused Evaluation
- Measuring Semantic Relations between Human Activities
- Universal Machine Learning Methods for Detecting and Temporal Anchoring of Events
- Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
- Scaled Pearson’s Correlation Coefficient for Evaluating Text Similarity Measures
- Multilingual Transformer Ensembles for Portuguese Natural Language Tasks
- Making Monolingual Sentence Embeddings Multilingual Using Knowledge Distillation
- On the nature of information access evaluation metrics: a unifying framework
- Source Attribution: Recovering the Press Releases Behind Health Science News
- An Unsupervised Sentence Embedding Method by Mutual Information Maximization
- Latte-Mix: Measuring Sentence Semantic Similarity with Latent Categorical Mixtures
- Natural Language Understanding for Argumentative Dialogue Systems in the Opinion Building Domain
- SimCSE: Simple Contrastive Learning of Sentence Embeddings
- AStitchInLanguageModels: Dataset and Methods for the Exploration of Idiomaticity in Pre-Trained Language Models
- Virtual Augmentation Supported Contrastive Learning of Sentence Representations
- PAUSE: Positive and Annealed Unlabeled Sentence Embedding
- Measuring Similarity of Opinion-bearing Sentences
- SemEval-2022 Task 2: Multilingual Idiomaticity Detection and Sentence Embedding
- A joint FrameNet and element focusing Sentence-BERT method of sentence similarity computation
- TSDAE: Using Transformer-based Sequential Denoising Auto-Encoder for Unsupervised Sentence Embedding Learning
Related papers
- A large annotated corpus for learning natural language inference
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
- SemEval-2016 Task 1: Semantic Textual Similarity, Monolingual and Cross-Lingual Evaluation
- SemEval-2012 Task 6: A Pilot on Semantic Textual Similarity
- GloVe: Global Vectors for Word Representation
- SemEval-2014 Task 10: Multilingual Semantic Textual Similarity