[Corpora-List] CFP: Dialogue Evaluation 2020: Taxonomy Enrichment for the Russian Language

Lidia Pivovarova lidia.pivovarova at gmail.com
Tue Dec 17 12:24:32 CET 2019

We invite you to participate in the Dialogue 2020 shared task on Taxonomy Enrichment for the Russian Language: https://competitions.codalab.org/competitions/22168.

Taxonomies are tree structures which organize terms into a semantic hierarchy. Taxonomic relations (or hypernyms) are “is-a” relations: cat is-a animal, banana is-a fruit, Microsoft is-a company, etc. This type of relations is useful in a wide range of natural language processing tasks for performing semantic analysis. The goal of this semantic task is to extend an existing taxonomy with relations of previously unseen words.

Multiple evaluation campaigns for hypernym extraction (SemEval-2018 task 9 <https://repositori.upf.edu/handle/10230/35249>), taxonomy induction (Semeval-2016 task 13 <https://www.aclweb.org/anthology/S16-1168>, SemEval 2015 task 17 <https://www.aclweb.org/anthology/S15-2151/>), and most notably for taxonomy enrichment (SemEval-2016 task 14 <https://www.aclweb.org/anthology/S16-1169/>) were organized for English and other western European languages in the past. However, this is the first evaluation campaign of this kind for Russian and any Slavic language. Moreover, the task has a more realistic setting as compared to the SemEval-2016 task 14 taxonomy enrichment task as the participants are not given the definitions of words but only new unseen words in context.

More concretely, the goal of this task is the following: Given words that are not yet included in the taxonomy, we need to associate each word with the appropriate hypernyms from an existing taxonomy. For example, given the input word “утка” (duck) we expect you to provide a list of its most probable 10 candidate hypernym synsets the word could be attached to, e.g. “animal”, “bird”, and so on. Here a word may refer to one, two or more “ancestors” (hypernym synsets) at the same time.

We believe that popular neural context-aware models (like ELMo and BERT) will be of particular use for this task as they can represent out-of-vocabulary words on the basis of their context. Therefore, everyone interested in testing these and other distributional semantic models are welcome to participate. We will provide several baselines based on distributional and neural language models.

Important dates:


First Call for Participation: December 15, 2019.


Release of the Training Data: December 15, 2019.


Release of the Test Data: January 31, 2020.


Submission of the Results: February 14, 2020.


Results of the Shared Task: February 28, 2020.

We will be grateful if you can spread the word about this shared task.


Irina.Nikishina at skoltech.ru

v.logacheva at skoltech.ru -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/html Size: 10800 bytes Desc: not available URL: <https://mailman.uib.no/public/corpora/attachments/20191217/1b7f1536/attachment.txt>

More information about the Corpora mailing list