[Corpora-List] Call for Participation: EVALUATING PHRASAL SEMANTICS (SemEval 2013, Task 5)

Lushan Han lushan1 at umbc.edu
Mon Feb 11 23:19:07 CET 2013


Regarding to the subtask A, I wonder if WordNet can be used in developing the semantic measure? In the homepage of the task, I saw that dictionaries are prohibited.



On Fri, Jan 11, 2013 at 7:25 PM, Eugenie Giesbrecht <eugeniegiz at gmx.de>wrote:

> -- as part of --
> ** SemEval-2013, TASK 5 **
> http://www.cs.york.ac.uk/semeval-2013/task5/
> Numerous tasks have focused on leveraging the meaning of words (noun
> categorization, TOEFL test), or of words in context (WSD, metonymy
> resolution, lexical substitution). These tasks have enjoyed a lot of
> success. A natural further step is to build up and evaluate models that can
> perform similar tasks in the face of multiword expressions and complex
> compositional structures.
> We suggest two subtasks that are designed to evaluate phrasal models.
> Participating systems may attempt any or all of the subtasks, in any or all
> of the languages provided in the datasets. The sub-tasks are the following:
> a) Semantic similarity of words and compositional phrases
> b) Evaluating the compositionality of phrases in context
> All subtasks are based on items drawn from the large-scale, freely
> available WaCky corpora (Baroni et al., 2009).
> ** Training data as well as further details are available ** @
> http://www.cs.york.ac.uk/semeval-2013/task5/
> (a) Semantic Similarity of Words and Compositional Phrases
> The aim of this subtask is to evaluate how well systems can judge the
> semantic similarity of a word and a short sequence of (two or more) words.
> For example, in the word-sequence pair: <contact, close interaction> the
> meaning of the sequence as a whole is semantically close to the meaning of
> the word. Contrarily, in the sequence-word pair: <megalomania, great
> madness> the meaning of the word is semantically different from the meaning
> of the sequence, although it is not entirely unrelated. This task is
> offered for English, German and Italian.
> == Subtask Organizers ==
> - Yannis Korkontzelos (National Centre for Text Mining, University of
> Manchester, UK)
> - Fabio Massimo Zanzotto (University of Rome "Tor Vergata", Italy)
> (b) Semantic Compositionality in Context
> An interesting sub-problem of semantic compositionality is to decide
> whether a phrase is used in its literal or figurative meaning in a **given
> context**. For example, “big picture” may be used literally as in “Click
> here for a bigger picture”, or figuratively as in “To solve this problem,
> you have to look at the bigger picture”.
> Another example is “old school” which can also be used literally (“During
> the 1970's the hall of the old school was converted into the library”) or
> figuratively (“He will go down in history as one of the old school, a true
> gentlemen”).
> Participants of this subtask are provided with a list of target phrases
> together with real usage examples sampled from WaCky (Baroni et al., 2009)
> corpora. For each usage example, the task is to make a binary decision
> whether the target phrase is used literally or figuratively in this
> context. There may be cases where a binary decision is not possible, then
> the classification as “both” is allowed.
> For English, we define two different subsets that will be scored
> separately and in combination. One subset of target phrases is accompanied
> by a large number of contexts, which allows learning single classifiers for
> each phrase. The validation/test sets will only contain phrases that occur
> in the training set. This is comparable to the lexical sample task, which
> encourages the participation of supervised systems.
> The other subset contains target phrases together with a smaller number of
> usage contexts, probably favoring unsupervised approaches. The
> validation/test sets will contain new target phrases. This is comparable to
> the all-word task. Here, systems must grasp a notion of literal vs.
> idiomatic use in general, without training classifiers for each phrase.
> For German, only the second setup is offered.
> == Subtask Organizers ==
> - Chris Biemann, UKP Lab, Technische Universitšt Darmstadt, Germany
> - Eugenie Giesbrecht, Karlsruhe Institute of Technology, Germany
> - Torsten Zesch, UKP Lab, Technische Universitšt Darmstadt, Germany
> SemEval-2013 schedule
> • February 15, 2013 - Registration Deadline [for Task Participants]
> • March 1, 2013 onwards - Start of evaluation period [Task
> Dependent]
> • March 15, 2013 - End of evaluation period
> • April 9, 2013 - Paper submission deadline [TBC]
> • April 23, 2013 - Reviews Due [TBC]
> • May 4, 2013 - Camera ready Due [TBC]
> _______________________________________________
> UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
> Corpora mailing list
> Corpora at uib.no
> http://mailman.uib.no/listinfo/corpora
-------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/html Size: 5909 bytes Desc: not available URL: <https://mailman.uib.no/public/corpora/attachments/20130211/0768c787/attachment.txt>

More information about the Corpora mailing list