One of the most difficult problems when assigning either positive or negative polarity in sentiment analysis tasks is to accurately determine what is the truth value of a certain statement. In the case of literal language, when there is not a secondary meaning, existing techniques already achieve good results. However, in case of figurative language such as irony and affective metaphor, when secondary or extended meanings are intentionally profiled, the affective polarity of the literal meaning may contrast sharply with the affect created by the figurative meaning. Nowhere is this effect more pronounced than in ironic or sarcastic language, which delights in using affirmative language to convey negative meanings. Metaphor, irony and figurative language more generally demonstrate the limits of conventional techniques for the sentiment analysis of literal texts.
So figurative language creates a significant challenge for a sentiment analysis system, as direct approaches based on words and their lexical semantics are often shown to be inadequate in the face of indirect meanings. It would be convenient then if such language were rare and confined to specific genres of text, such as poetry and literature. Yet the reality is that figurative language is pervasive in almost any genre of text, and is especially commonplace on the texts of the Web and in social media communications. Figurative language often draws attention to itself as a creative artifact, but is just as likely to be viewed as part of the general fabric of human communication. In any case, Web users employ figures of speech (both old and new) to project their personality through a text, especially when limited to the 140 characters of a tweet.
This significant challenge is the basis for SemEval Task 11, for which trial and training data are now available!
The task: given a set of tweets that are rich in irony and metaphor, the goal is to determine whether the user has expressed a positive, negative or neutral sentiment. A fine-grained sentiment scale (scores in the range -5...+5) is be used to capture the effect of irony and metaphor on the perceived sentiment of a tweet. Participating systems will have to assign sentiment scores from the same fine-grained scale to a set of given tweets.
Trial (1000 figurative tweets) and training data (8000 figurative tweets): annotated and available at http://alt.qcri.org/semeval2015/task11
Important dates: December 5, 2014 Evaluation starts December 20, 2014 Evaluation ends January 30, 2015 Paper submission due February 28, 2015 Paper reviews due March 30, 2015 Camera-ready papers due Summer 2015 SemEval workshop
Contact: tony.veale at ucd.ie
Tony Veale, University College Dublin
John Barnden, University of Birmingham
Antonio Reyes, ISIT
Paolo Rosso, Univiversitat Politècnica de València
Ekaterina Shutova, UC Berkeley