[Corpora-List] FINAL CFP: NAACL-HLT 2009 Workshop on Semantic Evaluations: Recent Achievements and Future Directions

Eneko Agirre e.agirre at ehu.es
Tue Feb 24 18:45:55 CET 2009

Apologies for cross-postings



NAACL-HLT 2009 Workshop on

Semantic Evaluations: Recent Achievements and Future Directions


June 4, 2009

Boulder, Colorado, USA

Paper Submission deadline: March 6, 2009

Endorsed by the following ACL Special Interest Groups:

SIGLEX, Special Interest Group on the Lexicon

SIGSEM, Special Interest Group on Computational Semantics

SIGANN, Special Interest Group for Annotation



Workshop Description

The main purpose of this workshop is to review, analyse and discuss the latest developments in semantic analysis of text. The fact that this workshop happens between the last Semantic Evaluation exercise (SemEval-2007) and the preparation for the next SemEval in 2010, presents an exciting opportunity to discuss practical and foundational aspects of semantic processing of text. The workshop targets papers describing both semantic processing systems and evaluation exercises, with special attention to foundational issues in both lexical and propositional semantics, including semantic representation and semantic corpus construction problems.


There are now many computer systems that do automatic semantic analysis of text. The purpose of SemEval-2007 was to evaluate the strengths and weaknesses of such systems with respect to different words, relations, types of texts, different varieties of language, and different languages. This workshop is a follow-up to the SemEval-2007 and Senseval series of workshops on semantic evaluation and a preparation for the next SemEval workshop to be held in 2010.

Senseval-1 included semantic evaluation tasks in English, French and Italian and culminating workshop held at Herstmonceux Castle, Sussex, England in September, 1998. The Senseval-2 evaluations took place in the summer of 2001, and was followed by a workshop held in July 2001 in Toulouse, in conjunction with ACL-2001. It included tasks for 12 languages. A follow-up workshop on the recent successes and future directions of word sense disambiguation was held at ACL 2002. The Senseval-3 evaluations took place in the spring of 2004, and was followed by a workshop held in July 2004 in Barcelona, in conjunction with ACL-2004. More than 55 teams participated with over 160 systems in its 16 tasks. SemEval-2007 organized 18 tasks, with over 100 teams and 125 systems participating. The results of the tasks and systems were presented in a workshop held in conjunction with ACL-2007.

SemEval-2007 was motivated by a desire to broaden the spectrum of accepted works to all aspects of computational semantic analysis of language. It especially encouraged the proposal of tasks for different languages, cross-lingual tasks, and tasks that are relevant to particular NLP applications such as machine translation, information retrieval and information extraction.

In recent years, the deployment of multiple semantic lexicons and accordingly tagged corpora (WordNet-SemCor, VerbNet-PropBank, FrameNet Prague Dependency Treebank and OntoNotes, to name a few) have radically changed the way semantic analysis is performed, especially at the disambiguation stage. Some of the tasks at SemEval-2007 tried to take this one step further by proposing multiple layers of semantic annotation of the same corpus (e.g., senses, roles, name-entity classes), allowing for novel research to be performed.

SemEval-2007 had a tight schedule, and the results and systems were usually not known until the workshop was held. This fact, and the growing number of tasks and systems made it difficult to fully analyse the tasks and systems. The Language Resources and Evaluation journal devoted an special issue to SemEval-2007, attracting 20 submissions. This workshop will also serve as a complementary follow-up analysis of that evaluation exercise, with special attention to foundational issues in both lexical and propositional semantics, including semantic representations and semantic corpus construction problems.

In the meantime SemEval-2010 is already rolling, with 18 tasks accepted (http://semeval2.fbk.eu), and a call for participation being prepared. We expect many of these tasks to be presented in this workshop.


The workshop invites original submissions of papers on systems for semantic processing and evaluable evaluation exercises on semantic processing in general, including, but not limited to, the following topics:

* foundational issues in both lexical and propositional semantics

* semantic corpus construction problems

* shallow and deep semantic analysis

* word sense disambiguation

* semantic role labelling

* named-entity classification

* analysis and disambiguation of prepositions

* metonymy resolution

* lexical substitution and paraphrasing

* textual entailment

* semantics in applications: IR, IE, MT, Summarisation, etc.

We welcome papers on the above from all theoretical, practical, algorithmic and corpus construction perspectives.


Authors are invited to submit two kinds of papers:

* full: for papers on original, unpublished work in the topic area of

this workshop.

* short: for papers on semantic evaluation tasks (especially those

from SemEval-2010) and papers describing ongoing work, possibly with

preliminary results.

Authors will be able to express their preference for short/long papers but the final decision is on the program chairs. Short papers will be presented in a poster session. Full papers should be up to 8 pages in length, plus one page for references. Short papers should be up to 6 pages in length. Both kinds of papers will appear in the proceedings indistinguishably.

As reviewing will be blind, the paper should not include the authors' names and affiliations. Furthermore, self-references that reveal the author's identity, e.g., "We previously showed (Smith, 1991) ...", should be avoided.

All submissions must be electronic in PDF. Please see the conference website for detailed typesetting specifications. Authors are strongly encouraged to use the LaTeX or Microsoft Word style files available at the conference website (http://www.naaclhlt2009.org/).

Submission will be electronic, using the START paper submission website. The START submission page will be available from http://www.naaclhlt2009.org/.

Important Dates

March 6, 2009 Paper Submission due

March 30, 2009 Notification of acceptance

April 12, 2009 Camera ready papers due

June 4, 2009 SEW-2009 Workshop

Program chairs:

Eneko Agirre (University of the Basque Country, UPV/EHU, Basque Country) Lluís Màrquez (Technical University of Catalonia, UPC, Spain) Richard Wicentowski (Swarthmore College, USA)

Address any queries regarding the workshop to:


Programme Committee

* Timothy Baldwin, Melbourne University, Australia * Nicoletta Calzolari, Istituto di Linguistica Computazionale, Italy * Walter Daelemans, University of Antwerp, Belgium * Katrin Erk, University of Texas, USA * Roxana Girju, University of Illinois at Urbana-Champaign * Veronique Hoste, University of Antwerp, Belgium * Eduard Hovy, Information Science Institute, USA * Nancy Ide, Vassar College, USA * Kenneth Litkowski, CL Research, USA * Bernardo Magnini, FBC, Italy * Katja Markert, Leeds University, UK * David Martínez, University of Melbourne, Australia * Diana McCarthy, University of Sussex, UK * Roberto Navigli, University of Rome "La Sapienza", Italy * Hwee Tou Ng, National University of Singapore, Singapore * Martha Palmer, University of Colorado, USA * Ted Pedersen, University of Minnesota in Duluth, USA * German Rigau, Basque Country University, Basque Country * Mark Stevenson, University of Sheffield, UK * Carlo Strapparava, FBK-irst, Italy * Mihai Surdeanu, Yahoo Research, Spain * Stan Szpakowicz, University of Ottawa, Canada * Dekai Wu, The Hong Kong University of Science & Technology, China * Deniz Yuret, Koc University, Turkey

More information about the Corpora mailing list