[Corpora-List] Call for Participation: MWE 2008 Shared Task on Multiword Ranking

Stefan Evert stefan.evert at uos.de
Thu Feb 21 14:51:54 CET 2008


################################################################

CALL FOR SHARED TASK PARTICIPATION

LREC2008 - Towards a Shared Task for Multiword Expressions (MWE 2008)

endorsed by the ACL Special Interest Group on the Lexicon (SIGLEX)

Date: Sunday, 1 June 2008 Location: Marrakech, Morocco Deadline: Friday, 14 March 2008 (Shared Task)

Workshop web page: http://multiword.sf.net/mwe2008/

#################################################################

The MWE 2008 workshop is primarily intended as a forum for collecting, sharing and exploiting MWE evaluation resources. Following an earlier call, numerous contributors have submitted data sets of various kinds, which can now be downloaded from the Multiword homepage: http://multiword.sourceforge.net/

With this call, we invite teams to participate in a SHARED TASK on MULTIWORD EXTRACTION, based on a subset of the new resources. For this first and exploratory evaluation campaign, we decided to focus on the simplest and most straightforward task -- ranking MWE candidates -- in order to make it easier for participating teams to adapt their existing systems before the deadline on March 14th.

Participating teams have to apply a MWE candidate ranking algorithm to at least two different data sets, provided in the form of a list of MWE candidates that have been manually annotated as true positives (TP) and false positives (FP). Evaluation is carried out by the teams themselves in terms of n-best precision and precision-recall graphs (sample R scripts for the evaluation will be provided on the shared task homepage soon). Further analysis and experiments are strongly encouraged (e.g. on different subsets of the candidates, or for different subtypes of MWE). Baseline results for standard association measures will be presented by the workshop organisers.

More information and downloads can be found on the shared task homepage:

http://multiword.sourceforge.net/mwe2008/shared_task.html

Teams planning to participate in the shared task should also join the multiword mailing list for up-to-date information and discussions:

https://lists.sourceforge.net/lists/listinfo/multiword-expressions

We expect a full-fledged competition with a more varied range of tasks to be organised in the near future, making use of the full collection of data sets from this workshop.

SUBMISSION INFORMATION

Shared task participants have to submit a short paper with complete evaluation results for their ranking algorithm on at least three different data sets. Submissions are not anonymous and will be briefly reviewed by the workshop organisers for complicance with the shared task requirements.

Papers must adhere to the format of LREC proceedings (preferably using the style files provided on the conference Web site) and must not exceed four (4) pages, including references. Only submissions in PDF format will be considered.

The papers must be submitted no later than 23:59 GMT on March 14, 2008.

Please submit your paper here: https://www.softconf.com/LREC2008/ MWE2008/submit.html

In addition, participants should make available their full candidate rankings and, if possible, an implementation of their ranking algorithm, to encourage further experiments. Details will be discussed on the multiword mailing list.

IMPORTANT DATES

8-page regular paper submission deadline: February 29, 2008 4-page resource paper submission deadline: February 29, 2008 4-page shared task paper submission deadline: March 14, 2008 Notification of acceptance: March 28, 2008 Camera ready papers due: April 4, 2008 Workshop date: June 1, 2008

WORKSHOP CHAIRS

Nicole Grégoire University of Utrecht, The Netherlands

Stefan Evert University of Osnabrueck, Germany

Brigitte Krenn Austrian Research Institute for Artificial Intelligence (ÖFAI), Austria

CONTACT

For any inquiries regarding the workshop please contact Nicole Grégoire (Nicole.Gregoire at let.uu.nl).



More information about the Corpora mailing list