[Corpora-List] EVALITA 2020: Final Call for tasks

Valerio Basile basile at di.unito.it
Tue Feb 4 11:13:17 CET 2020


The 7th evaluation campaign EVALITA 2020 will be held in Bologna, as a satellite event of the 7th Italian Conference on Computational Linguistics - CLiC-it 2020 (Nov 30th – Dec 3rd 2020). http://www.evalita.it/2020

EVALITA 2020 is an initiative of AILC (Associazione Italiana di Linguistica Computazionale, http://www.ai-lc.it/).

As in the previous editions (http://www.evalita.it/), EVALITA 2020 will be organized along few selected tasks, which provide participants with opportunities to discuss and explore both emerging and traditional areas of Natural Language Processing and Speech for Italian. The participation is encouraged for teams working both in academic institutions and industrial organizations.

TASK PROPOSAL SUBMISSION

Tasks proposals should be no longer than 4 pages and should include:

task title and acronym; names and affiliation of the organizers (minimum 2 organizers); brief task description, including motivations and state of the art showing the international relevance of the task; description and examples of the data, including information about their availability, development stage, and issues concerning privacy and data sensitivity; expected number of participants and attendees; names and contact information of the organizers.

In submitting your proposal, please bear in mind that we encourage:

challenging tasks involving linguistic analysis, e.g., beyond “simple” classification problems; tasks focused on multimodality, e.g., considering both textual and visual information; tasks characterized by different levels of complexity, e.g., with a straightforward main subtask and one or more sophisticated additional subtasks; the re-annotation of datasets from previous years with new annotation types, and texts from publicly available corpora; both new tasks and re-runs. For new tasks, you will have to specify in the proposal why it would attract a reasonable number of participants, and why it is needed; application-oriented tasks, that is tasks that have a clearly defined end-user application showcasing.

The organizers of the accepted tasks should take care of planning, according to the scheduled deadlines (see below):

- the development and distribution of data sets needed for the contest, i.e. data for training and development, and data for testing; the scorer to be used to evaluate the submitted systems should be included in the release of development data - the development of task guidelines, where all the instructions for the participation are made clear together with a detailed description of data and evaluation metrics applied for the evaluation of the participant results; the collection of participants results - the evaluation of participants results according to standard metrics and baseline(s) - the solicitation of submissions - the reviewing process of the papers describing the participants approach and results (according to the template to be made available by the EVALITA-2020 chairs) - the production of a paper describing the task (according to the template to be made available by the EVALITA-2020 chairs)

*** Email your proposal in PDF format to evalita2020 at gmail.com with "Evalita 2020 TASK Proposal" as the subject line by the submission deadline (February 7th 2020). ***

Please feel free to contact the EVALITA-2020 chairs at evalita2020 at gmail.com in case of any questions or suggestions.

Deadlines of the task proposal:

February 7th 2020: submission of task proposals March 6th 2020: notification of task proposal acceptance

Tentative timelines of Evalita 2020:

29th May 2020: development data available to participants 4th September 2020: test data available, registration closes 4th - 24th September 2020: evaluation window 2nd October 2018: assessment returned to participants 6th November 2018: technical reports due to organizers (camera ready)

EVALITA 2020 CHAIRS

Valerio Basile (University of Turin) Danilo Croce (University of Rome “Tor Vergata”) Maria Di Maro (University of Naples “Federico II”) Lucia Passaro (University of Pisa) Advisor: Nicole Novielli (University of Bari “A. Moro”) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/html Size: 5159 bytes Desc: not available URL: <https://mailman.uib.no/public/corpora/attachments/20200204/b067eda2/attachment.txt>



More information about the Corpora mailing list