[Corpora-List] Generation Challenges 2009: Final Call for Future Task Proposals

Anja Belz a.s.belz at itri.brighton.ac.uk
Mon Nov 24 16:37:19 CET 2008

Final Call for Future Task Proposals (Special Track)

*** Submission deadline for future task proposals: *** *** 30 November 2008 ***

GENERATION CHALLENGES 2009 --------------------------

To be held in conjunction with ENLG 2009.

Following the NSF/SIGGEN Workshop on Shared Tasks and Comparative Evaluation in NLG in April 2007 (http://www.ling.ohio-state.edu/~mwhite/nlgeval07/), and the INLG'06 Special Session on Sharing Data and Comparative Evaluation (http://www.nltg.brighton.ac.uk/home/Anja.Belz/inlg06-specsess.html), there has recently been a lot of activity in connection with shared tasks in Natural Language Generation (NLG). The Pilot Attribute Selection for Generating Referring Expressions (ASGRE) Challenge took place between May and September 2007; the Referring Expression Generation (REG) Challenge 2008 has just been completed; the GIVE Challenge is currently underway; and other challenges are being prepared. More information about all of these activities can be found via the links on the Generation Challenges homepage: http://www.nltg.brighton.ac.uk/research/genchal09

To provide a common forum for these activities, we are organising Generation Challenges 2009, an umbrella event designed to bring together a variety of shared-task evaluation efforts that involve the generation of natural language. Evaluation results and participating systems will be presented at the Generation Challenges 2009 Special Session at the 12th European Workshop on Natural Language Generation (ENLG'09).

1. Shared Tasks: - - - - - - - - -

There will be three shared tasks at Generation Challenges 2009 which will be organised independently (for details, please refer to the Call for Participation for each task):

1. The GIVE Challenge (Koller, Byron, Moore, Oberlander & Striegnitz): Generation of natural-language instructions to aid human task-solving in a virtual environment. This task has started, for more information go to:


2. GREC Tasks (Belz, Gatt, Kow & Viethen): Generation of references to named entities in discourse; (i) GREC-MSR Task: selection of referring expressions for references to the main subject of the text; (ii) GREC-NEG Task: generation of referring expressions for all mentions of people in a text. For more information go to:


3. TUNA Progress Test (Gatt, Belz & Kow): An opportunity to improve on REG'08 Task 3, i.e. mapping from TUNA domain representations to referring expressions. For more information go to:


2. Special Track - Proposals for Future Tasks: - - - - - - - - - - - - - - - - - - - - - - - -

We invite submissions of papers describing ideas for future shared tasks in the general area of language generation. Proposed tasks can be in the area of core NLG, or straddle more than one research area in which language is generated, e.g. core NLG and MT, or core NLG and text summarisation. Submissions should describe possible future tasks in detail, including information regarding organisers, task description, motivating theoretical interest and/or application context, size and state of completion of data to be used, and evaluation plans.

3. Open Track: - - - - - - - -

We invite submissions of papers reporting any language generation method and results obtained using the training and development data from any of the shared task tracks. The task definition should differ from that in the corresponding shared task track, and submissions will be reviewed according to standard workshop reviewing criteria.

4. Evaluation Methods Track: - - - - - - - - - - - - - - -

We invite submissions of papers describing any evaluation method that can be used to evaluate systems submitted to any of the shared tasks. Papers should report results for applying the evaluation method to one or more systems implementing a solution to one of the shared tasks (although different data may be used). Submissions will be reviewed according to standard workshop reviewing criteria.

5. Instructions for Paper Submissions: - - - - - - - - - - - - - - - - - - - -

Submissions in the Special Track, Open Track and Evaluation Methods Track should be no more than 4 (four) pages long, and should follow the ACL'09 guidelines using the style files provided via the ACL-IJCNLP 2009 homepage. Papers should be sent in PDF format by email to nlg-stec at itri.brighton.ac.uk. Special Track papers should be submitted no later than 30 November 2008. Papers in the Open Track and the Evaluation Methods Track should be submitted no later than 12 January 2009.

Submissions will be reviewed by at least 3 members of the review committee (see below). As reviewing will not be blind, there is no need to anonymise papers.

Accepted submissions will be included in the ENLG'09 proceedings, but the page limit for camera-ready versions has not yet been finalised (the final limit will depend on how many submissions there are).

6. Dates: - - - - -

Submission of task proposal papers 30 November 2008 Notification of acceptance of task proposals 31 December 2008 Submission of papers in Open and

Evaluation Methods tracks 12 January 2009 Deadline for camera-ready papers 07 February 2009 ENLG'09 30-31 March 2009

7. Review Committee: - - - - - - - - - - -

Robert Dale, Macquarie University, Australia Kevin Knight, ISI, University of Southern California, USA Mirella Lapata, ICCS, University of Edinburgh, UK Chris Mellish, University of Aberdeen, UK Amanda Stent, Stony Brook University, USA

8. Organisation: - - - - - - - - -

Anja Belz, NLTG, University of Brighton, UK Albert Gatt, Computing Science, University of Aberdeen, UK Eric Kow, NLTG, University of Brighton, UK

Generation Challenges homepage: http://www.nltg.brighton.ac.uk/research/genchal09 Generation Challenges email: nlg-stec at itri.brighton.ac.uk _______________________________________________ SIGGEN mailing list SIGGEN at ics.mq.edu.au http://www.ics.mq.edu.au/mailman/listinfo/siggen

More information about the Corpora mailing list