[Corpora-List] CfP to the Automatic Post-Editing shared task at WMT 2020

Rajen Chatterjee rajen.k.chatterjee at gmail.com
Thu Apr 2 19:36:26 CEST 2020


CALL FOR PARTICIPATION

in the

Sixth Automatic Post-Editing (APE) shared task

at the Fifth Conference on Machine Translation (WMT20) OVERVIEW

The sixth round of the APE shared task follows the success of the previous rounds organised from 2015 to 2019. The aim is to examine *automatic methods for correcting errors produced by an unknown machine translation (MT) system.* This has to be done by exploiting knowledge acquired from human post-edits, which are provided as training material. Goals

The aim of this task is to improve MT output in black-box scenarios, in which the MT system is used "as is" and cannot be modified. From the application point of view, APE components would make it possible to:

- Cope with systematic errors of an MT system whose decoding process is

not accessible

- Provide professional translators with improved MT output quality to

reduce (human) post-editing effort

- Adapt the output of a general-purpose system to the lexicon/style

requested in a specific application domain

Task Description

This year the task will use Wikipedia data for English --> German and English --> Chinese lanaguge pairs. In these datasets, the source sentences have been translated into the target language by using a state-of-the-art neural MT system unknown to the participants in terms of system configuration and then manually post-edited. This dataset is shared by both Automatic Post-Editing and Quality Estimation shared tasks.

At training stage, the collected human post-edits have to be used to learn correction rules for the APE systems. At test stage they will be used for system evaluation with automatic metrics (TER and BLEU). Differences from the 5th round (WMT 2019)

Compared to the the previous round, the main differences are:

- Addition of a new language pair (English --> Chinese)

- New source of data (Wikipedia)

- Maximum 2 submissions allowed (instead of 3)

Evaluation

Systems' performance will be evaluated with respect to their capability to reduce the distance that separates an automatic translation from its human-revised version. Such distance will be measured in terms of TER, which will be computed between automatic and human post-edits in case-sensitive mode. Also BLEU will be taken into consideration as a secondary evaluation metric.

Important dates Release of training and development data March 28, 2020 Release of test data June 8, 2020 APE system submission deadline June 15, 2020 Manual evaluation June, 2020 Paper submission deadline July 15, 2020 Notification of acceptance August 17, 2020 Camera-ready deadline August 31, 2020 Conference November 11-12, 2020--On behalf of the APE task organizers

-- -Regards,

Rajen Chatterjee. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/html Size: 6370 bytes Desc: not available URL: <https://mailman.uib.no/public/corpora/attachments/20200402/e8842bc9/attachment.txt>



More information about the Corpora mailing list