[Corpora-List] 1st CfP: NLP-TEA 2016 Shared Task for Chinese Grammatical Error Diagnosis

Lung-Hao Lee lunghaolee at gmail.com
Mon Jun 27 09:07:32 CEST 2016


(With apologies for cross-posting)

--------------------------------------------------------------------------------------------------------------- The 3rd Workshop on Natural Language Processing Techniques for Educational Applications (*NLP-TEA-3*) with a Shared Task for Chinese Grammatical Error Diagnosis (*CGED*) December 12, 2016 at Osaka, Japan (in conjunction with *COLING 2016*) *http://nlptea2016.weebly.com/ <http://nlptea2016.weebly.com/>* ---------------------------------------------------------------------------------------------------------------

*Call for Participation *

*NLP-TEA 2016 Shared Task: **Chinese Grammatical Error Diagnosis* *http://nlptea2016.weebly.com/shared-task.html <http://nlptea2016.weebly.com/shared-task.html>*

*Task Description*

The goal of this shared task is to develop computer-assisted systems to automatically diagnose Chinese sentences, in which may contains grammatical errors written by learners of Chinese as a foreign language. The input sentence may contain at least one of defined error types, i.e., redundant word (denoting as a capital letter ‘R’), missing word (‘M’), word selection error (‘S’), and word ordering error (‘W’). The developed system should indicate which kind of error type is embedded in the given sentence and its occurred positions. If an input sentences contain no grammatical errors, the system should return: sid, correct. The output format should be a quadruple: sid, start_off, end_off, error_type, if the input sentence consists of a grammatical error. In this format, sid means the unique sentence identifier, start_off and end_off represent the positions of starting and ending character where a grammatical error occurs, in which each character or punctuation occupies 1 for counting positions, and error_type should be one of the defined errors: R, M, S, W. Examples are shown as follows.

- *Example 1*:

Input: (sid=A2-0007-2) 聽說妳打算開一個慶祝會。可惜我不能參加。因為那個時候我有別的事。當然我也要參加給你慶祝慶祝。

Output: A2-0007-2, 38, 39, R

(Note: “參加” is a redundant word)

- *Example 2*:

Input: (sid=A2-0007-3) 我要送給你一個慶祝禮物。要是兩、三天晚了,請別生氣。

Output: A2-0007-3, 15, 20, W

(Note: "兩、三天晚了" should be "晚了兩、三天")

- *Example 3*:

Input: (sid=00038800464) 我真不明白。她们可能是追求一些前代的浪漫。

Output: 00038800464, correct

- *Example 4*:

Input: (sid=00038801261) 人战胜了饥饿,才努力为了下一代作更好的、更健康的东西。

Output: 00038801261, 9, 9, M

00038801261, 16, 16, S

(Notes: "能" is missing. The word "作" should be "做". The correct sentence

is "才能努力为了下一代做更好的")

*Data Sets*

We will provide mutually exclusive data sets selecting from the TOCFL Learner Corpus (Traditional Chinese) and the HSK Learner Corpus (Simplified Chinese).

- *Training Set*: The sentences contain grammatical errors accompanying

with their corrections will be provided for training purpose.

- *Test Set*: We will provide at least 3000 testing instances selecting

to cover different error types for official performance evaluation.

*Policy*: Shared task participating teams are allowed to use other publicly available data. Use of other data should be specified in the final system report. Here are the links to download the data sets of the two previous editions for this shared task.

NLP-TEA 2015 CGED Shared Task: http://ir.itc.ntnu.edu.tw/lre/nlptea15cged.htm NLP-TEA 2014 CFL Shared Task: http://ir.itc.ntnu.edu.tw/lre/nlptea14cfl.htm

*Evaluation Metrics*

The criteria for judging correctness are:

- *Detection level*: binary classification of a given sentence, i.e.,

correct or incorrect should be completely identical with the gold standard.

All error types will be regarded as incorrect.

- *Identification level*: this level could be considered as a

multi-class categorization problem. In addition to correct instances, all

error types should be clearly identified.

- *Position level*: besides identifying the error types, this level also

judges the positions of erroneous range. That is, the system results should

be perfectly identical with the quadruples of gold standard.

The evaluation metrics include:

- False Positive Rate

- Accuracy

- Precision

- Recall

- F-Score

*Registration*

Participants need to register in order to obtain the training and test data. To register, please send the following information to Lung-Hao Lee ( lhlee at ntnu.edu.tw <http://urlblockederror.aspx/>).

- Team Name (identified abbreviation of your organization)

- Organization (affiliation)

- Contact person (name and Email)

*Important Dates*

- Registration open: June 25, 2016

- Release of training data: June 25, 2016

- Registration close: September 20, 2016

- Release of test data: October 3, 2016

- Testing results submission due: October 5, 2016

- Release of evaluation results: October 7, 2016

- Technical report submission due: October 20 , 2016

- Report reviews returned: October 25, 2016

- Camera-ready due: October 30, 2016

- Workshop date: December 12, 2016

*Shared Task Organizers*

- Lung-Hao Lee (National Taiwan Normal University)

- Gaoqi Rao (Beijing Language and Culture University)

- Liang-Chih Yu (Yuan Ze University)

- Endong Xun (Beijing Language and Culture University)

- Baolin Zhang (Beijing Language and Culture University)

- Li-Ping Chang (National Taiwan Normal University)

-- Lung-Hao Lee (李龍豪), Ph.D. Postdoctoral Fellow Information Technology Center National Taiwan Normal University Email: lhlee at ntnu.edu.tw URL: http://web.ntnu.edu.tw/~lhlee/ -- -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/html Size: 12619 bytes Desc: not available URL: <https://mailman.uib.no/public/corpora/attachments/20160627/77578cce/attachment.txt>



More information about the Corpora mailing list