Hi everyone,
Last week I asked for papers about evaluating dependency annotation, and specifically reporting scores on UD inter-annotator agreement. I received many interesting references, some of which are direct evaluations of UD annotation and others which discuss specific issues and metrics for evaluating dependency annotations – here is the complete list:
– Bamman, David, Francesco Mambrini & Gregory Crane (2009), An ownership model of annotation: The Ancient Greek dependency treebank. In: Proceedings of the Eighth International Workshop on Treebanks and Linguistic Theories (TLT 8). Groningen, 5–15. Available at: http://www.perseus.tufts.edu/~ababeu/tlt8.pdf.
– Berzak, Yevgeni, Yan Huang, Andrei Barbu, Anna Korhonen & Boris Katz (2016), Anchoring and Agreement in Syntactic Annotations. In: Proceedings of EMNLP 2016. Austin, TX, 2215–2224.
– Berzak, Yevgeni, Jessica Kenney, Carolyn Spadine, Jing Xian Wang, Lucia Lam, Keiko Sophie Mori, Sebastian Garza & Boris Katz (2016), Universal Dependencies for Learner English. In: Proceedings of ACL 2016. Berlin, Germany, 737–746. Available at: http://www.aclweb.org/anthology/P16-1070.
– Liu, Yijia, Yi Zhu, Wanxiang Che, Bing Qin, Nathan Schneider & Noah A. Smith (2018), Parsing Tweets into Universal Dependencies. In: Proceedings of NAACL 2018. New Orleans, LA, 965–975. Available at: http://aclweb.org/anthology/N18-1088.
– Nguyen, Kiem-Hieu (2018), BKTreebank: Building a Vietnamese Dependency Treebank. In: Proceedings of LREC 2018. Miyazaki, Japan, 2164–2168. Available at: http://www.lrec-conf.org/proceedings/lrec2018/pdf/69.pdf.
– Seddah, Djamé, Eric De La Clergerie, Benoît Sagot, Héctor Martínez Alonso & Marie Candito (2018), Cheating a Parser to Death: Data-driven Cross-Treebank Annotation Transfer. In: Proceedings of LREC 2018. Miyazaki, Japan, 4535–4539. Available at: http://www.lrec-conf.org/proceedings/lrec2018/pdf/1101.pdf.
– Seyoum, Binyam Ephrem, Yusuke Miyao & Baye Yimam Mekonnen (2018), Universal Dependencies for Amharic. In: Proceedings of LREC 2018. Miyazaki, Japan, 2216–2222. Available at: http://www.lrec-conf.org/proceedings/lrec2018/pdf/565.pdf.
– Skjærholt, Arne (2014), A Chance-corrected Measure of Inter-annotator Agreement for Syntax. In: Proceedings of ACL 2014. Baltimore, MD, 934–944. Available at: http://www.aclweb.org/anthology/P14-1088.
Thanks to everyone who contributed!
Amir
------------
Dr. Amir Zeldes
Asst. Prof. of Computational Linguistics
Department of Linguistics
Georgetown University
1437 37th St. NW
Washington, DC 20057
<http://corpling.uis.georgetown.edu/amir> http://corpling.uis.georgetown.edu/amir
-------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/html Size: 10925 bytes Desc: not available URL: <https://mailman.uib.no/public/corpora/attachments/20180719/129c2a6b/attachment.txt>