[Corpora-List] WSD / # WordNet senses / Mechanical Turk

Angus Grieve-Smith grvsmth at panix.com
Tue Jul 16 20:03:11 CEST 2013

On 7/16/2013 2:43 AM, Adam Kilgarriff wrote:
> For either automatic WSD, or even for the gold standard, I agree
> entirely with John:
> Miss Elliott, my high-school English teacher, wouldn't give
> anyone a gold star [for work like that]

Well, now hang on a minute.

First of all, how unambiguous were these, originally? Real language is full of ambiguous uses. I'm surprised that the experts can even agree 95% of the time, and I'd guess that part of their expert training was aimed at exaggerating that agreement to satisfy brittle models that weren't built to handle persistent ambiguity.

Second, if the human experts didn't even agree, why would Miss Elliott expect her students to do any better? What kind of a sadist was she?


-Angus B. Grieve-Smith

grvsmth at panix.com

-------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/html Size: 1663 bytes Desc: not available URL: <https://mailman.uib.no/public/corpora/attachments/20130716/9ead5675/attachment.txt>

More information about the Corpora mailing list