[Corpora-List] Multiple category assignement

Angus Grieve-Smith grvsmth at panix.com
Sun Aug 25 17:32:08 CEST 2013

On 8/25/2013 10:55 AM, Aliabbas Petiwala wrote:
> So should such multiple categories be represented as bitstrings , such
> that for n categories there would be a whopping 2^n assignments ? This
> would surely make the inter annotator agreement (IAA) scores very low
> for minor differences.
> So what is the best way to compute annotation agreement for tasks that
> require multiple assignment to an item? And how to represent
> categories for such cases?

In my French negation study, there were only three categories that could overlap that way (predicate negation, presupposition denial and proposition denial), hence only eight possibilities. There were other classifications, but they were along separate dimensions.


I find it hard to imagine an annotation task with more than four possible overlapping categories. If anyone has an example, I'd be interested to see it.


-Angus B. Grieve-Smith

grvsmth at panix.com

More information about the Corpora mailing list