[Corpora-List] Precision and Recall

Angus Grieve-Smith grvsmth at panix.com
Sat Apr 19 20:30:17 CEST 2008


On Sat, 19 Apr 2008, Daniel Zeman wrote:


> the false positives/negatives are absolute numbers. If you evaluate, say,
> performance of a parser on two different data sets and you get fp=100 and
> fn=100 for both, you still cannot say that both sets are equally hard for the
> parser. It may well be that the sets were not the same size and that tp1=100
> while tp2=1000.

Okay, I see that you would want to know how many false negatives there are as a proportion - i.e. how many of the positives it found correctly - so I see the value in "recall," even if it doesn't make much sense as a name. But it seems to me that the raw number of false negatives is also valuable.

But false positives are false positives; why does it matter how many true positives there were? Because it's a measure of how muddy the water is? It seems like here, absolute numbers of false positives would be more valuable in many situations. As Google found, it often doesn't matter how many false positives you have, as long as the most valuable true positives are close to the top of the list.

Incidentally, this is not a purely academic line of questioning; I worked on an information retrieval project that failed in part because precision and recall did not accurately predict customer satisfaction.

-Angus B. Grieve-Smith

grvsmth at panix.com



More information about the Corpora mailing list