[Corpora-List] Comparing n-grams / authorship

Yuval Marton yuvalmarton at gmail.com
Wed Apr 18 19:53:12 CEST 2012

A few years ago, there was some controversy regarding whether one could use off-the-shelf compression techniques such as gzip for authorship attribution. If you are interested in trying out something like that, you might want to have a look at

Yuval Marton, Ning Wu, and Lisa Hellerstein. "On Compression-Based Text Classification". Proceedings of the 27th European Conference on Information Retrieval (ECIR), Spain, March 2005. Abstract<http://www1.ccls.columbia.edu/%7Eymarton/pub/ecir05/finalforwebabs.pdf>. Full paper here <http://www.springerlink.com/index/M24DHW0XDMREHE64> or here<http://www1.ccls.columbia.edu/%7Eymarton/pub/ecir05/final.pdf>. Click here for the errata note!<http://www1.ccls.columbia.edu/%7Eymarton/pub/ecir05/onCompressionForClassificationErrata4.htm>

... and the papers cited within.

Pro: very easy to use. Simply split each chapter to a "training" and "test" parts, and follow recipe in the paper. Con: Not the most standard way authorship attribution tasks are usually carried out.

Yuval Marton IBM Research

On Tue, Apr 17, 2012 at 3:47 PM, Mark Davies <Mark_Davies at byu.edu> wrote:

> I am sending the following question on behalf of a colleague at BYU.
> Thanks in advance for any suggestions you have; I'll forward them to the
> researcher who is working on this problem.
> Mark Davies, BYU
> -------------------------------------------
> I am working with a 250,000 word text. Within this text there are two
> chapters, A and B (1,200 and 2,400 words respectively). The authorship of
> these two chapters is unknown, but we have reason to believe to that the
> author(s) of A and B have a relationship that is different from the
> majority of the rest of the book. There are two 4-grams, three 6-grams, one
> 7-gram, one 8-gram, and one 9-gram shared in common in chapters A and B
> that appear nowhere else in the book. Intuitively it seems like there is a
> unique relationship between chapters A and B.
> The question is:
> Is there a statistical method of measuring whether the types of n-grams
> above establish a reasonable probability that the two texts are linked.
> _______________________________________________
> UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
> Corpora mailing list
> Corpora at uib.no
> http://mailman.uib.no/listinfo/corpora
-------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/html Size: 2912 bytes Desc: not available URL: <https://mailman.uib.no/public/corpora/attachments/20120418/31652a00/attachment.txt>

More information about the Corpora mailing list