I think we should acknowledge here that corpus linguistics has come to mean two things: * one is the use of corpora to gain quantitative data on phenomena that are
otherwise hard to pin down, in part or wholly because they are
dependent on genre or time frame. * the other is the use of corpora as an additional information source for
linguistic questions which are not meant to be genre-dependent and
additionally are corroborated with other sources of information (e.g.,
introspection/thought experiments, elicitation of speaker judgements,
or experiments related to processing effects).
Your claim that
> Dumps are about rubbish, either legal or fly dumping. The results is still
> rubbish in rubbish out. Corpus linguistics has to have a benchmark,
> otherwisde it would cease to exist.
are tantamount to a dialectologist saying that the way most people elicit speaker judgments is rubbish because they only ever ask people from one city.
It should be clear that there are linguistic issues which are dependent on dialect, genre, time frame, and other factors, and that having a well-balanced corpus can help reduce the risk of blindly running into overgeneralizing a phenomenon that actually is limited to some sub-part of language. Then again, you should always look at empirical data with a modicum of common sense, as even carefully balanced corpora tend not to be representative of the finest genre distinctions - e.g., car repair manuals standing in as representative for all kinds of repair manuals, one magazine (possibly containing the idiolect of only a small group of people) standing in for magazines in general, or one kind of talk interactions (people doing small talk with a linguist nearby) standing in for all "oral" language. That kind of "perfection" when it comes to balancing corpora would not merely be difficult to achieve, it is a pure utopia. Which means that any statistical regularities, even if you gained them from a carefully curated corpus, have to be taken with a large grain of salt. NLP practitioners using corpora are used to taking in large amounts of salt in that way, because statistical models work best if you throw lots and lots of data at them; similarly lexicographers, and linguists investigating rare phenomena that maybe occur only once every million words. If you're the first kind of corpus linguist, though, careful composition beats quantity (and Biber et al's Longman Grammar of English is a good example how this may make sense), but you shouldn't overgeneralize your own requirements to the population at large, or monopolize terms such as "corpus linguistics".
> However corpus linguistics does not have a monopoly on the word corpus
> which is somewhat polysemic. If a Wikipedia or other dump works for
> testing some commercial application or NLP project, then why not use it.
> On the other hand, don't say that corpus linguistics is being done.
> Corpora list is wealthy through the experience of many who use corpora in
> different ways. Not all are corpus linguists, neither should they be. It
> is esential however that those disciplines involved know their research
> protocols. This is what Martin's timely reminder is about.
> Perfection is difficult to achieve, but that does not make it a less
> worthwhile goal.
> >> ...and a "dump" such as this couldn't be further from qualifying as a
> > corpus, if defined as "a > collection of pieces of language, selected and
> > ordered according to explicit linguistic criteria > in order to be used
> > as a
> > sample of the language.”
> > Sorry, Martin, but your definition of 'corpus' reads like it's designed
> > to strangle the life out of corpus linguistics. It begs questions about
> > selection and ordering (?? how does ordering come into it?) and explicit
> > linguistic criteria, and demotes many things that people refer to as
> > 'corpora' to a lower form of life. Lexicographically, bad.
> > I think it's a dream of some corpus linguists as to what they think a
> > corpus
> > should be, not a fact about how the word is used. But, delete the middle
> > clause and we're in agreement:
> > "a collection of pieces of language, used as a sample of the language"
> > Adam
> > On 2 March 2010 22:55, Martin Wynne <martin.wynne at oucs.ox.ac.uk> wrote:
> >> Francis Tyers wrote:
> >>> El dt 02 de 03 de 2010 a les 12:38 +0100, en/na Xin Yan va escriure:
> >>>> Hello,
> >>>> can anyone tell me, if there are some free text corpora for commercial
> >>>> purpose?
> >>>> Thank you in advance!
> >>> You can download dumps of Wikipedia from http://download.wikimedia.org
> >>> -- they are licensed under the CC-BY-SA or GFDL -- both of which allow
> >>> commercial use, providing changes made are redistributed under the same
> >>> licence.
> >>> Best regards,
> >>> Fran
> >>> _______________________________________________
> >>> Corpora mailing list
> >>> Corpora at uib.no
> >>> http://mailman.uib.no/listinfo/corpora
> >> Dumps of wikipedia may be an interesting electronic text collection that
> >> can be used to help address various linguistic research questions, but I
> >> think that the request was for a corpus...and a "dump" such as this
> >> couldn't
> >> be further from qualifying as a corpus, if defined as "a collection of
> >> pieces of language, selected and ordered according to explicit
> >> linguistic
> >> criteria in order to be used as a sample of the language.”
> >> The good news is that corpora are available. If you let us know what
> >> sort
> >> of corpus you are looking for and for what sort of commercial uses you
> >> intend to put them to, I am sure that there are plenty of people here on
> >> the
> >> mailing list who can help you.
> >> Martin
> >> Oxford Text Archive
> >> _______________________________________________
> >> Corpora mailing list
> >> Corpora at uib.no
> >> http://mailman.uib.no/listinfo/corpora
> > --
> > ================================================
> > Adam Kilgarriff
> > http://www.kilgarriff.co.uk
> > Lexical Computing Ltd http://www.sketchengine.co.uk
> > Lexicography MasterClass Ltd http://www.lexmasterclass.com
> > Universities of Leeds and Sussex adam at lexmasterclass.com
> > ================================================
> > _______________________________________________
> > Corpora mailing list
> > Corpora at uib.no
> > http://mailman.uib.no/listinfo/corpora
-- SFB 833 "Bedeutungskonstitution" Nauklerstr. 35 - D-72074 Tübingen Tel.: +49-7071-77155; Fax: +49-7071-5830