This is an interesting observation.
Maybe one explanation for the lack of response to user-feedback is that it is much harder to make incremental changes to probabilistic / machine-learned systems than to rule-based ones. If a corpus user identifies systematic errors this can - in a rule-based parser - be used to remove errors or add rules, or introduce new lexical sets and categories, while in an ML-system this would have to be done by paying somebody to annotate the changes into a treebank, which is, as you say, unlikely.
Though my view is probably biased, I think this might be an example of the side-effects of using trained systems for corpus work rather than rule-based ones (like AGFL or CG, to name a couple).
Best regards, Eckhard Bick
Adam Kilgarriff wrote:
> My lexicography colleagues and I use POS-tagged corpora all the time,
> every day, and very frequently spot systematic errors. (This is for a
> range of languages, but particularly English.) We would dearly like
> to be in a dialogue with the developers of the POS-tagger and/or the
> relevant language models so the tagger+model could be improved in
> response to our feedback. (We have been using standard models rather
> than training our own.) However it seems, for the taggers and
> language models we use (mainly TreeTagger, also CLAWS) and also for
> other market leaders, all of which seem to be from Universities, the
> developers have little motivation for continuing the improvement of
> their tagger, since incremental improvements do not make for good
> research papers, so there is nowhere for our feedback to go, nor any
> real prospect of these taggers/models improving.
> Am I too pessimistic? Are there ways of improving language models
> other than developing bigger and better training corpora - not an
> exercise we have the resources to invest in? Are there commercial
> taggers I should be considering (as, in the commercial world, there is
> motivation for incremental improvements and responding to customer
> Responses and ideas most welcome
> Adam Kilgarriff
> Adam Kilgarriff
> Lexical Computing Ltd http://www.sketchengine.co.uk
> Lexicography MasterClass Ltd http://www.lexmasterclass.com
> Universities of Leeds and Sussex adam at lexmasterclass.com
> <mailto:adam at lexmasterclass.com>
> Corpora mailing list
> Corpora at uib.no
-- Eckhard Bick, cand.med., dr.phil. University of Southern Denmark e-mail: eckhard.bick at mail.dk web: http://beta.visl.sdu.dk