John F Sowa sowa at bestweb.net
Fri Nov 9 19:55:08 CET 2012

Matt, Darren, Alex, and Ele,

General point: Every version of statistics assumes some model of the subject matter. The statistics are stated and interpreted in terms of that model. For example, the typical model for statistical parsers is a kind of phrase structure, for which the rules are derived by a statistical methodology.

Before commenting on the comments on my note, I'll cite an article by the statistician Leo Breiman with discussion by other statisticians and a response by Breiman:


> I readily acknowledge that there are situations where a simple data
> model may be useful and appropriate; for instance, if the science of
> the mechanism producing the data is well enough known to determine
> the model apart from estimating parameters.

The phrase-structure model for grammar is well understood. But more models are required for the semantics and pragmatics of each subject. The question of how multiple models -- derived by diverse methods -- can interoperate for language understanding is not well understood.

> ... we are in a period where there has never been such a wealth of
> new statistical problems and sources of data. The danger is that
> if we define the boundaries of our field in terms of familiar tools
> and familiar problems, we will fail to grasp the new opportunities.

Yes. This is the danger of taking models and tools that have proved to be useful for one type of problem and applying them uncritically to problems for which the models are inappropriate.

> The human brain does not use statistical methods for learning,
> language understanding, or sentiment detection.

> Did the advertisement make such a claim?

No, but that is a preamble to my later points. My criticisms are not really about the advertisement by Massimo Poesio, but about the dangers discussed by Breiman and others.

> I thought you might be interested in this article:
> http://www.tor.com/blogs/2011/06/norvig-vs-chomsky-and-the-fight-for-the-future-of-ai

Thanks for the pointer.

Chomsky was criticizing the "grammar discovery procedures" that were being developed during the 1950s. Chomsky made a major contribution by developing valuable new models. But he did a major disservice to linguistics by dismissing data-driven methods in favor of "the intuitions of a native speaker."

There is a very wide range of options between Chomsky and Norvig. Instead of fighting for either alternative, I support the proposals by Marvin Minsky for a "Society of Mind" with a multiplicity of interoperating, heterogeneous agents.

> I think it is a matter of different ways to view the term
> "Human Language Technology":
> a. technology to handle human languages
> vs.
> b. the "technology" used by human (brains) to handle languages

Yes, but the approaches can support and reinforce each other. Evidence from brain scans can show what areas are active in the brain during various operations, and evidence from anatomy can show the pathways that interconnect them. That evidence can suggest new kinds of tools and ways of connecting them.

I believe that Massimo P. and his group are going in the right direction by encouraging interdisciplinary collaboration between neuroscience and computational linguistics.

But I agree with Breiman that a focus on "familiar tools and familiar problems" could prevent us from grasping "the new opportunities."

> What kind of 'technology' would/could a human brain use?

The term 'technology' may be questionable. Minsky called the brain "a computer made of meat." In any case, the brain does some very complex processing, and neuroscience is beginning to provide better insights into how it works.

Any neuroscientist will admit that the amount that is known about the brain is very small compared to the amount that is still unknown. But that small amount can provide important guidance and inspiration for designing AI systems.

John Sowa

More information about the Corpora mailing list