[Corpora-List] QM analogy and grammatical incompleteness

Rob Freeman lists at chaoticlanguage.com
Mon Dec 19 02:32:01 CET 2005

On Sunday 18 December 2005 19:00, Dominic Widdows wrote:

> Dear Rob,


> > For instance, famously, you can perfectly describe the momentum or the

> > position of a particle, but not both at the same time. This is

> > Heisenburg's Uncertainty Principle.


> It is, and the Uncertainty Principle is perhaps the clearest formal

> expression we have of the homely truth "you can't know everything at

> once". A more classical version of the the same argument would follow

> from the observation that, if you had a machine that tried to run a

> Laplacian / deterministic model of the universe, its physical size and

> the speed of light would limit the amount of information it could

> synchronously process.

If the speed of light were infinite would this still result in an Uncertainty
Principle? I'm wondering because I want to know exactly where the Uncertainty
Principle begins. My own model is just inconsistent orderings of a set, say
according to colour and size, where you know everything about both orderings
at every moment, but only one ordering is possible at any given moment. Such
orderings are fundamentally inconsistent so you get an uncertainty principle.
Is this the same thing as an information lag due to a finite speed of light?

> > So it is not so much the fact of going from a continuous quality

> > to a discrete quality which is interesting, it is the necessary

> > incompleteness of description in terms of discrete qualities

> > abstracted from a distribution which is where I think we should

> > be focusing, in analogy with the Uncertainty Principle of physics.


> Is this similar to asking whether all such quantization is a "lossy"

> transformation? Is this what you mean by incompleteness?

I think so. Yes.

> > Dominic, I have only read the publically available chapter

> > of your book. You mention a "vector model" for quantum

> > mechanics. Do you have anything on the Web which talks

> > about that? I can only recall ever having met descriptions of

> > QM in terms of functions.


> The article at http://plato.stanford.edu/entries/qm/ looks like a good

> place to begin for QM and vectors.

Hmm, blushes :-) It is of course trivial that 3-space is a vector, so any
explanation of the physical world is also a vector explanation.

Still, I was thinking more of a vector explanation which motivated the quantum
element. I was interested in this because it strikes me that any quantum
element in grammar, and particularly any kind of grammatical uncertainty
principle, is crucially dependent on a vector representation.

It hadn't struck me that vector representation is crucial to QM. Newton's laws
apply to vectors, but they are not QM.

I'll have to look at it more closely and see if QM can be predicted purely
from the fact that physical laws can be expressed in terms of vectors.

> In broad strokes, the history of vectors and functional analysis became

> very closely linked in the 1840s and 1850s, partly through Hamilton's

> work on quaternions and the theory of analytic functions on 4-space.

> Functions over the real numbers form a vector space - you can add two

> functions together, and multiply any function by a scalar. As a result,

> mathematicians came to realize that Fourier analysis could be described

> in vectors - each of the functions sin(nx) and cos(nx) (for x a real

> number, n an integer) is a basis vector, and any piecewise smooth

> function can be expanded (uniquely) as a vector, using these functions

> as a basis. The Fourier series coefficients are thus interpreted as the

> coordinates of a vector in this basis. This vector space is clearly

> infinite-dimensional, because a Fourier series expansion can be

> infinitely long. (Note again that this means you will never work with

> complete information once you've quantized your functions.)

So quantization always means a loss of information, complete information about
one quantization will always result in incomplete information about another
quantization, resulting in an uncertainty principle?

We still don't have any postulates, but we already have an Uncertainty

Is this dependent on the fact that the Fourier coefficients are

> Make your

> functions complex-valued, and introduce a metric based on

> complex-conjugation, and you've got Hilbert spaces, around 1900 I

> think.


> In the 1930's, Paul Dirac, John von Neumann, and others, used this

> formulation of functional analysis as the basis for formal quantum

> theory, much of which boils down to the analysis of self-adjoint

> operators on Hilbert space. Each function is a state-vectors, can be

> normalized and operated on. The resulting operator algebra (group of

> linear transformations under composition) is non-commutative, and this

> how the formal theory accounts for the Uncertainty Principle - the

> lower bound on the uncertainty of two observables is given by the

> magnitude of the commutator of their self-adjoint matrices.

That is what I was looking for. I didn't know that. It struck me later dealing
with language that you would get an uncertainty principle if you defined
grammar in terms of different ways of ordering word associations, but I
didn't know that people had made the connection in QM, motivating the
Uncertainty Principle from first principles dealing with vector
representations of physical qualities.

In fact the vector representations they deal with are representations of
functions though, right, functions expressed as sums of Fourier coefficients?
So it is the act of summing over functions (to specify new functions) which
gives you QM, not just the fact of vector representation? Well, I suppose
that _is_ vector representation (for functions) fair enough.

In point of fact I had forgotten that quantum wave functions are usually
expressed as vectors of Fourier expansion coefficients. Comes of too much
familiarity with Greek letters :-) The standard formulation of QM is actually
a vector formulation.

> Clear as mud? ;)

I think I am good with that. It has explained something I wondered about. The
key to the Uncertainty Principle is that you collect components together
(quantize) in QM, while in Classical Mechanics you don't. Is that right?

So, look, Dominic, moving from QM back to language, doesn't it then follow
that any instantaneous characterization of grammar, if it is defined
fundamentally as a clustering (quantization) of distributions of word
associations, will be necessarily incomplete, for the same reason that there
is inevitably a loss of information when you quantize state-vectors.

Just to underline that, it seems like quite an important result: doesn't this
then mean that all machine learning, HPSG, all attempts to comprehensively
describe grammar as a single comptete and consistent set of generalizations,
any consistent attempt to enumerate grammar at all, HMM's etc. which seek
complete representations for the grammar of a language in terms of a single
set of consistent classes, all of these are doomed to fail?

Doesn't this then mean that we will always be wasting our time trying to
completely characterize grammar, that the most complete description of
language will only ever be a corpus, that we must leave language in the form
of data, a corpus, and only think of grammar as coming into existence at the
time it is observed, by making grammatical generalizations relevant to each
problem, as that problem occurs?

So Dominic, by this result isn't a large part of contemporary linguistics (and
knowledge representation/ontology as it happens), tree-banks, taggers, what
have you, most of what gets talked about on this list, in fact, clearly
wasting its time, trying to enumerate the innumerable?

> > I agree completely with your message, but would only add

> > that while quantum analogies can be very informative for

> > lexis, where I think it really gets interesting is in

> > syntax, which responds very nicely to a kind of "quantum"

> > analysis in terms of generating new quantum qualities

> > (particles?), a new one for each new sentence.


> This may be partly because composition is so far modelled more robustly

> in syntax that it is in (parts of) semantics? Just trying to figure out

> what compositional rules apply a list of noun-noun compounds extracted

> from a corpus is very hard - and this is just combining 2 "particles"!

> Some of the most interesting structures that arise in QM involve

> entanglement, and I dare say that some of the structures in syntax are

> as rich in this "multiply composed, new particles / systems arise"

> property. I don't have the expertise to do any proper analysis here,

> though.

I think syntax is where it gets interesting firstly because while people have
done quite a lot with distributed representations for words, LSA, vector
information retrieval, SVD etc, they have tended to shy away from syntax.
Perhaps this is because with syntax language can no longer be seen as a
classification problem. All our tools for dealing with distributed
representations, neural-networks, case-based reasoning, have been
classification tools. With syntax for the first time you can't get away from
an ad-hoc character.

So syntax is firstly interesting because it has not been done. But secondly I
think syntax is interesting because the very ad-hoc nature of syntax treated
this way gives us vast new powers.

Because you can cluster contexts on words or words on contexts the analysis
can be seen interchangeably as the generation of new syntax or the generation
of new meaning. New syntax generates new meaning, and because of the quantum
aspect, because you might say of very lossyness of quantization, the number
of new meanings which can be generated is now greater than the size of the
language itself. (Because new grammatical/semantic qualities are defined as
one or other way of collecting sentences together, and you can fold lots of
new patterns out of the same set of sentences.)

The ability of a finite language to express almost infinite shades of meaning
is explained.

It rather reminds me of Goedel incompleteness, which allows a formal system to
contain and describe itself. I don't know if there is another parallel there.

Are you aware of any other work which looks at syntax from this point of view?



More information about the Corpora-archive mailing list