Home->
Concepts->Waves
Mt Kailash, Tibet; photo: digital-dharma.net
Waves and Information
The internal structure of information is all around us. It's
especially beautiful in the shape of trees and leaves. A binary
tree cascades down in a pyramid form. It's this form that
mathematicians have exploited to unveil the ubiquous, dynamic
patterns in the universe. Binary trees as lists are use
exclusively in structures on wave packet analysis. The binary
tree represents information in its purest dynamical
form.
Pyramids of Giza, Eygpt; photo: bbc.co.uk
Binary Tree and
the Unfolding of Possibilities
Order in a sequence containing a finite number of elements can be
intrinsically described by a binary tree. The unfolding of a path
which splits in two enables the creation of new possibilities as you
descend down a binary tree. This attribute of the binary tree describing
its size is called the depth. The size a binary tree is equal to the
size of a sequence because you can think of the binary tree as containing
the ordered temporary structure of a sequence inside the mechanics
of computation.
It's natural to visualize the size or length of a sequence when modelling
Nature rather than a tree structure. But inside the mechanics of
computation lies the structure of binary trees. The binary tree is the
computational dimension of a sequence inside our wave packet model of
a spike trains.
The Binary Tree
The wave packet transform, WPT, method of representating a signal lies
in the decomposition of the complete binary tree in the figure above into
parts or subsets of sequences as shown in the two figures below.
The ordered subset is a connected or contiguous group of elements
or paths of the complete binary tree [1, 2, 3].
The Branches of a Binary Trees
Figures (A) and (B) above contain parts of the complete binary tree
sequence. The sequences down at the fourth level are (A) (ddda, dddd),
and (B) (daaa, daad, dada, dadd).
The depth or size of the binary tree determines the resolution of
the signal. If the sequence represents the signal, then the minimum
length of the sequence containing the maximum amount of information
is a compressed sequence in the sense of Kolmogorov complexity.
The information holding capacity in a sequence is determined by the
size of the binary tree in terms of the number of branches and leaves.
In term of wave packet analysis, the minimum number of partitions in
the sequence you make (in terms of decomposing of the complete binary
tree) determines how well or accurately you can measure the signal
or sequence. This comes full circle back to the idea of the limit
in measuring the signal. The minimum number of descriptive
"Fourier lexicons" or wave packet coefficients you use to describe
the signal determines its resolution limit.
Wave packet analysis depends on decomposing a complete binary tree,
also called a complete basis set, into branches of a binary tree or
a basis subset. A wave packet represents a sequence which lasts for
a limited time period.
Wave Packet
Entropy in Waves
In studying the neuron as an information processor, we want to know
when the synapses occur as a sequence. More specifically, we want to
know the duration of time between the occurrance of each synapse in
the spike train. If we assume that the duration of time between
each synapse is constant then we can considered this to be a Poisson
process. Synapses as events occurring as a Poisson process are
independent of each other; this independence is equivalent to making
the temporal duration of each event relative to each other constant .
If the duration between the synapses is a variable, then this is a Markov
proces in which each synapse depends on other the synapses which occurred
previously in time.
As a note, it is known from past studies in statistical mechanics that
a Poisson process contains the maximum amount of entropy among the
more general time ordered processes. A Poisson process, having the
maximal entropy (uses the least amount of state varibles as a steady
state ergodic system), and contains the lowest amount of information.
But we know that the neuron changes its synaptic firing rate in the
spike train from listening to the chirps of individual neurons or their
frequency modulated clicks. The firing rate of the neuron varies.
So in general, from an information processing perspective, we say that
the neuron's synapses is a temporally ordered Markov process.
A sequence contains a variable set of elements, S = {a, b, c, ... }.
For example a sequence could be written s = (abcefd). In the case
of a Poisson process, an example sequence would be s = (aaaaaa), and
a sequence for a Markov process example would be s = (cdefab). In the
case of a pure Poisson process, a binary tree structure representation
of the synapse would not be needed. A binary tree provides the
computational order in a sequence. A binary tree can represent the
potential of possibilities in a temporal chain of events which is
called a Markov process.
Time-Frequency Spectrum of Waves
The general description of wave forms are done using time-frequency plots [4]
of wave packets. The internals of a wave packet maybe described using binary
trees grown as Markov processes, but to get a real feel of this you can
look at a program which displays a time-frequency graph [5].
Time-Frequency Graph of a 32-bit Sequence
The plot above is that of a simple binary sequence
S = (0 1 0 1 0 1 1 1 1 1 0 0 1 0 1 0 1 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0).
The plot on the top represents a time line in which the interval
between the elements of this sequence can be set arbitrarily or scaled to
any length. The plot below contain time-frequency rectangles, called
"diagram of information" by Dennis Gabor. These rectangles have a simple
intuitive meaning. For low oscillations the rectangles appears lower
on the plot, and correspondingly for higher frequencies, the rectangles
are on the top.
The sound of this very simple sequence played at 10K Hz sounds like
a drop of water hitting the ground. This sequence is easy to inspect
because it contains relatively little information. It contains about
the amount of information in an 8 letter word like, "Hi_There".
The synapses of neurons may be represented by Markov processes in
which the spike train sequences are constructed from binary trees.
The wave packet formulation is a practical methodology because it
combines the intrinsically intuitive wave model of energy propagation
with a built in natural structure for information processing using
the binary tree.
Reference
1.
The Wavelet Transform Beyond Fourier Transforms,
Mac A. Cody, Dr. Dobb's Journal, April 1992.
[PS].
2.
The Wavelet Packet Transform Extending the Wavelet Transform,
Mac A. Cody, Dr. Dobb's Journal, April 1994.
[PS].
3.
Entropy-Based Algorithms for Best Basis Selection,
Ronald Coifman and Victor Wickerhauser,
IEEE Transactions on Information Theory, 1992,
[PDF].
4.
Theory of Communication, Dennis Gabor, 1946,
The Journal of the Institution Of Electrical Engineers, 93(3):429-457.
5.
Fabian Brachere;
Guimauve graphical software;
Gaussian decomposition of signals.
next:
Thoughts
