Dynamic movement as represented in an abstract sequence is always greatly
simplified. For the past 2 days, I've been studying Satosi's 1953 article
on ergodicity and sequence correlation index after I received the hardcopy
monograph I ordered which has handwritten formulas (which must be Satosi's)
in it. There're 14 pages for mathematical formulas Satosi uses to derive
his correlation index, W. There's more discussion about the reasons for using
concepts like ergodicity and redundancy in this report than in his 1960 article
on Multi-Variate Correlation.
The formula for W above work in principle for ergodic Markov chains. Satosi's
criterion was that the symbols in the sequence, S, had to be unique (which also
meant the sequence progression eventually made any conditions on the starting
symbols not matter).
Anyway, the interesting part of a dynamic movement in Nature seems to be at
the beginning of the movement ... entropy at work always increasing. I took
snap shots of a wave packet simulation. The wave packet bounded in a square
box hits a very thin vertical potential barrier in the middle of the box.
The snap shots are taken at 3 secs, 10 secs, 30 secs, 5 mins, 30 mins, 8 hrs
and 10 hrs.
In terms of dynamic movement, the wave packet settled into a rather constant mode
after 5 minutes into the simulation. According to ergodicty, it would not matter
what type of initial boundary barrier we had setup in the beginning of the
simulation. Eventually, after a long time, the waves in the box would look
like white-noise; just a noisy speckle pattern. In terms of the simulation,
this would be true primarily because of the floating point number roundoff
errors in the simulation equations.
Jan 4, 2011
Today I read some articles on spike train synchrony.
In articles 1 and 2 above, the authors rigorously study the relationship of
temporal spike trains. It's an admirable scientific study trying to pin
down the real grounds for what's inside a spike. Their principal analytical
object or tool is what they call the Inter-Spike Interval, ISI, which is
"parameter free and time-scale adaptive (Kreuz et al., 2007)". Their
construction of the "SPIKE" distance which is the temporal interval
between two spike is interesting. They actually reconstruct the differential
and integral cells (what I used to think of as cellular spaces in calculus)
for analysing spikes. These are two excellent articles. But it's also
interesting that in article 1, the authors say in the conclusion that
"First, it is obvious that no measure that results in a single number
quantifying the synchrony between two or more spike trains can be adequate
to deal with all kinds of potential coding schemes (e.g, time coding,
rate coding and pattern coding; ..."
Article 3 is a good study of neuronal synchronous temporal-dependent
plasticity, STDP, and the highly synchronized network response to stimuli.
My questions settled arount the equations used in this article. The
neuronal circuit equations contained interconnection weights, w_ij, so
it may not matter what boundary conditions you set in the beginning;
you could get periodic behavior in the end if you look for it like I do.
I'd just eliminate the word "non-periodic" in this article and use
"irregular" as they do in their introduction. Synchronization can occur
without resonance. This article, while confusing in places, was very
interesting. This article is a fine, extremely detailed study
of the simulation of neuronal topology and some experimental data. I
couldn't absorb most of it today.
The first 50 pages, 1st chapter, of a recently published book, August 2010,
edited by Christoph von der Malsburg, William A. Phillips and Wolf Singer
is online at
Dynamic Coordination in the Brain. The editors comment,
on page 17 on the section called "Temporal Structure and Synchrony" which
the authors of article 1 above carefully avoid implying in any manner, that
"One possibility is that spike rate and spike synchronization operate in
a complementary way such that salience can be enhanced by increasing either
or both."
The editors, in this paragraph on temporal synchrony, suggests that
synchronized rate codes, high frequency circuits, and neuronal circuit
inhibition, leads to a host of higher level cognitive functions which
I've always assumed to be true or could be true from a computer programming
perspective. In writing software you a free to create, and communicate
to others whatever you think are the most optimal structures. You can't
really do this when you're bounded by experimental, laboratory evidence.
I think spiking rate, and temporal coding are synonymous. I have described
both with the same mathematics.
Today, I re-read Bruce Knight's 2008 article,
The Faithful Copy Neuron. The simplicity of the idea of
an ensembly of neurons containing an innate propensity for synchronization
is very appealing. There are other concepts which add elegance to
the model such as "revised time" which seems to play the role of
phase shifting the spike trains. I don't understand this yet. I have not
thought about it enough. But I used to wonder about entropic correlation of
a set of sequences. The dynamic change of a set of sequences with respect
to its order in time usually occurs in the begin as it is constrained by
its initial "boundary" condition. The sequence as it progresses ergodically
appears like the beginning of the sequence. This is just the way it is
with most simple sequences in Nature.
Article 4 referenced above, covers essential technical analysis. The
authors state at the end of the article that
"To accomplish this type of decoding, neurons need not do
anything more sophisticated than be sensitive to the durations of
individual ISIs. This sensitivity can be embodied in a single
synapse and does not require averaging across stimulus repeats,
stretches of time that may be long compared with the time scale
of firing rate modulation, or a large population of neurons that
carry similar information."
This is quite an insightful conclusion. It also remainds me of the
significance of the prefix in sequences in pattern matching strings.
Dec 24, 2010
I really found what I was looking for today. I was skimming through
articles on neural coincidence detection for the pass few days. When I
read this article's summary "The firing rate of a population of neurons is
related to the firing rate of a single member in a subtle way.", I thought
if this article explains this even a little bit, it'll make my day. Most
articles on this subject go nowhere. Or maybe I wasn't tuned in and prepared
to resonate with this article before. The way an article is written, the
simplicity of expression of the math, etc., all determines how one absorbs
the material.
The next sentence in the introduction really impressed me.
"In a nervous system it is usual for extremely precise over-all results to
arise from the functioning of a collection of components which have very
modest precision in the individual construction and behavior."
Again I thought, wow, I hope this gets explained. Well, I tell you, I read
this single article for over 3 hours. I can't believe I never found this
article before. But then again, I wasn't tune in and ready for this before.
This article sort of bridges the gap for me between understanding how a
single neuron in a single neural circuit expresses itself as a larger
ensemble of neurons.
Bruce Knight,
the author of this article, published it in 1972. It's called "Dynamics
of Encoding in a Population of Neurons." It's perfect.
Dec 20, 2010
Around 10 years ago I downloaded Wade Lutgen's beautifully coded wave
packet scattering program. I put the program's simulation on video today
(it's 5.7 minutes long).
The Granularsynthesis.com website
features music artistry composed of "grains" which are sound atoms. The
following video called "decay" is made up of images of sound atoms which
are decomposed wave packets in frequency-time streams you see in signal
processing. These sound atoms are the building blocks of speech and
musical sound streams at the micro and milli second time scale. It's the
time scale at which we study the synapsing signatures of neurons. Pretty
amazing.
decay by Nikola Jeremic
Dec 18, 2010
This is a tribute to
Walter Freeman. His work
inspires me. He's done what many people, or at least I, would have
liked to have done for a few years, that is, to study the brain in a
clinical laboratory setting. Recently, he published some notes on
the theoretical foundations of his lifetime work on EEG called the
Hibert transform. The Hibert transform explains how we can
use data samples, the observation data taken in experiments, to use in
signal processing. In doing wave packet analysis, the Hilbert transform
makes working with the math more intuitive. I'll write an article
about this in the next few days.
When you want to model brain functions, you need experimental work. So
I've really tried to understand what Walter Freeman published. But that's
hard to do because of all the details. Today I was studying a publication
of his called "Application of Hilbert transform to scalp EEG containing
EMG",
Freeman, Burke, Holmes (2003) [PDF].
I know, as I read through these works, that I'm not really getting the
optimal feel for what's written because I wasn't there doing the work.
But Walter Freeman has been incredible in giving us his interpretation
of what the brain is really doing in the books he's written. Thanks
for helping us understand ourselves.
Dec 7, 2010
I watched the beautiful sunset the other day wondering about what to add to
this site. I'm studying again how Satosi derived his clustering formulas
to get the entropy equation. I think that's something I can understand.
But then I began to wonder if I should look at Renyi entropy. That's
really strange. The formula for Renyi entropy is:
where as q -> 1 Renyi entropy tends to Shannon entropy.
I can sort of feel what the natural log of the summation is like,
but how did Renyi get 1/(1-q). That's a singularity at q=1! What
a strange behavior for a limiting singularity to approach Shannon's
entropy. Renyi was a mathematician who worked with Erdos.
To get the full implications of this formula will take me a long time
so I've just left it alone. I know my limitations. I know that this quest
is bounded, so I have to smile.
December 3, 2010
I just put this page back online today. Ten years ago, most friends
who were not specialists in this field, said they could not
understand much of what I had written. But a few, rather small number,
seemed to appreciate the "strangeness." There are some great and
beautiful concepts that are simple and elegant enough to be in
wonderment of like entropy. David Bohm calls this awareness of a
streaming universal energy a "movement" which he saw all around him.
When I first read what he wrote, I had some doubts about what he
was saying, but now it's what I have incorporated into my worldview
completely. It's was strange at first, but not now.
October 14, 2010
Began updating this webpage with new contents about computing paths in
programming code. I naively started this web page in 1998, inspired
by the beautiful design of recurrent neural networks. However, I soon
realized that I was getting nowhere studying classical error minimizing
matrices in neural networks. But it's hard to overcome old notions.
I knew the neuron used the temporal code to talk to each other.
But in my developmental work I kept using the old data structures
and algorithms of the rigid networks. About 2001 I gave up on
really developing my ideas on waves and neurons. In 2007 thinking
that I might never find an interesting developmental path, I took
this website offline.
But about a year ago I just started feeling more confident about
the "wave model" of the neuron. This is because I could integrate
aspects of entropy or information into what I thought might be happening
in neural circuits.
