School of Chemistry F11, University of Sydney, NSW, 2006, Australia.
Phys. Rev. E (submitted, June 16, 1997)
Abstract
Formally exact series expressions are derived
for the entropy (information content) of a time-series or signal
by making systematic expansions for the higher order correlation functions
using generalised Kirkwood and Markov superpositions.
Termination of the series after two or three terms provides
tractable and accurate approximations for calculating the entropy.
Signals generated by a Gaussian random process are simulated
using Lorentzian and Gaussian spectral densities,
(exponential and Gaussian covariance functions),
and the entropy is calculated as a function of the correlation length.
The validity of the truncated Kirkwood expansion
is restricted to weakly correlated signals,
whereas the truncated Markov expansion is uniformly accurate;
the leading two terms yield the entropy exactly
in both the limits of weak and of strong correlations.
The concept of entropy for a continuous signal is explored in detail,
and it is shown that it depends
upon the level of digitisation and the frequency of sampling.
The limiting forms are analysed for a continuous signal with
exponentially decaying covariance,
for which explicit results can be obtained.
Explicit results are also obtained for the binary discrete case that
is isomorphic to the Ising spin lattice model.
PACS:
89.70.+c (Information science)
05.50.+q (Lattice theory and Statistics; Ising problems)
02.50.Ga (Markov processes)
02.50.Cw (Probability theory)
Back to publications or to Phil Attard's home page.