Integer Distributions 

This section considers positive integers, n≥1, unless otherwise stated (if you have nonnegative integers, n≥0, just add one before encoding and subtract one after decoding. If you also have negative integers interleave them 0, +1, 1, +2, 2, ... , say.) Positive integers form the first, and perhaps the most fundamental, infinite dataspace. A code over positive integers can be used to transmit data from any enumerable dataspace. The Geometric and Poisson distributions are examples of parameterised probability distributions; the probability distributions below have no parameters. 1/(n(n+1))A parameterless probability distribution for positive integers:
The expectation of the distribution is infinite:
1/nNote that pr(n) ~ 1/n cannot be a proper probability distribution because
Elias omega, log_{2}^{*} and Relatives
If you know that an integer, n, lies in the interval [1,N] (or in [0,N1]) then it can be encoded in log_{2}(N) bits, (and this is an optimal code if the probability distribution is uniform). What to do when there is no such bound N? Obviously transmit the length of the codeword for n first. But how to transmit the length? Transmit its length first, of course! A sound code can in fact be based on this intuitive idea; note that log^{k}(n) decreases very rapidly as k increases. The leading bit of n is necessarily “1” so there is no need to transmit it, except that it can be used as a flag to determine whether the current value is a length or the final value of n proper; lengths are thus given a leading “0”. Such a prefix code can be used to code integers of arbitrary size. Unfortunately the length of a codeword as a function of n is neither convex nor smooth although it is monotonic increasing^{+}.
The code above is valid, but not at all efficient. In fact we can do better, that is, achieve a nonredundant code, by using lengths minus one:
The probability distribution pr(n) has an infinite expectation: the probability of n is greater than under the 1/(n.(n+1)) distribution (which has an infinite expectation) for large n. Use the HTML FORM below to encode an integer 'n': Rissanen (1983) gives, r(n),
Notes
Wallace Tree CodeThe definition of (strict) binary trees is:
Note, a codeword always contains one more zero than it has ones. This allows the end of a codeword to be detected. It also allows the word to be decoded. Note for example that '1' and '10' are not codewords. The code is efficient in the sense that the sum over all asuch words, w, of 2^{w} is one. The code is equivalent to giving a tree with code w a probability of 2^{w}: This would be difficult to prove combinatorially, but consider an infinitely long random string over {0,1}. Now 0 (pr=0.5) and 1 (pr=0.5) can be taken as the steps in a random walk – 0 left and 1 right, say. It is well known that a onedimensional random walk returns to the starting point with probability 1.0 (so it will also visit a point next to the start with probability 1.0). There will be a first point in the string at which there has been one more 0 than there have been 1s. Take the prefix up to and including that point as a codeword. Repeat. In this way codewords are generated, each such word w with probability 2^{w}. Every possible codeword appears eventually. The sum of the probabilities of all possible codewords is one. This can be used as the basis of algorthms to encode and decode positive integers: Enumerate codewords in order of increasing length and within that, for a given length, lexicographically say. Use the nth codeword as the codeword for integer n: The first code word of length 2k+3, k≥0, is 1(01)^{k}00 and the last is 1^{k+1}0^{k+2}. We see that the codeword lengths increase in smaller and more regular jumps (of two bits) than is the case for the Elias and log^{*} codes. Notes


↑ © L. Allison, www.allisons.org/ll/ (or as otherwise indicated). Created with "vi (Linux)", charset=iso88591, fetched Thursday, 02Feb2023 14:43:47 UTC. Free: Linux, Ubuntu operatingsys, OpenOffice officesuite, The GIMP ~photoshop, Firefox webbrowser, FlashBlock flash on/off. 