Poisson Distribution (1)

LA home
Computing
MML
 Discrete
  Integers
   Poisson1

Also see
 Poisson2
[Poisson (click)] by MML, formulated for several data points.

The Poisson distribution with parameter α>0, for n≥0:

P(n) =
e αn
n!
   n≥0
NB. n0. The expectation equals the variance equals α.

P(n) = (α/n) P(n-1), so P(n) increases while n<α and decreases when n>α, i.e. the mode is α; this is also the maximum likelihood estimate of α given observed data `n'.

The Poisson distribution can be derived (e.g. Meyer 1970) as the distribution of the number of particle decays in a radioactive source in unit time where α is the rate, i.e. where the probability of a decay in a small time interval, dt, is α.dt.

MML

We observe a value of `n' (e.g. n decays in unit time). The negative log likelhood, i.e. -log P(n) is

-log(P(n|α))
= α - n.log α + log n!
(Recalling Stirling's approximation, loge(n!) = n.loge(n)-n+0.5loge(n)+0.5loge(2pi)+..., we see that the message length goes up roughly in proportion with n.log(n).)

The second derivative with respect to the parameter α is n/α2.
The expectation of this over n, i.e. the Fisher information, is

α/α2 = 1/α
The MML estimate of α is that value that minimises the message length
-log(h(α)) -log(p(n|α)) + 1/2 log F(α) +(-log 12 + 1)/2
Expectation
of this prior = A.
For the prior h(α) = (1/A).e-α/A, differentiate the msgLen w.r.t. α and set to zero:
d/d α { -log 1/A + α/A   //from h
+ α - n.log α + log n! //from likelihood
- 1/2 log α //from F
+ (-log 12 +1)/2 }  
= 1/A + 1 - n/α - 1/(2.α)
= 1/A + 1 - (n+1/2)/α
= 0
  
i.e. we make the MML estimate (inference)  α' = (n+1/2) / (1+1/A).
The uncertainty region in the estimate of the parameter is about sqrt(12/F(α')), i.e. sqrt(12 α').

Poisson Process

The Poisson process models, for example, the number of radioactive decays in a given time t:

P(n,t) = 
e-α.t(α.t)n
n!
   n≥0
  
If we observe data `n' over time `t', the MML estimate of α [WD97] is  α' = (n+1/2)/(t+1/A)

Easier

For a more convenient to use MML formulation, with a JavaScript implementation, and for application to datasets of several data values, see [here(click)].

Notes

  • [WD97] C. S. Wallace & D. L. Dowe. MML Mixture Modelling of Multi-state, Poisson, von Mises Circular and Gaussian Distributions. Proc. 6th Int. Workshop on Artificial Intelligence and Statistics, pp.529-536, 1997.
www:

↑ © L. Allison, www.allisons.org/ll/   (or as otherwise indicated).
Created with "vi (Linux)",  charset=iso-8859-1,   fetched Sunday, 09-Dec-2018 20:02:39 EST.

Free: Linux, Ubuntu operating-sys, OpenOffice office-suite, The GIMP ~photoshop, Firefox web-browser, FlashBlock flash on/off.