|
|
The Poisson distribution with parameter α>0, for n≥0:
-
NB. n≥0.
The expectation equals the variance equals α.
P(n) = (α/n) P(n-1),
so P(n) increases while n<α and decreases when n>α,
i.e. the mode is α;
this is also the maximum likelihood estimate of α
given observed data `n'.
The Poisson distribution can be derived (e.g. Meyer 1970)
as the distribution of the number of particle decays in a radioactive source
in unit time where α is the rate,
i.e. where the probability of a decay
in a small time interval, dt, is α.dt.
MML
We observe a value of `n' (e.g. n decays in unit time).
The negative log likelhood, i.e. -log P(n) is
- -log(P(n|α))
- = α - n.log α + log n!
(Recalling
Stirling's
approximation,
loge(n!)
= n.loge(n)-n+0.5loge(n)+0.5loge(2pi)+...,
we see that the message length goes up roughly in proportion with n.log(n).)
The second derivative with respect to the parameter α is
n/α2.
The expectation of this over n, i.e. the
Fisher
information, is
- α/α2 = 1/α
The MML estimate of α is that value that minimises the message length
-
-log(h(α)) -log(p(n|α))
+ 1/2 log F(α) +(-log 12 + 1)/2
Expectation
of this prior = A.
|
For the prior h(α) = (1/A).e-α/A,
differentiate the msgLen w.r.t. α and set to zero:
-
| d/d α { |
-log 1/A + α/A | |
//from h |
| + α - n.log α + log n! |
//from likelihood |
| - 1/2 log α |
//from F |
| + (-log 12 +1)/2 } |
|
- = 1/A + 1 - n/α - 1/(2.α)
- = 1/A + 1 - (n+1/2)/α
- = 0
i.e. we make the MML estimate (inference)
α' = (n+1/2) / (1+1/A).
The uncertainty region in the estimate of the parameter is about
sqrt(12/F(α')),
i.e. sqrt(12 α').
Poisson Process
The Poisson process models, for example, the number of radioactive decays
in a given time t:
-
If we observe data `n' over time `t',
the MML estimate of α [WD97] is
α' = (n+1/2)/(t+1/A)
Easier
For a more convenient to use MML formulation,
with a JavaScript implementation, and
for application to datasets of several data values,
see [here(click)].
Notes
- [WD97] C. S. Wallace & D. L. Dowe.
MML Mixture Modelling of Multi-state, Poisson, von Mises Circular
and Gaussian Distributions.
Proc. 6th Int. Workshop on Artificial Intelligence and Statistics,
pp.529-536, 1997.
|
|