Poisson Distribution (2)

P(x|α) = e^-α α^x / x!,    integer x≥0,

has one parameter, α, and mean = variance = α.

Note, P(x|α) = P(x-1|α) . α / x

so P(x|α) increases with x while x<α and decreases when x>α.

 

Given n data, x₁, x₂, ..., x_n, the likelihood

P(x₁, x₂, ..., x_n | α) = e^-n.α.α^∑x_i / (x₁! ... x_n!)

 

neg log likelihood,

L = - log P(x₁, x₂, ..., x_n | α)

= n.α - (∑x_i)logα + log x₁! + ... + log x_n!

1st derivative

d L / d α = n - (∑x_i)/α

Equating this to zero, αmaxLH = (∑xi) / n.

2nd derivative

d² L / d α² = (∑x_i) / α²

which has expectation, i.e., Fisher information, F_α = nα/α² = n/α,

note   +log F_α = log n - log α

 

Assume prior,   h α = (1/A).e^-α/A,   which has mean A.

Note   - log(h α) = log A + α/A.

 

Message length,

m = - log(h α) + L + 1/2 log F_α + (-log 12 +1)/2

 

To estimate α, differentiate m with respect to α

d m / d α = 1/A + n - (∑x_i)/α - 1/(2α)

equate to zero

α_MML = (∑x_i + 1/2) / (n + 1/A)

uncertainty region sqrt(12/F_αMML) = sqrt(12 α_MML / n)

-- LA, 3/7/2007

Some sanity checks:

1. If it happens that x₁ = ... = x_n = x then α_MML -> x, and the uncertainty region ->0 as n->∞.
2. If x₁ = ... = x_n = 0 then α_MML->0 as n->∞.
3. α_MML -> α_maxLH = ∑x_i/n as n->∞.

See [IP 1.2] for an implementation.