Poisson Distribution (1)

[Poisson (click)] by MML, formulated for several data points.

The Poisson distribution with parameter α>0, for n≥0:

P(n) =
e-α αn
n!
   n≥0
NB. n≥0. The expectation equals the variance equals α.

P(n) = (α/n) P(n-1), so P(n) increases while n<α and decreases when n>α, i.e. the mode is α; this is also the maximum likelihood estimate of α given observed data `n'.

The Poisson distribution can be derived (e.g. Meyer 1970) as the distribution of the number of particle decays in a radioactive source in unit time where α is the rate, i.e. where the probability of a decay in a small time interval, dt, is α.dt.

MML

We observe a value of `n' (e.g. n decays in unit time). The negative log likelhood, i.e. -log P(n) is

-log(P(n|α))
= α - n.log α + log n!
(Recalling Stirling's approximation, loge(n!) = n.loge(n)-n+0.5loge(n)+0.5loge(2pi)+..., we see that the message length goes up roughly in proportion with n.log(n).)

The second derivative with respect to the parameter α is n/α2.
The expectation of this over n, i.e. the Fisher information, is

α/α2 = 1/α
The MML estimate of α is that value that minimises the message length
-log(h(α)) -log(p(n|α)) + 1/2 log F(α) +(-log 12 + 1)/2
Expectation
of this prior = A.
For the prior h(α) = (1/A).e-α/A, differentiate the msgLen w.r.t. α and set to zero:
d/d α { -log 1/A + α/A   //from h
+ α - n.log α + log n! //from likelihood
- 1/2 log α //from F
+ (-log 12 +1)/2 }  
= 1/A + 1 - n/α - 1/(2.α)
= 1/A + 1 - (n+1/2)/α
= 0
  
i.e. we make the MML estimate (inference)  α' = (n+1/2) / (1+1/A).
The uncertainty region in the estimate of the parameter is about sqrt(12/F(α')), i.e. sqrt(12 α').

Poisson Process

The Poisson process models, for example, the number of radioactive decays in a given time t:

P(n,t) = 
e-α.t(α.t)n
n!
   n≥0
  
If we observe data `n' over time `t', the MML estimate of α [WD97] is  α' = (n+1/2)/(t+1/A)

Easier

For a more convenient to use MML formulation, with a JavaScript implementation, and for application to datasets of several data values, see [here(click)].

Notes