I’ve been reading through Crispin Gardiner’s `Handbook on stochastic processes’. It’s a really dense but helpful book, designed sort of as a reference as opposed to a teaching tool. There are plenty of calculations worth going through, but the very first one I wanted to redo is using generating functions to derive the Poisson process, that’s the probability distribution generated from a process that has a mean rate of happening in a given time interval. The events must be independent. I’ll frame the process in terms of adding a single bean to a pot with mean chance in a given time interval .
Thus, the probability we consider is a function of time and number of beans . The probability that the number of beans is increased by 1 in the time interval is
That means the probability that stays constant in the time interval is made up of the addition of two probabilities: the complement of the probability of having and increasing the number, and the probability that we orginally had beans and one was added. Together then
we can expand and factor and then as we have a partial differential equation (PDE) for the probability:
We can solve this partial differential equation, for example, by using the method of generating functions. We define a probability generating function (pgf) as a power law sum:
The reason for calling it a generating function is intuitive. Taking the first derivative with respect to and evaluating at gives us
where we had to readjust the sum to avoid division by zero. It is clear then that
so taking derivatives of the pgf and evaluating at 0 gives us the probability of having 1 bean over time. Likewise the second derivative evaluated at zero gives us the probability of having 2 beans
so that the general formula for the probability of having beans is
and this function `generates’ the probabilities.
Now, we can return to solving the PDE. Writing the time derivative of the pgf
we now substitute in the time rate-change of the probability from the PDE we are interested in solving
the right hand side sum can be written out using the notation $p(s,t)=p_s$
and grouping terms
so that for this is approximately
thus the solution is
If we assume that at , we have and . That means that . We can write
and expand in powers of to obtain
so that we may identify obtaining the solution to the Poisson process for the number of beans in a jar if they are added randomly with a mean number in a time interval to be
The form of this probability distribution might be familiar.