This is a summary of what I covered in class April 8 2008 
This is a global recap of the continuous time processes we have worked with.

In class I covered PP, BP and BDP first, because they are natural processes 
to model many real life problems. So it is easy to motivate them. However, they are
all special cases of MC. 
The following presents a global framework to think about these processes.


RECAP for PP, BP, BDP, and cont time MC
=======================================

(1) The quantities we work with
-------------------------------

* p_{ij}(t) = P( X(t+s) = j | X(s) = i ) = P( X(t) = j | X(0) = i ) stationarity
  often hard to get except at the limit when t tends to infinity

* p_{ij}(h) = P( X(h+s) = j | X(s) = i ) = P( X(h) = j | X(0) = i )
  as h tends to 0
  often easier to set up by modelling the properties of the process under study

* Generator G = P'(t=0)
  Also useful to remember P(h) = I + G.h + o(h)

* length of stay in state r: Y_r ~ exp( - g_{rr} )
  where g_{rr} takes special values when the process is a "special" process
  such as a PP, BP, BDP

* The jump chain


All these quantities can be calculated for all the processes we 
worked with, even though, for example, we never used the generator of a
Poisson process before today.


(2) How do these quantities relate to the quantities we handled with discrete time MC
-------------------------------------------------------------------------------------

They don't, although the following relationships have the same flavor:

discrete time : mu(n) = mu(0) P^n
  where P^n is the n-step transition probability matrix
  and mu(n) = ( P(X_n=i), i in the state space S )


cont time: mu(t) = mu(0) P(t)
  where mu(t) = ( P(X(t)=i), i in the state space S )


(3) The jump chain
------------------

Look at the path of the process (draw the path) and focus on a time 
where a transition occurred. Call that time t, and say that the process
is in state i before the transition.
The probability that the process jumpED from i to j right after the jump happened
is

qij(h) = P( X(t+h)=j | X(t)=i , "the chain jumped") 
which is 
qij(h) = P( X(t+h)=j , "the chain jumped" | X(t)=i ) / P( "the chain jumped" | X(t)=i)
using basis laws of probability.

Then qii(h) = 0, since you cannot at the same time have a jump and no jump
For j different form i,
qij(h) = P( X(t+h)=j , "the chain jumped" | X(t)=i ) / P( "the chain jumped" | X(t)=i)
       = P( X(t+h)=j | X(t)=i ) / P( "the chain jumped" | X(t)=i)
since   X(t+h)=j implies "the chain jumped"

P( X(t+h)=j | X(t)=i ) = gij.h + o(h)
and P( "the chain jumped" | X(t)=i) = 1 - P( X(t+h)=i | X(t)=i )
                                    = 1 - (1 + gii.h + o(h) ) = - gii.h + o(h)
using the fact that P(h) = I + G.h + o(h)

Note that P( "the chain jumped" | X(t)=i) is also the sum over j different from i
  of the numerator P( X(t+h)=j | X(t)=i ) 

Hence 
qij(h) =  ( gij.h + o(h) ) / ( - gii.h + o(h) )

Divive by h and let h tend to 0 to get the instantaneous jumping probabilities

qij = gij / (-gii)


example 1: determine the jump chain for a PP
example 2: determine the jump chain for a BDP


(4) How to obtain the infinitesimal transitions P(h) from P(t)
--------------------------------------------------------------

P(h) = I + G.h +o(h)
where G = P'(0)

Or directly, take P(t) and let t tend to zero, and collect the leading term in t


example 1: Let P(t) be the matrix of probabilities P(X(t)=j | X(0)=i), where X(t)
  is the number of events of a PP
  Calculate G from P(t)
  Calculate P(h)
  Verify that it matches the infinitesimal definition we gave for the PP


(5) How to obtain P(t) from P(h)
--------------------------------

The only way is to set the forward differential equations
P( N(t+h) = j ) = \sum P( N(t+h) = j | X(t) = i )  P( X(t) = i )
etc... 
as done many times in class.

You may verify that for all the processes we worked with, the set of 
differential equations can be written as 

P'(t) = P(t) G

where G is the generator of the process. 


example 1: verify this for the PP
example 2: verify this for the BDP

Note: writing P'(t) = P(t) G is a convenient way to write all the diffential 
  equations at once.


Solving P'(t) = P(t) G
----------------------
Either directly or using P(t) = \sum t^n G^n / n!
See for example problem 6.9.1 in HW5


(6) How to obtain G
-------------------

Either G = P'(0)
This assumes that you know P(t), which will be rare

Or, if you know the infinitesimal set up of the process, identify G 
from P(h) = I + G.h +o(h)
This will likely be the possible route


(7) Limiting distributions
++++++++++++++++++++++++++

(a) How to find it
------------------

find the limit of  p_{ij}(t) when t tends to infinity to get pi_j
OR
solve PI.G = 0 to get all the pi_j at once

(b) When does it exist
----------------------
All states must communicate
and
PI must be a proper distribution, i.e. all pi_j must be positive and
they must sum to 1

example 1: the states of a PP do not all communicate. There is no proper limiting
  distribution (the process wanders off to infinity)

example 2: BDP: a limiting distribution exists depending on the values of the birth 
  and death rates. Several examples were covered in class. Generally, if the death rates
  are greater than the birth rates, the process will be pulled towards 0 and a limiting 
  distribution exists. If the birth rates are higher than the death rates, the process
  will wander away to infinity --> no limiting distribution.

  
==========================================================================================
==========================================================================================
Few more comments on the practice exam questions as they arose during my OH

(Q4) Similar calculations for (a,b,c,g) were done once in class and once in HW5.

 (e) We established in class that what we want is p_{i0}(t) = P( X(t)=0 | X(0)=i )
     You just derived g_i(s,t)
     In HW5, you expanded this generating function into a polynomial in s so you could identify
     at once p_{ij}(t) for all j.
     Here, all we want is p_{i0}(t), so no need for the expansion. You simply get it by evaluating 
     g_i(s,t) at s=0 

 (f) Here you need pi_0. Get it by taking the limit of p_{i0}(t) as t goes to infinity.

     The (non recommended here) alternative would be to get the whole limiting distribution 
     PI, and collect from it the first element pi_0. 
     To get PI, you would need to solve PI.G=0, which means you need G=P'(0), which means you need
     P(t), which means you need to expand the generating function as a polynomial in s to get
     all p_{ij}(t)
     Actually, you could also get G from the infinitesimal definition of the process since 
     P(h) = I + Gh + o(h)
     You know P(h), so you can identify G