Feller Processes and the Strong Markov Property

Markov Property

We go way back to the Part 1B short course Markov Chains. In the first lecture of this course, we met discrete time Markov Chains. A definition was given, in terms of conditional single period transition probabilities, and it was immediately proved that general transition probabilities are specified by sums of products of entries in a so-called transition matrix. This proves very useful for performing calculations. But the question will inevitably be asked: “Prove that this is a Markov process.” And the answer “because it obviously is” isn’t good enough.

The point is that all of the above is relevant to the setting, but the ideal definition of a Markov process is something like the very general statement:

Conditional on the present, the past and the future are independent.

This opens up the possibility of a large class of processes being Markov processes. A technical definition would be that for s<t and A a measurable subset of the state space.

$\mathbb{P}(X_t\in A|\mathcal{F}_s)=\mathbb{P}(X_t\in A|\sigma(X_s))$ .

It is easy to check that this is equivalent to the original definition in the that context.

Strong Markov Property

SMP states that given a stopping time T, conditional on the event $\{T<\infty\}$ :

$(X_{T+t}-X_T, t\geq 0)\stackrel{d}{=}(X_t^0,t\geq 0)$ ,

that is, the process started at time T has the same distribution as the original process started from 0 (in space as well as time). Furthermore, it is independent of $\mathcal{F}_T$ , which requires technical definition, but which informally is the sigma field of events defined by what happens up to time T.

For a discrete time Markov chain, prove SMP by pre-multiplying by the indicator function $1(T=n)$ , which reduces SMP to the normal Markov property. Then take the countable sum over n (which is permissible) to get SMP. For Brownian Motion in one dimension, make dyadic approximations to the stopping time from above. SMP applies to these approximations, and measure theoretic machinery and the (almost sure) continuity of paths allows the equivalence of distributions to hold in the limit. Independence follows by expressing $\mathcal{F}_T=\cap \mathcal{F}_{T_n}$ as the intersection of sigma fields corresponding to the approximations.

In both of these cases, an implicit countable substructure (discrete time and continuity respectively) have been required to deduce SMP. This suggests that there are Markov processes which do not have SMP.

Motivating Counterexample

Take B to be a Brownian Motion in one dimension, with $B_0$ a RV which contains 0 in its support. Now define the the process:

$X_t=B_t1_{B_0\neq 0}$ .

Then X is certainly not Strong Markov, by considering the hitting time of 0. Then the process started there is either identically 0, or a standard BM, but which is determined by time 0 properties rather than time T properties.

But X is Markov. Take s<t and A Borel. Then:

$\mathbb{P}(X_t\in A|\mathcal{F}_s)=\mathbb{E}[1(X_t\in A\cap X_0\neq 0)+1(X_t\in A\cap X_0=0)|\mathcal{F}_s]$

$=1(X_0\neq 0)\int_A \frac{1}{\sqrt{2\pi(t-s)}}\exp(-\frac{(X_s-y)^2}{2(t-s)})dy+1(X_0=0)1(0\in A)$

$1(X_s\neq 0)\int_A(\ldots)dy + 1(X_s=0,X_0\neq 0)\int_A(\ldots)dy + 1(0\in A)[1(X_s=0)-1(X_s=0,X_0\neq 0)]$

Now $1(X_s=0, X_0\neq 0)=0$ a.s. so

$= \mathbb{E}[1(X_t\in A)|X_s]$ , which is precisely the Markov property.

Feller Property

In general, it hard to verify the Strong Markov Property for a given stochastic process. Instead, we consider a property which is stronger, but easier to check.

Given a Markov process X(x,t) with initial distribution x. X has the Feller property if for all t,

whenever $x_n\rightarrow x$ then $X(x_n,t)\stackrel{d}{\rightarrow}X(x,t)$ .

In other words, the distribution of the process at time t is dependent continuously on the initial distribution under a suitable topology. We also demand that

As $t\rightarrow 0, X(x,t)\stackrel{P}{\rightarrow} x$ ,

which means that starting from any point, paths are cadlag. (Remember that convergence in probability to a constant is the same as almost sure convergence)

Examples

1) The discrete time, finite state space Markov Chain as initially discussed has the Feller property. Time m distributions are given by repeating applications of the transition matrix:

$X(x,m)\stackrel{d}{=} P^m x$ .

But P is a finite-dimensional linear map, and so is certainly continuous.

2) Brownian Motion can be described as the sum of a standard BM and a random variable describing the starting point. The standard BM disappears from the criterion for the Feller property as convergence in distribution is preserved under addition, and the remaining RV is time-independent so has the property.

Feller implies Strong Markov

Define the transition operator:

$P_tf(x)=E^x f(X_t)$ .

The Feller property say that for all bounded measurable f, this is continuous as a function of x, including when x=Z a random variable. P must also be right-continuous as a function of t at 0. We want to prove that for Z bounded $\mathcal{F}_T$ -measurable:

$E[f(X_{T+t}Z]=E[P_tf(X_T)Z]$ .

This holds for $T_n$ , discrete approximations to T from above by the discrete time SMP result. Then want:

$E[f(X_{T+t})Z]=\lim_n E[f(X_{T_n+t})Z]=\lim_n E[P_tf(X_{T_n})Z]=E[P_tf(X_T)Z]$ .

The first equality follows by right-continuity and bounded convergence, and the final equality by right-continuity and the Feller property.

Feller = Strong Markov?

A final remark worth making is that the Feller property is genuinely stronger than SMP. Consider a process defined for times $(t\geq 0)$ with some initial distribution and evolution given by:

$X_t=X_0$ when $X_0\geq 0$ and $X_t=X_0-t$ otherwise.

This is not Feller because if the initial state moves from being fixed and negative to fixed and zero, the (deterministic) distribution at time 1 has a discontinuity. But it is Strong Markov.

However, it is reasonable to say that this example is quite contrived. In general, we aim to prove a process is Feller in order to use the Strong Markov Property.

References

Dynkin, Yushkevich – Strong Markov Processes (1955)

Breiman – Probability (Chapter 16)

Pitman – Notes, in particular the Markov but not Strong Markov counterexample.

http://almostsure.wordpress.com/2010/07/19/properties-of-feller-processes/

4 thoughts on “Feller Processes and the Strong Markov Property”

Pingback: Strong Markov Property for BM | Eventually Almost Everywhere
Pingback: Dominic Yeo
Pingback: Reversing Markov Chains | Eventually Almost Everywhere
Matti Kiiski on February 16, 2020 at 10:29 am said:

A nice blog post. Thank you!

By the way, there exists a small typo in the second display of the subsection “Feller implies Strong Markov”, if you want to clean it out.

Reply ↓

Eventually Almost Everywhere

A blog about probability and olympiads by Dominic Yeo

Feller Processes and the Strong Markov Property

4 thoughts on “Feller Processes and the Strong Markov Property”

Leave a Reply Cancel reply

Share this:

Like this:

Related

4 thoughts on “Feller Processes and the Strong Markov Property”

Leave a Reply Cancel reply