Computational Stochastic Processes

This page contains some of the material for the course MATH97016/MATH97094: Computational Stochastic Processes. The course material was initially developed by Grigorios Pavliotis, and it later received improvements from Andrew Duncan and Susana Gomes. In particular, the lecture notes linked below were assembled by Andrew Duncan, and the handwritten part of the notes distributed weekly were written by Susana Gomes.

Lectures
Office hours: Tuesday 15:00-16:00, or by email appointment.

Lecture notes

Lecture notes (from 2016): [pdf]
Last modified: 27 Apr 2020.
List of examinable proofs: [pdf]
Last modified: 13 May 2020.
Introduction (based on slides by Susana Gomes): [pdf]
Last modified: 25 Jan 2020.
W1: Random number generation and introduction to Monte Carlo simulation: [pdf]
Last modified: 17 Jan 2022.
W2: Variance reduction techniques for Monte Carlo simulation: [pdf]
Last modified: 03 Feb 2020.
W3: Simulation of continuous-time Markov processes: [pdf]
Last modified: 03 Feb 2020.
W4: Simulation of Gaussian processes (cont'd) and SDEs: [pdf]
Last modified: 10 Feb 2020.
W5: Numerical methods for SDEs: [pdf]
Last modified: 06 Mar 2020.
W6: Numerical methods for SDEs (cont'd): [pdf]
Last modified: 24 Mar 2020.
W7: Inference for SDEs: [pdf]
Last modified: 04 May 2020.
W8: Inference (cont'd) and markov chains: [pdf]
Last modified: 10 May 2020.
W9: Markov chain Monte Carlo: [pdf]
Last modified: 15 Mar 2020.

Problem sheets and coursework

Problem sheet 1: questions: [pdf]
Last modified: 01 Oct 2020.
Problem sheet 1: solutions: [pdf]
Last modified: 01 Oct 2020.
Problem sheet 1: codes: [Python] [Jupyter] [HTML]
Last modified: 03 Feb 2020.
Problem sheet 2: questions: [pdf]
Last modified: 28 Apr 2020.
Problem sheet 2: solutions: [pdf]
Last modified: 28 Apr 2020.
Problem sheet 2: codes: [Python] [Jupyter] [HTML]
Last modified: 16 Apr 2020.
Assessed coursework: [pdf]
Last modified: 17 Jan 2022.
Assessed coursework: solution to Problem 1: [Python] [Jupyter] [HTML]
Last modified: 18 Apr 2020.
Assessed coursework: solution to Problem 2: [Python] [Jupyter] [HTML]
Last modified: 21 Apr 2020.
Assessed coursework: solution to Problem 3: [Python] [Jupyter] [HTML]
Last modified: 16 Apr 2020.
Mock exam: [pdf]
Last modified: 27 Apr 2020.
Mock exam: brief solutions: [pdf]
Last modified: 13 May 2020.

Hints for the coursework:

In the bonus question of Problem 1, asymptotic stability has a precise meaning different from mean-square stability. You may find exercise 2 in the weekly handout of w5 useful for this part.
For Question 2.3, the numerical scheme that you are asked to derive is not the Euler-Maruyama scheme. A formal proof that the weak error is zero is not required, but if you already wrote one I will factor in your work in the calculation of the bonus mark for the problem.

A hint for deriving the scheme is the following: write \( X(\Delta t) \) — the exact solution at time \( \Delta t \) — as a function of \( X(0) \), and notice that \( X(\Delta t) \) can be written as \( X(\Delta t) = \mu + a(\Delta t) \, (X(0) - \mu) + I \), where \( I \) is some Itô integral. Then, use Question 2.2 to calculate the mean and variance of \(I\) explicitly and thereby deduce that \( I = b(\Delta t) \, \xi \) in law, for some appropriate \(b(\Delta t)\). Therefore \( X(\Delta t) = \mu + a(\Delta t) \, (X(0) - \mu) + b(\Delta t) \, \xi \) in law, which suggests an update formula for the numerical scheme.
Several students encountered overflow errors for the numerical calculation of the likelihood in Problem 2. My suggestion, for those of you who encounter this issue, is that you obtain the likelihood by taking the exponential of the log-likelihood, possibly shifted vertically (which corresponds to multiplying the likelihood by a constant factor) to ensure that no overflow error occurs and that the exponential is not everywhere equal to zero.
There was a short discussion with some of you on the approach for Question 3.3, just before a lecture a few weeks ago. My initial suggestion, for the first identity, was that you apply the chain rule to the function \( f(X_t) = X_1(t) \, X_2(t) \), where \( X_t = (X_1(t), X_2(t)) \), with \( X_1(t) = t - s \) and \( X_2(t) = W_t - W_s \). To this end, notice that \( X_t \) is the solution to the following Stratonovich SDE: \begin{align} dX_1(t) = dt, \qquad X_1(s) = 0, \\ dX_2(t) = 1 \circ dW(t), \qquad X_2(s) = 0. \end{align}

As some of you pointed out, there is an alternative approach based on applying a more general version of the chain rule (to allow an explicit dependence of \(f\) on \(t\)) to the function \( f(t, X_t) = (t - s) \, X_t \), where \( X_t = W_t - W_s \). Both approaches are fine.

If you suspect that a bug in your code is preventing you from advancing to the next question, feel free to email me the problematic bit of code and I'll see if can spot possible typos.

Python examples

If you find bugs in the codes, or if you worked on a code example that could benefit the class, do not hesitate to drop me an email. Alternatively, you can open an issue or a pull request at the git repository.

Introduction - Calculating Pi: [Python]
Last modified: 03 Mar 2020.
Introduction: Simulation of Langevin dynamics: [Python]
Last modified: 23 Jan 2020.
W1 - Linear congruential generator: [Python] [Jupyter] [HTML]
Last modified: 29 Jan 2020.
W1 - Generating nonuniform random numbers: [Python] [Jupyter] [HTML]
Last modified: 29 Jan 2020.
W2 - Monte Carlo simulation: [Python] [Jupyter] [HTML]
Last modified: 29 Jan 2020.
W2 - Variance reduction: [Python] [Jupyter] [HTML]
Last modified: 03 Feb 2020.
W3 - Simulation of Gaussian processes: [Python] [Jupyter] [HTML]
Last modified: 27 Apr 2020.
W4 - Fast simulation of stationary Gaussian processes: [Python] [Jupyter] [HTML]
Last modified: 05 Feb 2020.
W5 - Numerical methods for SDEs: [Python] [Jupyter] [HTML]
Last modified: 17 Apr 2020.
W6 - Some applications of numerical methods for SDEs: [Python] [Jupyter] [HTML]
Last modified: 03 Mar 2020.
W7 - Inference for SDEs: [Python] [Jupyter] [HTML]
Last modified: 03 Mar 2020.
W8 - Introduction to Markov chains: [Python] [Jupyter] [HTML]
Last modified: 12 Mar 2020.
W9 - Markov chain Monte Carlo: [Python] [Jupyter] [HTML]
Last modified: 25 Mar 2020.