## Introduction to Stochastic Processes with R

CHAPTER 1

INTRODUCTION AND REVIEW

We demand rigidly defined areas of doubt and uncertainty!

-Douglas Adams, The Hitchhiker's Guide to the Galaxy

1.1 DETERMINISTIC AND STOCHASTIC MODELS

Probability theory, the mathematical science of uncertainty, plays an ever growing role in how we understand the world around us-whether it is the climate of the planet, the spread of an infectious disease, or the results of the latest news poll.

The word "stochastic" comes from the Greek stokhazesthai , which means to aim at, or guess at. A stochastic process, also called a random process, is simply one in which outcomes are uncertain. By contrast, in a deterministic system there is no randomness. In a deterministic system, the same output is always produced from a given input.

Functions and differential equations are typically used to describe deterministic processes. Random variables and probability distributions are the building blocks for stochastic systems.

Consider a simple exponential growth model. The number of bacteria that grows in a culture until its food source is exhausted exhibits exponential growth. A common deterministic growth model is to assert that the population of bacteria grows at a fixed rate, say 20% per minute. Let denote the number of bacteria present after minutes. As the growth rate is proportional to population size, the model is described by the differential equation

The equation is solved by the exponential function

where is the initial size of the population.

As the model is deterministic, bacteria growth is described by a function, and no randomness is involved. For instance, if there are four bacteria present initially, then after 15 minutes, the model asserts that the number of bacteria present is

The deterministic model does not address the uncertainty present in the reproduction rate of individual organisms. Such uncertainty can be captured by using a stochastic framework where the times until bacteria reproduce are modeled by random variables. A simple stochastic growth model is to assume that the times until individual bacteria reproduce are independent exponential random variables, in this case with rate parameter 0.20. In many biological processes, the exponential distribution is a common choice for modeling the times of births and deaths .

In the deterministic model, when the population size is , the number of bacteria increases by in 1 minute. Similarly, for the stochastic model, after bacteria arise the time until the next bacteria reproduces has an exponential probability distribution with rate per minute. (The stochastic process here is called a birth process , which is introduced in Chapter 7 .)

While the outcome of a deterministic system is fixed, the outcome of a stochastic process is uncertain. See Figure 1.1 to compare the graph of the deterministic exponential growth function with several possible outcomes of the stochastic process.

Figure 1.1 Growth of a bacteria population. The deterministic exponential growth curve (dark line) is plotted against six realizations of the stochastic process.

The dynamics of a stochastic process are described by random variables and probability distributions. In the deterministic growth model, one can say with certainty how many bacteria are present after minutes. For the stochastic model, questions of interest might include:

What is the average number of bacteria present at time ?

What is the probability that the number of bacteria will exceed some threshold after minutes?

What is the distribution of the time it takes for the number of bacteria to double in size?

In more sophisticated stochastic growth models, which allow for births and deaths, one might be interested in the lik