## Introduction to Probability,

1

The Concept of Probability

Andrey Nikolaevich Kolmogorov (Tambov, Russia 1903-Moscow 1987)

Source : Keystone-France / Getty Images.

Regarded as the founder of modern probability theory, Kolmogorov was a Soviet mathematician whose work was also influential in several other scientific areas, notably in topology, constructive logic, classical mechanics, mathematical ecology, and algorithmic information theory.

He earned his Doctor of Philosophy (PhD) degree from Moscow State University in 1929, and two years later, he was appointed a professor in that university. In his book, Foundations of the Theory of Probability , which was published in 1933 and which remains a classic text to this day, he built up probability theory from fundamental axioms in a rigorous manner, comparable to Euclid's axiomatic development of geometry.

1.1 Chance Experiments - Sample Spaces

In this chapter, we present the main ideas and the theoretical background to understand what probability is and provide some illustrations of the way it is used to tackle problems in everyday life. It is rather difficult to try to answer the question "what is probability?" in a single sentence. However, from our experience and the use of this word in common language, we understand that it is a way to deal with uncertainty in our lives. In fact, probability theory has been referred to as "the science of uncertainty"; although intuitively most people associate probability with the degree of belief that something may happen, probability theory goes far beyond that as it attempts to formalize uncertainty in a way that is universally accepted and is also subject to rigorous mathematical treatment.

Since the idea of uncertainty is paramount when we discuss probability, we shall first introduce a concept that is broad enough to deal with uncertainties in a wide-ranging context when we consider practical applications. A chance experiment or a random experiment is any process which leads to an outcome that is not known beforehand. So tossing a coin, selecting a person at random and asking their age, or testing the lifetime of a new machine are all examples of random experiments.

Definition 1.1

A sample space of a chance experiment is the set of all possible outcomes that may appear in a realization of this experiment. The elements of are called sample points for this experiment. A subset of is called an event .

An event , consisting of a single sample point, i.e. a single outcome , is called an elementary event . We use capital letters , and so on to denote events. 1 If an event consists of more than one outcome, then it is called a compound event .

The following simple examples illustrate the above concepts.

Example 1.1

Perhaps the simplest example of a chance experiment is tossing a coin. There are two possible outcomes - Heads (denoted by and Tails (denoted by . In this notation, the sample space of the experiment is

If we toss two coins instead, there are four possible outcomes, represented by the pairs . The sample space for this experiment is thus

Here, the symbol means that both coins land Heads, while means that the first coin lands Heads and the second lands Tails. Note in particular that we treat the two events and as distinguishable, rather than combining them into a single event. The main reason for this is that the events and are elementary events, while the event "one coin lands Heads and the other lands Tails," which contains both and , is no longer an elementary event. As we will see later on, when we assign probabilities to the events of a sample space, it is much easier to work with elementary events, since in many cases such events are equally likely, and so it is reasonable the same probability to be assigned to each of them.

Consid