The law of large numbers and limit theorems

The law of large numbers in a broad sense is understood as the general principle according to which, under the formulation of academician A.N. Kolmogorov, cumulative action of the large number of random factors leads (at the some rather general conditions) to a result almost independent from a case. In other words, at the large number of random variables their average result ceases to be random and can be predicted with the big degree of definiteness.

The law of large numbers in a narrow sense is understood as a series of mathematical theorems, in each of which for those or other conditions the fact of approximation of average characteristics of the large number of trials to some certain constants is established.

Theorem (The Markov inequality). If a random variable X takes only non-negative values and has a mathematical expectation M(X) then for every positive number A the following inequality holds:

P(X > A) £ M(X)/A

 

Since the events X > A and X £ A are opposite, changing P(X > A) by 1 – P(X £ A) we have another form of the Markov inequality:

P(X £ A) ³ 1 – M(X)/A

 

The Markov inequality is applicable for all non-negative random variables.

Example. The average amount of calls entering on the switchboard of a factory within one hour is equal to 300. Estimate the probability that the number of calls on the switchboard within a randomly chosen hour: a) will exceed 400; b) will be no more than 500.

Solution:

a) By the hypothesis M(X) = 300. By the Markov inequality we have P (X >400) £ 300/400, i.e. the probability that the number of calls will exceed 400 is no more than 0,75.

b) By another form of the Markov inequality P(X £ A) ³ 1 – 300/500 = 0,4, i.e. the probability that the number of calls will be no more than 500 is no less than 0,4.

Example. The sum of all deposits in a branch of bank makes 2 million roubles, and the probability that a randomly taken deposit will not exceed 10 thousand roubles is equal to 0,6. What is it possible to tell about the number of depositors?

Solution: Let X be the size of a randomly taken deposit, and n be the number of all deposits. Then by the hypothesis the average size of deposit M(X) = 2000/n (thousand roubles). By the Markov inequality:

Taking in account that P(X £ 10) = 0,6 we obtain 1 – 200/n £ 0,6; and consequently n £ 500, i.e. the number of depositors is no more than 500.

Theorem (The Chebyshev inequality). For every random variable X having a mathematical expectation M(X) and a dispersion D(X) the following inequality holds:

P(|X – a| > e) £ D(X)/e ²

where a = M(X), e > 0.

Since the events |X – a| > e and |X – a| £ e are opposite, the Chebyshev inequality can be written in another form:

P(|X – a| £ e) ³ 1 – D(X)/e ²

 

The Chebyshev inequality is applicable for all random variables.

Example. The average expense of water on a cattle-breeding farm makes 1000 litres in a day, and the mean square deviation of this random variable does not exceed 200 litres. Estimate the probability that the expense of water on the farm in a randomly chosen day will not exceed 2000 litres by using: a) the Markov inequality; b) the Chebyshev inequality.

Solution: a) Let X be an expense of water on a cattle-breeding farm. By the hypothesis M(X) = 1000. Using the Markov inequality we obtain P(X £ 2000) ³ 1 – 1000/2000 = 0,5, i.e. no less than 0,5.

b) The dispersion D(X) = s ² £ 200². Since the boundaries of the interval 0 £ X £ 2000 are symmetric according to the mathematical expectation M(X) = 1000, for estimating the probability of the required event one can apply the Chebyshev inequality:

i.e. no less than 0,96. In the present example an estimating the probability of the event found by the Markov inequality (P ³ 0,5) was made more precisely by the Chebyshev inequality (P ³ 0,96).

The Chebyshev theorem. If the dispersions of n independent random variables X ₁, X ₂, …, X _n are bounded by the same constant C then the arithmetic mean of random variables converges on probability to the arithmetic mean of their mathematical expectations a ₁, a ₂, …, a _n for an unbounded increasing the number n, i.e.

and

 

Underline the sense of the Chebyshev theorem. For a large number n of random variables X ₁, X ₂, …, X _n it is practically certain that their arithmetic mean – a random variable – very little differs from non-random quantity , i.e. it practically ceases to be random.

Corollary. If independent random variables X ₁, X ₂, …, X _n have identical mathematical expectations which equal a, and their dispersions are bounded by the same constant C then

and

 

The Chebyshev theorem and its corollary have the big practical value. For example, it is necessary for an insurance company to establish the size of an insurance payment which the insurant should pay; thus the insurance company undertakes to pay a certain insurance sum at occurrence of an insurance case. Considering frequency/losses of the insurant at occurrence of an insurance case as a random variable and possessing by known statistics of such cases, it is possible to determine the average number/average losses at happening of insurance cases which on the basis of the Chebyshev theorem with the big degree of confidence can be considered as almost non-random quantity. Then the size of an insurance payment is determined on the basis of these data and the prospective insurance sum. Without taking into account action of the law of large numbers (the Chebyshev theorem) essential losses of the insurance company are possible (at understating the size of an insurance payment), or loss of appeal of insurance services (at overestimating the size of a payment).

Example. For determination of the average duration of burning electric lamps in a batch of 200 identical boxes it has been taken on sample on one lamp from each box. Estimate the probability that the average duration of burning the selected 200 electric lamps differs from the average duration of burning lamps in all batch no more than on 5 hours (by absolute value) if it is known that the mean square deviation of duration of burning lamps in each box is less than 7 hours.

Solution: Let X_i be duration of burning a lamp taken from the i- th box (hours). By the hypothesis the dispersion D(X_i) < 7² = 49. Obviously, the average duration of burning the selected lamps is equal to (X ₁ + X ₂ + … + X ₂₀₀)/200, and the average duration of burning lamps in all batch is (M (X ₁) + M (X ₂) + … + M (X ₂₀₀))/200 = (a ₁ + a ₂ + … + a ₂₀₀)/200. Then the probability of the required event is

i.e. no less than 0,9902.

The Bernoulli theorem. The relative frequency of an event in n repeated independent trials, in each of which the event can occur with the same probability p, converges on probability to probability p of the event in a single trial for an unbounded increasing the number n: and

 

 

The Bernoulli theorem gives a theoretical substantiation of replacement of unknown probability of an event by its relative frequency obtained in n repeated independent trials spent at the same complex of conditions. So, for example, if the probability of a birth of boy isn’t known, as its value we can accept the relative frequency of this event which on long-term statistical data makes approximately 0,515.

Example. A coin is tossed 1000 times. Give a lower bound for the probability that the absolute value of a deviation of the relative frequency of appearance of heads from the probability of its appearance is less than 0,1.

Solution: Here n = 1000, p = q = 0,5; e = 0,1. Consequently,

 

The central limit theorem

The central limit theorem is one of the most remarkable results in probability theory. Loosely put, it states that the sum of a large number of independent random variables has a distribution that is approximately normal. Hence it not only provides a simple method for computing approximate probabilities for sums of independent random variables, but it also helps explain the remarkable fact that the empirical frequencies of so many natural populations exhibit bell-shaped (that is, normal) curves.

In its simplest form the central limit theorem is as follows:

Theorem (the central limit theorem). Let X₁, X₂, …, X_n be a sequence of independent and identically distributed random variables each having mathematical expectation a and dispersion s ². Then the distribution of

tends to the standard normal as n ® ¥, i.e. U is a normally distributed random variable with M(U) = 0 and D(U) = 1.

Thus, for – ¥ < t < ¥,

as n ® ¥

 

Example. 100 numbers have been randomly chosen on the segment [0; 1], more precisely 100 independent random variables X ₁, X ₂, …, X _n uniformly distributed over the segment [0; 1] are considered. Find the probability that their sum is between 51 and 60, i.e.

Solution: By the central limit theorem is a normally distributed random variablewith M(U) = 0 and D(U) = 1.

Then

By uniformity of the random variable X_i we have:

Then:

Thus, is a normally distributed random variable with the mathematical expectation na = 50 and the dispersion ns ² = 100/12.

Consequently,

 

Glossary

to cease – перестать, прекращать; cumulative – совокупный

approximation – приближение; switchboard – коммутатор

cattle-breeding farm – животноводческая ферма

to undertake – обязаться, ручаться

to understate – понижать, преуменьшать

appeal – привлекательность; substantiation – толкование

long-term – многолетний; loosely – свободно

Exercises for Seminar 11

11.1. The amount of electric power consumed by a settlement within day is a random variable of which the mathematical expectation is 4 thousands of kWt×h. Estimate the probability that for the nearest day the consumption of energy: a) will exceed 8 thousands of kWt×h; b) will not exceed 6 thousands of kWt×h (settlement – поселок; electric power – электроэнергия; consumption – потребление).

The answer: a) 0,5; b) 0,33.

 

11.2. From past experience a professor knows that the test score of a student taking his or her final examination is a random variable with mathematical expectation 75. Give an upper bound for the probability that a student’s test score will exceed 85. Suppose, in addition, the professor knows that the dispersion of a student’s test score is equal to 25. What can be said about the probability that a student will score between 65 and 85?

The answer: 0,882; 0,75.

 

11.3. A discrete random variable X is given by the following law of distribution:

 

X
P	0,1	0,4	0,3	0,2

 

By using the Chebyshev inequality estimate the probability that |X – M(X)| > 3.

The answer: 2/3.

 

11.4. A random variable is given by the integral function:

(a) By using the Chebyshev inequality estimate the probability that |X – M(X)| < a/2; (b) Determine the probability that |X – M(X)| < a/2.

The answer: a) 7/9; b) 35/36.

 

11.5. By using the Chebyshev inequality estimate the probability that the number of come up seeds among the sown 5000 will be from 3750 up to 4250 if the mathematical expectation M(X) = 4000 (seed – семя; to come up – всходить; sown – посеянный).

The answer: 0,9872.

 

11.6. There are 100 automobiles in an economy. The probability of non-failure operation of each of them during a certain period is equal to 0,9. By using the Chebyshev inequality estimate the probability that the deviation of the number of trouble-free working automobiles for the certain period from its mathematical expectation will not exceed 5 by absolute value (economy – хозяйство).

The answer: 0,64.

 

11.7. By a sampling way the weight of ears of barley is determined. How many ears need to select in order to assert with probability no less than 0,99 that the average weight of randomly selected ears will differ from the average weight of ears in all batch (accepted as a mathematical expectation) no more than on 0,1 g? It has been established that the mean square deviation of the weight does not exceed 0,2 g (ear – колос; barley – ячмень).

The answer: 400.

 

11.8. Let as a result of 100 independent trials values of a random variable X have been found: х₁, х₂, …, х₁₀₀. Let the mathematical expectation M(X) = 10 and the dispersion D(X) = 1. Give a lower bound for the probability that the absolute value of the difference between the arithmetic mean of observed values of the random variable and the mathematical expectation will be less than 0,5.

The answer: 0,96.

 

11.9. There are 100 white and 100 black balls in an urn. 50 balls have been taken out (with returning). Give a lower bound for the probability that the amount of white balls from the taken out balls satisfies to the double inequality: 15 < m < 35.

The answer: 0,875.

 

11.10. An astronomer is interested in measuring, in light years, the distance from his observatory to a distant star. Although the astronomer has a measuring technique, he knows that, because of changing atmospheric conditions and normal error, each time a measurement is made it will not yield the exact distance but merely an estimate. As a result the astronomer plans to make a series of measurements and then use the average value of these measurements as his estimated value of the actual distance. If the astronomer believes that the values of the measurements are independent and identically distributed random variables having a common mathematical expectation a (the actual distance) and a common dispersion of 4 (light years), how many measurements need he make to be guaranteed with probability 0,95 that the deviation of the arithmetic mean of these measurements from the mathematical expectation will be no more than 0,5 light year by absolute value?

The answer: 62.

 

11.11. Let X_i, i = 1, 2, …, 10, be independent random variables, each uniformly distributed over (0, 1). Calculate an approximation to by using the central limit theorem.

The answer: 0,1368.

 

Exercises for Homework 11

11.12. The amount of forages spent on a farm of large horned livestock in a day is a random variable of which the mathematical expectation is 6 tons. Estimate the probability that for the nearest day the expense of forages on the farm will exceed 10 tons (forage – корм; large horned livestock – крупный рогатый скот).

The answer: 0,6.

 

11.13. The number of automobiles sold weekly at a certain dealership is a random variable with mathematical expectation 16. Give an upper bound to the probability that next week’s sales exceed 25. Suppose, in addition, that the dispersion of the number of automobiles sold weekly is 9. Give a lower bound to the probability that next week’s sales are between 10 and 22 inclusively (dealership – представительство).

The answer: 0,64; 0,75.

 

11.14. A discrete random variable X is given by the following law of distribution:

 

X	–1
P	0,1	0,2	0,4	0,2	0,1

 

By the Chebyshev inequality estimate the probability that |X – M(X)| < 2,5.

The answer: 0,5456.

 

11.15. The probability of ripening seeds of vegetable culture in a given district is equal to 0,8. By using the Chebyshev inequality estimate the probability that the number of plants with ripened seeds will make of 1000 plants from 750 up to 850 (ripening – вызревание; plant – растение).

The answer: 0,936.

 

11.16. Suppose that it is known that the number of items produced in a factory during a week is a random variable with mathematical expectation 50.

(a) What can be said about the probability that this week’s production will exceed 75?

(b) If the dispersion of a week’s production is known to equal 25, then what can be said about the probability that this week’s production will be between 40 and 60?

The answer: a) 2/3; b) 3/4.

 

11.17. A random variable is given by the integral function:

(a) By the Chebyshev inequality estimate the probability that |X – M(X)| < a;

(b) Determine the probability that |X – M(X)| < a.

The answer: a) 7/9; b) 35/36.

 

11.18. From past experience a professor knows that the test score of a student taking his or her final examination is a random variable with mathematical expectation 75 and dispersion 25. How many students would have to take the examination to ensure, with probability at least 0,9, that the class average would be within 5 of 75?

The answer: 10.

 

11.19. As a result of 200 independent trials values of a random variable X: х₁, х₂, …, х₂₀₀ have been found, and M(X) = D(X) = 2. Give a lower bound for the probability that the absolute value of the difference between the arithmetic mean of the values of the random variable and the mathematical expectation will be less than 1/5.

The answer: 3/4.

 

11.20. A die is tossed 10000 times. Estimate the probability of the deviation of the relative frequency of occurrence of six aces from the probability of occurrence of the same number of aces will be less than on 0,01.

The answer: 0,861.

 

11.21. Use the central limit theorem to solve Exercise 11.18.

The answer: 3.

 

11.22. Let X₁, X₂, …, X₂₀ be independent Poisson random variables with mathematical expectation 1. Use the central limit theorem to approximate

The answer: 0,1318.

 

 

L E C T U R E 12