Probability/Combinatorics
What is combinatorics?Edit
Combinatorics involves the counting and enumeration of elements of sets and similar structures such as sequences and multisets. We have discussed set theory in the chapter about set theory, and we will briefly discuss what is sequence and multiset.
Roughly speaking, a sequence is like a set, but ordering of elements matters, and a multiset is also like a set, but repetition of an element is allowed.
Sequence corresponds to the discussion about ordered selection without replacement, while multiset corresponds to the discussion about unordered selection with replacement.
Fundamental counting principlesEdit
Proof. Idea:
 To find the cardinality of the union of sets:
 Include the cardinalities of each of the sets.
 Exclude the cardinalities of the pairwise intersections (if needed).
 Include the cardinalities of the triplewise intersections (if needed).
 Exclude the cardinalities of the quadruplewise intersections (if needed).
 Include the cardinalities of the quintuplewise intersections (if needed).
 Continue, until the cardinality of the tuplewise intersection is included (if is odd) or excluded (if is even).
Remark.
 The formula can be written more compactly as
 The definition of will be discussed later in this chapter.
 The formula is usually used for the case and .
 When , the formula becomes .
 When , the formula becomes .
 The name 'inclusionexclusion principle' comes from the idea that the principle is based on overgenerous inclusion, and then followed by compensating exclusion.
Example. Among 140 people, 110 of them speak at least one of English, French and German. Given that
 90, 30, 42 of them speak English, French, German respectively;
 23 speak English and French;
 25 speak English and German;
 16 speak French and German.
Then, the no. of people that speak English, French and German is .
Proof. Let , , be the set containing people speaking English, French and German respectively. Then, by inclusionexclusion principle,
Venn diagram
** 90131211=54  < E  ***  2512=13 4213124=13  < G *****   12 1612=4    ***  < F  2312=11  3011124=3  *** 140110=30
Theorem. (Multiplication counting principle) If trial has possible outcomes respectively, then the trials have possible outcomes.
Proof. First, consider the case for : we can enumerate each possible outcomes using ordered pair, as follows:
After establishing the case for , we can establish the case for positive integer inductively, e.g.:
Remark.
 It is also known as rule of product.
Example.
of two sets, with cardinality, ]]The tree diagram of Figure 3 illustrates this for , , and . The number of possible outcomes is
Remark.
 this might be visualized by imagining a flip of threesided die (with three outcomes, e.g. 1,2,3), followed by a flip of a twosided coin (with two outcomes, e.g. A,B).
Counting the number of elements in a power setEdit
Example. (Number of elements in a power set) The number of elements in a power set of set with elements is .
Proof. Consider the elements in one by one. For each of them, we can either include or do not include it in a subset of . Then, there are steps involved to construct a subset of , and each step has two outcomes. It follows from the multiplication counting principle that the steps have outcomes. That is, there are possible (distinct) subsets of . Since power set contains all subsets of by definition, it follows that the power set has elements.
Remark.
 is arbitrary nonnegative integer
The counting principle misusedEdit
Figures 4 and 5 illustrate the fact that the counting principle is not always useful. Figure 4 calculates the ways the three integers can be added to five, if the integers are restricted to the set . Since these three integers are choices (decisions), it is convenient to label the choice indices with capital letters:
E.g., means the second choice is the integer 3.
We cannot apply the counting principle to figure 4 because depends on In our case, and and This leads us to an important caveat about using the counting principle:
 the counting principle cannot be used if the number of outcomes at each step cannot be uniquely defined
Figure 5 exams two flips of a coin. It calculates the correct number of outcomes to be, but only if we carefully define the outcome. The counting principle is valid only if heads followed by a tails (HT) is a different outcome than tails followed by heads (TH). In other words:
 when counting outcomes it is important to understand the role that order (enumeration) plays in defining outcomes
But, if we instead are counting the outcomes in a fashion such that HT and TH are considered to be the same, then a formula such as cannot be used:
 the counting principle does not hold if two different decision paths lead to the same final outcome (in the theorem, we say 'trial , which implicitly assumes that the order matters in the outcomes for the trials)
Example. Suppose we throw two sixfaced dice, with colors red and blue respectively. The number of possible distinct pairs of number facing up is .
Proof. Since the dice are distinguishable, we can use multiplication principle of counting. To be more precise, we can let the possible numbers facing up of red dice to be the possible outcomes in 'trial 1', and that of blue dice to be the possible outcomes in 'trial 2'. Since each trial has six outcomes, it follows that the number of outcomes (i.e. possible distinct pairs) is .
Number of ways to select some objects from distinguishable objectsEdit
In this section, we will discuss number (no.) of ways to select some objects from distinguishable objects, in four types, classified by whether the selection is ordered, and whether the selection is with replacement.
Before discussing these four types of selection, we will introduce some preliminary mathematical concepts used in the following.
Preliminary mathematical conceptsEdit
Definition. (Factorial) For each nonnegative integer , the factorial of , denoted by , is
More generally, we have gamma function.
Definition. (Gamma function) The gamma function is
Proposition. (Relationship between gamma function and factorial) For each nonnegative integer , .
Proof. Using integration by parts,
Remark.
 The infinity in the proof can be regarded as extended real number, or be in limit sense.
 Another more general result shown in the proof is that for each positive .
Definition. (Binomial coefficient) The binomial coefficient, indexed by nonegative integers and such that . denoted by , is
Theorem. (Binomial series theorem) For each real number ,
Remark. The following are some special cases of this theorem:
 ;
 ;
 (negative binomial series);
 (binomial series).
Theorem. (Binomial theorem) For each nonegative integer ,
Proof. It can be proved combinatorially or inductively. Complete proof is omitted.
The binomial theorem can be illustrated by Pascal's triangle:
Ordered selection without replacementEdit
Theorem. The no. of ways for ordered selection of objects from distinguishable objects without replacement is
Proof. Consider an equivalent situation: selecting objects from distinguishable objects to be put into ordered boxes, labelled box , in which each box contains at most one object. By considering the boxes from box 1 to box ,
 for box 1, there are choices of object to be put into it
 for box 2, there are choices of object to be put into it, since the object put into box 1 cannot be simultaneously put into box 2
 ...
 for box , there are choices of object to be put into it, since each of the objects put into box cannot be simultaneously put into box
Thus, by multiplication principle of counting, the desired no. of ways is
Remark.
 is often denoted by (read n p r).
Example. The no. of distinct ways to select 3 objects to be put into 3 boxes, labelled and from 5 objects, labelled and is
Example. (Competition) There are candidates for a competition. The no. of ways to award winner, 1st and 2nd runnersup is
If, Amy and Bob are among the candidates, and it is given that Amy is awarded 1st runnerup, while Bob does not receive any award, the no. of ways to award winner, 1st and 2nd runnersup becomes . In particular, Amy and Bob cannot be awarded winner or 2nd runnerup.
A special case of ordered selection without replacement is when the no. of selected objects equals the no. of objects to be selected.
In this case, this selection is called permutation, and the no. of
ways for permutation of objects
(i.e. ordered selection of objects from objects)
is .
Unordered selection of distinguishable objects without replacementEdit
Theorem. The no. of ways for unordered selection to select objects from distinguishable objects without replacement is .
Proof. There are two ways to prove this.
First, consider an equivalent situation: selecting objects from distinguishable objects without replacement to be put into one box ^{[1]}. Then, we consider the no. of ways to do this in order, and then remove some ways that are regarded to be the same for unordered selection (i.e. regarded as the same when we put the objects into one box). The no. of ways to do this in order is (choice means the th selection of objects to be put into the box)
Among these ways, putting the same objects into the box in different orders counts as different ways, and we need to merge them together, into one way. To merge them, we consider how many different ways are counted for putting the same objects into the box in different orders. Indeed, this is permutation (ordered selection of objects from distinguishable objects), so the no. of different ways is . So, we count extra times of no. of ways (i.e. scale up the no. of ways by a factor ) , and thus we need to scale down the no. of ways, by dividing the no. by . Thus, the desired no. of ways is .
Second, we use the notion of generating function, by encoding the selection process into a binomial series, and then use the coefficients to determine the desired no. of ways. To be more precise, recall a special case of binomial series theorem:
Remark.
 The unordered selection without replacement is also known as combination.
 is read as 'n choose r', or 'n c r'.
Example.
For combination, the order in which the items are selected are not important, so each selection from a set can be regarded as a subset of the original set. Figure 8 illustrates for the set The number of elements in this set is From our earlier discussion of the power set, we know that the total number of subsets is . All 8 subsets are shown in the figure, organized by how many items are in each subset (for example, the subset in the upperleft corner contains 3 elements, while all subsets with 2 elements occupy the lowerright corner.) Let denote the number of elements "chosen" to be in each of the 8 subsets of set (where the number of elements in is, .)
 set has elements. It is the empty set: .
 sets have element. They are , ,and .
 sets have elements. They are , ,and .
 set has elements. It is the set itself: .
Example.
No. of ways to select 2 objects from 4 distinguishable objects without considering the order is
Example. (Competition) There are 16 candidates for a competition. The no. of ways to select 3 candidates to enter final is
Special cases worth rememberingEdit
The formula for counting combinations has special cases that are worth remembering:
 (There is only one way to pick no thing and only one way to pick all things.)
 (there are n ways to pick one thing or to leave one thing out)
 (There are the same number of ways of picking of things as there are of leaving out of things)
Ordered selection of distinguishable objects with replacementEdit
Theorem. The no. of ways for selecting objects from distinguishable objects in order, with replacement is .
Proof. Consider the equivalent situation: selecting objects from types of objects, in which each type of the objects has unlimited stock, to be put into ordered boxes (the same object may be selected more than once). Then, the no. of ways is , since for each box, there are types of objects that can be selected to be put into it.
Remark.
 can be greater than .
Example. (Setting password) The number of ways to set a password with 6 characters, with the following rules:
 (R1) numbers are allowed
 (R2) alphabets are allowed, and they are casesensitive ^{[2]}
 (R3) special characters (i.e. all characters other than numbers and alphabets) are not allowed
is
Proof. For each of the 6 positions available for the password, there are choices of characters. Also, the characters can be repeated in more than one positions, and order matters. So, this is a case of ordered selection of distinguishable objects with replacement. Thus, the desired number is
Unordered selection of distinguishable objects with replacementEdit
This type of selection is probably the most complicated.
Theorem. The number of ways for unordered selection of objects from distinguishable objects with replacement is .
Proof. There are two ways to prove this.
First, consider an equivalent situation: selecting objects from types of objects, in which each type of the objects has unlimited stock, to be put into one box (the same object may be selected more than once). Then, we use the stars and bars notation: e.g.
****...****
in which th gap created by the bars corresponds to the th type of object (the leftmost gap made by one bar is the 1st gap, the rightmost gap made by one bar is the last gap), and the number of * in each gaps represents the number of objects selected for the corresponding type of objects. E.g., 2 * in 2nd gap represents the 2 objects are selected from the 2nd type of objects. Then, the desired no. of ways is the no. of arrangements of * and bars ^{[3]}, which is the no. of ways to select from positions for * ^{[4]} (order does not matter), calculated by .
Second, we use the notion of generating function, by encoding the selection as follows:
 encoding the selection of each type of objects to , by treating , , , etc. (up to in the th as selecting 0, 1, 2, etc. (up to ) objects from the th type respectively
Then, the desired no. is the coefficient of in
Remark.
 can be greater than .
 is often denoted by (read 'n h r')
Example. There are 8 distinct food or drink items, namely hamburger, egg, fries, cake, apple pie, apple juice, orange juice and coke. The number of distinct 4item combos that must consist of distinct items (unordered selection without replacement) is , and that without restrictions (particularly, may consist of more than one same item) (unordered selection with replacement) is .
Example. (Number of integer solutions of a equation) The number of solutions to
Proof. Consider the following stars and bars graph:
**********
in which the no. of stars is 10, corresponding to the number at RHS of the equation, and no. of gaps created by the bars is 7, corresponding to the number of unknowns at LHS of the equation. The no. of stars in each gap represents the (nonnegative) number assigned to that unknown. So, the number of solutions is the no. of arrangements of these stars and bars, namely
Alternatively, we can interpret there are 10 (no. at RHS) balls selected from 7 (no. of unknowns at LHS) types of balls, labelled , with unlimited stock, to be put into a box, in which the number of balls labelled in the box represents the number assigned for the unknowns respectively. Then, the no. of solutions is the no. of ways to do this, namely .
SummaryEdit
with replacement  without replacement  

ordered  
unordered 
Exercise. Try to prove each of the above formulas, without looking the previous subsections. After that, you can compare your proofs against the proofs in the previous subsections.
PartitionsEdit
Theorem. The number of ways to partition distinguishable objects into groups with group containing exactly objects respectively (order does not matter) is .
Proof. There are two ways to prove this.
First, consider an equivalent situation: putting objects selected from distinguishable objects into box respectively.
Then, consider the boxes one by one:
 box 1: objects selected from distinguishable objects to be put into it, so no. of ways is
 box 2: objects selected from distinguishable objects ^{[7]}, so no. of ways is
 ...
 box : objects selected from ^{[8]} to be put into it, so no. of ways is
By multiplication principle of counting, the no. of ways for the whole process is
Second, we use the notion of generating function, by encoding the partition process as follows:
 in the th , represents the th object is put into box respectively
Then, the desired no. of ways is the coefficient of in , which is , by multinomial theorem (generalized version of binomial theorem) ^{[9]}.
Remark.
 partitioning objects into two groups is the same as unordered selection without replacement ^{[10]}
 is called the multinomial coefficient, and is denoted by
Example. (Sequence of dice outcomes) A sixfaced dice is rolled nine times. The number of distinct sequences in which 1,3 and 5 each comes up three times is .
Proof. Consider this situation as the partition of the nine (ordered) outcomes from the die to three groups, which represents 1,3 and 5 comes up in that outcome respectively. The three groups contains 3 outcomes each, so that each odd number comes up three times. It follows that the number of ways to partition the outcomes is .
For each partition of outcomes into different groups, we obtain a unique sequence of outcomes. ^{[11]}
Example. (Arrangement of letters) The number of letter arrangements of the word PROBABILITY is .
Proof. The word PROBABILITY has 2 letter B's and 2 letter I's. For other letters, they appear only once. So, we partition the 11 letter positions in the word into 9 groups, representing letter P,R,O,B,A,I,L,T and Y, respectively, and the group representing letter B and I contain 2 letter positions each, and other groups contain 1 letter position each.
Example. (Walking path) Consider the following diagram.
Proof. First, observe that we need 6 and only 6 steps to walk from to ^{[12]}, consisting 4 steps of walking rightward ( ) and 2 steps of walking downward ( ).
Thus, the number of distinct sequence of steps is equivalent to the number of distinct sequence of 4 's and 2 's.
A way to calculate this is to consider this as a partition problem: partition the 6 step positions into 2 groups, one of them represents (and contains 4 step positions), another represents (and contains 2 step positions).
Alternatively, we can consider this as combination: unordered selection of 4 step positions for (then the remaining is for ).
The result follows.
Stirling's approximation for large 

The number grows very rapidly as a function of . A good approximation for when is large is given by Stirling's formula, as . The notation signifies that the ratio approaches 1 as tends to infinity. Taking the natural logarithm of both sides and dropping some terms yields (at the expense of accuracy), . A changeofbase formula for logarithms permits us to write this as,
For more precision, the asymptotic expression with a precise range for

ExercisesEdit
Lottery ticketsEdit
The examples in this subsubsection require the knowledge of (combinatorial or classical) probability. 
Example. (Pick 3 Texas Lottery) The Texas Lottery game Pick 3 is easy to play. A player must pick three numbers from zero to nine, and choose how to play them: exact order, or any order. The Pick 3 balls are drawn using three airdriven machines. These machines use compressed air to mix and select each ball.
The probability of winning while playing the exact order is
The probability of winning while playing any order depends on the numbers selected. If three distinct numbers are selected then the probability of winning is 3/500. If a number is repeated twice, the probability of winning is 3/1000. While, if the same number is selected three times, the probability of winning becomes 1/1000.
Example. (Mega Millions Texas Lottery) To play the Mega Millions game, a player must select five numbers from 1 to 56 in the upper white play area of the play board, and one Mega Ball number from 1 to 46 in the lower yellow play area of the play board.
All drawing equipment is stored in a secured onsite storage room. Only authorized drawings department personnel have keys to this door. Upon entry of the secured room to begin the drawing process, a lottery drawing specialist examines the security seal to determine if any unauthorized access has occurred. For each drawing, the Lotto Texas balls are mixed by four acrylic mixing paddles rotating clockwise. High speed is used for mixing and low speed for ball selection. As each ball is selected, it rolls down a chute into an official number display area. We wish to compute the probability of winning the Mega Millions Grand Prize, which require the correct selection of the five white balls plus the gold Mega ball. The probability of winning the Mega Millions Grand Prize is
References and footnotesEdit
 ↑ since we put the objects into one box, the order of putting the objects does not matter (we only know which objects are put into the box, but do not know in what order)
 ↑ e.g., A a
 ↑ bars create gaps
 ↑ or select from positions for bars, and the no. of ways for these two are the same
 ↑ the terms with order higher than , e.g. , do not affect the coefficient of , since there is not term with negative power. So, for convenience, we can include those terms with higher orders without affecting the result.
 ↑ with value such that the following terms are defined
 ↑ the objects put into box 1 cannot be simultaneously to be put into it
 ↑ the objects put into boxes cannot be simultaneously to be put into it
 ↑ the theorem itself is not our main focus here
 ↑ one group contains the objects selected, and another group contains the objects unselected
 ↑ E.g. if we put 1st, 2nd and 4th outcomes to the group representing 1 coming up, 5th, 7th and 9th outcomes to the group representing 3 coming up, and the remaining outcomes are put to the group representing 5 coming up, then the sequence obtained is: (1st outcome) 1,1,5,1,3,5,3,5,3 (9th outcome) in this order.
 ↑ this can be observed from the diagram, and the assumption that we can only walk one cell rightward or one cell downward is important