Chapter3 Special Distributions

Bernoulli and Binomial

Bernoulli Distribution

Bernoulli experiment a random experiment that outcome are classified with two mutually exclusive and exhaustive ways
Bernoulli process a sequence of Bernoulli trials.

Let X be a random variable associated with a Bernoulli trial

1. The pmf of X is $p(x) = p^x(1-p)^{1-x}, x=0,1$
2. The expected value is $E[X] = p$
3. The variance is $var(X) = p^2(1-p)+(1-p)^2p = p(1-p)$

Binomial distribution

Binomial random variable $X$: $X$ = # of sucesses in $n$ Bernoulli trials, denoted by $b(n,p)$

1. The pmf of $X$ is $p(x) = \binom{n}{x} p^x(1-p)^{n-x}, x=0,1,...,n$
2. The mgf: $\begin{aligned}[t] E(e^t \cdot X) = \sum_{x=0}^{n}e^{tx} \binom{n}{x}p^x(1-p)^{n-x} = (1-p+pe^t)^n \end{aligned}$
3. The mean and variances: $E(X) = np,\ Var(X) = np(1-p)$

Proof
$\begin{aligned} & M'(t) = n((1-p)+pe^t)^{n-1} \times pe^t \\ & M''(t) = n(n-1)((1-p)+pe^t)^{n-2}(p^2e^{2t}) + pe^tn((1-p)+pe^t)^{n-1} \end{aligned}$
From 1st moment, $\mu = M'(0) = np$
Also, $\sigma^2 = M''(0)-(M'(0))^2 = np(1-p)$

Example 3.1.4. Weak Law of Large Numbers

Let $Y \sim b(n,p)$. We call $Y/n$ the relative frequency of success.
Using the Chebyshev’s inequality (Thm 1.10.3),

$$for\ \forall\epsilon>0,\quad \ P(|\frac{Y}{n}\leq \epsilon|)\geq \frac{Var(Y/n)}{\epsilon^2}=\frac{p(1-p)}{n\epsilon^2}$$

As such, for every fixed $\epsilon$, righthand is closed to zero if $n$ is sufficiently large.
By equaition, $\lim_{n \to \infty} P\big(\big|\frac{Y}{n}-p\big|\leq \epsilon\big)=0$ and $\lim_{n \to \infty} P\big(\big|\frac{Y}{n}-p\big|<\epsilon\big) =1$

Example 3.1.5.

Therorem 3.1.1. Sum of Binomial Distribution

Let independent random variables $X_1,..,X_m$ where $X_i \sim b(n_i, p)\ for\ i=1,...,m$. Then $Y = \sum_{i=1}^{m}X_i$ has a $b(\sum_{i=1}^{m}n_i, p)$ distribution.

Proof
mgf of binomial distribution is $M_{X_i}(t) = (1-p+pe^t)^{n_i}$.
By independence, $M_Y(t) = \prod_{i=1}^{m}(1-p+pe^t)^{n_i}=(1-p+pe^t)^{\sum_{i=1}^{m}n_i}$

Negative Binomial distribution

Y denote the total number of failures in the sequence before $r^{th}$ success.

• Y+r equals to the nuber of trials necessary to produce exactly r successes with the last trial as a success.
• Why Negative?: $P(y)$ is a general

1. pmf: $p(y) = \binom{y+r-1}{r-1}p^r(1-p)^y\, y=0,1...$
2. mgf: $M(t) = p^r(1-(1-p)e^t)^{-r}\ for\ t<-log(1-p)$

Geometric distribution

Y is a number of trials until a success
This is $r=1$ case of Negative Binomial distribution

1. pmf: $p(y) = p(1-p)^y\, y=0,1,2,...$
2. mfg: $M(t) = p(1-(1-p)e^t)^{-1}$

Multi-Nomial distribution

The experiments is held by $k$ mutually exclusive and exhaustive ways.

1. pmf: $\frac{n!}{X_1! \cdots X_k!}p_1^{X_1}\cdots p_k^{X_k}$ where $X_k = n-(X_1+\cdots+X_{k-1})$ and $\sum_{i=1}^{k}p_i =1$
2. mgf: $M(t_1, ... , t_{k-1}) = (p_1e^{t_1}+ \cdots + P_{k-1}e^{t_{k-1}}+p_k)^n$ for $t_1, ... , t_{k-1} \in \mathbb{R}$
3. Joint mfg: $M(0,\dots, t_i,\dots, 0) = (p_ie^{t_i}+(1-p_i))^n$

4. Trinomial distribution $(X_i, X_j)$ has mgf

   $M(0,\dots,0,t_i, 0,\dots,0, t_j,0\dots, 0) = (p_ie^{t_i}+p_je^{t_j}(1-p_i-p_j))^n$

5. Conditional distribution of $X_i$ given $X_j$:
   $p_{X_2|X_1}(x_2|x_1) = \binom{n-x_1}{x_2}\big(\frac{p_2}{1-p_1}\big)^{x_2}\big(1-\frac{p_2}{1-p_1}\big)^{n-x_1-x_2}$

Proof
$\begin{aligned} M(t_1,...,t_{k-1}) & = E(e^{t_1X_1+\cdots+t_{k-1}X_{k-1}}) = \sum_{} \cdots \sum_{} \frac{n!}{x_1 \cdots x_k}\\ & = (p_1e^{t_1})^{x_1} \cdots (p_{k-1}e^{t_{k-1}})^{x_{k-1}}p_k^{x_k} = (p_1e^{t_1}+ \cdots +p_{k-1}e^{t_{k-1}+p_k})^n \end{aligned}$

Hypergeometric distribution

Let N as number of items and D is a number of defective items amont them. Let X is a number of defective items in a sample of size n, without replacement.

• Binomial distribution is sampling with replacement.

1. pmf: $p(x) = \frac{\binom{N-D}{n-x}\binom{D}{x}}{\binom{N}{n}}$
2. mean: $n\frac{D}{N}$
3. variance: $n\frac{D}{N}\frac{N-D}{N}\frac{N-n}{N-1}$
   • It has correction term when sampling without replacement.

Poisson Distribution

Poision Process a process that generate a number of changes in a fixed interval