Methods of finding confidence sets Inverting tests
Idea: 當我們做假設檢定 H 0 : θ = θ 0 H_0:\theta=\theta_0 H 0 : θ = θ 0 H 0 H_0 H 0 θ = θ 0 \theta=\theta_0 θ = θ 0 θ 0 ∈ C ( X ~ ) \theta_0\in C(\utilde{X}) θ 0 ∈ C ( X ) θ 0 \theta_0 θ 0 x ~ \utilde{x} x H 0 H_0 H 0 θ 0 \theta_0 θ 0 C ( x ~ ) C(\utilde{x}) C ( x )
EX X 1 , ⋯ , X n ∼ iid U ( 0 , θ ) X_1, \cdots, X_n\overset{\text{iid}}{\sim}U(0, \theta) X 1 , ⋯ , X n ∼ iid U ( 0 , θ )
H 0 : θ = θ 0 vs. H 1 : θ ≠ θ 0 H_0:\theta=\theta_0\quad\text{vs.}\quad H_1:\theta\neq\theta_0 H 0 : θ = θ 0 vs. H 1 : θ = θ 0 ⟹ \implies ⟹ α \alpha α H 0 H_0 H 0 x ( n ) > θ 0 x_{(n)}>\theta_0 x ( n ) > θ 0 x ( n ) < θ 0 α 1 n x_{(n)}<\theta_0\alpha^\frac{1}{n} x ( n ) < θ 0 α n 1
⟺ \iff ⟺ H 0 H_0 H 0 x ( n ) ≤ θ 0 x_{(n)}\le\theta_0 x ( n ) ≤ θ 0 x ( n ) ≥ θ 0 α 1 n x_{(n)}\ge\theta_0\alpha^\frac{1}{n} x ( n ) ≥ θ 0 α n 1
Note.
P θ 0 ( rej. H 0 ) = α ∀ θ 0 > 0 ⟺ P θ 0 ( Not rej. H 0 ) = 1 − α ∀ θ 0 > 0 ⟺ P θ ( X ( n ) ≤ θ ≤ X ( n ) α − 1 n ) = 1 − α ∀ θ > 0 \begin{align*}
&P_{\theta_0}(\text{rej. }H_0) = \alpha \quad \forall\theta_0 > 0\\
\iff& P_{\theta_0}(\text{Not rej. }H_0)=1-\alpha \quad \forall\theta_0 > 0\\
\iff& P_{\theta}(X_{(n)}\le\theta\le X_{(n)}\alpha^{-\frac{1}{n}})=1-\alpha \quad \forall\theta > 0
\end{align*} ⟺ ⟺ P θ 0 ( rej. H 0 ) = α ∀ θ 0 > 0 P θ 0 ( Not rej. H 0 ) = 1 − α ∀ θ 0 > 0 P θ ( X ( n ) ≤ θ ≤ X ( n ) α − n 1 ) = 1 − α ∀ θ > 0 ⟹ [ X ( n ) , X ( n ) α − 1 n ] \implies [X_{(n)}, X_{(n)}\alpha^{-\frac{1}{n}}] ⟹ [ X ( n ) , X ( n ) α − n 1 ] 1 − α 1-\alpha 1 − α UMA confidence set for θ \theta θ
我們可以從 UMP α \alpha α 1 − α 1-\alpha 1 − α
Inverting tests Method :
Let a non-randomized level α \alpha α
ϕ ( X ~ ) = { 1 if X ~ ∈ B ( θ 0 ) = rejection region 0 if X ~ ∈ ( B ( θ 0 ) ) c ≜ A ( θ 0 ) = acceptance region \phi(\utilde{X}) = \begin{cases}
1 & \text{if } \utilde{X}\in B(\theta_0)=\text{ rejection region}\\
0 & \text{if } \utilde{X}\in (B(\theta_0))^c\triangleq A(\theta_0)=\text{ acceptance region}
\end{cases} ϕ ( X ) = { 1 0 if X ∈ B ( θ 0 ) = rejection region if X ∈ ( B ( θ 0 ) ) c ≜ A ( θ 0 ) = acceptance region Def C ( X ~ ) C(\utilde{X}) C ( X ) X ~ ∈ A ( θ 0 ) ⟺ θ 0 ∈ C ( X ~ ) \utilde{X}\in A(\theta_0)\iff\theta_0\in C(\utilde{X}) X ∈ A ( θ 0 ) ⟺ θ 0 ∈ C ( X )
∀ θ 0 P θ 0 ( θ 0 ∈ C ( X ~ ) ) = P θ 0 ( X ~ ∈ A ( θ 0 ) ) = 1 − P θ 0 ( X ~ ∈ B ( θ 0 ) ) = 1 − α \begin{align*}
\forall\theta_0\quad P_{\theta_0}(\theta_0\in C(\utilde{X})) &= P_{\theta_0}(\utilde{X}\in A(\theta_0))\\
&=1-P_{\theta_0}(\utilde{X}\in B(\theta_0))\\
&=1-\alpha
\end{align*} ∀ θ 0 P θ 0 ( θ 0 ∈ C ( X )) = P θ 0 ( X ∈ A ( θ 0 )) = 1 − P θ 0 ( X ∈ B ( θ 0 )) = 1 − α i.e. ∀ θ P θ ( θ ∈ C ( X ~ ) ) = 1 − α ⟹ C ( X ~ ) \forall\theta P_\theta(\theta\in C(\utilde{X}))=1-\alpha\implies C(\utilde{X}) ∀ θ P θ ( θ ∈ C ( X )) = 1 − α ⟹ C ( X ) 1 − α 1-\alpha 1 − α θ \theta θ
In fact, θ 0 ∈ C ( X ~ ) ⟺ X ~ ∈ A ( θ 0 ) \theta_0\in C(\utilde{X})\iff\utilde{X}\in A(\theta_0) θ 0 ∈ C ( X ) ⟺ X ∈ A ( θ 0 )
也就是說，there is Duality(一體兩面) relationship between confidence sets and hypothesis tests. 它們問的是同一個問題，只是從不同的角度來看。假設檢定是給定 θ \theta θ θ \theta θ θ \theta θ
因此我們可以利用已知結果的檢定來得到 confidence set，並且
UMP ⟹ \implies ⟹
UMPU ⟹ \implies ⟹
EX X 1 , ⋯ , X n ∼ iid U ( 0 , θ ) X_1,\cdots, X_n\overset{\text{iid}}{\sim}U(0, \theta) X 1 , ⋯ , X n ∼ iid U ( 0 , θ ) 1 − α 1-\alpha 1 − α θ \theta θ θ ≥ L ( X ~ ) \theta\ge L(\utilde{X}) θ ≥ L ( X )
⟹ \implies ⟹ α \alpha α H 0 : θ ≤ θ 0 H_0:\theta\le\theta_0 H 0 : θ ≤ θ 0 H 1 : θ > θ 0 H_1:\theta>\theta_0 H 1 : θ > θ 0 H 0 H_0 H 0 x ( n ) ≤ θ 0 x_{(n)}\le\theta_0 x ( n ) ≤ θ 0
i.e. Not reject H 0 H_0 H 0 x ( n ) ≤ θ 0 ( 1 − α ) 1 n x_{(n)}\le\theta_0(1-\alpha)^{\frac{1}{n}} x ( n ) ≤ θ 0 ( 1 − α ) n 1
i.e. { x ~ : x ( n ) ( 1 − α ) − 1 n ≤ θ 0 } = A ( θ 0 ) \set{\utilde{x}:x_{(n)}(1-\alpha)^{-\frac{1}{n}}\le\theta_0}=A(\theta_0) { x : x ( n ) ( 1 − α ) − n 1 ≤ θ 0 } = A ( θ 0 )
⟹ C ( X ~ ) = { θ 0 : X ~ ∈ A ( θ 0 ) } = { θ 0 : θ 0 ≥ X n ( 1 − α ) − 1 n } = [ X ( n ) ( 1 − α ) − 1 n , ∞ ) \implies C(\utilde{X})=\set{\theta_0:\utilde{X}\in A(\theta_0)}=\set{\theta_0:\theta_0\ge X_{n}(1-\alpha)^{-\frac{1}{n}}}=[X_{(n)}(1-\alpha)^{-\frac{1}{n}}, \infty) ⟹ C ( X ) = { θ 0 : X ∈ A ( θ 0 ) } = { θ 0 : θ 0 ≥ X n ( 1 − α ) − n 1 } = [ X ( n ) ( 1 − α ) − n 1 , ∞ )
is UMA(hence UMAU) 1 − α 1-\alpha 1 − α θ \theta θ
如果我們關注的是 upper bound，那麼我們同樣做鑒定，得到 reject H 0 H_0 H 0 x ( n ) < θ 0 α 1 n x_{(n)}<\theta_0\alpha^{\frac{1}{n}} x ( n ) < θ 0 α n 1 { x ~ : x ( n ) ≥ θ 0 α 1 n } \set{\utilde{x}:x_{(n)}\ge\theta_0\alpha^{\frac{1}{n}}} { x : x ( n ) ≥ θ 0 α n 1 } 1 − α 1-\alpha 1 − α θ \theta θ ( − ∞ , X ( n ) α − 1 n ] (-\infty, X_{(n)}\alpha^{-\frac{1}{n}}] ( − ∞ , X ( n ) α − n 1 ]
EX
⊥ < X 1 , ⋯ , X n ∼ iid N ( μ , σ 2 ) Y 1 , ⋯ , Y m ∼ iid N ( μ , σ 2 ) \perp \Big< \begin{align*}
&X_1, \cdots, X_n\overset{\text{iid}}{\sim}N(\mu, \sigma^2)\\
&Y_1, \cdots, Y_m\overset{\text{iid}}{\sim}N(\mu, \sigma^2)
\end{align*} ⊥ ⟨ X 1 , ⋯ , X n ∼ iid N ( μ , σ 2 ) Y 1 , ⋯ , Y m ∼ iid N ( μ , σ 2 ) C ( X ~ , Y ~ ) = { λ : λ ≥ L ( X ~ , Y ~ ) } C(\utilde{X}, \utilde{Y})=\set{\lambda:\lambda\ge L(\utilde{X}, \utilde{Y})} C ( X , Y ) = { λ : λ ≥ L ( X , Y ) } θ = ( μ x , μ y , σ x 2 , σ y 2 ) \theta=(\mu_x, \mu_y, \sigma^2_x, \sigma^2_y) θ = ( μ x , μ y , σ x 2 , σ y 2 )
⟹ \implies ⟹ 1 − α 1-\alpha 1 − α λ = σ x 2 σ y 2 \lambda=\frac{\sigma^2_x}{\sigma^2_y} λ = σ y 2 σ x 2
⟹ \implies ⟹ α \alpha α
H 0 : σ x 2 σ y 2 = λ = λ 0 vs. H 1 : σ x 2 σ y 2 = λ > λ 0 H_0:\frac{\sigma^2_x}{\sigma^2_y}=\lambda=\lambda_0\quad\text{vs.}\quad H_1:\frac{\sigma^2_x}{\sigma^2_y}=\lambda>\lambda_0 H 0 : σ y 2 σ x 2 = λ = λ 0 vs. H 1 : σ y 2 σ x 2 = λ > λ 0 rejects H 0 H_0 H 0 S x 2 / σ x 2 S y 2 / σ y 2 > F n − 1 , m − 1 ( α ) ⟺ S x 2 S Y 2 > λ 0 F n − 1 , m − 1 ( α ) \frac{S^2_x/\sigma^2_x}{S^2_y/\sigma^2_y}>F_{n-1, m-1}(\alpha)\iff \frac{S^2_x}{S^2_Y}>\lambda_0F_{n-1, m-1}(\alpha) S y 2 / σ y 2 S x 2 / σ x 2 > F n − 1 , m − 1 ( α ) ⟺ S Y 2 S x 2 > λ 0 F n − 1 , m − 1 ( α ) λ 0 = H 0 σ x 2 σ y 2 \lambda_0\overset{H_0}{=}\frac{\sigma^2_x}{\sigma^2_y} λ 0 = H 0 σ y 2 σ x 2
i.e. Not reject H 0 H_0 H 0 S x 2 S y 2 ≤ λ 0 F n − 1 , m − 1 ( α ) ⟹ λ ∈ C ( X ~ , Y ~ ) = { λ : λ ≤ S X 2 S Y 2 F n − 1 , m − 1 , α − 1 } \frac{S^2_x}{S^2_y}\le\lambda_0F_{n-1, m-1}(\alpha)\implies\lambda\in C(\utilde{X}, \utilde{Y})=\set{\lambda:\lambda\le\frac{S^2_X}{S^2_Y}F_{n-1,m-1,\alpha}^{-1}} S y 2 S x 2 ≤ λ 0 F n − 1 , m − 1 ( α ) ⟹ λ ∈ C ( X , Y ) = { λ : λ ≤ S Y 2 S X 2 F n − 1 , m − 1 , α − 1 }
i.e. [ S X 2 S Y 2 1 F n − 1 , m − 1 , α , ∞ ) [\frac{S^2_X}{S^2_Y}\frac{1}{F_{n-1,m-1,\alpha}},\infty) [ S Y 2 S X 2 F n − 1 , m − 1 , α 1 , ∞ ) 1 − α 1-\alpha 1 − α λ = σ x 2 σ y 2 \lambda=\frac{\sigma^2_x}{\sigma^2_y} λ = σ y 2 σ x 2
Pivot
A r.v. K ( X ~ ; θ ) K(\utilde{X};\theta) K ( X ; θ ) pivot ⟺ K ( X ~ ; θ ) \iff K(\utilde{X};\theta) ⟺ K ( X ; θ ) ⊥ θ \perp\theta ⊥ θ
EX X 1 , ⋯ , X n ∼ iid N ( μ , σ 2 ) X_1,\cdots, X_n\overset{\text{iid}}{\sim}N(\mu, \sigma^2) X 1 , ⋯ , X n ∼ iid N ( μ , σ 2 )
n ( X ˉ − μ ) σ ∼ N ( 0 , 1 ) ⊥ μ , σ 2 \frac{\sqrt{n}(\bar{X}-\mu)}{\sigma}\sim N(0, 1) \perp \mu, \sigma^2 σ n ( X ˉ − μ ) ∼ N ( 0 , 1 ) ⊥ μ , σ 2 If K ( X ~ , θ ) K(\utilde{X},\theta) K ( X , θ ) ∀ \forall ∀ B B B P ( K ( X ~ , θ ) ∈ B ) ⊥ θ P(K(\utilde{X},\theta)\in B)\perp\theta P ( K ( X , θ ) ∈ B ) ⊥ θ
⟹ \implies ⟹ α ∈ ( 0 , 1 ) , ∃ B α \alpha\in(0, 1), \exist B_\alpha α ∈ ( 0 , 1 ) , ∃ B α P ( K ( X ~ , θ ) ∈ B α ) = 1 − α P(K(\utilde{X},\theta)\in B_\alpha)=1-\alpha P ( K ( X , θ ) ∈ B α ) = 1 − α inf θ P ( K ∈ B α ) = 1 − α \inf_\theta P(K\in B_\alpha)=1-\alpha inf θ P ( K ∈ B α ) = 1 − α
C ( X ~ ) = { θ : K ( X ~ , θ ) ∈ B α } C(\utilde{X})=\set{\theta:K(\utilde{X},\theta)\in B_\alpha} C ( X ) = { θ : K ( X , θ ) ∈ B α } we have
∀ θ P θ ( θ ∈ C ( X ~ ) ) = P ( K ( X ~ , θ ) ∈ B α ) = 1 − α \forall\theta\quad P_\theta(\theta\in C(\utilde{X}))=P(K(\utilde{X},\theta)\in B_\alpha)=1-\alpha ∀ θ P θ ( θ ∈ C ( X )) = P ( K ( X , θ ) ∈ B α ) = 1 − α ⟹ inf θ P θ ( θ ∈ C ( X ~ ) ) = 1 − α \implies \inf_\theta P_\theta(\theta\in C(\utilde{X}))=1-\alpha ⟹ inf θ P θ ( θ ∈ C ( X )) = 1 − α C ( X ~ ) C(\utilde{X}) C ( X ) 1 − α 1-\alpha 1 − α θ \theta θ
EX X 1 , ⋯ , X n ∼ iid N ( μ , σ 0 2 ) X_1,\cdots, X_n\overset{\text{iid}}{\sim}N(\mu, \sigma^2_0) X 1 , ⋯ , X n ∼ iid N ( μ , σ 0 2 ) σ 0 2 \sigma^2_0 σ 0 2 α ∈ ( 0 , 1 ) \alpha\in(0, 1) α ∈ ( 0 , 1 ) 1 − α 1-\alpha 1 − α μ \mu μ
⟹ n ( X ˉ − μ ) σ 0 ∼ N ( 0 , 1 ) ⊥ μ \implies \frac{\sqrt{n}(\bar{X}-\mu)}{\sigma_0}\sim N(0, 1)\perp\mu ⟹ σ 0 n ( X ˉ − μ ) ∼ N ( 0 , 1 ) ⊥ μ Let B α = [ − Z α 1 , Z α 2 ] B_\alpha=[-Z_{\alpha_1}, Z_{\alpha_2}] B α = [ − Z α 1 , Z α 2 ] α 1 + α 2 = α ⟹ P ( n ( X ˉ − μ ) σ 0 ∈ B α ) = 1 − α \alpha_1+\alpha_2=\alpha\implies P\left(\frac{\sqrt{n}(\bar{X}-\mu)}{\sigma_0}\in B_\alpha\right)=1-\alpha α 1 + α 2 = α ⟹ P ( σ 0 n ( X ˉ − μ ) ∈ B α ) = 1 − α
C ( X ~ ) = { μ : − Z α 1 ≤ n ( X ˉ − μ ) σ 0 ≤ Z α 2 } = { μ : X ˉ − σ 0 Z α 1 n ≤ μ ≤ X ˉ + σ 0 Z α 2 n } = [ X ˉ − σ 0 Z α 1 n , X ˉ + σ 0 Z α 2 n ] \begin{align*}
C(\utilde{X})&=\set{\mu:-Z_{\alpha_1}\le\frac{\sqrt{n}(\bar{X}-\mu)}{\sigma_0}\le Z_{\alpha_2}}\\
&=\set{\mu:\bar{X}-\frac{\sigma_0Z_{\alpha_1}}{\sqrt{n}}\le\mu\le\bar{X}+\frac{\sigma_0Z_{\alpha_2}}{\sqrt{n}}}\\
&=\left[\bar{X}-\frac{\sigma_0Z_{\alpha_1}}{\sqrt{n}}, \bar{X}+\frac{\sigma_0Z_{\alpha_2}}{\sqrt{n}}\right]
\end{align*} C ( X ) = { μ : − Z α 1 ≤ σ 0 n ( X ˉ − μ ) ≤ Z α 2 } = { μ : X ˉ − n σ 0 Z α 1 ≤ μ ≤ X ˉ + n σ 0 Z α 2 } = [ X ˉ − n σ 0 Z α 1 , X ˉ + n σ 0 Z α 2 ] 所有 α 1 + α 2 = α \alpha_1+\alpha_2=\alpha α 1 + α 2 = α 1 − α 1-\alpha 1 − α
C ( X ~ ) C(\utilde{X}) C ( X ) = σ 0 n ( Z α 2 − Z α 1 ) =\frac{\sigma_0}{\sqrt{n}}(Z_{\alpha_2}-Z_{\alpha_1}) = n σ 0 ( Z α 2 − Z α 1 ) P ( Z α 1 ≤ Z ≤ Z α 2 ) = 1 − α P(Z_{\alpha_1}\le Z\le Z_{\alpha_2})=1-\alpha P ( Z α 1 ≤ Z ≤ Z α 2 ) = 1 − α
⟺ \iff ⟺ ( Z α 2 − Z α 1 ) (Z_{\alpha_2}-Z_{\alpha_1}) ( Z α 2 − Z α 1 ) Φ ( Z α 2 ) − Φ ( Z α 1 ) = 1 − α \Phi(Z_{\alpha_2})-\Phi(Z_{\alpha_1})=1-\alpha Φ ( Z α 2 ) − Φ ( Z α 1 ) = 1 − α
我們可以使用 Largrange multiplier method 來求解這個問題。如果我們要使 h ( a , b ) h(a,b) h ( a , b ) g ( a , b ) = k g(a,b)=k g ( a , b ) = k a , b a,b a , b
∇ h ( a , b ) = λ ∇ g ( a , b ) and g ( a , b ) = k \nabla h(a,b)=\lambda\nabla g(a,b)\quad \text{and } g(a,b)=k ∇ h ( a , b ) = λ ∇ g ( a , b ) and g ( a , b ) = k 這裡 h ( a , b ) = b − a , g ( a , b ) = Φ ( b ) − Φ ( a ) , k = 1 − α h(a,b)=b-a, g(a,b)=\Phi(b)-\Phi(a), k=1-\alpha h ( a , b ) = b − a , g ( a , b ) = Φ ( b ) − Φ ( a ) , k = 1 − α
{ ∂ ∂ a ( b − a ) = λ ∂ ∂ a Φ ( a ) ∂ ∂ b ( b − a ) = λ ∂ ∂ b Φ ( b ) Φ ( b ) − Φ ( a ) = 1 − α ⟹ { − 1 = − λ ϕ ( a ) 1 = λ ϕ ( b ) Φ ( b ) − Φ ( a ) = 1 − α \begin{cases}
\frac{\partial}{\partial a}(b-a)=\lambda\frac{\partial}{\partial a}\Phi(a)\\
\frac{\partial}{\partial b}(b-a)=\lambda\frac{\partial}{\partial b}\Phi(b)\\
\Phi(b)-\Phi(a)=1-\alpha
\end{cases}\implies\begin{cases}
-1=-\lambda\phi(a)\\
1=\lambda\phi(b)\\
\Phi(b)-\Phi(a)=1-\alpha
\end{cases} ⎩ ⎨ ⎧ ∂ a ∂ ( b − a ) = λ ∂ a ∂ Φ ( a ) ∂ b ∂ ( b − a ) = λ ∂ b ∂ Φ ( b ) Φ ( b ) − Φ ( a ) = 1 − α ⟹ ⎩ ⎨ ⎧ − 1 = − λ ϕ ( a ) 1 = λ ϕ ( b ) Φ ( b ) − Φ ( a ) = 1 − α ⟹ ϕ ( a ) = ϕ ( b ) and Φ ( b ) − Φ ( a ) = 1 − α ⟹ a = − b \implies \phi(a)=\phi(b) \text{ and } \Phi(b)-\Phi(a)=1-\alpha \implies a=-b ⟹ ϕ ( a ) = ϕ ( b ) and Φ ( b ) − Φ ( a ) = 1 − α ⟹ a = − b i.e. Z α 1 = Z α 2 , Z α 2 = Z 1 − α 2 ⟹ Z_{\alpha_1}=Z_{\frac{\alpha}{2}}, Z_{\alpha_2}=Z_{1-\frac{\alpha}{2}}\implies Z α 1 = Z 2 α , Z α 2 = Z 1 − 2 α ⟹ μ \mu μ [ X ˉ ± σ 0 n Z α 2 ] \left[\bar{X}\pm \frac{\sigma_0}{\sqrt{n}}Z_{\frac{\alpha}{2}}\right] [ X ˉ ± n σ 0 Z 2 α ] 2 σ 0 n Z α 2 \frac{2\sigma_0}{\sqrt{n}}Z_{\frac{\alpha}{2}} n 2 σ 0 Z 2 α
同樣，我們也可以用 X 1 − μ σ 0 \frac{X_1-\mu}{\sigma_0} σ 0 X 1 − μ [ X 1 ± σ 0 Z α 2 ] \left[X_1\pm\sigma_0Z_{\frac{\alpha}{2}}\right] [ X 1 ± σ 0 Z 2 α ] 2 σ 0 Z α 2 2\sigma_0Z_{\frac{\alpha}{2}} 2 σ 0 Z 2 α
比較兩個 confidence set 的長度，我們可以發現 X ˉ \bar{X} X ˉ X 1 X_1 X 1 X ˉ \bar{X} X ˉ
Remark : 我們會使用基於 sufficient statistics 的 pivot 來構建 confidence set。
Note
X 1 , ⋯ , X n ∼ iid f ( x ; θ ) = f ( x − θ ) X_1, \cdots, X_n\overset{\text{iid}}{\sim}f(x;\theta)=f(x-\theta) X 1 , ⋯ , X n ∼ iid f ( x ; θ ) = f ( x − θ )
Let Y i = X i − θ ⟺ X i = Y i + θ Y_i=X_i-\theta\iff X_i=Y_i+\theta Y i = X i − θ ⟺ X i = Y i + θ
⟹ f Y i ( y ) = f X i ( y + θ ) ⋅ ∣ d d y ( y + θ ) ∣ = f ( y + θ − θ ) = f ( y ) ⊥ θ \implies f_{Y_i}(y)=f_{X_i}(y+\theta)\cdot |\frac{d}{dy}(y+\theta)|=f(y+\theta-\theta)=f(y) \perp \theta ⟹ f Y i ( y ) = f X i ( y + θ ) ⋅ ∣ d y d ( y + θ ) ∣ = f ( y + θ − θ ) = f ( y ) ⊥ θ ⟹ Y 1 , ⋯ , Y n ∼ iid f ( y ) \implies Y_1,\cdots,Y_n\overset{\text{iid}}{\sim} f(y) ⟹ Y 1 , ⋯ , Y n ∼ iid f ( y ) ⟹ ∀ h , h ( Y 1 , ⋯ , Y n ) \implies\forall h, h(Y_1,\cdots, Y_n) ⟹ ∀ h , h ( Y 1 , ⋯ , Y n )
X 1 , ⋯ , X n ∼ iid f ( x ; σ ) = 1 σ f ( x σ ) X_1, \cdots, X_n\overset{\text{iid}}{\sim}f(x;\sigma)=\frac{1}{\sigma}f(\frac{x}{\sigma}) X 1 , ⋯ , X n ∼ iid f ( x ; σ ) = σ 1 f ( σ x )
Let Y i = X i σ ⟺ X i = Y i σ Y_i=\frac{X_i}{\sigma}\iff X_i=Y_i\sigma Y i = σ X i ⟺ X i = Y i σ
⟹ f Y i ( y ) = f X i ( y σ ) ⋅ ∣ d d y ( y σ ) ∣ = 1 σ f ( y σ σ ) ⋅ σ = f ( y ) ⊥ σ \implies f_{Y_i}(y)=f_{X_i}(y\sigma)\cdot |\frac{d}{dy}(y\sigma)|=\frac{1}{\sigma}f(\frac{y\sigma}{\sigma})\cdot\sigma=f(y)\perp\sigma ⟹ f Y i ( y ) = f X i ( y σ ) ⋅ ∣ d y d ( y σ ) ∣ = σ 1 f ( σ y σ ) ⋅ σ = f ( y ) ⊥ σ ⟹ Y 1 , ⋯ , Y n ∼ iid f ( y ) \implies Y_1,\cdots,Y_n\overset{\text{iid}}{\sim}f(y) ⟹ Y 1 , ⋯ , Y n ∼ iid f ( y ) ⟹ ∀ h , h ( Y 1 , ⋯ , Y n ) \implies\forall h, h(Y_1,\cdots, Y_n) ⟹ ∀ h , h ( Y 1 , ⋯ , Y n )
X 1 , ⋯ , X n ∼ iid f ( x ; θ , σ ) = 1 σ f ( x − θ σ ) X_1, \cdots, X_n\overset{\text{iid}}{\sim}f(x;\theta, \sigma)=\frac{1}{\sigma}f(\frac{x-\theta}{\sigma}) X 1 , ⋯ , X n ∼ iid f ( x ; θ , σ ) = σ 1 f ( σ x − θ )
Let Y i = X i − θ σ ⟺ X i = Y i σ + θ Y_i=\frac{X_i-\theta}{\sigma}\iff X_i=Y_i\sigma+\theta Y i = σ X i − θ ⟺ X i = Y i σ + θ
similar Y 1 , ⋯ , Y n ∼ iid f ( y ) Y_1,\cdots,Y_n\overset{\text{iid}}{\sim}f(y) Y 1 , ⋯ , Y n ∼ iid f ( y ) ⟹ ∀ h , h ( Y 1 , ⋯ , Y n ) \implies\forall h, h(Y_1,\cdots, Y_n) ⟹ ∀ h , h ( Y 1 , ⋯ , Y n )
EX X 1 , ⋯ , X n ∼ iid N ( μ , σ 2 ) X_1, \cdots, X_n\overset{\text{iid}}{\sim}N(\mu, \sigma^2) X 1 , ⋯ , X n ∼ iid N ( μ , σ 2 )
1 2 π σ e − 1 2 σ 2 ( x − μ ) 2 = 1 σ ϕ ( x − μ σ ) is a location-scale family \frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{1}{2\sigma^2}(x-\mu)^2}=\frac{1}{\sigma}\phi(\frac{x-\mu}{\sigma}) \text{ is a location-scale family} 2 π σ 1 e − 2 σ 2 1 ( x − μ ) 2 = σ 1 ϕ ( σ x − μ ) is a location-scale family Note: ( X ˉ , S 2 ) (\bar{X},S^2) ( X ˉ , S 2 ) ( μ , σ 2 ) (\mu, \sigma^2) ( μ , σ 2 )
對於 θ \theta θ n ( X ˉ − θ ) S ∼ t n − 1 ⊥ ( θ , σ 2 ) \frac{\sqrt{n}(\bar{X}-\theta)}{S}\sim t_{n-1}\perp (\theta, \sigma^2) S n ( X ˉ − θ ) ∼ t n − 1 ⊥ ( θ , σ 2 )
∀ a < b \forall a<b ∀ a < b
1 − α = P ( a ≤ t n − 1 ≤ b ) = P θ , σ 2 ( a ≤ n ( X ˉ − θ ) S ≤ b ) = P θ , σ 2 ( a S ≤ n ( X ˉ − θ ) ≤ b S ) = P θ , σ 2 ( X ˉ − b S n ≤ θ ≤ X ˉ − a S n ) = P θ , σ 2 ( θ ∈ C a , b ( X ~ ) ) \begin{align*}
1-\alpha&=P(a\le t_{n-1}\le b)\\
&=P_{\theta,\sigma^2}(a\le\frac{\sqrt{n}(\bar{X}-\theta)}{S}\le b)\\
&=P_{\theta,\sigma^2}(aS\le\sqrt{n}(\bar{X}-\theta)\le bS)\\
&=P_{\theta,\sigma^2}(\bar{X}-\frac{bS}{\sqrt{n}}\le\theta\le\bar{X}-\frac{aS}{\sqrt{n}})\\
&=P_{\theta,\sigma^2}(\theta\in C_{a,b}(\utilde{X}))\\
\end{align*} 1 − α = P ( a ≤ t n − 1 ≤ b ) = P θ , σ 2 ( a ≤ S n ( X ˉ − θ ) ≤ b ) = P θ , σ 2 ( a S ≤ n ( X ˉ − θ ) ≤ b S ) = P θ , σ 2 ( X ˉ − n b S ≤ θ ≤ X ˉ − n a S ) = P θ , σ 2 ( θ ∈ C a , b ( X )) with C a , b ( X ~ , S 2 ) = [ X ˉ − b S n , X ˉ + a S n ] C_{a,b}(\utilde{X},S^2)=[\bar{X}-\frac{bS}{\sqrt{n}}, \bar{X}+\frac{aS}{\sqrt{n}}] C a , b ( X , S 2 ) = [ X ˉ − n b S , X ˉ + n a S ] 1 − α 1-\alpha 1 − α θ \theta θ
To min ( b − a ) (b-a) ( b − a ) a < b a<b a < b G ( b ) − G ( a ) = 1 − α G(b)-G(a)=1-\alpha G ( b ) − G ( a ) = 1 − α G G G t n − 1 t_{n-1} t n − 1
{ − 1 = λ ( − g ( a ) ) 1 = λ g ( b ) G ( b ) − G ( a ) = 1 − α ⟹ a = − b = t n − 1 , α 2 \begin{cases}
-1=\lambda(-g(a))\\
1=\lambda g(b)\\
G(b)-G(a)=1-\alpha
\end{cases}
\implies a=-b=t_{n-1,\frac{\alpha}{2}} ⎩ ⎨ ⎧ − 1 = λ ( − g ( a )) 1 = λ g ( b ) G ( b ) − G ( a ) = 1 − α ⟹ a = − b = t n − 1 , 2 α 如果用 C ∗ ( X ˉ , S 2 ) = [ X ˉ − S n t n − 1 , X ˉ + S n t n − 1 ] C^*(\bar{X}, S^2)=[\bar{X}-\frac{S}{\sqrt{n}}t_{n-1}, \bar{X}+\frac{S}{\sqrt{n}}t_{n-1}] C ∗ ( X ˉ , S 2 ) = [ X ˉ − n S t n − 1 , X ˉ + n S t n − 1 ]
H 0 : θ = θ 0 vs. H 1 : θ ≠ θ 0 H_0:\theta=\theta_0\quad\text{vs.}\quad H_1:\theta\neq\theta_0 H 0 : θ = θ 0 vs. H 1 : θ = θ 0 ⟹ \implies ⟹ H 0 ⟺ θ 0 ∉ C ∗ ( X ˉ , S 2 ) ⟺ ∣ n ( X ˉ − θ 0 ) S ∣ > t n − 1 , α 2 H_0\iff\theta_0\notin C^*(\bar{X}, S^2)\iff \left|\frac{\sqrt{n}(\bar{X}-\theta_0)}{S}\right|>t_{n-1,\frac{\alpha}{2}} H 0 ⟺ θ 0 ∈ / C ∗ ( X ˉ , S 2 ) ⟺ S n ( X ˉ − θ 0 ) > t n − 1 , 2 α α \alpha α θ \theta θ
⟹ [ X ˉ ± S n t n − 1 , α 2 ] \implies [\bar{X}\pm \frac{S}{\sqrt{n}}t_{n-1,\frac{\alpha}{2}}] ⟹ [ X ˉ ± n S t n − 1 , 2 α ] 1 − α 1-\alpha 1 − α θ \theta θ
σ 2 \sigma^2 σ 2 S 2 σ 2 \frac{S^2}{\sigma^2} σ 2 S 2 ( n − 1 ) S 2 σ 2 ∼ χ n − 1 2 ⊥ ( θ , σ 2 ) ⟹ ∀ 0 ≤ a < b \frac{(n-1)S^2}{\sigma^2}\sim \chi^2_{n-1}\perp(\theta, \sigma^2)\implies \forall 0\le a<b σ 2 ( n − 1 ) S 2 ∼ χ n − 1 2 ⊥ ( θ , σ 2 ) ⟹ ∀0 ≤ a < b
1 − α = P ( a ≤ χ n − 1 2 ≤ b ) = P θ , σ 2 ( a ≤ ( n − 1 ) S 2 σ 2 ≤ b ) ∀ θ , σ 2 = P θ , σ 2 ( ( n − 1 ) S 2 b ≤ σ 2 ≤ ( n − 1 ) S 2 a ) = P θ , σ 2 ( σ 2 ∈ C a , b ( X ~ ) ) \begin{align*}
1-\alpha&=P(a\le\chi^2_{n-1}\le b)\\
&=P_{\theta,\sigma^2}(a\le\frac{(n-1)S^2}{\sigma^2}\le b) \quad\forall\theta, \sigma^2\\
&=P_{\theta,\sigma^2}\left(\frac{(n-1)S^2}{b}\le\sigma^2\le\frac{(n-1)S^2}{a}\right)\\
&=P_{\theta,\sigma^2}(\sigma^2\in C_{a,b}(\utilde{X}))\\
\end{align*} 1 − α = P ( a ≤ χ n − 1 2 ≤ b ) = P θ , σ 2 ( a ≤ σ 2 ( n − 1 ) S 2 ≤ b ) ∀ θ , σ 2 = P θ , σ 2 ( b ( n − 1 ) S 2 ≤ σ 2 ≤ a ( n − 1 ) S 2 ) = P θ , σ 2 ( σ 2 ∈ C a , b ( X )) with C a , b ( X ~ , S 2 ) = [ ( n − 1 ) S 2 b , ( n − 1 ) S 2 a ] C_{a,b}(\utilde{X}, S^2)=\left[\frac{(n-1)S^2}{b}, \frac{(n-1)S^2}{a}\right] C a , b ( X , S 2 ) = [ b ( n − 1 ) S 2 , a ( n − 1 ) S 2 ] 1 − α 1-\alpha 1 − α σ 2 \sigma^2 σ 2 χ 2 \chi^2 χ 2
a = 0 , b = χ n − 1 , α 2 ⟹ C a , b ( S 2 ) = [ ( n − 1 ) S 2 χ n − 1 , α 2 , ∞ ] a=0, b=\chi^2_{n-1,\alpha}\implies C_{a,b}(S^2)=\left[\frac{(n-1)S^2}{\chi^2_{n-1,\alpha}}, \infty\right] a = 0 , b = χ n − 1 , α 2 ⟹ C a , b ( S 2 ) = [ χ n − 1 , α 2 ( n − 1 ) S 2 , ∞ ] 1 − α 1-\alpha 1 − α lower conf. bound for σ 2 \sigma^2 σ 2
a = χ n − 1 , 1 − α 2 , b = ∞ ⟹ C a , b ( S 2 ) = ( 0 , ( n − 1 ) S 2 χ n − 1 , 1 − α 2 ] a=\chi^2_{n-1,1-\alpha}, b=\infty\implies C_{a,b}(S^2)=\left(0, \frac{(n-1)S^2}{\chi^2_{n-1,1-\alpha}}\right] a = χ n − 1 , 1 − α 2 , b = ∞ ⟹ C a , b ( S 2 ) = ( 0 , χ n − 1 , 1 − α 2 ( n − 1 ) S 2 ] 1 − α 1-\alpha 1 − α upper conf. bound for σ 2 \sigma^2 σ 2
Usually , used a = χ n − 1 , α 2 2 , b = χ n − 1 , 1 − α 2 2 ⟹ C a , b ( S 2 ) = [ ( n − 1 ) S 2 χ n − 1 , 1 − α 2 2 , ( n − 1 ) S 2 χ n − 1 , α 2 2 ] a=\chi^2_{n-1,\frac{\alpha}{2}}, b=\chi^2_{n-1,1-\frac{\alpha}{2}}\implies C_{a,b}(S^2)=\left[\frac{(n-1)S^2}{\chi^2_{n-1,1-\frac{\alpha}{2}}}, \frac{(n-1)S^2}{\chi^2_{n-1,\frac{\alpha}{2}}}\right] a = χ n − 1 , 2 α 2 , b = χ n − 1 , 1 − 2 α 2 ⟹ C a , b ( S 2 ) = [ χ n − 1 , 1 − 2 α 2 ( n − 1 ) S 2 , χ n − 1 , 2 α 2 ( n − 1 ) S 2 ] σ 2 \sigma^2 σ 2
⟹ \implies ⟹ H 0 : σ 2 = σ 0 2 H_0:\sigma^2=\sigma^2_0 H 0 : σ 2 = σ 0 2 H 1 : σ 2 ≠ σ 0 2 H_1:\sigma^2\neq\sigma^2_0 H 1 : σ 2 = σ 0 2
但對於最佳的 C a , b ( S 2 ) C_{a,b}(S^2) C a , b ( S 2 ) ( 1 a − 1 b ) (\frac{1}{a}-\frac{1}{b}) ( a 1 − b 1 ) G ( b ) − G ( a ) = 1 − α G(b)-G(a)=1-\alpha G ( b ) − G ( a ) = 1 − α G G G χ n − 1 2 \chi^2_{n-1} χ n − 1 2
⟹ { − 1 a 2 = λ ( − g ( a ) ) 1 b 2 = λ g ( b ) G ( b ) − G ( a ) = 1 − α ⟹ { a 2 g ( a ) = b 2 g ( b ) G ( b ) − G ( a ) = 1 − α \implies\begin{cases}
-\frac{1}{a^2}=\lambda(-g(a))\\
\frac{1}{b^2}=\lambda g(b)\\
G(b)-G(a)=1-\alpha
\end{cases}\implies
\begin{cases}
a^2g(a)=b^2g(b)\\
G(b)-G(a)=1-\alpha
\end{cases} ⟹ ⎩ ⎨ ⎧ − a 2 1 = λ ( − g ( a )) b 2 1 = λ g ( b ) G ( b ) − G ( a ) = 1 − α ⟹ { a 2 g ( a ) = b 2 g ( b ) G ( b ) − G ( a ) = 1 − α 解出來的 a , b a, b a , b
上面找的都是 σ 2 \sigma^2 σ 2 σ = σ 2 \sigma=\sqrt{\sigma^2} σ = σ 2
1 − α = P ( A ≤ σ 2 ≤ B ) = P ( A ≤ σ ≤ B ) ⟹ 1 − α = P θ ( θ ∈ C ( X ~ ) ) = P θ ( h ( θ ) ∈ h ( C ( X ~ ) ) ) \begin{align*}
1-\alpha&=P(A\le\sigma^2\le B)\\
&=P(\sqrt{A}\le\sigma\le\sqrt{B})\\
\end{align*}\implies
\begin{align*}
1-\alpha&=P_\theta(\theta\in C(\utilde{X}))\\
&=P_\theta(h(\theta)\in h(C(\utilde{X})))\\
\end{align*} 1 − α = P ( A ≤ σ 2 ≤ B ) = P ( A ≤ σ ≤ B ) ⟹ 1 − α = P θ ( θ ∈ C ( X )) = P θ ( h ( θ ) ∈ h ( C ( X ))) EX X 1 , ⋯ , X n ∼ iid e − ( x − θ ) X_1,\cdots, X_n\overset{\text{iid}}{\sim}e^{-(x-\theta)} X 1 , ⋯ , X n ∼ iid e − ( x − θ ) x ≥ θ x\ge\theta x ≥ θ ⟹ X ( 1 ) \implies X_{(1)} ⟹ X ( 1 ) θ \theta θ X ( 1 ) − θ X_{(1)}-\theta X ( 1 ) − θ
Y = 2 n ( X ( 1 ) − θ ) ∼ 1 2 e − y 2 , y > 0 = d Gamma ( 1 , 2 ) = d χ 2 2 ⊥ θ Y=2n(X_{(1)}-\theta)\sim \frac{1}{2}e^{-\frac{y}{2}}, y>0\overset{d}{=}\text{Gamma}(1,2)\overset{d}{=}\chi^2_2\perp\theta Y = 2 n ( X ( 1 ) − θ ) ∼ 2 1 e − 2 y , y > 0 = d Gamma ( 1 , 2 ) = d χ 2 2 ⊥ θ ⟹ ∀ 0 ≤ a < b \implies \forall 0\le a<b ⟹ ∀0 ≤ a < b
1 − α = ∫ a b 1 2 e − y 2 d y = P ( a ≤ Y ≤ b ) = P θ ( a ≤ 2 n ( X ( 1 ) − θ ) ≤ b ) = P θ ( X ( 1 ) − b 2 n ≤ θ ≤ X ( 1 ) − a 2 n ) = P θ ( θ ∈ C a , b ( X ( 1 ) ) ) \begin{align*}
1-\alpha&=\int_a^b\frac{1}{2}e^{-\frac{y}{2}}dy\\
&=P(a\le Y\le b)\\
&=P_{\theta}(a\le 2n(X_{(1)}-\theta)\le b)\\
&=P_{\theta}\left(X_{(1)}-\frac{b}{2n}\le\theta\le X_{(1)}-\frac{a}{2n}\right)\\
&=P_{\theta}\left(\theta\in C_{a,b}(X_{(1)})\right)\\
\end{align*} 1 − α = ∫ a b 2 1 e − 2 y d y = P ( a ≤ Y ≤ b ) = P θ ( a ≤ 2 n ( X ( 1 ) − θ ) ≤ b ) = P θ ( X ( 1 ) − 2 n b ≤ θ ≤ X ( 1 ) − 2 n a ) = P θ ( θ ∈ C a , b ( X ( 1 ) ) ) with C a , b ( X ( 1 ) ) = [ X ( 1 ) − b 2 n , X ( 1 ) − a 2 n ] C_{a,b}(X_{(1)})=\left[X_{(1)}-\frac{b}{2n}, X_{(1)}-\frac{a}{2n}\right] C a , b ( X ( 1 ) ) = [ X ( 1 ) − 2 n b , X ( 1 ) − 2 n a ] 1 − α 1-\alpha 1 − α θ \theta θ
為了找到最短的 confidence set，我們要找到最小的 b − a b-a b − a G ( b ) − G ( a ) = 1 − α G(b)-G(a)=1-\alpha G ( b ) − G ( a ) = 1 − α G G G χ 2 2 \chi^2_2 χ 2 2
{ − 1 = λ ( − g ( a ) ) 1 = λ g ( b ) G ( b ) − G ( a ) = 1 − α ⟹ g ( a ) = g ( b ) ⟹ a = − b \begin{cases}
-1=\lambda(-g(a))\\
1=\lambda g(b)\\
G(b)-G(a)=1-\alpha
\end{cases}\implies
g(a)=g(b)\implies a=-b ⎩ ⎨ ⎧ − 1 = λ ( − g ( a )) 1 = λ g ( b ) G ( b ) − G ( a ) = 1 − α ⟹ g ( a ) = g ( b ) ⟹ a = − b 但注意到 g ( y ) = 1 2 e − y 2 g(y)=\frac{1}{2}e^{-\frac{y}{2}} g ( y ) = 2 1 e − 2 y a = 0 , b = χ 2 , α 2 a=0, b=\chi^2_{2,\alpha} a = 0 , b = χ 2 , α 2
∫ 0 b 1 2 e − y 2 d y = 1 − α ⟹ 1 − e − b 2 = 1 − α ⟹ b = − 2 log ( 1 − α ) \int_0^b \frac{1}{2}e^{-\frac{y}{2}}dy=1-\alpha\implies 1-e^{-\frac{b}{2}}=1-\alpha\implies b=-2\log(1-\alpha) ∫ 0 b 2 1 e − 2 y d y = 1 − α ⟹ 1 − e − 2 b = 1 − α ⟹ b = − 2 log ( 1 − α ) i.e. [ X ( 1 ) + 2 log ( 1 − α ) 2 n , X ( 1 ) ] [X_{(1)}+\frac{2\log(1-\alpha)}{2n}, X_{(1)}] [ X ( 1 ) + 2 n 2 l o g ( 1 − α ) , X ( 1 ) ] 1 − α 1-\alpha 1 − α θ \theta θ H 0 : θ = θ 0 H_0:\theta=\theta_0 H 0 : θ = θ 0 H 1 : θ ≠ θ 0 H_1:\theta\neq\theta_0 H 1 : θ = θ 0
reject H 0 ⟺ θ 0 ∉ C a , b ( X ( 1 ) ) ⟺ θ 0 < X ( 1 ) − a 2 n or θ 0 > X ( 1 ) − b 2 n \text{reject } H_0\iff\theta_0\notin C_{a,b}(X_{(1)})\iff \theta_0<X_{(1)}-\frac{a}{2n} \text{ or } \theta_0>X_{(1)}-\frac{b}{2n} reject H 0 ⟺ θ 0 ∈ / C a , b ( X ( 1 ) ) ⟺ θ 0 < X ( 1 ) − 2 n a or θ 0 > X ( 1 ) − 2 n b 事實上這個檢定是 UMP ，因此這個信賴區間是 UMA 。
信賴區間的長度最短並不代表是 UMA 或 UMAU 的。
要說明一個信賴區間是 UMA 或 UMAU 的，我們需要通過檢定的結果來說明。
EX X 1 , ⋯ , X n ∼ iid U ( 0 , θ ) X_1, \cdots, X_n\overset{\text{iid}}{\sim}U(0, \theta) X 1 , ⋯ , X n ∼ iid U ( 0 , θ ) f ( x ; θ ) = 1 θ ⋅ I ( 0 ≤ x θ ≤ 1 ) f(x;\theta)=\frac{1}{\theta}\cdot I(0\le\frac{x}{\theta}\le 1) f ( x ; θ ) = θ 1 ⋅ I ( 0 ≤ θ x ≤ 1 )
T = X ( n ) T=X_{(n)} T = X ( n ) θ ⟹ \theta\implies θ ⟹ W = X ( n ) θ ∈ [ 0 , 1 ] W=\frac{X_{(n)}}{\theta}\in[0,1] W = θ X ( n ) ∈ [ 0 , 1 ] 0 ≤ a < b ≤ 1 0\le a<b\le 1 0 ≤ a < b ≤ 1
1 − α = P ( a ≤ W ≤ b ) = P θ ( a ≤ X ( n ) θ ≤ b ) = P θ ( a θ ≤ X ( n ) ≤ b θ ) = ( b θ θ ) n − ( a θ θ ) n = b n − a n = P θ ( X ( n ) b ≤ θ ≤ X ( n ) a ) \begin{align*}
1-\alpha&=P(a\le W\le b)\\
&=P_\theta(a\le\frac{X_{(n)}}{\theta}\le b)\\
&=P_\theta(a\theta\le X_{(n)}\le b\theta)=\left(\frac{b\theta}{\theta}\right)^n-\left(\frac{a\theta}{\theta}\right)^n=b^n-a^n\\
&=P_\theta\left(\frac{X_{(n)}}{b}\le\theta\le\frac{X_{(n)}}{a}\right)\\
\end{align*} 1 − α = P ( a ≤ W ≤ b ) = P θ ( a ≤ θ X ( n ) ≤ b ) = P θ ( a θ ≤ X ( n ) ≤ b θ ) = ( θ b θ ) n − ( θ a θ ) n = b n − a n = P θ ( b X ( n ) ≤ θ ≤ a X ( n ) ) Let C a , b ( X ( n ) ) = [ X ( n ) b , X ( n ) a ] ⟹ ∀ 0 ≤ a < b ≤ 1 C_{a,b}(X_{(n)})=[\frac{X_{(n)}}{b}, \frac{X_{(n)}}{a}]\implies \forall 0\le a<b\le 1 C a , b ( X ( n ) ) = [ b X ( n ) , a X ( n ) ] ⟹ ∀0 ≤ a < b ≤ 1 b n − a n = 1 − α b^n-a^n=1-\alpha b n − a n = 1 − α C a , b ( X ( n ) ) C_{a,b}(X_{(n)}) C a , b ( X ( n ) ) 1 − α 1-\alpha 1 − α θ \theta θ
接下來，為了得到最短的信賴區間，我們要最小化 1 a − 1 b \frac{1}{a}-\frac{1}{b} a 1 − b 1 b n − a n = 1 − α b^n-a^n=1-\alpha b n − a n = 1 − α
∵ b n − a n = 1 − α ⟹ a = a ( b ) \because b^n-a^n=1-\alpha\implies a = a(b) ∵ b n − a n = 1 − α ⟹ a = a ( b ) l ( b ) = 1 a ( b ) − 1 b l(b)=\frac{1}{a(b)}-\frac{1}{b} l ( b ) = a ( b ) 1 − b 1
n ⋅ b n − 1 − n ⋅ a n − 1 ⋅ a ′ ( b ) = 0 a ′ ( b ) = b n − 1 a n − 1 ⟹ l ′ ( b ) = − a ′ ( b ) a 2 + 1 b 2 = − 1 a 2 b n − 1 a n − 1 + 1 b 2 = a n + 1 − b n + 1 a n + 1 b 2 < 0 \begin{align*}
n\cdot b^{n-1}-n\cdot a^{n-1}\cdot a'(b)&=0\\
a'(b)&=\frac{b^{n-1}}{a^{n-1}}\\
\end{align*}
\implies
\begin{align*}
l'(b)&=-\frac{a'(b)}{a^2}+\frac{1}{b^2}=-\frac{1}{a^2}\frac{b^{n-1}}{a^{n-1}}+\frac{1}{b^2}\\
&=\frac{a^{n+1}-b^{n+1}}{a^{n+1}b^2}<0
\end{align*} n ⋅ b n − 1 − n ⋅ a n − 1 ⋅ a ′ ( b ) a ′ ( b ) = 0 = a n − 1 b n − 1 ⟹ l ′ ( b ) = − a 2 a ′ ( b ) + b 2 1 = − a 2 1 a n − 1 b n − 1 + b 2 1 = a n + 1 b 2 a n + 1 − b n + 1 < 0 ⟹ l ( b ) ↘ \implies l(b) \searrow ⟹ l ( b ) ↘ b ≤ 1 ⟹ l ( b ) b\le 1\implies l(b) b ≤ 1 ⟹ l ( b ) b = 1 ⟹ a = α 1 n b=1\implies a=\alpha^{\frac{1}{n}} b = 1 ⟹ a = α n 1 [ X ( n ) , X ( n ) α − 1 n ] [X_{(n)}, X_{(n)}\alpha^{-\frac{1}{n}}] [ X ( n ) , X ( n ) α − n 1 ]
For testing H 0 : θ = θ 0 H_0:\theta=\theta_0 H 0 : θ = θ 0 H 1 : θ ≠ θ 0 H_1:\theta\neq\theta_0 H 1 : θ = θ 0
⟹ reject H 0 ⟺ θ 0 ∉ [ X ( n ) , X ( n ) α − 1 n ] ⟺ θ 0 < X ( n ) or θ 0 > X ( n ) α − 1 n ⟺ X ( n ) > θ 0 or X ( n ) < θ 0 α 1 n is UMP level α test \begin{align*}
\implies\text{reject }H_0 &\iff \theta_0\notin [X_{(n)}, X_{(n)}\alpha^{-\frac{1}{n}}]\\
&\iff \theta_0<X_{(n)}\text{ or }\theta_0>X_{(n)}\alpha^{-\frac{1}{n}}\\
&\iff X_{(n)} > \theta_0 \text{ or } X_{(n)} < \theta_0\alpha^{\frac{1}{n}}\\
&\text{ is UMP level }\alpha\text{ test}
\end{align*} ⟹ reject H 0 ⟺ θ 0 ∈ / [ X ( n ) , X ( n ) α − n 1 ] ⟺ θ 0 < X ( n ) or θ 0 > X ( n ) α − n 1 ⟺ X ( n ) > θ 0 or X ( n ) < θ 0 α n 1 is UMP level α test ⟹ [ X ( n ) , X ( n ) α − 1 n ] \implies [X_{(n)}, X_{(n)}\alpha^{-\frac{1}{n}}] ⟹ [ X ( n ) , X ( n ) α − n 1 ] UMA 1 − α 1-\alpha 1 − α θ \theta θ
EX X 1 , ⋯ , X n ∼ iid 1 λ e − x λ ∈ X_1, \cdots, X_n\overset{\text{iid}}{\sim}\frac{1}{\lambda}e^{-\frac{x}{\lambda}}\in X 1 , ⋯ , X n ∼ iid λ 1 e − λ x ∈ ∑ X i \sum X_i ∑ X i λ \lambda λ
Note that
∑ i = 1 n X i ∼ Gamma ( n , λ ) ⟹ 2 ∑ X i λ ∼ Gamma ( n , 2 ) = χ 2 n 2 ⊥ λ \sum_{i=1}^n X_i \sim \text{Gamma}(n, \lambda)\implies \frac{2\sum X_i}{\lambda}\sim\text{Gamma}(n, 2)=\chi^2_{2n}\perp\lambda i = 1 ∑ n X i ∼ Gamma ( n , λ ) ⟹ λ 2 ∑ X i ∼ Gamma ( n , 2 ) = χ 2 n 2 ⊥ λ ⟹ \implies ⟹ 2 ∑ X i λ ∼ χ 2 n 2 \frac{2\sum X_i}{\lambda}\sim \chi^2_{2n} λ 2 ∑ X i ∼ χ 2 n 2
⟹ 1 − α = P λ ( χ 2 n , 1 − α 2 2 ≤ 2 ∑ X i λ ≤ χ 2 n , α 2 2 ) = P λ ( 2 ∑ X i χ 2 n , α 2 2 ≤ λ ≤ 2 ∑ X i χ 2 n , 1 − α 2 2 ) \begin{align*}
\implies 1-\alpha&=P_\lambda(\chi^2_{2n, 1-\frac{\alpha}{2}}\le\frac{2\sum X_i}{\lambda}\le\chi^2_{2n, \frac{\alpha}{2}})\\
&=P_\lambda\left(\frac{2\sum X_i}{\chi^2_{2n,\frac{\alpha}{2}}}\le\lambda\le\frac{2\sum X_i}{\chi^2_{2n, 1-\frac{\alpha}{2}}}\right)\\
\end{align*} ⟹ 1 − α = P λ ( χ 2 n , 1 − 2 α 2 ≤ λ 2 ∑ X i ≤ χ 2 n , 2 α 2 ) = P λ ( χ 2 n , 2 α 2 2 ∑ X i ≤ λ ≤ χ 2 n , 1 − 2 α 2 2 ∑ X i ) ⟹ [ 2 ∑ X i χ 2 n , 1 − α 2 2 , 2 ∑ X i χ 2 n , α 2 2 ] \implies \left[\frac{2\sum X_i}{\chi^2_{2n, 1-\frac{\alpha}{2}}}, \frac{2\sum X_i}{\chi^2_{2n, \frac{\alpha}{2}}}\right] ⟹ [ χ 2 n , 1 − 2 α 2 2 ∑ X i , χ 2 n , 2 α 2 2 ∑ X i ] 1 − α 1-\alpha 1 − α λ \lambda λ
EX
⊥ < X 1 , ⋯ , X n ∼ iid 1 λ e − x λ , x > 0 Y 1 , ⋯ , Y m ∼ iid 1 θ e − y θ , y > 0 ⟹ ⊥ < 2 ∑ X i λ ∼ χ 2 n 2 ⊥ λ 2 ∑ Y i θ ∼ χ 2 m 2 ⊥ θ \perp\Big<\begin{align*}
&X_1, \cdots, X_n\overset{\text{iid}}{\sim}\frac{1}{\lambda}e^{-\frac{x}{\lambda}}, x>0\\
&Y_1, \cdots, Y_m\overset{\text{iid}}{\sim}\frac{1}{\theta}e^{-\frac{y}{\theta}}, y>0
\end{align*}
\quad\implies\quad
\perp\Big<\begin{align*}
&\frac{2\sum X_i}{\lambda}\sim\chi^2_{2n}\perp\lambda\\
&\frac{2\sum Y_i}{\theta}\sim\chi^2_{2m}\perp\theta
\end{align*} ⊥ ⟨ X 1 , ⋯ , X n ∼ iid λ 1 e − λ x , x > 0 Y 1 , ⋯ , Y m ∼ iid θ 1 e − θ y , y > 0 ⟹ ⊥ ⟨ λ 2 ∑ X i ∼ χ 2 n 2 ⊥ λ θ 2 ∑ Y i ∼ χ 2 m 2 ⊥ θ 2 ∑ X i λ ( 2 n ) / 2 ∑ Y i θ ( 2 m ) ∼ F 2 n , 2 m = X ˉ λ / Y ˉ θ = X ˉ Y ˉ θ λ \begin{align*}
&\frac{2\sum X_i}{\lambda(2n)}\Bigg/\frac{2\sum Y_i}{\theta(2m)}\sim F_{2n, 2m}\\
=&\frac{\bar{X}}{\lambda}\bigg/\frac{\bar{Y}}{\theta}=\frac{\bar{X}}{\bar{Y}}\frac{\theta}{\lambda}
\end{align*} = λ ( 2 n ) 2 ∑ X i / θ ( 2 m ) 2 ∑ Y i ∼ F 2 n , 2 m λ X ˉ / θ Y ˉ = Y ˉ X ˉ λ θ 1 − α = P ( F 2 n , 2 m , 1 − α 2 ≤ F 2 n , 2 m ≤ F 2 n , 2 m , α 2 ) = P θ , λ ( F 2 n , 2 m , 1 − α 2 ≤ X ˉ Y ˉ θ λ ≤ F 2 n , 2 m , α 2 ) = P θ , λ ( Y ˉ X ˉ F 2 n , 2 m , 1 − α 2 ≤ θ λ ≤ Y ˉ X ˉ F 2 n , 2 m , α 2 ) \begin{align*}
1-\alpha&=P(F_{2n,2m,1-\frac{\alpha}{2}}\le F_{2n,2m}\le F_{2n,2m,\frac{\alpha}{2}})\\
&=P_{\theta,\lambda}\left(F_{2n,2m,1-\frac{\alpha}{2}}\le\frac{\bar{X}}{\bar{Y}}\frac{\theta}{\lambda}\le F_{2n,2m,\frac{\alpha}{2}}\right)\\
&=P_{\theta,\lambda}\left(\frac{\bar{Y}}{\bar{X}}F_{2n,2m,1-\frac{\alpha}{2}}\le\frac{\theta}{\lambda}\le\frac{\bar{Y}}{\bar{X}}F_{2n,2m,\frac{\alpha}{2}}\right)\\
\end{align*} 1 − α = P ( F 2 n , 2 m , 1 − 2 α ≤ F 2 n , 2 m ≤ F 2 n , 2 m , 2 α ) = P θ , λ ( F 2 n , 2 m , 1 − 2 α ≤ Y ˉ X ˉ λ θ ≤ F 2 n , 2 m , 2 α ) = P θ , λ ( X ˉ Y ˉ F 2 n , 2 m , 1 − 2 α ≤ λ θ ≤ X ˉ Y ˉ F 2 n , 2 m , 2 α ) ⟹ [ Y ˉ X ˉ F 2 n , 2 m , 1 − α 2 , Y ˉ X ˉ F 2 n , 2 m , α 2 ] \implies \left[\frac{\bar{Y}}{\bar{X}}F_{2n,2m,1-\frac{\alpha}{2}}, \frac{\bar{Y}}{\bar{X}}F_{2n,2m,\frac{\alpha}{2}}\right] ⟹ [ X ˉ Y ˉ F 2 n , 2 m , 1 − 2 α , X ˉ Y ˉ F 2 n , 2 m , 2 α ] 1 − α 1-\alpha 1 − α θ λ \frac{\theta}{\lambda} λ θ
⟹ \implies ⟹ H 0 : λ = θ H_0:\lambda=\theta H 0 : λ = θ H 1 : λ ≠ θ H_1:\lambda\neq\theta H 1 : λ = θ ⟺ \iff ⟺ H 0 : θ λ = 1 H_0:\frac{\theta}{\lambda}=1 H 0 : λ θ = 1 H 1 : θ λ ≠ 1 H_1:\frac{\theta}{\lambda}\neq 1 H 1 : λ θ = 1
Reject H 0 H_0 H 0 1 ∉ [ Y ˉ X ˉ F 2 n , 2 m , 1 − α 2 , Y ˉ X ˉ F 2 n , 2 m , α 2 ] 1\notin\left[\frac{\bar{Y}}{\bar{X}}F_{2n,2m,1-\frac{\alpha}{2}}, \frac{\bar{Y}}{\bar{X}}F_{2n,2m,\frac{\alpha}{2}}\right] 1 ∈ / [ X ˉ Y ˉ F 2 n , 2 m , 1 − 2 α , X ˉ Y ˉ F 2 n , 2 m , 2 α ]
EX
< X 1 , ⋯ , X n ∼ iid N ( μ x , σ 2 ) Y 1 , ⋯ , Y m ∼ iid N ( μ y , σ 2 ) \Big < \begin{align*}
&X_1, \cdots, X_n\overset{\text{iid}}{\sim}N(\mu_x, \sigma^2)\\
&Y_1, \cdots, Y_m\overset{\text{iid}}{\sim}N(\mu_y, \sigma^2)
\end{align*} ⟨ X 1 , ⋯ , X n ∼ iid N ( μ x , σ 2 ) Y 1 , ⋯ , Y m ∼ iid N ( μ y , σ 2 ) Note that S p 2 ≜ ( n − 1 ) S X 2 + ( m − 1 ) S Y 2 n + m − 2 S_p^2\triangleq\frac{(n-1)S^2_X+(m-1)S^2_Y}{n+m-2} S p 2 ≜ n + m − 2 ( n − 1 ) S X 2 + ( m − 1 ) S Y 2 σ 2 \sigma^2 σ 2
X ˉ − Y ˉ − ( μ x − μ y ) ( 1 n + 1 m ) S p 2 ∼ t n + m − 2 ⊥ ( μ , σ 2 ) \frac{\bar{X}-\bar{Y}-(\mu_x-\mu_y)}{\sqrt{(\frac{1}{n}+\frac{1}{m})S_p^2}}\sim t_{n+m-2}\perp(\mu, \sigma^2) ( n 1 + m 1 ) S p 2 X ˉ − Y ˉ − ( μ x − μ y ) ∼ t n + m − 2 ⊥ ( μ , σ 2 ) ⟹ [ ( X ˉ − Y ˉ ) ± t n + m − 2 , α 2 ( 1 n + 1 m ) S p 2 ] \implies [(\bar{X}-\bar{Y})\pm t_{n+m-2,\frac{\alpha}{2}}\sqrt{(\frac{1}{n}+\frac{1}{m})S_p^2}] ⟹ [( X ˉ − Y ˉ ) ± t n + m − 2 , 2 α ( n 1 + m 1 ) S p 2 ] 1 − α 1-\alpha 1 − α μ x − μ y \mu_x-\mu_y μ x − μ y
To test H 0 : μ x = μ y H_0:\mu_x=\mu_y H 0 : μ x = μ y H 1 : μ x ≠ μ y H_1:\mu_x\neq\mu_y H 1 : μ x = μ y H 0 H_0 H 0 0 ∉ [ ( X ˉ − Y ˉ ) ± t n + m − 2 , α 2 ( 1 n + 1 m ) S p 2 ] 0\notin\left[(\bar{X}-\bar{Y})\pm t_{n+m-2,\frac{\alpha}{2}}\sqrt{(\frac{1}{n}+\frac{1}{m})S_p^2}\right] 0 ∈ / [ ( X ˉ − Y ˉ ) ± t n + m − 2 , 2 α ( n 1 + m 1 ) S p 2 ]
⟹ [ ( X ˉ − Y ˉ ) ± t n + m − 2 , α 2 ( 1 n + 1 m ) S p 2 ] \implies \left[(\bar{X}-\bar{Y})\pm t_{n+m-2,\frac{\alpha}{2}}\sqrt{(\frac{1}{n}+\frac{1}{m})S_p^2}\right] ⟹ [ ( X ˉ − Y ˉ ) ± t n + m − 2 , 2 α ( n 1 + m 1 ) S p 2 ] 1 − α 1-\alpha 1 − α μ x − μ y \mu_x-\mu_y μ x − μ y
Asymptotic
這個方法會讓 n → ∞ n\to\infty n → ∞ Large Sample method 。
As n → ∞ n\to\infty n → ∞ W ( x ~ ) − θ V ( X ~ ) → D N ( 0 , 1 ) \frac{W(\utilde{x})-\theta}{V(\utilde{X})}\xrightarrow{D}N(0,1) V ( X ) W ( x ) − θ D N ( 0 , 1 )
P ( a ≤ W ( X ~ ) − θ V ( X ~ ) ≤ b ) ≈ P ( a ≤ Z ≤ b ) P\left(a\le\frac{W(\utilde{X})-\theta}{V(\utilde{X})}\le b\right)\approx P\left(a\le Z\le b\right) P ( a ≤ V ( X ) W ( X ) − θ ≤ b ) ≈ P ( a ≤ Z ≤ b ) i.e. 1 − α ≈ P θ ( − Z α 2 ≤ W ( X ~ ) − θ V ( X ~ ) ≤ Z α 2 ) = P θ ( W ( X ~ ) − Z α 2 V ( X ~ ) ≤ θ ≤ W ( X ~ ) + Z α 2 V ( X ~ ) ) = P θ ( θ ∈ [ W ( X ~ ) ± Z α 2 V ( X ~ ) ] ) ∀ θ \begin{align*}
\text{i.e. }1-\alpha&\approx P_\theta\left(-Z_{\frac{\alpha}{2}}\le\frac{W(\utilde{X})-\theta}{V(\utilde{X})}\le Z_{\frac{\alpha}{2}}\right)\\
&=P_\theta\left(W(\utilde{X})-Z_{\frac{\alpha}{2}}V(\utilde{X})\le\theta\le W(\utilde{X})+Z_{\frac{\alpha}{2}}V(\utilde{X})\right)\\
&=P_\theta\left(\theta\in [W(\utilde{X})\pm Z_{\frac{\alpha}{2}}V(\utilde{X})]\right)\quad \forall\theta
\end{align*} i.e. 1 − α ≈ P θ ( − Z 2 α ≤ V ( X ) W ( X ) − θ ≤ Z 2 α ) = P θ ( W ( X ) − Z 2 α V ( X ) ≤ θ ≤ W ( X ) + Z 2 α V ( X ) ) = P θ ( θ ∈ [ W ( X ) ± Z 2 α V ( X )] ) ∀ θ ⟹ [ W ( X ~ ) ± Z α 2 V ( X ~ ) ] \implies [W(\utilde{X})\pm Z_{\frac{\alpha}{2}}V(\utilde{X})] ⟹ [ W ( X ) ± Z 2 α V ( X )] approximated 1 − α 1-\alpha 1 − α θ \theta θ
Remark : 當有確定的樣本數時，永遠都是推導精確 的信賴區間。只有當被要求時，才用 approximated 的信賴區間。
C.L.T(中央極限定理)
X 1 , ⋯ , X n X_1,\cdots,X_n X 1 , ⋯ , X n E ( X i ) = μ , V a r ( X i ) ∈ ( 0 , ∞ ) E(X_i)=\mu, Var(X_i)\in(0, \infty) E ( X i ) = μ , Va r ( X i ) ∈ ( 0 , ∞ ) X ˉ = 1 n ∑ i = 1 n X i \bar{X}=\frac{1}{n}\sum_{i=1}^n X_i X ˉ = n 1 ∑ i = 1 n X i
n ( X ˉ − μ ) σ → n → ∞ D N ( 0 , 1 ) \frac{\sqrt{n}(\bar{X}-\mu)}{\sigma}\xrightarrow[n\to\infty]{D}N(0,1) σ n ( X ˉ − μ ) D n → ∞ N ( 0 , 1 ) i.e. P ( n ( X ˉ − μ ) σ ≤ t ) ≈ Φ ( t ) ∀ t for large enough n \text{i.e. }P\left(\frac{\sqrt{n}(\bar{X}-\mu)}{\sigma}\le t\right)\approx\Phi(t) \quad \forall t\text{ for large enough } n i.e. P ( σ n ( X ˉ − μ ) ≤ t ) ≈ Φ ( t ) ∀ t for large enough n ⟹ if σ 2 known ⟹ P ( − Z α 2 ≤ n ( X ˉ − μ ) σ ≤ Z α 2 ) ≈ 1 − α ⟺ P μ ( μ ∈ [ X ˉ − σ n Z α 2 , X ˉ + σ n Z α 2 ] ) ≈ 1 − α ∀ μ \begin{align*}
&\implies \text{ if }\sigma^2\text{ known}\\
&\implies P\left(-Z_{\frac{\alpha}{2}}\le\frac{\sqrt{n}(\bar{X}-\mu)}{\sigma}\le Z_{\frac{\alpha}{2}}\right)\approx 1-\alpha\\
&\iff P_\mu\left(\mu\in[\bar{X}-\frac{\sigma}{\sqrt{n}}Z_{\frac{\alpha}{2}}, \bar{X}+\frac{\sigma}{\sqrt{n}}Z_{\frac{\alpha}{2}}]\right)\approx 1-\alpha\quad\forall\mu
\end{align*} ⟹ if σ 2 known ⟹ P ( − Z 2 α ≤ σ n ( X ˉ − μ ) ≤ Z 2 α ) ≈ 1 − α ⟺ P μ ( μ ∈ [ X ˉ − n σ Z 2 α , X ˉ + n σ Z 2 α ] ) ≈ 1 − α ∀ μ i.e. [ X ˉ ± σ n Z α 2 ] [\bar{X}\pm\frac{\sigma}{\sqrt{n}}Z_{\frac{\alpha}{2}}] [ X ˉ ± n σ Z 2 α ] 1 − α 1-\alpha 1 − α μ \mu μ
Also P μ ( n ( X ˉ − μ ) σ ≤ Z α ) = P μ ( μ ≥ X ˉ − σ n Z α ) ≈ P ( Z ≤ Z α ) = 1 − α ∀ μ \begin{align*}
\text{Also } &P_\mu\left(\frac{\sqrt{n}(\bar{X}-\mu)}{\sigma}\le Z_{\alpha}\right)=P_\mu\left(\mu\ge\bar{X}-\frac{\sigma}{\sqrt{n}}Z_{\alpha}\right)\\
\approx& P(Z\le Z_{\alpha})=1-\alpha\quad\forall\mu
\end{align*} Also ≈ P μ ( σ n ( X ˉ − μ ) ≤ Z α ) = P μ ( μ ≥ X ˉ − n σ Z α ) P ( Z ≤ Z α ) = 1 − α ∀ μ i.e. [ X ˉ − σ n Z α , ∞ ) [\bar{X}-\frac{\sigma}{\sqrt{n}}Z_{\alpha}, \infty) [ X ˉ − n σ Z α , ∞ ) 1 − α 1-\alpha 1 − α μ \mu μ
但以上結論都是在 σ 2 \sigma^2 σ 2 σ 2 \sigma^2 σ 2
Slutsky's Theorem
r.v.'s Y n → n → ∞ D W Y_n\xrightarrow[n\to\infty]{D}W Y n D n → ∞ W P ( Y n ≤ t ) → n → ∞ P ( W ≤ t ) P(Y_n\le t)\xrightarrow[n\to\infty]{}P(W\le t) P ( Y n ≤ t ) n → ∞ P ( W ≤ t ) ∀ t \forall t ∀ t F w F_w F w t t t
r.v. T n → n → ∞ P c T_n\xrightarrow[n\to\infty]{P}c T n P n → ∞ c
⟹ T n Y n → n → ∞ D c W \implies T_nY_n\xrightarrow[n\to\infty]{D}cW ⟹ T n Y n D n → ∞ c W EX X 1 , ⋯ , X n X_1,\cdots, X_n X 1 , ⋯ , X n E ( X i ) = μ , V a r ( X i ) = σ 2 ∈ ( 0 , ∞ ) E(X_i)=\mu, Var(X_i)=\sigma^2\in(0, \infty) E ( X i ) = μ , Va r ( X i ) = σ 2 ∈ ( 0 , ∞ )
If σ 2 \sigma^2 σ 2
n ( X ˉ − μ ) S = n ( X ˉ − μ ) σ → D N ( 0 , 1 ) S σ → P 1 → D N ( 0 , 1 ) \frac{\sqrt{n}(\bar{X}-\mu)}{S}=\frac{\frac{\sqrt{n}(\bar{X}-\mu)}{\sigma}\xrightarrow{D}N(0,1)}{\frac{S}{\sigma}\xrightarrow{P}1} \xrightarrow{D}N(0,1) S n ( X ˉ − μ ) = σ S P 1 σ n ( X ˉ − μ ) D N ( 0 , 1 ) D N ( 0 , 1 ) i.e. If σ 2 \sigma^2 σ 2
[ X ˉ ± S n Z α 2 ] [\bar{X}\pm\frac{S}{\sqrt{n}}Z_{\frac{\alpha}{2}}] [ X ˉ ± n S Z 2 α ] 1 − α 1-\alpha 1 − α μ \mu μ [ X ˉ − S n Z α , ∞ ) [\bar{X}-\frac{S}{\sqrt{n}}Z_{\alpha}, \infty) [ X ˉ − n S Z α , ∞ ) 1 − α 1-\alpha 1 − α μ \mu μ
EX X 1 , ⋯ , X n ∼ iid B ( 1 , p ) X_1, \cdots, X_n\overset{\text{iid}}{\sim}B(1, p) X 1 , ⋯ , X n ∼ iid B ( 1 , p ) E ( X i ) = p , V a r ( X i ) = p ( 1 − p ) E(X_i)=p, Var(X_i)=p(1-p) E ( X i ) = p , Va r ( X i ) = p ( 1 − p )
By C.L.T n ( X ˉ − p ) p ( 1 − p ) → D N ( 0 , 1 ) \text{By C.L.T}\qquad\frac{\sqrt{n}(\bar{X}-p)}{\sqrt{p(1-p)}}\xrightarrow{D}N(0,1) By C.L.T p ( 1 − p ) n ( X ˉ − p ) D N ( 0 , 1 ) 但其中分母部分有 p p p [ L ( X ~ ) , U ( X ~ ) ] [L(\utilde{X}),U(\utilde{X})] [ L ( X ) , U ( X )]
L.L.N X ˉ → P p ⟹ X ˉ → P p ⟹ X ˉ ( 1 − X ˉ ) → P p ( 1 − p ) \text{L.L.N }\bar{X}\xrightarrow{P}p\implies\bar{X}\xrightarrow{P}p\implies\sqrt{\bar{X}(1-\bar{X})}\xrightarrow{P}\sqrt{p(1-p)} L.L.N X ˉ P p ⟹ X ˉ P p ⟹ X ˉ ( 1 − X ˉ ) P p ( 1 − p ) ⟹ [ X ˉ ± Z α 2 X ˉ ( 1 − X ˉ ) / n ] \implies \left[\bar{X}\pm Z_{\frac{\alpha}{2}}\sqrt{\bar{X}(1-\bar{X})}/\sqrt{n}\right] ⟹ [ X ˉ ± Z 2 α X ˉ ( 1 − X ˉ ) / n ] 1 − α 1-\alpha 1 − α p p p
Recall : ∀ a n → 1 , b n → 0 \forall a_n\to 1, b_n\to 0 ∀ a n → 1 , b n → 0 a n X ˉ + b n → P p a_n\bar{X}+b_n\xrightarrow{P}p a n X ˉ + b n P p
S 2 = 1 n − 1 ∑ i = 1 n ( X i − X ˉ ) 2 = 1 n − 1 ∑ i = 1 n X i 2 − n n − 1 X ˉ 2 = 1 n − 1 ∑ i = 1 n X i − n n − 1 X ˉ 2 X i ∈ { 0 , 1 } in Bernoulli = n n − 1 X ˉ − n n − 1 X ˉ 2 = n n − 1 X ˉ ( 1 − X ˉ ) \begin{align*}
S^2&=\frac{1}{n-1}\sum_{i=1}^n(X_i-\bar{X})^2\\
&=\frac{1}{n-1}\sum_{i=1}^nX_i^2-\frac{n}{n-1}\bar{X}^2\\
&=\frac{1}{n-1}\sum_{i=1}^nX_i-\frac{n}{n-1}\bar{X}^2\quad X_i\in\{0,1\}\text{ in Bernoulli}\\
&=\frac{n}{n-1}\bar{X}-\frac{n}{n-1}\bar{X}^2\\
&=\frac{n}{n-1}\bar{X}(1-\bar{X})\\
\end{align*} S 2 = n − 1 1 i = 1 ∑ n ( X i − X ˉ ) 2 = n − 1 1 i = 1 ∑ n X i 2 − n − 1 n X ˉ 2 = n − 1 1 i = 1 ∑ n X i − n − 1 n X ˉ 2 X i ∈ { 0 , 1 } in Bernoulli = n − 1 n X ˉ − n − 1 n X ˉ 2 = n − 1 n X ˉ ( 1 − X ˉ ) ⟹ [ X ˉ ± X ˉ ( 1 − X ˉ ) n − 1 Z α 2 ] \implies\left[\bar{X}\pm \frac{\sqrt{\bar{X}(1-\bar{X})}}{\sqrt{n-1}}Z_{\frac{\alpha}{2}}\right] ⟹ [ X ˉ ± n − 1 X ˉ ( 1 − X ˉ ) Z 2 α ] 1 − α 1-\alpha 1 − α p p p
Remark : 如果 X ˉ = 1 \bar{X}=1 X ˉ = 1 0 0 0
EX X 1 , ⋯ , X n ∼ iid P ( λ ) X_1, \cdots, X_n\overset{\text{iid}}{\sim}P(\lambda) X 1 , ⋯ , X n ∼ iid P ( λ ) E ( X i ) = λ = V a r ( X i ) E(X_i)=\lambda=Var(X_i) E ( X i ) = λ = Va r ( X i ) X ˉ \bar{X} X ˉ S 2 S^2 S 2 λ \lambda λ
⟹ [ X ˉ ± Z α 2 X ˉ or S 2 n ] \implies \left[\bar{X}\pm Z_{\frac{\alpha}{2}}\sqrt{\frac{\bar{X}\text{ or }S^2}{n}}\right] ⟹ [ X ˉ ± Z 2 α n X ˉ or S 2 ] 1 − α 1-\alpha 1 − α λ \lambda λ
我們還可以直接假設區間 C ( X ~ ) = { λ : − Z α 2 ≤ n ( X ˉ − λ ) λ ≤ Z α 2 } C(\utilde{X})=\set{\lambda: -Z_{\frac{\alpha}{2}}\le\frac{\sqrt{n}(\bar{X}-\lambda)}{\sqrt{\lambda}}\le Z_{\frac{\alpha}{2}}} C ( X ) = { λ : − Z 2 α ≤ λ n ( X ˉ − λ ) ≤ Z 2 α } λ \lambda λ
δ \delta δ 如果我們關注參數 μ \mu μ n ( X ˉ − μ ) σ → D N ( 0 , 1 ) , S → P σ \frac{\sqrt{n}(\bar{X}-\mu)}{\sigma}\xrightarrow{D}N(0,1), S\xrightarrow{P}\sigma σ n ( X ˉ − μ ) D N ( 0 , 1 ) , S P σ μ \mu μ g ( μ ) g(\mu) g ( μ ) δ \delta δ
δ \delta δ n → ∞ n\to\infty n → ∞ ∀ g \forall g ∀ g g ′ g' g ′ g ′ ( μ ) ≠ 0 g'(\mu)\neq 0 g ′ ( μ ) = 0
If n ( T n − μ ) → D N ( 0 , σ 2 ) \sqrt{n}(T_n-\mu)\xrightarrow{D}N(0, \sigma^2) n ( T n − μ ) D N ( 0 , σ 2 )
then n ( g ( T n ) − g ( μ ) ) → D N ( 0 , σ 2 ⋅ [ g ′ ( μ ) ] 2 ) \sqrt{n}(g(T_n)-g(\mu))\xrightarrow{D}N(0, \sigma^2\cdot[g'(\mu)]^2) n ( g ( T n ) − g ( μ )) D N ( 0 , σ 2 ⋅ [ g ′ ( μ ) ] 2 )
EX X 1 , ⋯ , X n ∼ iid B ( 1 , p ) X_1,\cdots, X_n\overset{\text{iid}}{\sim}B(1,p) X 1 , ⋯ , X n ∼ iid B ( 1 , p )
我們關心 p 1 − p \frac{p}{1-p} 1 − p p p p p p 1 − p \frac{p}{1-p} 1 − p p
Let g ( p ) = log ( p 1 − p ) ≜ logit ( p ) = log ( p ) − log ( 1 − p ) \begin{align*}
\text{Let }g(p)&=\log\left(\frac{p}{1-p}\right)\triangleq \text{logit}(p)\\
&=\log(p)-\log(1-p)\\
\end{align*} Let g ( p ) = log ( 1 − p p ) ≜ logit ( p ) = log ( p ) − log ( 1 − p ) ⟹ g ′ ( p ) = 1 p − 1 1 − p = 1 p ( 1 − p ) \implies g'(p)=\frac{1}{p}-\frac{1}{1-p}=\frac{1}{p(1-p)} ⟹ g ′ ( p ) = p 1 − 1 − p 1 = p ( 1 − p ) 1 n ( X ˉ − p ) p ( 1 − p ) → D N ( 0 , 1 ) ⟺ n ( X ˉ − p ) → D N ( 0 , p ( 1 − p ) ) δ -methpd with g ( p ) = log ( p 1 − p ) ⟹ n ( g ( X ˉ ) − g ( p ) ) → D N ( 0 , [ p ( 1 − p ) ] ⋅ [ 1 p ( 1 − p ) ] 2 ) \begin{align*}
\frac{\sqrt{n}(\bar{X}-p)}{\sqrt{p(1-p)}}\xrightarrow{D}N(0,1) &\iff\sqrt{n}(\bar{X}-p)\xrightarrow{D}N(0, p(1-p))\\
\delta\text{-methpd with }g(p)=\log(\frac{p}{1-p})&\implies \sqrt{n}(g(\bar{X})-g(p))\xrightarrow{D}N(0, [p(1-p)]\cdot[\frac{1}{p(1-p)}]^2)\\
\end{align*} p ( 1 − p ) n ( X ˉ − p ) D N ( 0 , 1 ) δ -methpd with g ( p ) = log ( 1 − p p ) ⟺ n ( X ˉ − p ) D N ( 0 , p ( 1 − p )) ⟹ n ( g ( X ˉ ) − g ( p )) D N ( 0 , [ p ( 1 − p )] ⋅ [ p ( 1 − p ) 1 ] 2 ) i.e. n ( log ( T n − T ) − log ( p 1 − p ) ) → D N ( 0 , 1 ) with T = ∑ X i ⟺ n ( log ( T n − T ) − log ( p 1 − p ) ) 1 p ( 1 − p ) → D N ( 0 , 1 ) ⟺ log ( T n − T ) − log ( p 1 − p ) 1 n ( 1 p + 1 1 − p ) → D N ( 0 , 1 ) \begin{align*}
\text{i.e.} &\sqrt{n}\left(\log\left(\frac{T}{n-T}\right)-\log\left(\frac{p}{1-p}\right)\right)\xrightarrow{D}N(0, 1)\qquad\text{with }T=\sum X_i\\
\iff &\frac{\sqrt{n}\left(\log\left(\frac{T}{n-T}\right)-\log\left(\frac{p}{1-p}\right)\right)}{\sqrt{\frac{1}{p(1-p)}}}\xrightarrow{D}N(0, 1)\\
\iff &\frac{\log\left(\frac{T}{n-T}\right)-\log\left(\frac{p}{1-p}\right)}{\sqrt{\frac{1}{n}\left(\frac{1}{p}+\frac{1}{1-p}\right)}}\xrightarrow{D}N(0, 1)\\
\end{align*} i.e. ⟺ ⟺ n ( log ( n − T T ) − log ( 1 − p p ) ) D N ( 0 , 1 ) with T = ∑ X i p ( 1 − p ) 1 n ( log ( n − T T ) − log ( 1 − p p ) ) D N ( 0 , 1 ) n 1 ( p 1 + 1 − p 1 ) log ( n − T T ) − log ( 1 − p p ) D N ( 0 , 1 ) 分母部分仍有 p p p X ˉ \bar{X} X ˉ p p p Slutsky's Theorem ，收斂性不會受到影響。
⟹ log ( T n − T ) − log ( p 1 − p ) 1 n ( n T + n n − T ) → D N ( 0 , 1 ) ⟹ P p ( log ( p 1 − p ) ∈ [ log ( T n − T ) ± Z α 2 1 T + 1 n − T ] ) ≈ 1 − α \begin{align*}
\implies& \frac{\log\left(\frac{T}{n-T}\right)-\log\left(\frac{p}{1-p}\right)}{\sqrt{\frac{1}{\cancel{n}}\left(\frac{\cancel{n}}{T}+\frac{\cancel{n}}{n-T}\right)}}\xrightarrow{D}N(0, 1)\\
\implies& P_p\left(\log\left(\frac{p}{1-p}\right)\in\left[\log\left(\frac{T}{n-T}\right)\pm Z_{\frac{\alpha}{2}}\sqrt{\frac{1}{T}+\frac{1}{n-T}}\right]\right)\approx 1-\alpha
\end{align*} ⟹ ⟹ n 1 ( T n + n − T n ) log ( n − T T ) − log ( 1 − p p ) D N ( 0 , 1 ) P p ( log ( 1 − p p ) ∈ [ log ( n − T T ) ± Z 2 α T 1 + n − T 1 ] ) ≈ 1 − α ⟹ \implies ⟹ [ log ( T n − T ) ± Z α 2 1 T + 1 n − T ] [\log\left(\frac{T}{n-T}\right)\pm Z_{\frac{\alpha}{2}}\sqrt{\frac{1}{T}+\frac{1}{n-T}}] [ log ( n − T T ) ± Z 2 α T 1 + n − T 1 ] 1 − α 1-\alpha 1 − α log ( p 1 − p ) \log\left(\frac{p}{1-p}\right) log ( 1 − p p )
⟹ \implies ⟹ [ exp ( log ( T n − T ) ± Z α 2 1 T + 1 n − T ) ] \left[\exp\left(\log\left(\frac{T}{n-T}\right)\pm Z_{\frac{\alpha}{2}}\sqrt{\frac{1}{T}+\frac{1}{n-T}}\right)\right] [ exp ( log ( n − T T ) ± Z 2 α T 1 + n − T 1 ) ] 1 − α 1-\alpha 1 − α p 1 − p \frac{p}{1-p} 1 − p p
因為在 B ( 1 , p ) B(1,p) B ( 1 , p ) ∑ X i = T \sum X_i=T ∑ X i = T n − T n-T n − T T n − T \frac{T}{n-T} n − T T p 1 − p \frac{p}{1-p} 1 − p p
for large n n n
log ( T x n − T x ) − log ( p 1 1 − p 1 ) ≈ 1 T x + 1 n − T x Z 1 log ( T y m − T y ) − log ( p 2 1 − p 2 ) ≈ 1 T y + 1 m − T y Z 2 with Z 1 , Z 2 ∼ N ( 0 , 1 ) iid \begin{align*}
&\log\left(\frac{T_x}{n-T_x}\right)-\log\left(\frac{p_1}{1-p_1}\right)\approx \sqrt{\frac{1}{T_x}+\frac{1}{n-T_x}}Z_1\\
&\log\left(\frac{T_y}{m-T_y}\right)-\log\left(\frac{p_2}{1-p_2}\right)\approx \sqrt{\frac{1}{T_y}+\frac{1}{m-T_y}}Z_2
\end{align*}\qquad\text{with }Z_1, Z_2\sim N(0,1) \text{ iid} log ( n − T x T x ) − log ( 1 − p 1 p 1 ) ≈ T x 1 + n − T x 1 Z 1 log ( m − T y T y ) − log ( 1 − p 2 p 2 ) ≈ T y 1 + m − T y 1 Z 2 with Z 1 , Z 2 ∼ N ( 0 , 1 ) iid log ( T x n − T x m − T y T y ) − log ( p 1 1 − p 1 p 2 1 − p 2 ) ≈ 1 T x + 1 n − T x Z 1 + 1 T y + 1 m − T y Z 2 = N ( 0 , 1 T x + 1 n − T x + 1 T y + 1 m − T y ) \begin{align*}
\log\left(\frac{T_x}{n-T_x}\frac{m-T_y}{T_y}\right)-\log\left(\frac{p_1}{1-p_1}\frac{p_2}{1-p_2}\right)&\approx \sqrt{\frac{1}{T_x}+\frac{1}{n-T_x}}Z_1+\sqrt{\frac{1}{T_y}+\frac{1}{m-T_y}}Z_2\\
&=N(0,\frac{1}{T_x}+\frac{1}{n-T_x}+\frac{1}{T_y}+\frac{1}{m-T_y})
\end{align*} log ( n − T x T x T y m − T y ) − log ( 1 − p 1 p 1 1 − p 2 p 2 ) ≈ T x 1 + n − T x 1 Z 1 + T y 1 + m − T y 1 Z 2 = N ( 0 , T x 1 + n − T x 1 + T y 1 + m − T y 1 ) 如果我們關注的是勝算比（odds ratio）p 1 1 − p 1 / p 2 1 − p 2 ≜ θ \frac{p_1}{1-p_1}\big/\frac{p_2}{1-p_2}\triangleq\theta 1 − p 1 p 1 / 1 − p 2 p 2 ≜ θ
⟹ P p 1 , p 2 ( log ( θ ) ∈ [ log ( T x n − T x m − T y T y ) ± Z α 2 1 T x + 1 n − T x + 1 T y + 1 m − T y ] ) ≈ 1 − α \implies P_{p_1,p_2}\left(\log(\theta)\in\left[\log\left(\frac{T_x}{n-T_x}\frac{m-T_y}{T_y}\right)\pm Z_{\frac{\alpha}{2}}\sqrt{\frac{1}{T_x}+\frac{1}{n-T_x}+\frac{1}{T_y}+\frac{1}{m-T_y}}\right]\right)\approx 1-\alpha ⟹ P p 1 , p 2 ( log ( θ ) ∈ [ log ( n − T x T x T y m − T y ) ± Z 2 α T x 1 + n − T x 1 + T y 1 + m − T y 1 ] ) ≈ 1 − α 正面 反面 總和 X T x T_x T x n − T x n-T_x n − T x n Y T y T_y T y m − T y m-T_y m − T y m
EX ：對於阿司匹林是否能預防心臟病這一問題收集到以下數據：
是否有心臟病 有 沒有 阿司匹林 104 10933 安慰劑 189 10845
我們關心 odds ratio θ = p 1 1 − p 1 / p 2 1 − p 2 \theta=\frac{p_1}{1-p_1}\big/\frac{p_2}{1-p_2} θ = 1 − p 1 p 1 / 1 − p 2 p 2 p 2 p_2 p 2 p 1 p_1 p 1
⟹ [ ln ( 189 × 10933 10845 × 104 ) ± 1.96 1 189 + 1 10845 + 1 104 + 1 10933 ] = [ 0.365 , 0.846 ] is 95 % conf. int. for ln ( θ ) ⟹ [ e 0.365 , e 0.846 ] = [ 1.44 , 2.33 ] is 95 % conf. int. for θ \begin{align*}
&\implies \left[\ln\left(\frac{189\times 10933}{10845\times 104}\right)\pm 1.96\sqrt{\frac{1}{189}+\frac{1}{10845}+\frac{1}{104}+\frac{1}{10933}} \right]=[0.365, 0.846]\text{ is } 95\%\text{ conf. int. for }\ln(\theta)\\
&\implies \left[e^{0.365}, e^{0.846}\right]=[1.44, 2.33]\text{ is } 95\%\text{ conf. int. for }\theta
\end{align*} ⟹ [ ln ( 10845 × 104 189 × 10933 ) ± 1.96 189 1 + 10845 1 + 104 1 + 10933 1 ] = [ 0.365 , 0.846 ] is 95% conf. int. for ln ( θ ) ⟹ [ e 0.365 , e 0.846 ] = [ 1.44 , 2.33 ] is 95% conf. int. for θ 如果阿司匹林明顯有預防心臟病的效果，那麼 p 2 p_2 p 2 p 1 p_1 p 1
H 0 : p 1 = p 2 vs. H 1 : p 1 > p 2 ⟺ H 0 : θ = 1 vs. H 1 : θ > 1 \begin{align*}
& H_0: p_1=p_2\text{ vs. }H_1: p_1>p_2\\
\iff& H_0: \theta=1\text{ vs. }H_1: \theta>1\\
\end{align*} ⟺ H 0 : p 1 = p 2 vs. H 1 : p 1 > p 2 H 0 : θ = 1 vs. H 1 : θ > 1 1 ∉ [ 1.44 , 2.33 ] ⟹ 1\notin [1.44, 2.33]\implies 1 ∈ / [ 1.44 , 2.33 ] ⟹ H 0 H_0 H 0 5 % 5\% 5%
EX :
⊥ < X ∼ B ( n , p ) Y ∼ B ( m , p ) θ = p 1 ( 1 − p 2 ) ( 1 − p 1 ) p 2 \perp\Big<\begin{align*}
&X\sim B(n,p)\\
&Y\sim B(m,p)
\end{align*}\qquad\theta=\frac{p_1(1-p_2)}{(1-p_1)p_2} ⊥ ⟨ X ∼ B ( n , p ) Y ∼ B ( m , p ) θ = ( 1 − p 1 ) p 2 p 1 ( 1 − p 2 ) 將獲得的 Binomal 數據整理成以下 2 × 2 2\times 2 2 × 2
Data S F Total x p 1 x_{p_1} x p 1 n − x ( 1 − p 1 ) n-x_{(1-p_1)} n − x ( 1 − p 1 ) n y p 2 y_{p_2} y p 2 m − x ( 1 − p 2 ) m-x_{(1-p_2)} m − x ( 1 − p 2 ) m
ln θ ∈ [ ln ( x ( m − y ) y ( n − x ) ± Z α 2 1 x + 1 n − x + 1 y + 1 m − y ) ] = [ A , B ] with conf ≈ 1 − α ⟹ θ ∈ [ e A , e B ] with conf ≈ 1 − α \begin{align*}
&\ln\theta\in \left[\ln\left(\frac{x(m-y)}{y(n-x)}\pm Z_{\frac{\alpha}{2}}\sqrt{\frac{1}{x}+\frac{1}{n-x}+\frac{1}{y}+\frac{1}{m-y}} \right)\right]=[A,B] \text{ with conf }\approx 1-\alpha\\
\implies& \theta\in [e^A, e^B] \text{ with conf }\approx 1-\alpha
\end{align*} ⟹ ln θ ∈ [ ln ( y ( n − x ) x ( m − y ) ± Z 2 α x 1 + n − x 1 + y 1 + m − y 1 ) ] = [ A , B ] with conf ≈ 1 − α θ ∈ [ e A , e B ] with conf ≈ 1 − α Remark ：這個結論對於從 multinomial / Poisson 獲得樣本所組成的列聯表也是成立的。
EX :
收集到以下數據。A, B 是關心的兩個不同的事件（e.g. 天氣、路口），中間表格則是交集事件發生的次數，以及發生幾率。這組數據服從 multinomial distribution。
B 1 B_1 B 1 B 2 B_2 B 2 A 1 A_1 A 1 X 11 , P 11 X_{11}, P_{11} X 11 , P 11 X 12 , P 12 X_{12}, P_{12} X 12 , P 12 A 2 A_2 A 2 X 21 , P 21 X_{21}, P_{21} X 21 , P 21 X 22 , P 22 X_{22}, P_{22} X 22 , P 22
∑ i = 1 2 ∑ j = 1 2 X i j = n ∑ i = 1 2 ∑ j = 1 2 P i j = 1 \sum_{i=1}^2\sum_{j=1}^2 X_{ij}=n\quad \sum_{i=1}^2\sum_{j=1}^2 P_{ij}=1 ∑ i = 1 2 ∑ j = 1 2 X ij = n ∑ i = 1 2 ∑ j = 1 2 P ij = 1
X ~ = ( X 11 , X 12 , X 21 , X 22 ) P ~ = ( P 11 , P 12 , P 21 , P 22 ) ∼ Multinomial n ( k , P ~ ) k = col × row = 4 \begin{align*}
&\utilde{X}=(X_{11}, X_{12}, X_{21}, X_{22})\\
&\utilde{P}=(P_{11}, P_{12}, P_{21}, P_{22})
\end{align*}\sim \text{Multinomial}_n(k, \utilde{P})\quad k=\text{ col }\times \text{ row }=4 X = ( X 11 , X 12 , X 21 , X 22 ) P = ( P 11 , P 12 , P 21 , P 22 ) ∼ Multinomial n ( k , P ) k = col × row = 4 θ ≜ P 11 P 22 P 12 P 21 : odds ratio \theta\triangleq \frac{P_{11}P_{22}}{P_{12}P_{21}}: \text{ odds ratio} θ ≜ P 12 P 21 P 11 P 22 : odds ratio 如果 A ⊥ B A\perp B A ⊥ B A 1 A_1 A 1 A 2 A_2 A 2 B 1 B_1 B 1 B 2 B_2 B 2
H 0 : A ⊥ B ⟺ H 0 : P 11 P 12 = P 21 P 22 ⟺ H 0 : P 11 P 22 P 12 P 21 = θ = 1 ⟺ H 0 : ln θ = 0 \begin{align*}
& H_0: A\perp B\\
\iff& H_0: \frac{P_{11}}{P_{12}}=\frac{P_{21}}{P_{22}}\\
\iff& H_0: \frac{P_{11}P_{22}}{P_{12}P_{21}}=\theta=1\\
\iff& H_0: \ln\theta=0
\end{align*} ⟺ ⟺ ⟺ H 0 : A ⊥ B H 0 : P 12 P 11 = P 22 P 21 H 0 : P 12 P 21 P 11 P 22 = θ = 1 H 0 : ln θ = 0 ⟹ ln θ ∈ [ ln ( X 11 X 22 X 12 X 21 ) ± Z α 2 1 X 11 + 1 X 12 + 1 X 21 + 1 X 22 ] = [ A , B ] with conf ≈ 1 − α ⟹ θ ∈ [ e A , e B ] with conf ≈ 1 − α \begin{align*}
&\implies \ln\theta\in \left[\ln\left(\frac{X_{11}X_{22}}{X_{12}X_{21}}\right)\pm Z_{\frac{\alpha}{2}}\sqrt{\frac{1}{X_{11}}+\frac{1}{X_{12}}+\frac{1}{X_{21}}+\frac{1}{X_{22}}}\right]=[A,B] \text{ with conf }\approx 1-\alpha\\
&\implies \theta\in [e^A, e^B] \text{ with conf }\approx 1-\alpha
\end{align*} ⟹ ln θ ∈ [ ln ( X 12 X 21 X 11 X 22 ) ± Z 2 α X 11 1 + X 12 1 + X 21 1 + X 22 1 ] = [ A , B ] with conf ≈ 1 − α ⟹ θ ∈ [ e A , e B ] with conf ≈ 1 − α 廣泛來說，X ~ ∼ Multinomial n ( k , P ~ ) \utilde{X}\sim \text{Multinomial}_n(k,\utilde{P}) X ∼ Multinomial n ( k , P ) k = I × J k=I\times J k = I × J I × J I\times J I × J
A\B B 1 B_1 B 1 B 2 B_2 B 2 ⋯ \cdots ⋯ B J B_J B J A 1 A_1 A 1 X 11 X_{11} X 11 X 12 X_{12} X 12 ⋯ \cdots ⋯ X 1 J X_{1J} X 1 J A 2 A_2 A 2 X 21 X_{21} X 21 X 22 X_{22} X 22 ⋯ \cdots ⋯ X 2 J X_{2J} X 2 J ⋮ \vdots ⋮ ⋮ \vdots ⋮ ⋮ \vdots ⋮ ⋱ \ddots ⋱ ⋮ \vdots ⋮ A I A_I A I X I 1 X_{I1} X I 1 X I 2 X_{I2} X I 2 ⋯ \cdots ⋯ X I J X_{IJ} X I J
而關注的問題通常是 H 0 : A ⊥ B H_0:A\perp B H 0 : A ⊥ B H 1 : A ⊥̸ B H_1:A\not\perp B H 1 : A ⊥ B 2 × 2 2\times 2 2 × 2
Multivariate
C.L.T
X ~ 1 , ⋯ , X ~ n iid with E ( X ~ i ) = θ ~ , V a r ( X ~ i ) = Σ ~ \utilde{X}_1, \cdots, \utilde{X}_n \text{ iid with } E(\utilde{X}_i)=\utilde{\theta}, Var(\utilde{X}_i)=\bcancel{\utilde{\Sigma}} X 1 , ⋯ , X n iid with E ( X i ) = θ , Va r ( X i ) = Σ 如果每個隨機向量 X ~ i \utilde{X}_i X i m m m X ˉ = 1 n ∑ X ~ i \bar{X}=\frac{1}{n}\sum\utilde{X}_i X ˉ = n 1 ∑ X i m m m
⟹ n ( X ˉ − θ ~ ) → n → ∞ D N m ( 0 ~ , Σ ~ ) \implies \sqrt{n}(\bar{X}-\utilde{\theta})\xrightarrow[n\to\infty]{D}N_m(\utilde{0}, \bcancel{\utilde{\Sigma}}) ⟹ n ( X ˉ − θ ) D n → ∞ N m ( 0 , Σ ) EX : Y ~ ∼ Multinomial n ( k , P ~ ) \utilde{Y}\sim \text{Multinomial}_n(k,\utilde{P}) Y ∼ Multinomial n ( k , P )
1 2 ⋯ \cdots ⋯ k total data Y 1 Y_1 Y 1 Y 2 Y_2 Y 2 ⋯ \cdots ⋯ Y k Y_k Y k n w) prob P 1 P_1 P 1 P 2 P_2 P 2 ⋯ \cdots ⋯ P k P_k P k 1
⟹ \implies ⟹ P j P_j P j Y j n \frac{Y_j}{n} n Y j P ^ ~ = Y ~ n \utilde{\hat{P}}=\frac{\utilde{Y}}{n} P ^ = n Y
如果令 X ~ i \utilde{X}_i X i i i i X ~ i \utilde{X}_i X i
如果單看 X ~ i \utilde{X}_i X i X i j X_{ij} X ij X i j ∼ B ( 1 , P j ) X_{ij}\sim B(1,P_j) X ij ∼ B ( 1 , P j ) E [ X i j ⋅ X i j ′ ] = 0 V a r ( X i j ) = P j ( 1 − P j ) ⟹ ∀ j ≠ j ′ C o v ( X i j , X i j ′ ) = − P j P j ′ E[X_{ij}\cdot X_{ij'}]=0\quad Var(X_{ij})=P_j(1-P_j)\implies\forall j\neq j'\quad Cov(X_{ij},X_{ij'})=-P_jP_{j'} E [ X ij ⋅ X i j ′ ] = 0 Va r ( X ij ) = P j ( 1 − P j ) ⟹ ∀ j = j ′ C o v ( X ij , X i j ′ ) = − P j P j ′
I.e. X ~ i ∼ Multinomial 1 ( k , P ~ ) \utilde{X}_i\sim \text{Multinomial}_1(k, \utilde{P}) X i ∼ Multinomial 1 ( k , P )
E ( X i ) = P ~ σ 2 { X ~ i } = [ P 1 ( 1 − P 1 ) − P 1 P 2 ⋯ − P 1 P k − P 2 P 1 P 2 ( 1 − P 2 ) ⋯ − P 2 P k ⋮ ⋮ ⋱ ⋮ − P k P 1 − P k P 2 ⋯ P k ( 1 − P k ) ] = diag ( P ~ ) − P ~ P ~ t = Σ \begin{align*}
E(X_i)=\utilde{P}\quad \sigma^2\set{\utilde{X}_i}&=\begin{bmatrix}
P_1(1-P_1) & -P_1P_2 & \cdots & -P_1P_k\\
-P_2P_1 & P_2(1-P_2) & \cdots & -P_2P_k\\
\vdots & \vdots & \ddots & \vdots\\
-P_kP_1 & -P_kP_2 & \cdots & P_k(1-P_k)
\end{bmatrix}\\
&=\text{diag}(\utilde{P})-\utilde{P}\utilde{P}^t=\bcancel{\Sigma}
\end{align*} E ( X i ) = P σ 2 { X i } = P 1 ( 1 − P 1 ) − P 2 P 1 ⋮ − P k P 1 − P 1 P 2 P 2 ( 1 − P 2 ) ⋮ − P k P 2 ⋯ ⋯ ⋱ ⋯ − P 1 P k − P 2 P k ⋮ P k ( 1 − P k ) = diag ( P ) − P P t = Σ ⟹ Y ~ = ∑ X ~ i ⟹ P ^ ~ = Y ~ n = X ˉ \implies \utilde{Y}=\sum \utilde{X}_i\implies \utilde{\hat{P}}=\frac{\utilde{Y}}{n}=\bar{X} ⟹ Y = ∑ X i ⟹ P ^ = n Y = X ˉ ⇒ C.L.T n ( P ^ ~ − P ~ ) → D N m ( 0 ~ , Σ ) \xRightarrow{\text{C.L.T}} \sqrt{n}(\utilde{\hat{P}}-\utilde{P})\xrightarrow{D}N_m(\utilde{0}, \bcancel{\Sigma}) C.L.T n ( P ^ − P ) D N m ( 0 , Σ ) δ \delta δ n ( X ˉ − θ ~ ) → D N m ( 0 ~ , Σ ) ⟹ n ( g ( X ˉ ) − g ( θ ~ ) ) → D N m ( 0 ~ , ∇ g ( θ ~ ) t Σ ∇ g ( θ ~ ) ) \begin{align*}
&\sqrt{n}(\bar{X}-\utilde{\theta})\xrightarrow{D}N_m(\utilde{0}, \bcancel{\Sigma})\\
\implies& \sqrt{n}(g(\bar{X})-g(\utilde{\theta}))\xrightarrow{D}N_m(\utilde{0},\nabla g(\utilde{\theta})^t\bcancel{\Sigma}\nabla g(\utilde{\theta}))
\end{align*} ⟹ n ( X ˉ − θ ) D N m ( 0 , Σ ) n ( g ( X ˉ ) − g ( θ )) D N m ( 0 , ∇ g ( θ ) t Σ ∇ g ( θ )) ∀ g : R m → R with ∇ g ( θ ~ ) = ( ∂ g ∂ θ 1 ( θ ~ ) ⋮ ∂ g ∂ θ m ( θ ~ ) ) exists and ≠ 0 ~ \forall g:\R^m\to \R \quad\text{with } \nabla g(\utilde{\theta})=\begin{pmatrix}
\frac{\partial g}{\partial \theta_1}(\utilde{\theta})\\
\vdots\\
\frac{\partial g}{\partial \theta_m}(\utilde{\theta})
\end{pmatrix}\text{ exists and }\neq \utilde{0} ∀ g : R m → R with ∇ g ( θ ) = ∂ θ 1 ∂ g ( θ ) ⋮ ∂ θ m ∂ g ( θ ) exists and = 0 k ≥ 4 n ( P ^ ~ − P ~ ) → D N k ( 0 ~ , Σ ) Σ = diag ( P ~ ) − P ~ P ~ t k\ge 4\quad \sqrt{n}(\utilde{\hat{P}}-\utilde{P})\xrightarrow{D}N_k(\utilde{0}, \bcancel{\Sigma})\quad \bcancel{\Sigma}=\text{diag}(\utilde{P})-\utilde{P}\utilde{P}^t k ≥ 4 n ( P ^ − P ) D N k ( 0 , Σ ) Σ = diag ( P ) − P P t Take g ( P ~ ) = ln ( P 1 P 2 P 3 P 4 ) = ln ( P 1 ) + ln ( P 2 ) − ln ( P 3 ) − ln ( P 4 ) ⟹ ∇ g ( P ~ ) = ( 1 P 1 , 1 P 2 , − 1 P 3 , − 1 P 4 , 0 , ⋯ , 0 ) t : k × 1 \begin{align*}
\text{Take}&\quad g(\utilde{P})=\ln\left(\frac{P_1P_2}{P_3P_4}\right)=\ln(P_1) + \ln(P_2) - \ln(P_3) - \ln(P_4)\\
\implies& \nabla g(\utilde{P})=(\frac{1}{P_1}, \frac{1}{P_2}, -\frac{1}{P_3}, -\frac{1}{P_4}, 0, \cdots, 0)^t: k\times 1
\end{align*} Take ⟹ g ( P ) = ln ( P 3 P 4 P 1 P 2 ) = ln ( P 1 ) + ln ( P 2 ) − ln ( P 3 ) − ln ( P 4 ) ∇ g ( P ) = ( P 1 1 , P 2 1 , − P 3 1 , − P 4 1 , 0 , ⋯ , 0 ) t : k × 1 ( ∇ g ( P ~ ) ) t Σ ∇ g ( P ~ ) = ( ∇ g ( P ~ ) ) t ( diag ( P ~ ) − P ~ P ~ t ) ∇ g ( P ~ ) = ( ∇ g ( P ~ ) ) t diag ( P ~ ) ∇ g ( P ~ ) − ( ∇ g ( P ~ ) ) t P ~ P ~ t ∇ g ( P ~ ) = ∑ i = 1 k P i ( ∇ g ( P ~ ) ) i 2 − [ ( ∇ g ( P ~ ) ) t P ~ ] 2 = 1 P 1 + 1 P 2 + 1 P 3 + 1 P 4 − ( 1 + 1 − 1 − 1 + 0 + ⋯ + 0 ) 2 = 1 P 1 + 1 P 2 + 1 P 3 + 1 P 4 \begin{align*}
&(\nabla g(\utilde{P}))^t\bcancel{\Sigma}\nabla g(\utilde{P})\\
=&(\nabla g(\utilde{P}))^t\left(\text{diag}(\utilde{P})-\utilde{P}\utilde{P}^t\right)\nabla g(\utilde{P})\\
=&(\nabla g(\utilde{P}))^t\text{diag}(\utilde{P})\nabla g(\utilde{P}) - (\nabla g(\utilde{P}))^t\utilde{P}\utilde{P}^t\nabla g(\utilde{P})\\
=&\sum_{i=1}^kP_i(\nabla g(\utilde{P}))_i^2 - \left[(\nabla g(\utilde{P}))^t \utilde{P} \right]^2\\
=&\frac{1}{P_1} + \frac{1}{P_2} + \frac{1}{P_3} + \frac{1}{P_4} - (1+1-1-1+0+\cdots+0)^2\\
=&\frac{1}{P_1} + \frac{1}{P_2} + \frac{1}{P_3} + \frac{1}{P_4}
\end{align*} = = = = = ( ∇ g ( P ) ) t Σ ∇ g ( P ) ( ∇ g ( P ) ) t ( diag ( P ) − P P t ) ∇ g ( P ) ( ∇ g ( P ) ) t diag ( P ) ∇ g ( P ) − ( ∇ g ( P ) ) t P P t ∇ g ( P ) i = 1 ∑ k P i ( ∇ g ( P ) ) i 2 − [ ( ∇ g ( P ) ) t P ] 2 P 1 1 + P 2 1 + P 3 1 + P 4 1 − ( 1 + 1 − 1 − 1 + 0 + ⋯ + 0 ) 2 P 1 1 + P 2 1 + P 3 1 + P 4 1 ⇒ δ -method n ( g ( P ^ ~ ) − g ( P ~ ) ) → D N ( 0 , 1 P 1 + 1 P 2 + 1 P 3 + 1 P 4 ) ⟹ g ( P ^ ~ ) − g ( P ~ ) 1 n ( 1 P 1 + 1 P 2 + 1 P 3 + 1 P 4 ) → D N ( 0 , 1 ) ⇒ slutsky g ( P ^ ~ ) − g ( P ~ ) 1 Y 1 + 1 Y 2 + 1 Y 3 + 1 Y 4 → D N ( 0 , 1 ) P ^ j = Y j n ⟹ ln ( Y 1 Y 2 Y 3 Y 4 ) − ln ( P 1 P 2 P 3 P 4 ) 1 n ( 1 P 1 + 1 P 2 + 1 P 3 + 1 P 4 ) → D N ( 0 , 1 ) \begin{align*}
\xRightarrow{\delta\text{-method}}& \sqrt{n}(g(\utilde{\hat{P}})-g(\utilde{P}))\xrightarrow{D}N\left(0, \frac{1}{P_1} + \frac{1}{P_2} + \frac{1}{P_3} + \frac{1}{P_4}\right)\\
\implies& \frac{g(\utilde{\hat{P}})-g(\utilde{P})}{\sqrt{\frac{1}{n}\left(\frac{1}{P_1}+\frac{1}{P_2}+\frac{1}{P_3}+\frac{1}{P_4} \right)}} \xrightarrow{D}N(0,1)\\
\xRightarrow{\text{slutsky}}& \frac{g(\utilde{\hat{P}})-g(\utilde{P})}{\sqrt{\frac{1}{Y_1}+\frac{1}{Y_2}+\frac{1}{Y_3}+\frac{1}{Y_4}}} \xrightarrow{D}N(0,1)\quad \hat{P}_j=\frac{Y_j}{n}\\
\implies& \frac{\ln\left(\frac{Y_1Y_2}{Y_3Y_4} \right)-\ln\left(\frac{P_1P_2}{P_3P_4} \right)}{\sqrt{\frac{1}{n}\left(\frac{1}{P_1}+\frac{1}{P_2}+\frac{1}{P_3}+\frac{1}{P_4} \right)}} \xrightarrow{D}N(0,1)
\end{align*} δ -method ⟹ slutsky ⟹ n ( g ( P ^ ) − g ( P )) D N ( 0 , P 1 1 + P 2 1 + P 3 1 + P 4 1 ) n 1 ( P 1 1 + P 2 1 + P 3 1 + P 4 1 ) g ( P ^ ) − g ( P ) D N ( 0 , 1 ) Y 1 1 + Y 2 1 + Y 3 1 + Y 4 1 g ( P ^ ) − g ( P ) D N ( 0 , 1 ) P ^ j = n Y j n 1 ( P 1 1 + P 2 1 + P 3 1 + P 4 1 ) ln ( Y 3 Y 4 Y 1 Y 2 ) − ln ( P 3 P 4 P 1 P 2 ) D N ( 0 , 1 ) 因此如果 θ = ln P 1 P 2 P 3 P 4 \theta=\ln\frac{P_1P_2}{P_3P_4} θ = ln P 3 P 4 P 1 P 2
P ( θ ∈ [ ln ( Y 1 Y 2 Y 3 Y 4 ) ± Z α 2 1 Y 1 + 1 Y 2 + 1 Y 3 + 1 Y 4 ] ) ≈ 1 − α for large enough n P\left(\theta\in\left[\ln\left(\frac{Y_1Y_2}{Y_3Y_4} \right)\pm Z_{\frac{\alpha}{2}}\sqrt{\frac{1}{Y_1}+\frac{1}{Y_2}+\frac{1}{Y_3}+\frac{1}{Y_4}} \right] \right)\approx 1-\alpha\quad\text{for large enough }n P ( θ ∈ [ ln ( Y 3 Y 4 Y 1 Y 2 ) ± Z 2 α Y 1 1 + Y 2 1 + Y 3 1 + Y 4 1 ] ) ≈ 1 − α for large enough n Sample size determination with Binomial sample
我們通常會在收集資料之前就定下目標，然後為了驗證目標而收集資料。因此我們會需要知道需要多少樣本才能驗證是否達到目標。而不是收集資料後發現樣本數不夠達到足夠的信心水平。
X 1 , ⋯ , X n ∼ iid B ( 1 , p ) X_1,\cdots, X_n\overset{\text{iid}}{\sim}B(1,p) X 1 , ⋯ , X n ∼ iid B ( 1 , p )
⇒ C.L.T p ∈ [ X ˉ ± X ˉ ( 1 − X ˉ ) n Z α 2 ] with conf ≈ 1 − α \xRightarrow{\text{C.L.T}} p\in \left[\bar{X}\pm \frac{\sqrt{\bar{X}(1-\bar{X})}}{\sqrt{n}}Z_{\frac{\alpha}{2}} \right] \text{ with conf }\approx 1-\alpha C.L.T p ∈ [ X ˉ ± n X ˉ ( 1 − X ˉ ) Z 2 α ] with conf ≈ 1 − α i.e. ∀ p ∈ ( 0 , 1 ) 1 − α ≈ P p ( X ˉ − X ˉ ( 1 − X ˉ ) n Z α 2 ≤ p ≤ X ˉ + X ˉ ( 1 − X ˉ ) n Z α 2 ) = P p ( ∣ X ˉ − p ∣ ≤ X ˉ ( 1 − X ˉ ) n Z α 2 ) \begin{align*}
\text{i.e. }\forall p\in(0,1)\quad 1-\alpha&\approx P_p\left(\bar{X}-\frac{\sqrt{\bar{X}(1-\bar{X})}}{\sqrt{n}}Z_{\frac{\alpha}{2}}\le p\le\bar{X}+\frac{\sqrt{\bar{X}(1-\bar{X})}}{\sqrt{n}}Z_{\frac{\alpha}{2}} \right)\\
&=P_p\left(|\bar{X}-p|\le\frac{\sqrt{\bar{X}(1-\bar{X})}}{\sqrt{n}}Z_{\frac{\alpha}{2}} \right)
\end{align*} i.e. ∀ p ∈ ( 0 , 1 ) 1 − α ≈ P p ( X ˉ − n X ˉ ( 1 − X ˉ ) Z 2 α ≤ p ≤ X ˉ + n X ˉ ( 1 − X ˉ ) Z 2 α ) = P p ( ∣ X ˉ − p ∣ ≤ n X ˉ ( 1 − X ˉ ) Z 2 α ) 其中 X ˉ \bar{X} X ˉ p p p 1 − α 1-\alpha 1 − α X ˉ \bar{X} X ˉ p p p E E E
因為這是在收集數據之前確定要收集的樣本量，因此我們還不知道 X ˉ \bar{X} X ˉ X ˉ ∈ [ 0 , 1 ] \bar{X}\in[0,1] X ˉ ∈ [ 0 , 1 ] X ˉ ( 1 − X ˉ ) ≤ 1 2 \sqrt{\bar{X}(1-\bar{X})}\le \frac{1}{2} X ˉ ( 1 − X ˉ ) ≤ 2 1
X ˉ ( 1 − X ˉ ) n Z α 2 ≤ 1 2 Z α 2 = E ⟺ n = 1 2 E Z α 2 ⟺ n = ( 1 2 E Z α 2 ) 2 \begin{align*}
&\sqrt{\frac{\bar{X}(1-\bar{X})}{n}} Z_{\frac{\alpha}{2}}\le \frac{1}{2}Z_{\frac{\alpha}{2}}=E\\
\iff& \sqrt{n}=\frac{1}{2E}Z_{\frac{\alpha}{2}}\\
\iff& n=\left(\frac{1}{2E}Z_{\frac{\alpha}{2}}\right)^2
\end{align*} ⟺ ⟺ n X ˉ ( 1 − X ˉ ) Z 2 α ≤ 2 1 Z 2 α = E n = 2 E 1 Z 2 α n = ( 2 E 1 Z 2 α ) 2 e.g. α = 0.05 , E = 0.03 ⟹ n ≥ ( 1.96 ) 2 4 ⋅ ( 0.03 ) 2 = 1068 \alpha=0.05, E=0.03\implies n\ge \frac{(1.96)^2}{4\cdot(0.03)^2}=1068 α = 0.05 , E = 0.03 ⟹ n ≥ 4 ⋅ ( 0.03 ) 2 ( 1.96 ) 2 = 1068