Likelihood Ratio Test(LRT) Recall : L ( θ ; x ~ ) = f ( x ~ ; θ ) L(\theta;\utilde{x})=f(\utilde{x};\theta) L ( θ ; x ) = f ( x ; θ ) θ \theta θ x ~ \utilde{x} x θ \theta θ
Note : MP test rejects H 0 = θ = θ 0 H_0=\theta=\theta_0 H 0 = θ = θ 0
f ( x ~ ; θ 1 ) > c ⋅ f ( x ~ ; θ 0 ) ⟺ L ( θ 1 ; x ~ ) > c ⋅ L ( θ 0 ; x ~ ) ⟺ L ( θ 0 ; x ~ ) L ( θ 1 ; x ~ ) < 1 c = c ′ ⟺ L ( θ 0 ; x ~ ) max { L ( θ 0 ; x ~ ) , L ( θ 1 ; x ~ ) } < k some k ∈ [ 0 , 1 ] = { L ( θ 0 ; x ~ ) L ( θ 1 ; x ~ ) if L ( θ 1 ; x ~ ) ≥ L ( θ 0 ; x ~ ) 1 if L ( θ 1 ; x ~ ) < L ( θ 0 ; x ~ ) ⟺ sup θ ∈ ω 0 L ( θ ; x ~ ) sup θ ∈ Ω L ( θ ; x ~ ) < k Ω = ω 0 ∪ ω 1 and ω 0 = { θ 0 } , ω 1 = { θ 1 } \begin{align*}
&f(\utilde{x};\theta_1)>c\cdot f(\utilde{x};\theta_0)\\
\iff &L(\theta_1;\utilde{x})>c\cdot L(\theta_0;\utilde{x})\\
\iff &\frac{L(\theta_0;\utilde{x})}{L(\theta_1;\utilde{x})}<\frac{1}{c}=c'\\
\iff &\frac{L(\theta_0;\utilde{x})}{\max\set{L(\theta_0;\utilde{x}), L(\theta_1;\utilde{x})}}<k\quad\text{some } k\in[0,1]\\
&=\begin{cases}
\frac{L(\theta_0;\utilde{x})}{L(\theta_1;\utilde{x})} &\text{if } L(\theta_1;\utilde{x}) \ge L(\theta_0;\utilde{x})\\
1 &\text{if } L(\theta_1;\utilde{x})<L(\theta_0;\utilde{x})
\end{cases}\\
\iff &\frac{\sup_{\theta\in\omega_0 L(\theta;\utilde{x})}}{\sup_{\theta\in\Omega L(\theta;\utilde{x})}}<k \quad \Omega=\omega_0\cup\omega_1\text{ and } \omega_0=\set{\theta_0}, \omega_1=\set{\theta_1}
\end{align*} ⟺ ⟺ ⟺ ⟺ f ( x ; θ 1 ) > c ⋅ f ( x ; θ 0 ) L ( θ 1 ; x ) > c ⋅ L ( θ 0 ; x ) L ( θ 1 ; x ) L ( θ 0 ; x ) < c 1 = c ′ max { L ( θ 0 ; x ) , L ( θ 1 ; x ) } L ( θ 0 ; x ) < k some k ∈ [ 0 , 1 ] = { L ( θ 1 ; x ) L ( θ 0 ; x ) 1 if L ( θ 1 ; x ) ≥ L ( θ 0 ; x ) if L ( θ 1 ; x ) < L ( θ 0 ; x ) sup θ ∈ Ω L ( θ ; x ) sup θ ∈ ω 0 L ( θ ; x ) < k Ω = ω 0 ∪ ω 1 and ω 0 = { θ 0 } , ω 1 = { θ 1 }
For testing H 0 : θ ∈ ω 0 H_0:\theta\in\omega_0 H 0 : θ ∈ ω 0 H 1 : θ ∈ ω 1 H_1:\theta\in\omega_1 H 1 : θ ∈ ω 1 ϕ \phi ϕ H 0 H_0 H 0 λ ( x ~ ) < k \lambda(\utilde{x})<k λ ( x ) < k with k ∈ [ 0 , 1 ] and λ ( x ~ ) = sup θ ∈ ω 0 L ( θ ; x ~ ) sup θ ∈ Ω L ( θ ; x ~ ) where Ω = ω 0 ∪ ω 1 \text{with }k\in[0,1]\text{ and }\lambda(\utilde{x})=\frac{\sup_{\theta\in\omega_0}L(\theta;\utilde{x})}{\sup_{\theta\in\Omega}L(\theta;\utilde{x})}\text{ where }\Omega=\omega_0\cup\omega_1 with k ∈ [ 0 , 1 ] and λ ( x ) = sup θ ∈ Ω L ( θ ; x ) sup θ ∈ ω 0 L ( θ ; x ) where Ω = ω 0 ∪ ω 1
A level α \alpha α ⟹ \implies ⟹ sup θ ∈ ω 0 E θ ϕ ( x ~ ) = α \sup_{\theta\in\omega_0}E_\theta\phi(\utilde{x})=\alpha sup θ ∈ ω 0 E θ ϕ ( x ) = α
EX X ∼ f ( x ; θ ) X\sim f(x;\theta) X ∼ f ( x ; θ )
H 0 : θ ∈ { θ 1 , θ 2 } v.s. H 1 : θ ∈ { θ 3 , θ 4 } H_0:\theta\in\set{\theta_1,\theta_2} \text{ v.s. }H_1:\theta\in\set{\theta_3,\theta_4} H 0 : θ ∈ { θ 1 , θ 2 } v.s. H 1 : θ ∈ { θ 3 , θ 4 } with level 0.05
x 2 3 4 5 6 7 8 9 10 11 12 θ 1 \theta_1 θ 1 (0.01) (0.01) (0.01) (0.01) (0.01) (0.01) (0.01) (0.01) (0.01) (0.01) 0.90 θ 2 \theta_2 θ 2 (0.01) 0.009 0.008 0.007 0.006 0.005 0.006 0.007 0.008 0.009 [(0.925)] θ 3 \theta_3 θ 3 0.20 [0.10] [0.09] 0.08 0.06 0.05 [0.07] 0.05 0.05 [0.05] 0.20 θ 4 \theta_4 θ 4 [0.26] [0.10] [0.09] [0.11] [0.07] [0.08] [0.07] [0.06] [0.06] [0.05] 0.05 λ = \lambda= λ = 1 26 \frac{1}{26} 26 1 1 10 \frac{1}{10} 10 1 1 9 \frac{1}{9} 9 1 1 11 \frac{1}{11} 11 1 1 7 \frac{1}{7} 7 1 1 8 \frac{1}{8} 8 1 1 7 \frac{1}{7} 7 1 1 6 \frac{1}{6} 6 1 1 6 \frac{1}{6} 6 1 1 5 \frac{1}{5} 5 1 1 1 1
λ ( x ) = sup θ ∈ ω 0 L ( θ ; x ) sup θ ∈ Ω L ( θ ; x ) < k ⟹ k = 1 7 i.e. x = 2 , 3 , 4 , 5 , 7 \lambda(x)=\frac{\sup_{\theta\in\omega_0}L(\theta;x)}{\sup_{\theta\in\Omega}L(\theta;x)}<k\implies k=\frac{1}{7}\quad\text{i.e. }x=2,3,4,5,7 λ ( x ) = sup θ ∈ Ω L ( θ ; x ) sup θ ∈ ω 0 L ( θ ; x ) < k ⟹ k = 7 1 i.e. x = 2 , 3 , 4 , 5 , 7 P θ 1 ( x = 2 or 3 or 4 or 5 or 7 ) = 0.05 P_{\theta_1}(x=2\text{ or }3\text{ or }4\text{ or }5\text{ or }7)=0.05 P θ 1 ( x = 2 or 3 or 4 or 5 or 7 ) = 0.05 Note :
Let θ Ω ^ = arg sup θ ∈ Ω L ( θ ; x ~ ) , θ ω 0 ^ = arg sup θ ∈ ω 0 L ( θ ; x ~ ) \widehat{\theta_\Omega}=\arg\sup_{\theta\in\Omega}L(\theta;\utilde{x}), \widehat{\theta_{\omega_0}}=\arg\sup_{\theta\in\omega_0}L(\theta;\utilde{x}) θ Ω = arg sup θ ∈ Ω L ( θ ; x ) , θ ω 0 = arg sup θ ∈ ω 0 L ( θ ; x )
⟹ λ ( x ~ ) = sup θ ∈ ω 0 L ( θ ; x ) sup θ ∈ Ω L ( θ ; x ) = L ( θ ω 0 ^ ; x ~ ) L ( θ Ω ^ ; x ~ ) \implies\lambda(\utilde{x})=\frac{\sup_{\theta\in\omega_0}L(\theta;x)}{\sup_{\theta\in\Omega}L(\theta;x)}=\frac{L(\widehat{\theta_{\omega_0}};\utilde{x})}{L(\widehat{\theta_\Omega};\utilde{x})} ⟹ λ ( x ) = sup θ ∈ Ω L ( θ ; x ) sup θ ∈ ω 0 L ( θ ; x ) = L ( θ Ω ; x ) L ( θ ω 0 ; x )
T = T ( X ~ ) T=T(\utilde{X}) T = T ( X ) θ \theta θ ⟹ f ( x ~ ; θ ) = g ( t ; θ ) h ( x ~ ) \implies f(\utilde{x};\theta)=g(t;\theta)h(\utilde{x}) ⟹ f ( x ; θ ) = g ( t ; θ ) h ( x )
如果最大值發生在 ω 0 \omega_0 ω 0 λ ( x ~ ) = 1 \lambda(\utilde{x})=1 λ ( x ) = 1 ω 1 \omega_1 ω 1 ω 0 \omega_0 ω 0 ∂ ( ω 0 ) \partial(\omega_0) ∂ ( ω 0 )
λ ( x ~ ) = g ( t ; θ ω 0 ^ ) g ( t ; θ Ω ^ ) = { 1 if θ Ω ^ ∈ ω 0 g ( t ; ∂ ( ω 0 ) ) g ( t ; θ Ω ^ ) if θ Ω ^ ∈ ω 1 \begin{align*}
\lambda(\utilde{x})&=\frac{g(t;\widehat{\theta_{\omega_0}})}{g(t;\widehat{\theta_\Omega})}\\
&=\begin{cases}
1 &\text{if } \widehat{\theta_\Omega}\in\omega_0\\
\frac{g(t;\partial(\omega_0))}{g(t;\widehat{\theta_\Omega})} &\text{if } \widehat{\theta_\Omega}\in\omega_1
\end{cases}
\end{align*} λ ( x ) = g ( t ; θ Ω ) g ( t ; θ ω 0 ) = { 1 g ( t ; θ Ω ) g ( t ; ∂ ( ω 0 )) if θ Ω ∈ ω 0 if θ Ω ∈ ω 1
EX : X 1 , ⋯ , X n ∼ iid N ( θ , σ 0 2 ) X_1, \cdots, X_n\overset{\text{iid}}{\sim} N(\theta, \sigma^2_0) X 1 , ⋯ , X n ∼ iid N ( θ , σ 0 2 )
H 0 : θ = θ 0 v.s. H 1 : θ ≠ θ 0 H_0:\theta=\theta_0\quad\text{ v.s. }\quad H_1:\theta\ne\theta_0 H 0 : θ = θ 0 v.s. H 1 : θ = θ 0 note that ω 0 = { θ 0 } ⟹ θ ω 0 ^ = θ 0 \omega_0=\set{\theta_0}\implies\widehat{\theta_{\omega_0}}=\theta_0 ω 0 = { θ 0 } ⟹ θ ω 0 = θ 0 Ω = { θ 0 } ∪ { θ : θ ≠ θ } = R ⟹ θ Ω ^ = x ˉ \Omega=\set{\theta_0}\cup\set{\theta:\theta\neq\theta}=\R\implies\widehat{\theta_\Omega}=\bar{x} Ω = { θ 0 } ∪ { θ : θ = θ } = R ⟹ θ Ω = x ˉ x ˉ \bar{x} x ˉ
⟹ \implies ⟹ H 0 H_0 H 0
⟺ λ ( x ~ ) = L ( θ 0 ; x ~ ) L ( x ˉ ; x ~ ) < k ⟺ exp ( − n ( x ˉ − θ 0 ) 2 2 σ 0 2 ) < k ′ ⟺ − n ( x ˉ − θ 0 ) 2 2 σ 0 2 < ln k ′ ⟺ n ( x ˉ − θ 0 ) 2 2 σ 0 2 > C ⟺ ∣ n ( x ˉ − θ 0 ) σ 0 ∣ > D \begin{align*}
\iff &\lambda(\utilde{x})=\frac{L(\theta_0;\utilde{x})}{L(\bar{x};\utilde{x})}<k\\
\iff&\exp\left(-\frac{n(\bar{x}-\theta_0)^2}{2\sigma^2_0}\right)<k'\\
\iff&-\frac{n(\bar{x}-\theta_0)^2}{2\sigma^2_0}<\ln k'\\
\iff&\frac{n(\bar{x}-\theta_0)^2}{2\sigma^2_0}>C\\
\iff& \left|\frac{\sqrt{n}(\bar{x}-\theta_0)}{\sigma_0}\right|>D
\end{align*} ⟺ ⟺ ⟺ ⟺ ⟺ λ ( x ) = L ( x ˉ ; x ) L ( θ 0 ; x ) < k exp ( − 2 σ 0 2 n ( x ˉ − θ 0 ) 2 ) < k ′ − 2 σ 0 2 n ( x ˉ − θ 0 ) 2 < ln k ′ 2 σ 0 2 n ( x ˉ − θ 0 ) 2 > C σ 0 n ( x ˉ − θ 0 ) > D and
α = P θ 0 ( ∣ n ( x ˉ − θ 0 ) σ 0 ∣ > D ) = P ( ∣ Z ∣ > D ) \alpha=P_{\theta_0}\left(\left|\frac{\sqrt{n}(\bar{x}-\theta_0)}{\sigma_0}\right|>D\right)=P(|Z|>D) α = P θ 0 ( σ 0 n ( x ˉ − θ 0 ) > D ) = P ( ∣ Z ∣ > D ) I.e., A level α \alpha α H 0 H_0 H 0 ∣ n ( x ˉ − θ 0 ) σ 0 ∣ > Z α / 2 |\frac{\sqrt{n}(\bar{x}-\theta_0)}{\sigma_0}|>Z_{\alpha/2} ∣ σ 0 n ( x ˉ − θ 0 ) ∣ > Z α /2
EX : X 1 , ⋯ , X n ∼ iid N ( θ , σ 2 ) X_1, \cdots, X_n\overset{\text{iid}}{\sim}N(\theta, \sigma^2) X 1 , ⋯ , X n ∼ iid N ( θ , σ 2 ) θ , σ 2 \theta, \sigma^2 θ , σ 2
To testH 0 : θ = θ 0 H_0:\theta=\theta_0 H 0 : θ = θ 0 H 1 : θ ≠ θ 0 H_1:\theta\neq\theta_0 H 1 : θ = θ 0
ω 0 = { ( θ , σ 2 ) : θ = θ 0 , σ 2 > 0 } ⟹ θ ω 0 ^ = θ 0 , σ ω 0 2 ^ = 1 n ∑ i = 1 n ( x i − θ 0 ) 2 \omega_0=\set{(\theta, \sigma^2):\theta=\theta_0, \sigma^2>0}\implies \widehat{\theta_{\omega_0}}=\theta_0, \widehat{\sigma^2_{\omega_0}}=\frac{1}{n}\sum_{i=1}^n(x_i-\theta_0)^2 ω 0 = { ( θ , σ 2 ) : θ = θ 0 , σ 2 > 0 } ⟹ θ ω 0 = θ 0 , σ ω 0 2 = n 1 i = 1 ∑ n ( x i − θ 0 ) 2 Ω = { ( θ , σ 2 ) : θ ≠ θ 0 , σ 2 > 0 } ⟹ θ Ω ^ = x ˉ , σ Ω 2 ^ = 1 n ∑ i = 1 n ( x i − x ˉ ) 2 \Omega=\set{(\theta, \sigma^2):\theta\neq\theta_0, \sigma^2>0}\implies \widehat{\theta_\Omega}=\bar{x}, \widehat{\sigma^2_\Omega}=\frac{1}{n}\sum_{i=1}^n(x_i-\bar{x})^2 Ω = { ( θ , σ 2 ) : θ = θ 0 , σ 2 > 0 } ⟹ θ Ω = x ˉ , σ Ω 2 = n 1 i = 1 ∑ n ( x i − x ˉ ) 2 L ( θ ω 0 ^ , σ ω 0 2 ^ ; x ~ ) = ( 1 2 π σ ω 0 2 ^ ) n / 2 exp ( − 1 2 σ ω 0 2 ^ ∑ i = 1 n ( x i − θ 0 ) 2 ) = ( 1 2 π ) n / 2 ( 1 σ ω 0 2 ^ ) n / 2 exp ( − n 2 ) L ( θ Ω ^ , σ Ω 2 ^ ; x ~ ) = ( 1 2 π σ Ω 2 ^ ) n / 2 exp ( − 1 2 σ Ω 2 ^ ∑ i = 1 n ( x i − x ˉ ) 2 ) = ( 1 2 π ) n / 2 ( 1 σ Ω 2 ^ ) n / 2 exp ( − n 2 ) \begin{align*}
&\begin{align*}
L(\widehat{\theta_{\omega_0}}, \widehat{\sigma^2_{\omega_0}};\utilde{x})&=\left(\frac{1}{2\pi\widehat{\sigma^2_{\omega_0}}}\right)^{n/2}\exp\left(-\frac{1}{2\widehat{\sigma^2_{\omega_0}}}\sum_{i=1}^n(x_i-\theta_0)^2\right)\\
&=\left(\frac{1}{2\pi}\right)^{n/2}\left(\frac{1}{\widehat{\sigma^2_{\omega_0}}}\right)^{n/2}\exp\left(-\frac{n}{2}\right)
\end{align*}\\
&\begin{align*}
L(\widehat{\theta_\Omega}, \widehat{\sigma^2_\Omega};\utilde{x})&=\left(\frac{1}{2\pi\widehat{\sigma^2_\Omega}}\right)^{n/2}\exp\left(-\frac{1}{2\widehat{\sigma^2_\Omega}}\sum_{i=1}^n(x_i-\bar{x})^2\right)\\
&=\left(\frac{1}{2\pi}\right)^{n/2}\left(\frac{1}{\widehat{\sigma^2_\Omega}}\right)^{n/2}\exp\left(-\frac{n}{2}\right)
\end{align*}
\end{align*} L ( θ ω 0 , σ ω 0 2 ; x ) = ( 2 π σ ω 0 2 1 ) n /2 exp ( − 2 σ ω 0 2 1 i = 1 ∑ n ( x i − θ 0 ) 2 ) = ( 2 π 1 ) n /2 ( σ ω 0 2 1 ) n /2 exp ( − 2 n ) L ( θ Ω , σ Ω 2 ; x ) = ( 2 π σ Ω 2 1 ) n /2 exp ( − 2 σ Ω 2 1 i = 1 ∑ n ( x i − x ˉ ) 2 ) = ( 2 π 1 ) n /2 ( σ Ω 2 1 ) n /2 exp ( − 2 n ) reject H 0 ⟺ λ ( x ~ ) = L ( θ ω 0 ^ , σ ω 0 2 ^ ; x ~ ) L ( θ Ω ^ , σ Ω 2 ^ ; x ~ ) = ( σ Ω 2 ^ σ ω 0 2 ^ ) n / 2 < k ⟺ ( ∑ i = 1 n ( x i − x ˉ ) 2 ∑ i = 1 n ( x i − θ 0 ) 2 ) n / 2 < k ⟺ ∑ ( x i − x ˉ ) 2 ∑ ( x i − x ˉ + x ˉ − θ 0 ) 2 = ∑ ( x i − x ˉ ) 2 ∑ ( x i − x ˉ ) 2 + n ( x ˉ − θ 0 ) 2 = 1 1 + n ( x ˉ − θ 0 ) 2 ∑ ( x i − x ˉ ) 2 < k ′ ⟺ n ( x ˉ − θ 0 ) 2 ∑ ( x i − x ˉ ) 2 n − 1 > C ⟺ n ( x ˉ − θ 0 ) 2 s 2 > C X ˉ ∼ N ( θ , σ 2 / n ) ⊥ ( n − 1 ) S 2 σ 2 ∼ χ n − 1 2 ⟺ ∣ n ( x ˉ − θ 0 ) s ∣ > D ∣ n ( x ˉ − θ 0 ) s ∣ ∼ t n − 1 \begin{align*}
\text{reject }H_0&\iff\lambda(\utilde{x})=\frac{L(\widehat{\theta_{\omega_0}}, \widehat{\sigma^2_{\omega_0}};\utilde{x})}{L(\widehat{\theta_\Omega}, \widehat{\sigma^2_\Omega};\utilde{x})}=\left(\frac{\widehat{\sigma^2_\Omega}}{\widehat{\sigma^2_{\omega_0}}}\right)^{n/2}<k\\
&\iff\left(\frac{\sum_{i=1}^n(x_i-\bar{x})^2}{\sum_{i=1}^n(x_i-\theta_0)^2}\right)^{n/2}<k\\
&\iff\frac{\sum(x_i-\bar{x})^2}{\sum(x_i-\bar{x}+\bar{x}-\theta_0)^2}=\frac{\sum(x_i-\bar{x})^2}{\sum(x_i-\bar{x})^2+n(\bar{x}-\theta_0)^2}=\frac{1}{1+\frac{n(\bar{x}-\theta_0)^2}{\sum(x_i-\bar{x})^2}}<k'\\
&\iff\frac{n(\bar{x}-\theta_0)^2}{\frac{\sum(x_i-\bar{x})^2}{n-1}}>C\\
&\iff\frac{n(\bar{x}-\theta_0)^2}{s^2}>C\qquad \bar{X}\sim N(\theta, \sigma^2/n)\perp \frac{(n-1)S^2}{\sigma^2}\sim\chi^2_{n-1}\\
&\iff\left|\frac{\sqrt{n}(\bar{x}-\theta_0)}{s}\right|>D \qquad \left|\frac{\sqrt{n}(\bar{x}-\theta_0)}{s}\right| \sim t_{n-1}
\end{align*} reject H 0 ⟺ λ ( x ) = L ( θ Ω , σ Ω 2 ; x ) L ( θ ω 0 , σ ω 0 2 ; x ) = ( σ ω 0 2 σ Ω 2 ) n /2 < k ⟺ ( ∑ i = 1 n ( x i − θ 0 ) 2 ∑ i = 1 n ( x i − x ˉ ) 2 ) n /2 < k ⟺ ∑ ( x i − x ˉ + x ˉ − θ 0 ) 2 ∑ ( x i − x ˉ ) 2 = ∑ ( x i − x ˉ ) 2 + n ( x ˉ − θ 0 ) 2 ∑ ( x i − x ˉ ) 2 = 1 + ∑ ( x i − x ˉ ) 2 n ( x ˉ − θ 0 ) 2 1 < k ′ ⟺ n − 1 ∑ ( x i − x ˉ ) 2 n ( x ˉ − θ 0 ) 2 > C ⟺ s 2 n ( x ˉ − θ 0 ) 2 > C X ˉ ∼ N ( θ , σ 2 / n ) ⊥ σ 2 ( n − 1 ) S 2 ∼ χ n − 1 2 ⟺ s n ( x ˉ − θ 0 ) > D s n ( x ˉ − θ 0 ) ∼ t n − 1 I.e. a level α \alpha α H 0 H_0 H 0 ∣ n ( x ˉ − θ 0 ) s ∣ > t n − 1 , α / 2 |\frac{\sqrt{n}(\bar{x}-\theta_0)}{s}|>t_{n-1,\alpha/2} ∣ s n ( x ˉ − θ 0 ) ∣ > t n − 1 , α /2
EX : X 1 , ⋯ , X n ∼ iid f ( x ; θ ) = e − ( x − θ ) , x ≥ θ X_1, \cdots, X_n\overset{\text{iid}}{\sim}f(x;\theta)=e^{-(x-\theta)}, x\ge\theta X 1 , ⋯ , X n ∼ iid f ( x ; θ ) = e − ( x − θ ) , x ≥ θ
A level α \alpha α H 0 : θ ≤ θ 0 H_0:\theta\le\theta_0 H 0 : θ ≤ θ 0 H 1 : θ > θ 0 H_1:\theta>\theta_0 H 1 : θ > θ 0
Note L ( θ ; x ~ ) = ∏ i = 1 n ( e − ( x i − θ ) I ( x i ≥ θ ) ) = e − ∑ x i e n θ I ( x ( 1 ) ≥ θ ) L(\theta;\utilde{x})=\prod_{i=1}^n\left(e^{-(x_i-\theta)}I(x_i\ge\theta)\right)=e^{-\sum x_i}e^{n\theta}I(x_{(1)}\ge\theta) L ( θ ; x ) = ∏ i = 1 n ( e − ( x i − θ ) I ( x i ≥ θ ) ) = e − ∑ x i e n θ I ( x ( 1 ) ≥ θ )
在 θ ≤ x ( 1 ) \theta\le x_{(1)} θ ≤ x ( 1 ) L ( θ ; x ~ ) L(\theta;\utilde{x}) L ( θ ; x ) θ \theta θ θ > x ( 1 ) \theta>x_{(1)} θ > x ( 1 ) L ( θ ; x ~ ) = 0 L(\theta;\utilde{x})=0 L ( θ ; x ) = 0 θ 0 > x 1 \theta_0>x_{1} θ 0 > x 1 θ < θ 0 \theta<\theta_0 θ < θ 0 L ( θ = x ( 1 ) , x ~ ) L(\theta=x_{(1)},\utilde{x}) L ( θ = x ( 1 ) , x ) θ 0 ≤ x ( 1 ) \theta_0\le x_{(1)} θ 0 ≤ x ( 1 ) L ( θ 0 , x ~ ) L(\theta_0,\utilde{x}) L ( θ 0 , x )
⟹ sup θ ≤ θ 0 L ( θ ; x ~ ) = { e − ∑ x i e n x ( 1 ) if θ 0 > x ( 1 ) e − ∑ x i e n θ 0 if θ 0 ≤ x ( 1 ) ⟹ sup θ ∈ Ω L ( θ ; x ~ ) = e − ∑ x i e n x ( 1 ) \begin{align*}
&\implies\sup_{\theta\le\theta_0}L(\theta;\utilde{x})=\begin{cases}
e^{-\sum x_i}e^{nx_{(1)}} &\text{if }\theta_0 > x_{(1)}\\
e^{-\sum x_i}e^{n\theta_0} &\text{if }\theta_0\le x_{(1)}
\end{cases}\\
&\implies\sup_{\theta\in\Omega}L(\theta;\utilde{x})=e^{-\sum x_i}e^{nx_{(1)}}\\
\end{align*} ⟹ θ ≤ θ 0 sup L ( θ ; x ) = { e − ∑ x i e n x ( 1 ) e − ∑ x i e n θ 0 if θ 0 > x ( 1 ) if θ 0 ≤ x ( 1 ) ⟹ θ ∈ Ω sup L ( θ ; x ) = e − ∑ x i e n x ( 1 ) ⟹ \implies ⟹ H 0 H_0 H 0
λ ( x ~ ) = sup θ ≤ θ 0 L ( θ ; x ~ ) sup θ ∈ Ω L ( θ ; x ~ ) = { 1 if θ 0 > x ( 1 ) e n ( θ 0 − x ( 1 ) ) if θ 0 ≤ x ( 1 ) < k ∈ ( 0 , 1 ) ⟺ e n ( θ 0 − x ( 1 ) ) < k and θ 0 ≤ x ( 1 ) ⟺ θ 0 ≤ c < x ( 1 ) \begin{align*}
&\lambda(\utilde{x})=\frac{\sup_{\theta\le\theta_0}L(\theta;\utilde{x})}{\sup_{\theta\in\Omega}L(\theta;\utilde{x})}=\begin{cases}
1 &\text{if }\theta_0>x_{(1)}\\
e^{n(\theta_0-x_{(1)})} &\text{if }\theta_0\le x_{(1)}
\end{cases}<k\in(0,1)\\
\iff & e^{n(\theta_0-x_{(1)})}<k\quad\text{and }\theta_0\le x_{(1)}\\
\iff & \theta_0\le c < x_{(1)}
\end{align*} ⟺ ⟺ λ ( x ) = sup θ ∈ Ω L ( θ ; x ) sup θ ≤ θ 0 L ( θ ; x ) = { 1 e n ( θ 0 − x ( 1 ) ) if θ 0 > x ( 1 ) if θ 0 ≤ x ( 1 ) < k ∈ ( 0 , 1 ) e n ( θ 0 − x ( 1 ) ) < k and θ 0 ≤ x ( 1 ) θ 0 ≤ c < x ( 1 ) α = P θ 0 ( X ( 1 ) > c ) = ( P θ 0 ( X 1 > c ) ) n = ( ∫ c ∞ e − ( x − θ 0 ) d x ) n = ( e − ( c − θ 0 ) ) n = e n ( θ 0 − c ) \begin{align*}
\alpha&=P_{\theta_0}(X_(1)>c)\\
&=\left(P_{\theta_0}(X_1>c)\right)^n\\
&=\left(\int_c^\infty e^{-(x-\theta_0)}dx\right)^n\\
&=\left(e^{-(c-\theta_0)}\right)^n\\
&=e^{n(\theta_0-c)}\\
\end{align*} α = P θ 0 ( X ( 1 ) > c ) = ( P θ 0 ( X 1 > c ) ) n = ( ∫ c ∞ e − ( x − θ 0 ) d x ) n = ( e − ( c − θ 0 ) ) n = e n ( θ 0 − c ) ⟹ c = θ 0 − ln α n > θ 0 \implies c = \theta_0-\frac{\ln\alpha}{n} > \theta_0 ⟹ c = θ 0 − n ln α > θ 0 這與我們用 MLR 得到的 UMP test 是一樣的。
EX :
⊥ < X 1 , ⋯ , X n ∼ iid θ 1 e − θ 1 x , x > 0 Y 1 , ⋯ , Y m ∼ iid θ 2 e − θ 2 y , y > 0 \perp\Big < \begin{align*}
X_1, \cdots, X_n\overset{\text{iid}}{\sim}\theta_1e^{-\theta_1x}, x>0\\
Y_1, \cdots, Y_m\overset{\text{iid}}{\sim}\theta_2e^{-\theta_2y}, y>0
\end{align*} ⊥ ⟨ X 1 , ⋯ , X n ∼ iid θ 1 e − θ 1 x , x > 0 Y 1 , ⋯ , Y m ∼ iid θ 2 e − θ 2 y , y > 0 L ( θ 1 , θ 2 ; x ~ , y ~ ) = f X ~ ( x ~ ; θ 1 ) ⋅ f Y ~ ( y ~ ; θ 2 ) = θ 1 n e − θ 1 ∑ x i ⋅ θ 2 m e − θ 2 ∑ y i L(\theta_1, \theta_2;\utilde{x}, \utilde{y})=f_{\utilde{X}}(\utilde{x};\theta_1)\cdot f_{\utilde{Y}}(\utilde{y};\theta_2)=\theta_1^ne^{-\theta_1\sum x_i}\cdot\theta_2^me^{-\theta_2\sum y_i} L ( θ 1 , θ 2 ; x , y ) = f X ( x ; θ 1 ) ⋅ f Y ( y ; θ 2 ) = θ 1 n e − θ 1 ∑ x i ⋅ θ 2 m e − θ 2 ∑ y i ω 0 = { ( θ 1 , θ 2 ) ; θ 1 = θ 2 > 0 } Ω = { ( θ 1 , θ 2 ) ; θ 1 > 0 , θ 2 > 0 } \omega_0=\set{(\theta_1, \theta_2);\theta_1=\theta_2>0}\qquad\Omega=\set{(\theta_1, \theta_2);\theta_1>0, \theta_2>0} ω 0 = { ( θ 1 , θ 2 ) ; θ 1 = θ 2 > 0 } Ω = { ( θ 1 , θ 2 ) ; θ 1 > 0 , θ 2 > 0 } Note
W ∼ θ k e − θ ∑ w i ln W ∼ k ln θ − θ ∑ w i ⟹ ∂ ∂ θ ( k ln θ − θ ∑ w i ) = 0 ⟹ k θ − ∑ w i = 0 ⟹ θ = k ∑ w i = 1 W ˉ \begin{align*}
&W\sim\theta^ke^{-\theta \sum w_i}\qquad\ln W\sim k\ln\theta-\theta\sum w_i\\
\implies&\frac{\partial}{\partial\theta}(k\ln\theta-\theta\sum w_i)=0\implies\frac{k}{\theta}-\sum w_i=0\implies\theta=\frac{k}{\sum w_i}=\frac{1}{\bar{W}}
\end{align*} ⟹ W ∼ θ k e − θ ∑ w i ln W ∼ k ln θ − θ ∑ w i ∂ θ ∂ ( k ln θ − θ ∑ w i ) = 0 ⟹ θ k − ∑ w i = 0 ⟹ θ = ∑ w i k = W ˉ 1 在 ω 0 \omega_0 ω 0
L ( θ 1 , θ 2 , x ~ , y ~ ) = L ( θ , θ ; x ~ , y ~ ) = θ n + m e − θ ( ∑ x i + ∑ y i ) ⟹ θ ω 0 ^ = n + m ∑ i = 1 n x i + ∑ i = 1 m y i ⟹ L ( θ ω 0 ^ , θ ω 0 ^ ; x ~ , y ~ ) = ( n + m ∑ i = 1 n x i + ∑ i = 1 m y i ) n + m e − ( n + m ) \begin{align*}
&L(\theta_1, \theta_2, \utilde{x}, \utilde{y})=L(\theta, \theta;\utilde{x}, \utilde{y})=\theta^{n+m}e^{-\theta(\sum x_i+\sum y_i)}\\
\implies&\widehat{\theta_{\omega_0}}=\frac{n+m}{\sum_{i=1}^{n}x_i+\sum_{i=1}^{m}y_i}\\
\implies&L(\widehat{\theta_{\omega_0}}, \widehat{\theta_{\omega_0}};\utilde{x}, \utilde{y})=\left(\frac{n+m}{\sum_{i=1}^{n}x_i+\sum_{i=1}^{m}y_i}\right)^{n+m}e^{-(n+m)}\\
\end{align*} ⟹ ⟹ L ( θ 1 , θ 2 , x , y ) = L ( θ , θ ; x , y ) = θ n + m e − θ ( ∑ x i + ∑ y i ) θ ω 0 = ∑ i = 1 n x i + ∑ i = 1 m y i n + m L ( θ ω 0 , θ ω 0 ; x , y ) = ( ∑ i = 1 n x i + ∑ i = 1 m y i n + m ) n + m e − ( n + m ) 在 Ω \Omega Ω
L ( θ Ω ^ , θ Ω ^ ; x ~ , y ~ ) = ( n ∑ i = 1 n x i ) n ( m ∑ i = 1 m y i ) m e − n e − m L(\widehat{\theta_{\Omega}}, \widehat{\theta_{\Omega}};\utilde{x}, \utilde{y})=\left(\frac{n}{\sum_{i=1}^{n}x_i}\right)^n\left(\frac{m}{\sum_{i=1}^{m}y_i}\right)^me^{-n}e^{-m} L ( θ Ω , θ Ω ; x , y ) = ( ∑ i = 1 n x i n ) n ( ∑ i = 1 m y i m ) m e − n e − m ⟹ \implies ⟹ H 0 H_0 H 0
k > λ ( x ~ , y ~ ) = L ( θ ω 0 ^ , θ ω 0 ^ ; x ~ , y ~ ) L ( θ Ω ^ , θ Ω ^ ; x ~ , y ~ ) = ( n + m ∑ x i + ∑ y i ) n + m ( ∑ x i n ) n ( ∑ y i m ) m e n + m = ( ∑ x i ∑ x i + ∑ y i ⏟ = T ∈ ( 0 , 1 ) ) n ( ∑ y i ∑ x i + ∑ y i ⏟ = 1 − T ) m ( n + m ) n + m 1 n n 1 m m ⟺ T n ( 1 − T ) m < c ⟺ T < k 1 or T > k 2 \begin{align*}
&\begin{align*}
k>\lambda(\utilde{x}, \utilde{y})&=\frac{L(\widehat{\theta_{\omega_0}}, \widehat{\theta_{\omega_0}};\utilde{x}, \utilde{y})}{L(\widehat{\theta_{\Omega}}, \widehat{\theta_{\Omega}};\utilde{x}, \utilde{y})}\\
&=\left(\frac{n+m}{\sum x_i+\sum y_i}\right)^{n+m}\left(\frac{\sum x_i}{n}\right)^n\left(\frac{\sum y_i}{m}\right)^me^{n+m}\\
&=\left(\underbrace{\frac{\sum x_i}{\sum x_i+\sum y_i}}_{=T\in(0,1)}\right)^n\left(\underbrace{\frac{\sum y_i}{\sum x_i+\sum y_i}}_{=1-T}\right)^m(n+m)^{n+m}\frac{1}{n^n}\frac{1}{m^m}\\
\end{align*}\\
\iff& T^n(1-T)^m<c\\
\iff& T<k_1\text{ or }T>k_2
\end{align*} ⟺ ⟺ k > λ ( x , y ) = L ( θ Ω , θ Ω ; x , y ) L ( θ ω 0 , θ ω 0 ; x , y ) = ( ∑ x i + ∑ y i n + m ) n + m ( n ∑ x i ) n ( m ∑ y i ) m e n + m = = T ∈ ( 0 , 1 ) ∑ x i + ∑ y i ∑ x i n = 1 − T ∑ x i + ∑ y i ∑ y i m ( n + m ) n + m n n 1 m m 1 T n ( 1 − T ) m < c T < k 1 or T > k 2 Note X ∼ exp ( θ 1 ) = d Gamma ( 1 , 1 θ 1 ) ⟹ ∑ n x i ∼ G ( n , 1 θ 1 ) ⟹ 2 θ 1 ∑ n x i ∼ G ( n , 2 ) = d χ 2 n 2 X\sim \exp(\theta_1)\xlongequal{\text{d}}\text{Gamma}(1, \frac{1}{\theta_1})\implies\sum^n x_i\sim \text{G}(n,\frac{1}{\theta_1})\implies 2\theta_1\sum^n x_i\sim\text{G}(n,2)\xlongequal{\text{d}}\chi^2_{2n} X ∼ exp ( θ 1 ) d Gamma ( 1 , θ 1 1 ) ⟹ ∑ n x i ∼ G ( n , θ 1 1 ) ⟹ 2 θ 1 ∑ n x i ∼ G ( n , 2 ) d χ 2 n 2
T = ∑ x i ∑ x i + ∑ y i = 2 θ ∑ x i 2 θ ∑ x i + 2 θ ∑ y i = 1 1 + 2 θ ∑ m y / 2 m 2 θ ∑ n x / 2 n m n where 2 θ ∑ m y / 2 m 2 θ ∑ n x / 2 n = d χ 2 m 2 / 2 m χ 2 n 2 / 2 n ∼ F 2 m , 2 n \begin{align*}
T&=\frac{\sum x_i}{\sum x_i+\sum y_i}=\frac{2\theta\sum x_i}{2\theta\sum x_i+2\theta\sum y_i}\\
&=\frac{1}{1+\frac{2\theta\sum^m y/2m}{2\theta\sum^n x/2n}\frac{m}{n}}\quad\text{where }\frac{2\theta\sum^m y/2m}{2\theta\sum^n x/2n}\xlongequal{\text{d}}\frac{\chi^2_{2m}/2m}{\chi^2_{2n}/2n}\sim F_{2m, 2n}
\end{align*} T = ∑ x i + ∑ y i ∑ x i = 2 θ ∑ x i + 2 θ ∑ y i 2 θ ∑ x i = 1 + 2 θ ∑ n x /2 n 2 θ ∑ m y /2 m n m 1 where 2 θ ∑ n x /2 n 2 θ ∑ m y /2 m d χ 2 n 2 /2 n χ 2 m 2 /2 m ∼ F 2 m , 2 n ⟹ T < k 1 or T > k 2 ⟺ Y ˉ X ˉ < C 1 or Y ˉ X ˉ > C 2 \implies T<k_1\text{ or }T>k_2\iff \frac{\bar{Y}}{\bar{X}}<C_1 \text{ or }\frac{\bar{Y}}{\bar{X}}>C_2 ⟹ T < k 1 or T > k 2 ⟺ X ˉ Y ˉ < C 1 or X ˉ Y ˉ > C 2 I.e. LRT is reject H_0 at level α \alpha α Y ˉ X ˉ < F 2 m , 2 n , 1 − α / 2 \frac{\bar{Y}}{\bar{X}}<F_{2m,2n,1-\alpha/2} X ˉ Y ˉ < F 2 m , 2 n , 1 − α /2 Y ˉ X ˉ > F 2 m , 2 n , α / 2 \frac{\bar{Y}}{\bar{X}}>F_{2m,2n,\alpha/2} X ˉ Y ˉ > F 2 m , 2 n , α /2