Anova Model 通常在分析定性型（Qualitative）變數時使用，例如：教育程度，血型等。在這種情況下，我們將變數稱為因素（factor） 。
Remark: ANOVA Model 只是回歸模型的一種特例。
One-way ANOVA Model
One-way 是指在單個變數上進行變異數分析。我們會使用以下模型
Y = E ( Y ∣ X ) + ε Y=E(Y|X)+\varepsilon Y = E ( Y ∣ X ) + ε 在第 i i i E ( Y ∣ X = i ) = μ i , i = 1 , ⋯ , k E(Y|X=i)=\mu_i,i=1,\cdots,k E ( Y ∣ X = i ) = μ i , i = 1 , ⋯ , k i = i= i =
而我們的假設檢定也會是
H 0 : X 對 Y 無影響 ⟺ H 0 : μ 1 = μ 2 = ⋯ = μ k H_0:\text{ X 對 Y 無影響 }\iff H_0:\mu_1=\mu_2=\cdots=\mu_k H 0 : X 對 Y 無影響 ⟺ H 0 : μ 1 = μ 2 = ⋯ = μ k 我們在收集數據時，每個種類 i i i n i n_i n i n n n
Y i j = μ i + ε i j , i = 1 , ⋯ , k , j = 1 , ⋯ , n i , n = ∑ i = 1 k n i Y_{ij}=\mu_i+\varepsilon_{ij},\quad i=1,\cdots,k,j=1,\cdots,n_i, n=\sum_{i=1}^k n_i Y ij = μ i + ε ij , i = 1 , ⋯ , k , j = 1 , ⋯ , n i , n = i = 1 ∑ k n i ( Y 11 ⋮ Y 1 n 1 Y 21 ⋮ Y 2 n 2 ⋮ Y k n k ) = ( μ 1 ⋮ μ 1 μ 2 ⋮ μ 2 ⋮ μ k ) + ( ε 11 ⋮ ε 1 n 1 ε 21 ⋮ ε 2 n 2 ⋮ ε k n k ) ⟹ Y ~ n × 1 = [ 1 ~ n 1 0 ⋯ 0 0 1 ~ n 2 ⋯ 0 ⋮ ⋮ ⋱ ⋮ 0 0 ⋯ 1 ~ n k ] ⏟ D [ μ 1 μ 2 ⋮ μ k ] ⏟ β ~ + ε ~ n × 1 \begin{pmatrix*}
Y_{11}\\
\vdots\\
Y_{1n_1}\\
Y_{21}\\
\vdots\\
Y_{2n_2}\\
\vdots\\
Y_{kn_k}
\end{pmatrix*}= \begin{pmatrix*}
\mu_1\\
\vdots\\
\mu_1\\
\mu_2\\
\vdots\\
\mu_2\\
\vdots\\
\mu_k
\end{pmatrix*}+\begin{pmatrix*}
\varepsilon_{11}\\
\vdots\\
\varepsilon_{1n_1}\\
\varepsilon_{21}\\
\vdots\\
\varepsilon_{2n_2}\\
\vdots\\
\varepsilon_{kn_k}
\end{pmatrix*}\implies
\utilde{Y}_{n\times 1}=\underbrace{\begin{bmatrix*}
\utilde{1}_{n_1}&0&\cdots&0\\
0&\utilde{1}_{n_2}&\cdots&0\\
\vdots&\vdots&\ddots&\vdots\\
0&0&\cdots&\utilde{1}_{n_k}
\end{bmatrix*}}_{D}
\underbrace{\begin{bmatrix*}
\mu_1\\
\mu_2\\
\vdots\\
\mu_k
\end{bmatrix*}}_{\utilde{\beta}}+\utilde{\varepsilon}_{n\times 1} Y 11 ⋮ Y 1 n 1 Y 21 ⋮ Y 2 n 2 ⋮ Y k n k = μ 1 ⋮ μ 1 μ 2 ⋮ μ 2 ⋮ μ k + ε 11 ⋮ ε 1 n 1 ε 21 ⋮ ε 2 n 2 ⋮ ε k n k ⟹ Y n × 1 = D 1 n 1 0 ⋮ 0 0 1 n 2 ⋮ 0 ⋯ ⋯ ⋱ ⋯ 0 0 ⋮ 1 n k β μ 1 μ 2 ⋮ μ k + ε n × 1 ⟹ D t D = diag ( n 1 , ⋯ , n k ) D t Y ~ = ( ∑ j = 1 n 1 Y 1 j ⋮ ∑ j = 1 n k Y k j ) = ( Y 1 ⋅ ⋮ Y k ⋅ ) where Y i ⋅ = ∑ j = 1 n i Y i j called treatment totals \begin{align*}
\implies & D^tD=\text{diag}(n_1,\cdots,n_k)\\
&D^t\utilde{Y}=\begin{pmatrix*}
\sum_{j=1}^{n_1}Y_{1j}\\
\vdots\\
\sum_{j=1}^{n_k}Y_{kj}
\end{pmatrix*}=
\begin{pmatrix*}
Y_{1\cdot}\\
\vdots\\
Y_{k\cdot}
\end{pmatrix*}\quad \text{where } Y_{i\cdot}=\sum_{j=1}^{n_i}Y_{ij} \text{ called treatment totals}\\
\end{align*} ⟹ D t D = diag ( n 1 , ⋯ , n k ) D t Y = ∑ j = 1 n 1 Y 1 j ⋮ ∑ j = 1 n k Y kj = Y 1 ⋅ ⋮ Y k ⋅ where Y i ⋅ = j = 1 ∑ n i Y ij called treatment totals LSE of μ ~ \utilde{\mu} μ
b ~ = μ ^ ~ = ( D t D ) − 1 D t Y ~ = ( n 1 0 ⋯ 0 0 n 2 ⋯ 0 ⋮ ⋮ ⋱ ⋮ 0 0 ⋯ n k ) − 1 ( Y 1 ⋅ ⋮ Y k ⋅ ) = ( Y ˉ 1 ⋅ ⋮ Y ˉ k ⋅ ) = treatment sample mean \utilde{b}=\utilde{\hat{\mu}}=(D^tD)^{-1}D^t\utilde{Y}=\begin{pmatrix*}
n_1&0&\cdots&0\\
0&n_2&\cdots&0\\
\vdots&\vdots&\ddots&\vdots\\
0&0&\cdots&n_k
\end{pmatrix*}^{-1}\begin{pmatrix*}
Y_{1\cdot}\\
\vdots\\
Y_{k\cdot}
\end{pmatrix*}=\begin{pmatrix*}
\bar{Y}_{1\cdot}\\
\vdots\\
\bar{Y}_{k\cdot}
\end{pmatrix*}=\text{ treatment sample mean} b = μ ^ = ( D t D ) − 1 D t Y = n 1 0 ⋮ 0 0 n 2 ⋮ 0 ⋯ ⋯ ⋱ ⋯ 0 0 ⋮ n k − 1 Y 1 ⋅ ⋮ Y k ⋅ = Y ˉ 1 ⋅ ⋮ Y ˉ k ⋅ = treatment sample mean Y ^ ~ = D b ~ \utilde{\hat{Y}}=D\utilde{b} Y ^ = D b Y ^ i j = Y ˉ i ⋅ , ∀ i , j \hat{Y}_{ij}=\bar{Y}_{i\cdot}, \forall i,j Y ^ ij = Y ˉ i ⋅ , ∀ i , j
SSR = ∑ ∑ ( Y ^ i j − Y ˉ ⋅ ⋅ ) = ∑ ∑ ( Y ˉ i ⋅ − Y ˉ ⋅ ⋅ ) 2 \sum\sum(\hat{Y}_{ij}-\bar{Y}_{\cdot\cdot})=\sum\sum(\bar{Y}_{i\cdot}-\bar{Y}_{\cdot\cdot})^2 ∑∑ ( Y ^ ij − Y ˉ ⋅⋅ ) = ∑∑ ( Y ˉ i ⋅ − Y ˉ ⋅⋅ ) 2
SSE = ∑ ∑ ( Y i j − Y ^ i j ) 2 = ∑ ∑ ( Y i j − Y ˉ i ⋅ ) 2 \sum\sum(Y_{ij}-\hat{Y}_{ij})^2=\sum\sum(Y_{ij}-\bar{Y}_{i\cdot})^2 ∑∑ ( Y ij − Y ^ ij ) 2 = ∑∑ ( Y ij − Y ˉ i ⋅ ) 2
One-way ANOVA with
Y i j = μ i + ε i j = E ( Y ∣ X = i ) + ε i j is called cell-mean model Y_{ij}=\mu_i+\varepsilon_{ij}=E(Y|X=i)+\varepsilon_{ij} \text{ is called cell-mean model} Y ij = μ i + ε ij = E ( Y ∣ X = i ) + ε ij is called cell-mean model Let μ = 1 k ∑ i = 1 k μ i = μ , τ i = μ i − μ , i = 1 , ⋯ , k \mu=\frac{1}{k}\sum_{i=1}^k\mu_i=\mu, \tau_i=\mu_i-\mu,i=1,\cdots,k μ = k 1 ∑ i = 1 k μ i = μ , τ i = μ i − μ , i = 1 , ⋯ , k ∑ i = 1 k τ i = 0 \sum_{i=1}^k\tau_i=0 ∑ i = 1 k τ i = 0
Y i j = μ i + ε i j = μ + μ i − μ + ε i j = μ + τ i + ε i j is called factor-effect model Y_{ij}=\mu_i+\varepsilon_{ij}=\mu+\mu_i-\mu+\varepsilon_{ij}=\mu+\tau_i+\varepsilon_{ij} \text{ is called factor-effect model} Y ij = μ i + ε ij = μ + μ i − μ + ε ij = μ + τ i + ε ij is called factor-effect model Two-way ANOVA Model
Two factors:
A with levels a
B with levels b
Y = E ( Y ∣ A = i , B = j ) + ε Y=E(Y|A=i,B=j)+\varepsilon Y = E ( Y ∣ A = i , B = j ) + ε cell-mean model: Y i j k = μ i j + ε i j k Y_{ijk}=\mu_{ij}+\varepsilon_{ijk} Y ijk = μ ij + ε ijk ε i j k ∼ N ( 0 , σ 2 ) , k = 1 , ⋯ , n i j \varepsilon_{ijk}\sim N(0,\sigma^2), k=1,\cdots,n_{ij} ε ijk ∼ N ( 0 , σ 2 ) , k = 1 , ⋯ , n ij
factor-effect model: Y i j k = μ + A i + B j + ( α β ) i j + ε i j k Y_{ijk}=\mu+A_i+B_j+(\alpha\beta)_{ij}+\varepsilon_{ijk} Y ijk = μ + A i + B j + ( α β ) ij + ε ijk