Regression¶
Suggested Prerequisites¶
Overview¶
Linear Regression Modeling¶
In this case, sample data is fit by a linear function as formalized by:
where \(p\) is the number of features, \(n\) is the number of samples and \(\epsilon\) is an error term with mean of zero and finite variance. Or in vector notation:
where \(y\) is a response vector \([y_1, y_2, ..., y_n]^\mathbf{T}\) of length \(n\), \(\mathbf{X}\) is a \(n \times (p + 1)\) design matrix of features \([\mathbf{1}, \mathbf{x_1}, \mathbf{x_2}, ..., \mathbf{x_p}]\), and \(\mathbf{\beta}\) is a length \((p+1)\) coefficient vector \([\beta_0, \beta_1, \beta_2, ..., \beta_p]\) with \(\beta_0\) an intercept term. This intercept term is included in the model through data augmentation of the column of \(\mathbf{1}\)s to the design matrix. When an intercept is not sought, this column can be omitted and \(\mathbf{\beta}\) is length \(p\).