dailylog 7-13-20

Questions from ISL CH.6

Q4. Suppose we estimate the regression coefficients in a linear regression model by minimizing ∑i=1n(yi−β0−∑j=1pβjxij)−λ∑j=1pβ2j for a particular value of λ. For parts (a) through (e), indicate which of i. through v. is correct. Justify your answer.

As we increase λ from 0, the training RSS will Steadily increase. As we increase λ from 0, we are restricting the βj coefficients more and more (the coefficients will deviate from their least squares estimates), and so the model is becoming less and less flexible which provokes a steady increase in training RSS.

Repeat (a) for test RSS. Decrease initially, and then eventually start increasing in a U shape. As we increase λ from 0, we are restricting the βj coefficients more and more (the coefficients will deviate from their least squares estimates), and so the model is becoming less and less flexible which provokes at first a decrease in the test RSS before increasing again after that in a typical U shape.

Repeat (a) for variance. Steadily decrease. As we increase λ from 0, we are restricting the βj coefficients more and more (the coefficients will deviate from their least squares estimates), and so the model is becoming less and less flexible which provokes a steady decrease in variance.

Repeat (a) for (squared) bias. Steadily increase. As we increase λ from 0, we are restricting the βj coefficients more and more (the coefficients will deviate from their least squares estimates), and so the model is becoming less and less flexible which provokes a steady increase in bias.

Repeat (a) for the irreducible error. Remain constant. By definition, the irreducible error is independant of the model, and consequently independant of the value of λ.