HW 5.1: fig1.m
HW 5.2: hw52.m
HW 5.3: hw53.m
Subroutines: plotdata.m, vcm.m, vcmbound.m (vcmline.m. Infact, using contour() is a simpler way to draw the boundaries.)
Jeopardy - a variation of weight decay: run.m traintst.m epoch.m (newnet.m, initnet.m, forward.m, backward.m)
The average errors of 3 trials with different combinations of $\beta$ and $\lambda$ are not listed in the homework:
| $\beta^2$ | $\lambda=0.001$ | $\lambda=0.01$ | $\lambda=0.1$ | |||||||||||
| $\beta^1$ | 0.1 | 0.5 | 1 | 2 | 0.1 | 0.5 | 1 | 2 | 0.1 | 0.5 | 1 | 2 | ||
| 0.1 | 0.0211 | 0.0210 | 0.0210 | 0.0210 | 1.0686 | 1.0683 | 1.0685 | 1.0687 | 1.0665 | 1.0652 | 1.0671 | 1.0682 | ||
| 0.5 | 0.0212 | 0.0214 | 0.0214 | 0.0214 | 1.0686 | 0.0225 | 0.0219 | 0.0218 | 1.0665 | 1.0652 | 1.0671 | 1.0682 | ||
| 1 | 0.0213 | 0.0163 | 0.0163 | 0.0163 | 1.0686 | 0.0229 | 0.0225 | 0.0225 | 1.0665 | 1.0652 | 1.0671 | 1.0682 | ||
| 2 | 0.0215 | 0.0164 | 0.0129 | 0.0128 | 1.0686 | 0.0254 | 0.0251 | 0.0252 | 1.0665 | 1.0652 | 1.0667 | 0.1919 | ||
| $\beta^2$ | $\lambda=0.001$ | $\lambda=0.01$ | $\lambda=0.1$ | |||||||||||
| $\beta^1$ | 0.1 | 0.5 | 1 | 2 | 0.1 | 0.5 | 1 | 2 | 0.1 | 0.5 | 1 | 2 | ||
| 0.1 | 0.1006 | 0.1006 | 0.1006 | 0.1006 | 1.0005 | 1.0017 | 1.0019 | 1.0020 | 0.9984 | 0.9996 | 1.0011 | 1.0018 | ||
| 0.5 | 0.1006 | 0.1006 | 0.1006 | 0.1005 | 1.0005 | 0.1005 | 0.1004 | 0.1004 | 0.9984 | 0.9996 | 1.0011 | 1.0018 | ||
| 1 | 0.1006 | 0.1136 | 0.1134 | 0.1134 | 1.0005 | 0.1004 | 0.1004 | 0.1004 | 0.9984 | 0.9996 | 1.0011 | 1.0018 | ||
| 2 | 0.1005 | 0.1142 | 0.1189 | 0.1209 | 1.0005 | 0.1005 | 0.1004 | 0.1004 | 0.9984 | 0.9996 | 0.9977 | 0.1942 | ||