Kernel Machines II — some practical details

36-465/665, Spring 2021

13 April 2021 (Lecture 20)

\[ \newcommand{\myexp}[1]{\exp{\left( #1 \right)}} \]

Housekeeping

Kernel machines (reprise)

Kernels and their associated functions

Kernel machines vs. basis expansions

Towards being more concrete about all this

Look at the data

Priors vs. age for the training set, color-coded for recidivism. Points are “jittered” so that multiple individuals with equal features aren’t superposed. The “rugs” along the axes give a sense of the marginal distributions of the two attributes for the two classes.

Bringing the kernel in

Picking a kernel

The kernel matrix and (approximate) basis functions

Visualizing the kernel matrix

Visualizing the basis functions

Visualizing the basis functions

The first four eigenfunctions

What about the eigenvalues?

Scale the eigenvectors by the eigenvalues

Notice that lots of the eigenvectors don’t even show up any more, because they’re being multiplied by very small (square roots of) eigenvalues

Back to fitting

Fits on the training set, with color indicating fitted value (darker being lower predictions of recidivism, lighter higher predictions), and shape indicating actual outcome

Look at the testing set