Kernel Methods

Abstract

Remember the xor example of a classification problem that is not linearly separable. If we map every example into a new representation, then the problem becomes linearly separable. Specifically, ... The major disadvantage of mapping points into a new space is that the new space may have very high dimension. For example, if points lie in d-dimensional Euclidean space, and we include the product of every pair of dimensions then we have quadratic blowup with the mapping f : R 7→ Rd2. We can avoid this explosion if we can achieve two objectives: