Kernel ridge regression (KRR)

• Combina el modelo ridge (OLS + regularización L2) con el kernel trick.

• Corresponde a un modelo lineal que aprende en el espacio (no lineal dependiendo del kernel) generado por el kernel y los datos.

• Este modelo es equivalente a una SVR con algunas diferencias.

• En las SVR se usa la pérdida \epsilon-insensitiva + regularización L2.

• En el KRR se usa la pérdida cuadrática + regularización L2.

import numpy as np

n_samples, n_features = 10, 5

rng = np.random.RandomState(0)

y = rng.randn(n_samples)
X = rng.randn(n_samples, n_features)

from sklearn.kernel_ridge import KernelRidge

kernelRidge = KernelRidge(
    # --------------------------------------------------------------------------
    # Regularization strength; must be a positive float. Regularization improves
    # the conditioning of the problem and reduces the variance of the estimates.
    # Larger values specify stronger regularization. Alpha corresponds to
    # 1 / (2C) in other linear models such as LogisticRegression or LinearSVC.
    alpha=1.0,
    # --------------------------------------------------------------------------
    # Kernel mapping used internally.
    # * 'additive_chi2'
    # * 'chi2'
    # * 'linear'
    # * 'poly'
    # * 'polynomial'
    # * 'rbf'
    # * 'laplacian'
    # * 'sigmoid'
    # * 'cosine'
    kernel="linear",
    # --------------------------------------------------------------------------
    # Gamma parameter for the RBF, laplacian, polynomial, exponential chi2 and
    # sigmoid kernels.
    gamma=None,
    # --------------------------------------------------------------------------
    # Degree of the polynomial kernel. Ignored by other kernels.
    degree=3,
    # --------------------------------------------------------------------------
    # Zero coefficient for polynomial and sigmoid kernels. Ignored by other
    # kernels.
    coef0=1,
    # --------------------------------------------------------------------------
    # Additional parameters (keyword arguments) for kernel function passed as
    # callable object.
    kernel_params=None,
)

kernelRidge.fit(X, y)

KernelRidge(alpha=1.0)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

kernelRidge.dual_coef_

array([ 1.53839505,  0.20049053, -0.33063188,  1.15477888,  1.28601901,
       -0.49046154,  0.46714476,  0.77765325, -0.78298407,  0.82155474])