DC Neural Networks avoid overfitting in one-dimensional nonlinear regression

Fecha

2024-01-11

Título de la revista

ISSN de la revista

Título del volumen

Editor

Elsevier

Enlace externo

Resumen

In this paper, we analyze Difference of Convex Neural Networks in the context of one-dimensional nonlinear regression. Specifically, we show the surprising ability of the Difference of Convex Multilayer Perceptron (DC-MLP) to avoid overfitting in nonlinear regression. Otherwise said, DC-MLPs self-regularize (do not require additional regularization techniques). Thus, DC-MLPs could result very useful for practical purposes based on one-dimensional nonlinear regression. It turns out that shallow MLPs with a convex activation (ReLU, softplus, etc.) fall in the class of DC-MLPs. On the other hand, we call SQ-MLP the shallow MLP with a Squashing activation (logistic, hyperbolic tangent, etc.). In the numerical experiments, we show that DC-MLPs used for nonlinear regression avoid overfitting, in contrast with SQ-MLPs. We also compare DC-MLPs and SQ-MLPs from a theoretical point of view

Descripción

Palabras clave

Citación

Cesar Beltran-Royo, Laura Llopis-Ibor, Juan J. Pantrigo, Iván Ramírez, DC Neural Networks avoid overfitting in one-dimensional nonlinear regression, Knowledge-Based Systems, Volume 283, 2024, 111154, ISSN 0950-7051, https://doi.org/10.1016/j.knosys.2023.111154
license logo
Excepto si se señala otra cosa, la licencia del ítem se describe como Attribution-NonCommercial-NoDerivs 4.0 International