Parameter-free surrounding neighborhood based regression methods

Abstract

In machine learning, nearest neighbor (NN) regression is one of the most prominent methods for numeric prediction. It estimates the output variable of a new data point by averaging the output variables of the neighboring points. The selection of the neighborhood and its parameter(s) is crucial for the performance of NN regression, however this is still an open issue. This study contributes to the literature by adopting the parameter-free surrounding neighborhood (PSN) concept for NN regression. PSNs are based on proximity graphs, i.e. minimum spanning tree, relative neighborhood graph, and Gabriel graph. They yield a unique neighborhood for each point by combining proximity, connectivity and spatial distribution. The performances of the PSN regression methods are compared with k-nearest neighbors, k-nearest centroid neighbors, and support vector regression using real-world data sets. The statistical tests show that the PSN regression methods perform significantly better than most of the competing approaches. Also, the proposed approaches do not have any parameters to be set.