Local feature selection using gaussian process regression

Abstract

Most feature selection algorithms determine a global subset of features, where all
data instances are projected in order to improve classification accuracy. An attractive
alternative solution is to adaptively find a local subset of features for each data instance,
such that, the classification of each instance is performed according to its own selective
subspace. This paper presents a novel application of Gaussian Processes that improves
classification performance by learning discriminative local subsets of features for each
instance in a dataset. Gaussian Processes are used to build for each available feature
a function that estimates the discriminative power of the feature over all the input
space. Using these functions, we are able to determine a discriminative subspace for
each possible instance by locally joining the features that present the highest levels
of discriminative power. New instances are then classified by using a K-NN classifier
that operates in the local subspaces. Experimental results show that by using local
discriminative subspaces, we are able to reach higher levels of accuracy than alternative
state-of-the-art feature selection approaches.