Systems Seminar

On Feature Selection for Supervised and Unsupervised Learning

Prof. Mario A. T. Figueiredo
Department of Electrical and Computer Engineering
Instituto Superior Tecnico (IST)

Abstract

It has recently been showed that feature selection in supervised learning can be embedded in the learning algorithm by using sparsity-promoting priors/penalties that encourage the coefficient estimates to be either significantly large or exactly zero. In the first half of this talk, I will review this type of approach (which includes the well-known LASSO criterion for regression) and present some recent developments: (i) simple and efficient algorithms (with both parallel and sequential updates) for both binary and multi-class problems; (ii) generalization bounds; (iii) feature selection "inside" the kernel for kernel-based formulations. Experimental results (on standard benchmark data-sets and also on gene expression data) reveal that this class of methods achieves state-of-the-art performance.

While feature selection is a well studied problem in supervised learning, the important issue of determining what attributes of the data better reveal its cluster structure is rarely touched upon. Feature selection for clustering is a difficult because, in the absence of class labels, there is no obvious optimality criterion. In the second half of my talk I will describe two recently proposed approaches to feature selection for mixture-based clustering. One of the approaches uses a new concept of "feature saliency" which can be estimated using an EM algorithm. The second approach extends the mutual-information-based feature relevance criterion to the unsupervised learning case. The result is an algorithm which "wraps" mixture estimation in an outer layer that performs feature selection. Experiments show that both methods have promising performance.

Time and Place: Tues., Jan. 20, at 4 pm in 3609 Engr. Hall.       *** NOTE SPECIAL DAY, TIME, & PLACE ***

SYSTEMS SEMINAR WEB PAGE: http://www.cae.wisc.edu/~gubner/seminar/

File "figueiredo.shtml" last modified Tue 15 Oct 2019, 01:45 PM, CDT
Web Page Contact: John (dot) Gubner (at) wisc (dot) edu