In many scientific domains as diverse as engineering, astronomy, biology, remote sensing, economics, and consumer transactions, the ability to model large amounts of high-dimensional data is fundamental to the success of the application. Typically, from all the high-dimensional measurements, only a few measured variables are important to understanding the underlying phenomena of interest. Moreover, because of the curse of dimensionality, it has been observed that the use of many irrelevant or redundant features might harm the performance of the algorithms (e.g. classification). Feature selection, a discrete version of dimensionality reduction, is the task of selecting a subset of the variables relevant to a given task (e.g. clustering, modeling). Feature selection is often a combinatorial problem and typically greedy sub-optimal methods are used. The aim of this project is to develop a convex optimization relaxation framework for feature selection. In particular, we are interested in feature selection in Component Analysis methods such as kernel principal component analysis, multivariate regression and support vector machines.