Feature construction is a key step in any data analysis process, largely conditioning the success of any signal processing, computer vision, pattern recognition, and machine learning algorithm. Over the last several decades, Component Analysis (CA) methods (e.g. Kernel Principal Component Analysis, spectral clustering) have been extensively used as a feature extraction step for modeling, classification, and clustering in numerous visual, graphics, and signal processing tasks. CA algorithms decompose a given signal into relevant components which explicitly or implicitly (e.g. kernel methods) define the representation of the signal. However, most clustering (e.g. spectral graph methods), classification (e.g. Support Vector Machine), and modeling (e.g. Active Appearance Models) algorithms are optimized independently of the CA feature extraction methods. This independent feature extraction step might result in a loss of information that is relevant for the final task. This work proposes an extension of CA techniques to jointly learn optimal signal representations and parameters of the models.