Flexibility through Incremental Learning: Neural Networks for Text Categorization

Abstract

In this paper we show an adaptive incremental learning algorithm that learns interactively to classify text messages (here: emails) into categories without the need for lengthy batch training runs. The algorithm was evaluated on a large database of email messages that fall into five subjective categories. As control experiment best human categorization performance was established at 79.4% for this task. The best of all connectionist architectures presented here achieves near human performance (79.1%). This architecture acquires its language model and dictionary adaptively and hence avoids handcoding of either. The learning algorithm combines an adaptive phase which instantly updates dictionary and weights during interaction and a tuning phase which fine tunes for performance using previously seen data. Such systems can be deployed in various applications where instantaneous interactive learning is necessary such as on-line email or news categorization, text summarization and information filtering in general.