Event prediction with learning algorithms – A study of events surrounding the Egyptian revolution of 2011 on the basis of micro blog data

Abstract

We aim to predict activities of political nature influencing or reflecting societal-scale behavior and beliefs by applying learning algorithms to Twitter data. This study focuses on capturing domestic events in Egypt from November 2009 to November 2013. To this extent we study un­derlying communication patterns by evaluating content and metadata of 1.3 million tweets through computationally supported classification,
with­out targeting specific keywords or users from the Twitter stream. Support Vector Machine (SVM) and Support Distribution Machine (SDM) classification algorithms are applied to detect and predict societal-scale unrest. Latent Dirichlet Allocation (LDA) is used to create content-based input patterns for the SVM while the SDM is used to classify sets of features created from meta-data. The experiments reveal that user cen­tric approaches based on meta-data outperform methods employing content-based input despite the use of well established natural language pro­cessing algorithms. The results show that distri­butions over user centric meta information pro­vide an important signal when detecting and pre­dicting events. Applying this approach can assist policymakers and stakeholders in their efforts toward proactive community management.