Online Ensemble based Reinforcement learning architecture to detect novel class and recurring classes on emerging data streams

Abstract

The extensive growth of digital technologies has led to new challenges regarding computational complexity on mining streaming data. The data classification is not trivial due to the high volume of data and limited time available for the classification. It is particularly difficult in dealing with data streams, where each instance of data is typically processed once on its arrival (i.e. online) while the underlying data distribution often changes due to the changing environment. In this paper, we propose a novel online ensemble based reinforcement learning for effective data stream classification in the context of changing environment leading to concept drifts (i.e. evolution of data streams). Proposed reinforcement learning uses three strategies to flexibly adapt to different types of concept drifts when performing data stream classification. Proposed architecture initially extract the a set of random feature combinations to form the pool of features on data streams using feature extraction technique such as kernel principle component analysis and Increment Non – Linear discriminant analysis . Pools of feature are classified using ensemble of Reinforcement learning such as Monte Carlo, Genetic algorithm and Q learning. Reinforcement mechanism is proposed to increase the weights of the base classifiers that perform better on the minority class and decrease the weights of the classifiers that perform worse. A resampling buffer is used for storing the instances of the minority class to balance the imbalanced distribution over time. On those classifiers, base classifier has been constructed weighting boosting technique. Extensive result on performance analysis proves that proposed model outperforms existing state of art approaches against various real datasets. The results demonstrated that the proposed framework can provide the best accuracy and computation time on average when comparing with existing models.