Data-Driven Decision Support

This section presents possible future research directions of real-time decision support under concept drift.

Adaptive decision support systems under concept drift

Streaming data are a set of continuous record of events. The volume of data is expanded by its time stamps, which can be infinite in number. Nowadays, streaming data has the capacity to track events for long periods at high frequency from mobile and/or embedded devices (e.g., sensors). It can thus continuously capture the potential risk of an event by analyzing its data stream. If a potential risk is detected that may result in a significant decision-making failure, the existing decision-making results need to be immediately updated to prevent loss being caused by old decisions. We refer to this as Adaptive Decision Support, and it has application in such environments as the IoT, emergency management, industrial control systems and online decision-making. An example of the applications is situation awareness-based decision support systems which can improve human decision-makers' performance and reduce error in dynamic environments.

Multi-stream decision support under concept drift

Huge amounts of streaming data are now generated by government and industry from multiple sources, such as sensors and marketing activities. They are known as multi-streams. Disruptive technologies and unique user experiences, e.g., new markets and new customer behaviors, have inevitably resulted in changes in the underlying data distribution in almost all streaming data. In addition, high-volume streaming data commonly have undiscovered correlations across data streams, and a drift in one stream may cause drift in other streams. A data-driven decision support system on a single stream could be highly related to decision support systems on other streams, thus efficient learning methods on streaming data, such as identifying correlations between streams and constructing adaptive correlation networks, are urgently needed to support the timely prediction of drift and aid decision-making. In the finance industry, for example, the bid/offer rate in the inter-bank lending market always involves the behaviors of more than two banks. The rate needs to be determined based on the interrelationship between banks to benefit the involved banks; in telecommunications, smartphone producers are competitors between each other. The marketing strategy of one producer affects other producers' strategies, especially for large companies, such as Samsung and Apple. Therefore, it is import for the Apple company to analyze marketing behaviors of Samsung to make efficient strategies and maximize the profit. How to take advantage of this interrelationship for decision support to benefit these network groups individually or as a whole is a promising future research direction.

Recommender systems under concept drift

Recommender systems have attracted great attention and achieved great success in the last decade. Nevertheless, the dynamic characteristics of high-volume streaming data have not been adequately addressed. Current recommender systems treat user preferences as static, in spite of the fact that preferences change with increased expertise, personal experiences, or social popularity. The performance of recommender systems will be impaired in many aspects, such as accuracy, novelty and diversity, if these dynamic changes in user profiles, item analysis, or user preferences are not considered. Recommendation should consider the consistency of customer behaviors, customer interactions, and changes in customer preferences; adopting concept drift detection and reaction techniques are, therefore, promising directions in recommender system research for both academia and industry.

Data-driven decision-making under uncertainty

A significant challenge of using large quantities of streaming data collected from different sources in different time frames is uncertainty. Uncertainty in high-volume streaming data takes a number of different forms. We consider that four main layers are impacted by uncertainty issues in streaming data-driven decision support: the data layer, the stream layer, the concept drift detection layer, and the decision-making layer. The first two layers correspond to Component I in Fig. 1. Layers three and four correspond, respectively, to Component II and Component III. Uncertainty problems in the data layer concern data insufficiency, outdatedness, incompletion, and ambiguity. In the stream layer, uncertainty may exist in the relationship between streams, such as whether two streams convey the same information, and may also exist in the correlation of concept drift between streams, such as the likelihood of drift in one stream causing drift in other streams. In the concept drift layer, uncertainty may take the form of noise, false alarms caused by outliers, and new emerging classes. Uncertainty issues also need to be considered in the generation of drift early warning. Lastly, in the decision-making layer, both the model adaptation and decision optimization processes may be subject to uncertainty issues, since there is no universal decision model to fit all situations. The research problem is to develop a general guidance framework for addressing uncertainty issues and to use uncertainty characteristics to aid decision support.