DDDM helps management make better-informed decisions, but when the data is streaming in from a number of sources, it can complicate the process as management attempts to match and keep up with the velocity of the incoming data. Read this article to explore the challenges of making decisions with streaming data and the adaptation needed in the decision-making framework to continue making informed decisions.
Adaptive Decision-Making Framework
This section presents the general framework of adaptive decision-making.
Data-driven decision-making
Data-driven decision-making uses a variety of machine learning approaches for data analysis by characterizing a decision problem and ascertaining the connections between the problem variables (input, internal and output variables) without having explicit knowledge of the physical behavior of the decision model.
Adaptive decision-making to address the concept drift problem has gained considerable attention. Concept drift detection and adaptation is an effective strategy for improving the accuracy of decision-making in a dynamic data streaming environment. When a drift is detected, machine learning techniques are applied to adapt decision models to new concepts. The components of an adaptive data-driven decision-making framework are introduced in the next section.
An adaptive data-driven decision-making framework under concept drift
Fig. 1
A general framework of adaptive M
Data-driven decision support under concept drift in high-volume streaming data has three major components as shown in Fig. 1. The first collects raw data from various sources and reformats them to unify the time frame and feature space, so that they can be applied to modeling and constructing the training data. The second detects and interprets the changes in data streams over time. If the most recently arrived data significantly conflicts with the historical data, a concept drift will be reported and an adaptation process will be triggered. The third component is adaptive decision-making. In this component, DSSs are actively updated according to the results of the drift detection and understanding. The data-driven decision process under this framework is as follows. Drift detection part will detect drift, once a drift is identified, it will notify the system. The drift understanding will be then initialized to target the drift and propose possible drift resolve solutions. To help finding a better solution, the system will interpret the drift from When the drift occurred, How significant the drift is, and Where the drift is located. Drift responses and adaptation are dependent on the types of DSSs. For a model-based DSS, adaptive decision-making could be able to react to drift by, e.g., updating an optimization model's parameters; for a knowledge-based DSS, adaptive decision-making could be able to react to drift by, e.g., updating a knowledge base. In a data-driven DSS, adaptive decision-making involves, e.g., retraining a prediction model. In this paper, we only focus on data-driven DSSs.