Goal-Oriented BI

Although several "standard" GRL evaluation algorithms (qualitative, quantitative, and hybrid) already exist, as described by Amyot et al., none of them provides the formula-based KPI aggregation required for the type of cause-effect analysis performed in our decision making context. The current algorithms allow modellers to specify the contribution level of a KPI on another GRL intentional element and to calculate the satisfaction level of that target element. However, these algorithms prevent one KPI from driving the computation of the current value of another KPI. Although the current evaluation methods allow computing the impact of one KPI on another KPI in terms of satisfaction levels, when it comes to showing the impact of several KPIs on one KPI (e.g., their aggregate effect), the current evaluation methods quickly become a bottleneck and thus obstruct the cause-effect analysis. Furthermore, the current algorithms have no means to consider the impact of situations (threats and opportunities) in the calculation of the evaluation values. There is also a need to connect KPIs to raw data values in goal models. These three features (KPI aggregation, KPI calculation using raw data values, and situation impact) are essential as they allow modellers and managers to explore and modify the way KPIs impact each other inside the goal model. This may happen frequently, especially when little data or knowledge about the enterprise is available or when the organization is small and does not want to have a large infrastructure in place. The alternative is to rely on heavy computing at the BI tool level and hence involves complex modifications outside the modelling environment that often require implementation by IT staff.

Other modelling languages and enterprise modelling methodologies exist that can be used to model KPIs, however many have a limited computational power and do not allow one to define proper relationships between KPIs for advanced analysis. In addition, there have been recent efforts in industry to use strategy maps and measurable objectives to help with decision making and process improvement. However, the mutual influence of KPIs on each other has not been discussed. More recently, and inspired by work on GRL and KPIs, the Business Intelligence Model (BIM) proposed by Jiang et al. also suggests KPI aggregation in goal models. However, BIM's formal semantics is mainly based on Description Logic, and hence semantics and tools are currently unable to consider the impact of situations in models in a quantitative way.

In order to address these issues, we introduce further extensions to GRL and a novel evaluation algorithm that allow analysts and decision-makers to define precise mathematical formulae describing relationships between the model elements. This method extends the bottom-up propagation algorithm defined in (Amyot et al.). Modellers and analysts gain full control of the model and can change the impact of one element on another as desired.

The algorithm uses current/evaluation values of the source KPIs as inputs for the formula (described as metadata, see Figure 4) and calculates the target KPI evaluation value using these inputs. Then, the satisfaction level of the KPI is calculated using the KPI's target/threshold/worst values as discussed previously. The impact of KPIs on other types of intentional elements (e.g., goals, soft goals, and tasks) is computed using conventional GRL quantitative and qualitative algorithms. This unique combination allows one to have both quantifiable KPIs and strategic-level soft goals that are hard to quantify together in the same model and to show and monitor the impact of KPIs on the goals of the organization.

Figure 4

Formula-based KPI evaluation.

Figure 4 shows a simple example where the current KPI values are displayed, with their units, above the usual satisfaction values. Note that the inputs can be of different units; the formula in the target KPI must take this into consideration. In this example, the current value of Profit is computed as Revenue – Costs – Stolen*50 (the first two are in dollars and the third is a number of items). Note also that the contributions have no weight; the satisfaction of the Profit KPI is based on the normalization of its computed current value ($39,000) against its specified target, threshold, and worse values. We have implemented this new formula-based algorithm in the jUCMNav tool.

Another benefit of this formula-based approach is the ability to account for situations. In organizations, cause-effect analysis, and decision making usually involve an element of threat or opportunity that we capture as situations influencing the business. Even though we can easily show situations as model elements in GRL diagrams (e.g., using soft goals stereotyped with «Situation»), it is rather hard to quantify the impact of situations on the value of a KPI and consider it in evaluation algorithms.

It has been argued that integrating risks, threats, and opportunities into KPI frameworks can contribute to effective management. We therefore propose such integration in GRL with a new model element called situation (a stereotyped softgoal) with an evaluation value between 0 and 100. This element allows us to easily model and evaluate threats and opportunities in the models. The situation element can be connected to KPI nodes using either positive or negative contributions. Table 3 discusses one potential example of the expected impact on a target KPI. The method is not limited to this example (where linear interpolation is used) and one can define any formula that is suitable for their situation.

Table 3 An example heuristic for situation impact on a target KPI

The target KPI threshold value changes based on the evaluation level and level of contribution of the situation factor on the target element as indicated in Table 4. This enables the modeller to vary the acceptable range of values for a KPI when there is an expected situation involved.

Table 4 Situation formula based on the example situation heuristic

The formalization of this algorithm and its implementation are further detailed by Pourshahid.

Although we believe situations give more power to the analyst to set expectations and define the expected outcome of a decision made on the business goals, the usage of situations is optional in our methodology. More organizations are, however, seeking to integrate risk and performance management methods, therefore, this capability is provided for those that have risk information available and find it useful to integrate this information into their decision making approaches.

As an example of application, Figure 5 uses the same formula-based algorithm described in Figure 4, only this time the model also contains a new situation element, namely a risk with a negative contribution (−75). Therefore, the threshold value of the profit KPI has changed and increased the acceptable range for the evaluation value of the KPI. In this example, the threat has somehow been mitigated (satisfaction level of 20, which means weakly denied). Therefore, even though the evaluation value of the KPI (i.e., 39,000$) has not changed from Figure 4, the evaluation level of the KPI has slightly improved and went up from 45 to 52.

Figure 5

Situation impact on KPI evaluation (with jUCMNav).