Business Process Performance Measurement

Abstract

Measuring the performance of business processes has become a central issue in both academia and business, since organizations are challenged to achieve effective and efficient results. Applying performance measurement models to this purpose ensures alignment with a business strategy, which implies that the choice of performance indicators is organization-dependent. Nonetheless, such measurement models generally suffer from a lack of guidance regarding the performance indicators that exist and how they can be concretized in practice. To fill this gap, we conducted a structured literature review to find patterns or trends in the research on business process performance measurement. The study also documents an extended list of 140 process-related performance indicators in a systematic manner by further categorizing them into 11 performance perspectives in order to gain a holistic view. Managers and scholars can consult the provided list to choose the indicators that are of interest to them, considering each perspective. The structured literature review concludes with avenues for further research.

Source: Amy Van Looy and Aygun Shafagatova, https://springerplus.springeropen.com/articles/10.1186/s40064-016-3498-1
This work is licensed under a Creative Commons Attribution 4.0 License.

Background

Since organizations endeavor to measure what they manage, performance measurement is a central issue in both the literature and in practice. Performance measurement is a multidisciplinary topic that is highly studied by both the management and information systems domains (business process management or BPM in particular). Different performance measurement models, systems and frameworks have been developed by academia and practitioners. While measurement models were initially limited to financial performance (e.g., traditional controlling models), a more balanced and integrated approach was needed beginning in the 1990s due to the challenges of the rapidly changing society and technology; this approach resulted in multi-dimensional models. Perhaps the best known multi-dimensional performance measurement model is the Balanced Scorecard (BSC) developed by Kaplan and Norton, which takes a four-dimensional approach to organizational performance: (1) financial perspective, (2) customer perspective, (3) internal business process perspective, and (4) "learning and growth" perspective. The BSC helps translate an organization’s strategy into operational performance indicators (also called performance measures or metrics) and objectives with targets for each of these performance perspectives. Even today, the BSC is by far the most used performance measurement approach in the business world.

Equally important for measuring an organization’s performance is process-oriented management or business process management (BPM), which is "about managing entire chains of events, activities and decisions that ultimately add value to the organization and its customers. These ‘chains of events, activities and decisions’ are called processes". In particular, an organization can do more with its current resources by boosting the effectiveness and efficiency of its way of working (i.e., its business processes). In this regard, academic research also suggests a strong link between business process performance and organizational performance, either in the sense of a causal relationship or as distinctive indicators that co-exist, as in the BSC.

Nonetheless, performance measurement models tend to give little guidance on how business (process) performance indicators can be chosen and operationalized. They are limited to mainly defining performance perspectives, possibly with some examples or steps to derive performance indicators, but without offering concrete indicators. Whereas fairly large bodies of research exist for both performance models and business processes, no structured literature review of (process) performance measurement has been carried out thus far. To the best of our knowledge, existing reviews cover one or another aspect of performance measurement; for instance, reviews on measurement models or evaluation criteria for performance indicators. Despite the considerable importance of a comprehensive and holistic approach to business (process) performance measurement, little is known regarding the state of the research on alternative performance indicators and their operationalization with respect to evaluating the performance of an organization’s work routines. To some extent, this lack of guidance can be explained by the fact that performance indicators are considered organization-dependent, given that strategic alignment is claimed by many measurement models such as the BSC. Although the selection of appropriate performance indicators is challenging for practitioners due to the lack of best practices, it is also highly relevant for performance measurement.

The gap that we are studying is the identification and, in particular, the concretization/operationalization of process-related performance indicators. This study enhances the information systems literature, which focuses on the design and development of measurement systems without paying much attention to essential indicators. To fill this gap, our study presents a structured literature review in order to describe the current state of business process performance measurement and related performance indicators. The choice to focus on the business process management (BPM) discipline is motivated by the close link between organizational performance and business process performance, as well as to ensure a clear scope (specifically targeting an organization’s way of working). Accordingly, the study addresses the following research questions.

RQ₁. What is the current state of the research on business process performance measurement?
RQ₂. Which indicators, measures and metrics are used or mentioned in the current literature related to business process performance?

The objective of RQ₁ is to identify patterns in the current body of knowledge and to note weaknesses, whereas RQ₂ mainly intends to develop an extended list of measurable process performance indicators, categorized into recognized performance perspectives, which can be tailored to diverse purposes. This list could, for instance, serve as a supplement to existing performance measurement models. Practitioners can use the list as a source for best practice indicators from academic research to find and select a subset of performance indicators that fit their strategy. The study will thus not address the development of specific measurement systems but rather the indicators to be used within such systems. To make our intended list system-independent, we will begin with the BSC approach and extend its performance perspectives. Given this generic approach, the research findings can also be used by scholars when building and testing theoretical models in which process performance is one of the factors that must be concretized.

The remainder of this article is structured as follows. "Theoretical background" section describes the theoretical background of performance measurement models and performance indicators. Next, the methodology for our structured literature review is detailed in "Methods" section. The subsequent sections present the results for RQ₁ ("Results for RQ₁" section) and RQ₂ ("Results for RQ₂" section). The discussion of the results in provided in "Discussion" section, followed by concluding comments ("Conclusion" section).

Theoretical background

This section addresses the concepts of performance measurement models and performance indicators separately in order to be able to differentiate them further in the study.

Performance measurement models

According to overviews in the performance literature, some of the most cited performance measurement models are the Balanced Scorecard, self-assessment excellence models such as the EFQM, and the models by Cross and Lynch, Kueng and Neely et al. A distinction should, however, be made between models focusing on the entire business and models focusing on a single business process.

Organizational performance measurement models

Organizational performance measurement models typically intend to provide a holistic view of an organization's performance by considering different performance perspectives. As mentioned earlier, the BSC provides four perspectives for which objectives and performance indicators ensure alignment between strategies and operations (Fig. 1). Other organizational performance measurement models provide similar perspectives. For instance, Cross and Lynch offer a four-level performance pyramid: (1) a top level with a vision, (2) a second level with objectives per business unit in market and financial terms, (3) a third level with objectives per business operating system in terms of customer satisfaction, flexibility and productivity, and (4) a bottom level with operational objectives for quality, delivery, process time and costs. Another alternative view on organizational performance measurement is given in business excellence models, which focus on an evaluation through self-assessment rather than on strategic alignment, albeit by also offering performance perspectives. For instance, the EFQM distinguishes enablers [i.e., (1) leadership, (2) people, (3) strategy, (4) partnerships and resources, and (5) processes, products and services] from results [i.e., (1) people results, (2) customer results, (3) society results, and (4) key results], and a feedback loop for learning, creativity and innovation.

Fig. 1

An overview of the performance perspectives in Kaplan and Norton (1996, 2001)

Since the BSC is the most used performance measurement model, we have chosen it as a reference model to illustrate the function of an organizational performance measurement model. The BSC is designed to find a balance between financial and non-financial performance indicators, between the interests of internal and external stakeholders, and between presenting past performance and predicting future performance. The BSC encourages organizations to directly derive (strategic) long-term objectives from the overall strategy and to link them to (operational) short-term targets. Concrete performance measures or indicators should be defined to periodically measure the objectives. These indicators are located on one of the four performance perspectives in Fig. 1 (i.e., ideally with a maximum of five indicators per perspective).

Table 1 illustrates how an organizational strategy can be translated into operational terms using the BSC.

Table 1 An example of translating an organizational strategy into operational terms using the BSC

Perspective	Strategy	Objective	Indicator, measure or metric	Target	Initiative
Customer	Operational excellence	Industry-leading customer loyalty	Customer satisfaction rating	80	85	90	Mystery shopper program Customer loyalty program

Perspective

Strategy

Objective

Indicator, measure or metric

Target

Initiative

Year 1 (%)

Year 2 (%)

Year 3 (%)

Customer

Operational excellence

Industry-leading customer loyalty

Customer satisfaction rating

80

85

90

Mystery shopper program

Customer loyalty program

During periodical measurements using the BSC, managers can assign color-coded labels according to actual performance on short-term targets: (1) a green label if the organization has achieved the target, (2) an orange label if it is almost achieved, or (3) a red label if it is not achieved. Orange and red labels thus indicate areas for improvement.

Furthermore, the BSC assumes a causal or logical relationship between the four performance perspectives. An increase in the competences of employees (i.e., performance related to "learning and growth") is expected to positively affect the quality of products and services (i.e., internal business process performance), which in turn will lead to improved customer perceptions (i.e., customer performance). The results for the previous perspectives will then contribute to financial performance to ultimately realize the organization's strategy, mission and vision. Hence, indicators belonging to the financial and customer perspectives are assumed to measure performance outcomes, whereas indicators from the perspectives of internal business processes and "learning and growth" are considered as typical performance drivers.

Despite its widespread use and acceptance, the BSC is also criticized for appearing too general by managers who are challenged to adapt it to the culture of their organization or find suitable indicators to capture the various aspects of their organization's strategy. Additionally, researchers question the choice of four distinct performance perspectives (i.e., which do not include perspectives related to inter-organizational performance or sustainability issues). Further, the causal relationship among the BSC perspectives has been questioned. To some degree, Kaplan and Norton responded to this criticism by introducing strategy maps that focus more on the causal relationships and the alignment of intangible assets.

Business process performance measurement models

In addition to organizational models, performance measurement can also focus on a single business process, such as statistical process control, workflow-based monitoring or process performance measurement systems. The approach taken in business process performance measurement is generally less holistic than the BSC. For instance, in an established BPM handbook, Dumas et al. position time, cost, quality and flexibility as the typical performance perspectives of business process performance measurement (Fig. 2). Similar to organizational performance measurement, concrete performance measures or indicators should be defined for each process performance perspective. In this sense, the established perspectives of Dumas et al. seem to further refine the internal business process performance perspective of the BSC.

Fig. 2

An overview of the performance perspectives in Dumas et al. (2013)

Neely et al., on the other hand, present ten steps to develop or define process performance indicators. The process performance measurement system of Kueng is also of high importance, which is visualized as a "goal and performance indicator tree" with five process performance perspectives: (1) financial view, (2) customer view, (3) employee view, (4) societal view, and (5) innovation view. Kueng thus suggests a more holistic approach towards process performance, similar to organizational performance, given the central role of business processes in an organization. He does so by focusing more on the different stakeholders involved in certain business processes.

Performance indicators

Section "Performance measurement models" explained that performance measurement models typically distinguish different performance perspectives for which performance indicators should be further defined. We must, however, note that we consider performance measures, performance metrics and (key) performance indicators as synonyms. For reasons of conciseness, this work will mainly refer to performance indicators without mentioning the synonyms. In addition to a name, each performance indicator should also have a concretization or operationalization that describes exactly how it is measured and that can result in a value to be compared against a target. For instance, regarding the example in Table 1, the qualitative statements to measure customer satisfaction constitute an operationalization. Nonetheless, different ways of operationalization can be applied to measure the same performance indicator. Since organizations can profit from reusing existing performance indicators and the related operationalization instead of inventing new ones (i.e., to facilitate benchmarking and save time), this work investigates which performance indicators are used or mentioned in the literature on business process performance and how they are operationalized.

Neely et al. and Richard et al. both present evaluation criteria for performance indicators (i.e., in the sense of desirable characteristics or review implications), which summarize the general consensus in the performance literature. First, the literature strongly agrees that performance indicators are organization-dependent and should be derived from an organization's objectives, strategy, mission and vision. Secondly, consensus in the literature also exists regarding the need to combine financial and non-financial performance indicators. Nonetheless, disagreement still seems to exist in terms of whether objective and subjective indicators need to be combined, with objective indicators preferred by most advocates. Although subjective (or quasi-objective) indicators face challenges from bias, their use has some advantages; for instance, to include stakeholders in an assessment, to address latent constructs or to facilitate benchmarking when a fixed reference point is missing. Moreover, empirical research has shown that subjective (or quasi-objective) indicators are more or less correlated with objective indicators, depending on the level of detail of the subjective question. For instance, a subjective question can be made more objective by using clear definitions or by selecting only well-informed respondents to reduce bias.

Methods

We conducted a structured literature review (SLR) to find papers dealing with performance measurement in the business process literature. SLR can be defined as "a means of evaluating and interpreting all available research relevant to a particular research question, topic area, or phenomenon of interest". An SLR is a meta study that identifies and summarizes evidence from earlier research or a way to address a potentially large number of identified sources based on a strict protocol used to search and appraise the literature. It is systematic in the sense of a systematic approach to finding relevant papers and a systematic way of classifying the papers. Hence, according to Boellt and Cecez-Kecmanovic, SLR as a specific type of literature review can only be used when two conditions are met. First, the topic should be well-specified and closely formulated (i.e., limited to performance measurement in the context of business processes) to potentially identify all relevant literature based on inclusion and exclusion criteria. Secondly, the research questions should be answered by extracting and aggregating evidence from the identified literature based on a high-level summary or bibliometric-type of content analysis. Furthermore, King and He also refer to a statistical analysis of existing literature.

Informed by the established guidelines proposed by Kitchenham, we undertook the review in distinct stages: (1) formulating the research questions and the search strategy, (2) filtering and extracting data based on inclusion and exclusion criteria, and (3) synthesizing the findings. The remainder of this section describes the details of each stage.

Formulating the research questions and search strategy

A comprehensive and unbiased search is one of the fundamental factors that distinguish a systematic review from a traditional literature review. For this purpose, a systematic search begins with the identification of keywords and search terms that are derived from the research questions. Based on the research questions stipulated in the introduction, the SLR protocol for our study was defined, as shown in Table 2.

Table 2 The structured literature review protocol for this study, based on Boellt and Cecez-Kecmanovic (2015)

Protocol elements	Translation to this study
1/Research question	RQ1. What is the current state of the research on business process performance measurement? RQ2. Which indicators, measures and metrics are used or mentioned in the current literature related to business process performance?
2/Sources searched	Web of science database (until November 2015)
3/Search terms	Combining "business process" and "performance indicator"/"performance metric"/"performance measur"
4/Search strategy	Different search queries, with keywords in topic and title (Table 3)
5/Inclusion criteria	Include only papers containing a combination of search terms, defined in the search queries Include only papers indexed in the Web of Science from all periods until November 2015 Include only papers written in English
6/Exclusion criteria	Exclude unrelated papers, i.e., if they do not explicitly claim addressing the measurement of business process performance
7/Quality criteria	Only peer-reviewed papers are indexed in the web of science database

The ISI Web of Science (WoS) database was searched using predetermined search terms in November 2015. This database was selected because it is used by many universities and results in the most outstanding publications, thus increasing the quality of our findings. An important requirement was that the papers focus on "business process*" (BP). This keyword was used in combination with at least one of the following: (1) "performance indicator*", (2) "performance metric*", (3) "performance measur*". All combinations of "keyword in topic" (TO) and "keyword in title" (TI) have been used.

Table 3 shows the degree to which the initial sample sizes varied, with 433 resulting papers for the most permissive search query (TOxTO) and 19 papers for the most restrictive one (TIxTI). The next stage started with the most permissive search query in an effort to select and assess as many relevant publications as possible.

Table 3 The number of papers in the web of science per search query (until November 2015)

	(1) "Performance indicator*"	(2) "Performance metric*"	(3) "Performance measur*"	TOTAL
Column keywords in TO
BP-TO	153	30	250	433
BP-TI	31	4	64	99
Column keywords in TI
BP-TO	19	2	62	83
BP-TI	5	0	14	19

Filtering and extracting data

Figure 3 summarizes the procedure for searching and selecting the literature to be reviewed. The list of papers found in the previous stage was filtered by deleting 35 duplicates, and the remaining 398 papers were further narrowed to 153 papers by evaluating their title and abstract. After screening the body of the texts, 76 full-text papers were considered relevant for our scope and constituted the final sample.

Fig. 3

Exclusion of papers and number of primary studies

More specifically, studies were excluded if their main focus was not business process performance measurement or if they did not refer to indicators, measures or metrics for business performance. The inclusion of studies was not restricted to any specific type of intervention or outcome. The SLR thus included all types of research studies that were written in English and published up to and including November 2015. Furthermore, publication by peer-reviewed publication outlets (e.g., journals or conference proceedings) was considered as a quality criterion to ensure the academic level of the research papers.

Synthesizing the findings

The analysis of the final sample was performed by means of narrative and descriptive analysis techniques. For RQ1, the 76 papers were analyzed on the basis of bibliometric data (e.g., publication type, publication year, geography) and general performance measurement issues by paying attention to the methodology and focus of the study.

For RQ2, all the selected papers were screened to identify concrete performance indicators in order to generate a comprehensive list or checklist. The latter was done in different phases. In the first phase, the structured literature review allowed us to analyze which performance indicators are mainly used in the process literature and how they are concretized (e.g., in a question or mathematical formulation), resulting in an unstructured list of potential performance indicators. The indicators were also synthesized by combining similar indicators and rephrasing them into more generic terms.

The next phase was a comparative study to categorize the output of phase 1 into the commonly used measurement models in the performance literature. For the purpose of this study, we specifically looked for those organizational performance models, mentioned in "Theoretical background" section, that are cited the most and that suggest categories, dimensions or performance perspectives that can be re-used. Since the BSC is the most commonly used of these measurement models, we began with the BSC as the overall framework to categorize the observed indicators related to business (process) performance, supplemented with an established view on process performance from the process literature. Subsequently, a structured list of potential performance indicators was obtained.

In the third and final phase, an evaluation study was performed to validate whether the output of phase 2 is sufficiently comprehensive according to other performance measurement models, i.e., not included in our sample and differing from the most commonly used performance measurement models. Therefore, we investigated the degree to which our structured list covers the items in two variants or concretizations of the BSC. Hence, a validation by other theoretical models is provided. We note that a validation by subject-matter experts is out of scope for a structured literature review but relates to an opportunity for further research.

Results for RQ1

The final sample of 76 papers consists of 46 journal papers and 30 conference papers (Fig. 4), indicating a wide variety of outlets to reach the audience via operations and production-related journals in particular or in lower-ranked information systems journals.

Fig. 4

The distribution of the sampled papers per publication type (N = 76)

When considering the chronological distribution of the sampled papers, Fig. 5 indicates an increase in the uptake of the topic in recent years, particularly for conference papers but also for journal publications since 2005.

Fig. 5

The chronological distribution of the sampled papers per publication type (N = 76)

This uptake seems particularly situated in the Western world and Asia (Fig. 6). The countries with five or more papers in our sample are Germany (12 papers), the US (6 papers), Spain (5 papers), Croatia (5 papers) and China (5 papers). Figure 6 shows that business process performance measurement is a worldwide topic, with papers across the different continents. Nonetheless, a possible explanation for the higher coverage in the Western world could be due to its long tradition of measuring work (i.e., BSC origins).

Fig. 6

The geographical distribution of the sampled papers per continent, based on a paper's first author (N = 76)

The vast majority of the sampled papers address artifacts related to business (process) performance measurement. When looking at the research paradigm in which the papers are situated (Fig. 7), 71 % address design-science research, whereas 17 % conduct research in behavioral science and 12 % present a literature review. This could be another explanation for the increasing uptake in the Western world, as many design-science researchers are from Europe or North America.

Fig. 7

The distribution of the sampled journal papers per research paradigm (N = 76)

Figure 8 supplements Fig. 7 by specifying the research methods used in the papers. For the behavioral-science papers, case studies and surveys are equally used. The 54 papers that are situated within the design-science paradigm explicitly refer to models, meta-models, frameworks, methods and/or tools. When mapping these 54 papers to the four artifact types of March and Smith, the vast majority present (1) methods in the sense of steps to perform a task (e.g., algorithms or guidelines for performance measurement) and/or (2) models to describe solutions for the topic. The number of papers dealing with (3) constructs or a vocabulary and/or (4) instantiations or tools is much more limited, with 14 construct-related papers and 9 instantiations in our sample. We also looked at which evaluation methods, defined by Peffers et al., are typically used in the sampled design-science papers. While 7 of the 54 design-science papers do not seem to report on any evaluation effort, our sample confirms that most papers apply one or another evaluation method. Case studies and illustrative scenarios appear to be the most frequently used methods to evaluate design-science research on business (process) performance measurement.

Fig. 8

The distribution of the sampled journal papers per research method (N = 76)

The sampled design-science research papers typically build and test performance measurement frameworks, systems or models or suggest meta-models and generic templates to integrate performance indicators into the process models of an organization. Such papers can focus on the process level, organizational level or even cross-organizational level. Nonetheless, the indicators mentioned in those papers are illustrative rather than comprehensive. An all-inclusive list of generic performance indicators seems to be missing. Some authors propose a set of indicators, but those indicators are specific to a certain domain or sector instead of being generic. For instance, Table 4 shows that 36 of the 76 sampled papers are dedicated to a specific domain or sector, such as technology-related aspects or supply chain management.

Table 4 The number of sampled papers dedicated to a specific domain or sector (N = 76)

Domain or sector	Number of papers
IS/IT	7
Supply chain	5
Business network	3
Manufacturing	3
Services	3
Automobile	2
Banking/financial	2
Government	2
Health	2
Helpdesk/maintenance	2
Construction	1
HR	1
SME	1
Strategic planning	1
Telecom	1
Total	36

Furthermore, the reviewed literature was analyzed with regard to its (1) scope, (2) functionalities, (3) terminology, and (4) foundations.

Starting with scope, it is observed that nearly two-thirds of the sampled papers can be categorized as dealing with process-oriented performance measurement, whereas one-third focuses more on general performance measurement and management issues. Nonetheless, most of the studies of process performance also include general performance measurement as a supporting concept. A minor cluster of eight research papers specifically focuses on business process reengineering and measurement systems to evaluate the results of reengineering efforts. Furthermore, other researchers focus on the measurement and assessment of interoperability issues and supply chain management measurements.

Secondly, while analyzing the literature, two groups of papers were identified based on their functionalities: (1) focusing on performance measurement systems or frameworks, and (2) focusing on certain performance indicators and their categorization. Regarding the first group, it should be mentioned that while the process of building or developing a performance measurement system (PMS) or framework is well-researched, only a small number of papers explicitly address process performance measurement systems (PPMS). The papers in this first group typically suggest concrete steps or stages to be followed by particular organizations or discuss the conceptual characteristics and design of a performance measurement system. Regarding the second group of performance indicators, we can differentiate two sub-groups. Some authors focus on the process of defining performance indicators by listing requirements or quality characteristics that an indicator should meet. However, many more authors are interested in integrating performance indicators into the process models or the whole architecture of an organization, and they suggest concrete solutions to do so. Compared to the first group of papers, this second group deals more with the categorization of performance indicators into domains (financial/non-financial, lag/lead, external/internal, BSC dimensions) or levels (strategic, tactical, operational).

Thirdly, regarding terminology, different terms are used by different authors to discuss performance measurement. Performance "indicator" is the most commonly used term among the reviewed papers. For instance, it is frequently used in reference to a key performance indicator (KPI), a KPI area or a performance indicator (PI). The concept of a process performance indicator (PPI) is also used, mainly in the process-oriented literature. Performance "measure" is another prevalent term in the papers. The least-used term is performance "metric" (i.e., in only nine papers). Although the concepts of performance indicators, measures and metrics are used interchangeably throughout most of the papers, the concepts are sometimes defined in different ways. For instance, paper 17 defines a performance indicator as a metric, and paper 49 defines a performance measure as an indicator. On the other hand, paper 7 defines a performance indicator as a set of measures. Yet another perspective is taken in paper 74, which defines a performance measure as "a description of something that can be directly measured (e.g., number of reworks per day)", while defining a performance indicator as "a description of something that is calculated from performance measures (e.g., percentage reworks per day per direct employee" (p. 386). Inconsistencies exist not only in defining indicators but also in describing performance goals. For instance, some authors include a sign (e.g., minus or plus) or a verb (e.g., decrease or increase) in front of an indicator. Other authors attempt to describe performance goals in a SMART way - for instance, by including a time indication (e.g., "within a certain period") and/or target (e.g., "5 % of all orders") - whereas most of the authors are less precise. Hence, a great degree of ambiguity exists in the formulation of performance objectives among to the reviewed papers.

Finally, regarding the papers' foundations, "Performance measurement models" section already indicated that the BSC plays an important role in the general literature on performance management systems (PMS), while Kueng (2000) also offers influential arguments on process performance measurement systems (PPMS). In our literature review, we observed that the BSC was mentioned in 43 of the 76 papers and that the results of 19 papers were mainly based on the BSC (Fig. 9). This finding provides additional evidence that the BSC can be considered the most frequently used performance model in academia as well. However, the measurement model of Kueng (2000) was also mentioned in the sampled papers on PPMS, though less frequently (i.e., in six papers).

Fig. 9

The importance of the BSC according to the sampled papers (N = 76)

Interestingly, the BSC is also criticized by the sampled papers for not being comprehensive; for instance, due to the exclusion of environmental aspects, supply chain management aspects or cross-organizational processes. In response, some of the sampled papers also define sector-specific BSC indicators or suggest additional steps or indicators to make the process or business more sustainable (see Table 4). Nonetheless, the majority of the papers agree on the need for integrated and multidimensional measurement systems, such as the BSC, and on the importance of directly linking performance measurement to an organization's strategy. However, while these papers mention the required link with strategy, the prioritization of indicators according to their strategic importance has been studied very little thus far.

Results for RQ2

For RQ₂, the sampled papers were reviewed to distinguish papers with performance indicators from papers without performance indicators. A further distinction was made between indicators found with operationalization (i.e., concretization by means of a question or formula) and those without operationalization. We note that for many indicators, no operationalization was available. We discovered that only 30 of the 76 sampled papers contained some type of performance indicator (namely 3, 5, 6, 7, 11, 16, 17, 18, 20, 22, 26, 27, 30, 35, 37, 40, 43, 46, 49, 51, 52, 53, 55, 57, 58, 59, 60, 66, 71, 73). In total, approximately 380 individual indicators were found throughout all the sampled papers (including duplicates), which were combined based on similarities and modified to use more generic terms. This resulted in 87 indicators with operationalization and 48 indicators without operationalization.

The 87 indicators with operationalization were then categorized according to the four perspectives of the BSC (i.e., financial, customer, business processes, and "learning and growth") and the four established dimensions of process performance (i.e., time, cost, quality, and flexibility) (Dumas et al. 2013). In particular, based in the identified indicators, we revealed 11 sub-perspectives within the initial BSC perspectives to better emphasize the focus of the indicators and the different target groups (Table 5): (1) financial performance for shareholders and top management, (2) customer-related performance, (3) supplier-related performance, (4) society-related performance, (5) general process performance, (6) time-related process performance, (7) cost-related process performance, (8) process performance related to internal quality, (9) flexibility-related process performance, (10) (digital) innovation performance, and (11) employee-related performance.

Table 5 A description of the observed performance perspectives, linked to the Balanced scorecard

Initial BSC perspectives	Observed perspectives based on target groups and focus	Scope of the performance indicators
1. Financial performance	1.1 Financial performance for shareholders and top management	Strategic financial data
2. Customer-related performance	2.1 Customer performance	Outcomes of external quality or meeting end user needs
	2.2 Supplier performance	External collaboration and process dependencies
	2.3 Society performance	Outcomes for other stakeholders and the environment during process work
3. Internal business process performance	3.1 General process performance	Descriptive data of process work, not related to time, costs, quality or flexibility
	3.2 Time-related process performance	Time-related data of process work
	3.3 Cost-related process performance	Operational financial data
	3.4 Process performance related to internal quality	Capability of meeting end user needs and internal user needs
	3.5 Flexibility-related process performance	Data of changes or variants in process work
4. Performance related to "learning and growth"	4.1 (Digital) innovation performance	Innovation of processes and innovation projects
4. Performance related to "learning and growth"	4.2 Employee performance	Staff contributions to process work and personal development

For reasons of objectivity, the observed performance indicators were assigned to a single perspective starting from recognized frameworks. Bias was further reduced by following the definitions of Table 5. Furthermore, the authors of this article first classified the indicators individually and then reached consensus to obtain a more objective categorization.

Additional rationale for the identification of 11 performance perspectives is presented in Table 6, which compares our observations with the perspectives adopted by the most commonly used performance measurement models (see "Theoretical background" section). This comparison allows us to highlight similarities and differences with other respected models. In particular, Table 6 shows that we did not observe a dedicated perspective for strategy and that we did not differentiate between financial indicators and market indicators. Nonetheless, the similarities in Table 6 prevail. For instance, Cross and Lynch also acknowledge different process dimensions. Further, Kueng and the EFQM also differentiate employee performance from innovation performance, and they both add a separate perspective for results related to the entire society.

Table 6 The comparison of our observed performance perspectives with the perspectives taken in the most commonly used performance measurement models in the literature

Balanced scorecard	EFQM	Kueng	Cross and Lynch	Our observed performance perspectives
Financial perspective	Key results	Financial view	Financial measures Market measures	Financial performance for shareholders and top management
Customer perspective	Customer results	Customer view	Customer satisfaction	Customer performance Supplier performance Society performance
Internal business processes perspective	Enablers (processes/products/services, people, strategy, partnerships/resources, leadership)	Overall process performance based on the other views as driving forces	Flexibility Productivity Quality Delivery Process time Cost	General process performance Time-related process performance Cost-related process performance Process performance related to internal quality Flexibility-related process performance
"Learning and growth" perspective	People results Learning, creativity and innovation	Employee view Innovation view	–	(Digital) innovation performance Employee performance
–	Society results	Societal view	–	Society performance as a sub-perspective of customer performance (see above)

Figure 10 summarizes the number of performance indicators that we identified in the process literature per observed performance perspective. Not surprisingly, the initial BSC perspective of internal business process performance contains most of the performance indicators: 29 of 87 indicators. However, the other initial BSC perspectives are also covered by a relatively high number of indicators: 16 indicators for both financial performance and customer-related performance and 26 indicators for "learning and growth". This result confirms the close link between process performance and organizational performance, as mentioned in the introduction.

Fig. 10

The number of performance indicators with operationalization per performance perspective

A more detailed comparison of the perspectives provides interesting refinements to the state of the research. More specifically, Fig. 10 shows that five performance perspectives have more than ten indicators in the sample, indicating that academic research focuses more on financial performance for shareholders and top management and performance related to customers, process time, innovation and employees. On the other hand, fewer than five performance indicators were found in the sample for the perspectives related to suppliers, society, process costs and process flexibility, indicating that the literature focuses less on those perspectives. The latter remains largely overlooked by academic research, possibly due to the newly emerging character of these perspectives.

We must, however, note that the majority of the performance indicators are mentioned in only a few papers. For instance, 59 of the 87 indicators were cited in a single paper, whereas the remainder are mentioned in more than one paper. Eleven performance indicators are frequently mentioned in the process literature (i.e., by five or more papers). These indicators include four indicators of customer-related performance (i.e., customer complaints, perceived customer satisfaction, query time, and delivery reliability), three indicators of time-related process performance (i.e., process cycle time, sub-process turnaround time, and process waiting time), one cost-related performance indicator (i.e., process cost), two indicators of process performance related to internal quality (i.e., quality of internal outputs and deadline adherence), and one indicator of employee performance (i.e., perceived employee satisfaction).

Consistent with "Performance indicators" section, the different performance perspectives are a combination of financial or cost-related indicators with non-financial data. The latter also take the upper hand in our sample. Furthermore, the sample includes a combination of objective and subjective indicators, and the vast majority are objective indicators. Only eight indicators explicitly refer to qualitative scales; for instance, to measure the degree of satisfaction of the different stakeholder groups. For all the other performance indicators, a quantifiable alternative is provided.

It is important to remember that a distinction was made between the indicators with operationalization and those without operationalization. The list of 87 performance indicators, can thus be extended with those indicators for which operationalization is missing in the reviewed literature. Specifically, we found 48 additional performance indicators that mainly address supplier performance, process performance related to costs and flexibility, and the employee-related aspects of digital innovation. Consequently, this structured literature review uncovered a total of 135 performance indicators that are directly or indirectly linked to business process performance.

Finally, the total list of 135 performance indicators was evaluated for its comprehensiveness by comparing the identified indicators with other BSC variants that were not included in our sample. More specifically, based on a random search, we looked for two BSC variants in the Web of Science that did not fit the search strategy of this structured literature review: one that did not fit the search term of "business process*" and another that did not fit any of the performance-related search terms of "performance indicator*", "performance metric*" or "performance measur*". These two BSC variants cover 30 and 17 performance indicators, respectively, and are thus less comprehensive than the extended list presented in this study. Most of the performance indicators suggested by the two BSC variants are either directly covered in our findings or could be derived after recalculations. Only five performance indicators could not be linked to our list of 135 indicators, and these suggest possible refinements regarding (1) the growth potential of employees, (2) new markets, (3) the social performance of suppliers, (4) philanthropy, or (5) industry-specific events.

Discussion

This structured literature review culminated in an extended list of 140 performance indicators: 87 indicators with operationalization, 48 indicators without operationalization and 5 refinements derived from two other BSC variants. The evaluation of our findings against two BSC variants validated our work in the sense that we present a more exhaustive list of performance indicators, with operationalization for most, and that only minor refinements could be added. However, the comprehensiveness of our findings can be claimed only to a certain extent given the limitations of our predefined search strategy and the lack of empirical validation by subject-matter experts or organizations. Notwithstanding these limitations, conclusions can be drawn from the large sample of 76 papers to respond to the research questions (RQs).

Regarding RQ₁ on the state of the research on business process performance measurement, the literature review provided additional evidence for the omnipresence of the BSC. Most of the sampled papers mentioned or used the BSC as a starting point and basis for their research and analysis. The literature study also showed a variety of research topics, ranging from behavioral-science to design-science research and from a focus on performance measurement models to a focus on performance indicators. In addition to inconsistencies in the terminology used to describe performance indicators and targets, the main weakness uncovered in this literature review deals with the concretization of performance indicators supplementing performance measurement systems. The SLR results suggest that none of the reviewed papers offers a comprehensive measurement framework, specifically one that includes and extends the BSC perspectives, is process-driven and encompasses as many concrete performance indicators as possible. Such a comprehensive framework could be used as a checklist or a best practice for reference when defining specific performance indicators. Hence, the current literature review offers a first step towards such a comprehensive framework by means of an extended list of possible performance indicators bundled in 11 performance perspectives (RQ₂).

Regarding RQ₂ on process performance indicators, the literature study revealed that scholars measure performance in many different ways and without sharing much detail regarding the operationalization of the measurement instruments, which makes a comparison of research results more difficult. As such, the extended list of performance indicators is our main contribution and fills a gap in the literature by providing a detailed overview of performance indicators mentioned or used in the literature on business process performance. Another novel aspect is that we responded to the criticism of missing perspectives in the original BSC and identified the narrow view of performance typically taken in the process literature. Figures 1 and 2 are now combined and extended in a more exhaustive way, namely by means of more perspectives than are offered by other attempts (Table 6), by explicitly differentiating between performance drivers (or lead indicators) and performance outcomes (or lag indicators), and by considering concrete performance indicators.

Our work also demonstrated that all perspectives in the BSC relate to business process performance to some degree. In other words, while the BSC is a strategic tool for organizational performance measurement, it is actually based on indicators that originate from business processes. More specifically, in addition to the perspective of internal business processes, the financial performance perspective typically refers to sales or revenues gained while doing business, particularly after executing business processes. The customer perspective relates to the implications of product or service delivery, specifically to the interactions throughout business processes, whereas the "learning and growth" perspective relates to innovations in the way of working (i.e., business processes) and the degree to which employees are prepared to conduct and innovate business processes. The BSC, however, does not present sub-perspectives and thus takes a more high-level view of performance. Hence, the BSC can be extended based on other categorizations made in the reviewed literature; for instance, related to internal/external, strategic/operational, financial/non-financial, or cost/time/quality/flexibility.

Therefore, this study refined the initial BSC perspectives into eleven performance perspectives (Fig. 11) by applying three other performance measurement models and the respected Devil’s quadrangle for process performance. Additionally, a more holistic view of business process performance can be obtained by measuring each performance perspective of Fig. 11 than can be achieved by using the established dimensions of time, cost, quality and flexibility as commonly proposed in the process literature. As such, this study demonstrated a highly relevant synergy between the disciplines of process management, organization management and performance management.

Fig. 11

An overview of the observed performance perspectives in the business process literature

We also found out that not all the performance perspectives in Fig. 11 are equally represented in the studied literature. In particular, the perspectives related to suppliers, society, process costs and process flexibility seem under-researched thus far.

The eleven performance perspectives (Fig. 11) can be used by organizations and scholars to measure the performance of business processes in a more holistic way, considering the implications for different target groups. For each perspective, performance indicators can be selected that fit particular needs. Thus, we do not assert that every indicator in the extended list of 140 performance indicators should always be measured, since "Theoretical background" section emphasized the need for organization-dependent indicators aligned with an organization’s strategy. Instead, our extended list can be a starting point for finding and using appropriate indicators for each performance perspective, without losing much time reflecting on possible indicators or ways to concretize those indicators. Similarly, the list can be used by scholars, since many studies in both the process literature and management literature intend to measure the performance outcomes of theoretical constructs or developed artifacts.

Consistent with the above, we acknowledge that the observed performance indicators originate from different models and paradigms or can be specific to certain processes or sectors. Since our intention is to provide an exhaustive list of indicators that can be applied to measure business process performance, the indicators are not necessarily fully compatible. Instead, our findings allow the recognition of the role of a business context (i.e., the peculiarities of a business activity, an organization or other circumstances). For instance, a manufacturing organization might choose different indicators from our list than a service or non-profit organization (e.g., manufacturing lead time versus friendliness, or carbon dioxide emission versus stakeholder satisfaction).

Another point of discussion is dedicated to the difference between the performance of specific processes (known as "process performance") and the performance of the entire process portfolio (also called "BPM performance"). While some indicators in our extended list clearly go beyond a single process (e.g., competence-related indicators or employee absenteeism), it is our opinion that the actual performance of multiple processes can be aggregated to obtain BPM performance (e.g., the sum of process waiting times). This distinction between (actual) process performance and BPM performance is useful; for instance, for supplementing models that try to predict the (expected) performance based on capability development, such as process maturity models (e.g., CMMI) and BPM maturity models. Nonetheless, since this study has shown a close link between process performance, BPM performance, and organizational performance, it seems better to refer to different performance perspectives than to differentiate between such performance types.

In future research, the comprehensiveness of the extended list of performance indicators can be empirically validated by subject-matter experts. Additionally, case studies can be conducted in which organizations apply the list as a supplement to performance measurement models in order to facilitate the selection of indicators for their specific business context. The least covered perspectives in the academic research also seem to be those that are newly emerging (namely, the perspectives related to close collaboration with suppliers, society/sustainability and process flexibility or agility), and these need more attention in future research. Another research avenue is to elaborate on the notion of a business context; for instance, by investigating what it means to have a strategic fit in terms of performance measurement and which strategies are typically associated with which performance indicators. Additionally, the impact of environmental aspects, such as market velocity, on the choice of performance indicators can be taken into account in future research.

Conclusion

Business quotes such as "If you cannot measure it, you cannot manage it" or "What is measured improves" (P. Drucker) are sometimes criticized because not all important things seem measurable. Nonetheless, given the perceived need of managers to measure their business and the wide variety of performance indicators (i.e., ranging from quantitative to qualitative and from financial to non-financial), this structured literature review has presented the status of the research on business process performance measurement. This structured approach allowed us to detect weaknesses or inadequacies in the current literature, particularly regarding the definition and concretization of possible performance indicators. We continued by taking a holistic view of the categorization of the observed performance indicators (i.e., measures or metrics) into 11 performance perspectives based on relevant performance measurement models and established process performance dimensions.

The identified performance indicators within the 11 perspectives constitute an extended list from which practitioners and researchers can select appropriate indicators depending on their needs. In total, the structured literature review resulted in 140 possible performance indicators: 87 indicators with operationalization, 48 additional indicators that need further concretization, and 5 refinements based on other Balanced Scorecard (BSC) variants. As such, the 11 performance perspectives with related indicators can be considered a conceptual framework that was derived from the current process literature and theoretically validated by established measurement approaches in organization management.

Future research can empirically validate the conceptual framework by involving subject-matter experts to assess the comprehensiveness of the extended list and refine the missing concretizations, and by undertaking case studies in which the extended list can be applied by specific organizations. Other research avenues exist to investigate the link between actual process performance and expected process performance (as measured in maturity models) or the impact of certain strategic or environmental aspects on the choice of specific performance indicators. Such findings are needed to supplement and enrich existing performance measurement systems.

Site:	Saylor Academy
Course:	BUS631: Brand Management
Book:	Business Process Performance Measurement

Printed by:	Guest user
Date:	Wednesday, October 16, 2024, 8:22 AM