The Incompleteness Theorem of Performance Measurement in Service Delivery

Introduction

In boardrooms around the globe, the famous words of former General Electric CEO Jack Welch (2001) are frequently touted: “What you measure is what you get”. This quote is used so often that it has the status of a business axiom and is accepted without further deliberation. From a common sense point of view, this statement is considered as an absolute truth and has led to a proliferation of complex, quantitative performance measurement systems. However, Welch’s simple statement belies the complexity of an effective performance measurement system. The literature on this topic is vast, with each author claiming to address the shortcomings of competing models (Williams, 2006).

What you measure is what you get

This paper challenges the idea that quantitative assessment is a sufficient condition for providing a full understanding of performance in service provision. A quantitative evaluation does not adequately describe “what you get”. This paper argues that, due to the precise nature of service delivery, performance measurement can never be complete and consistent at the same time. A performance measurement system is considered consistent only when it does not contain internal contradictions, and it is considered complete only when it provides a full description of service delivery.1

The incompleteness theorem has implications for the epistemology of the measurement of service quality. Using insights from hermeneutic philosophy, this paper argues that a qualitative approach is required to enhance the performance measurement of services and to develop a complete understanding of service provision.

The Nature of Service Delivery

The marketing of services is, due to their specific properties, managed differently to the marketing of products. The literature on the marketing of services identifies a range of differences between products and services, each with specific implications for marketing strategy (Lovelock, Patterson, & Walker, 2007). Services are distinguished from products by (among other aspects) the heterogeneity of the quality of the service, the intangibility of the offering, and the inseparability of their production and consumption (Rushton & Carson, 1985). Based on the distinguishing differences between products and services, a range of classification schemes has been proposed to distinguish services from each other (Lovelock, 1983; Schmenner, 1986; Shostack, 1977). These classification schemes can be used to develop targeted marketing strategies and inform performance measurement systems for service delivery.

Inseparability

The provision of services is inseparable from the personal interaction between the customer and the service provider because the customer always interacts directly with the service provider’s staff or with its technology. Due to this human element, the provision of services has fewer options for standardisation than the manufacturing of products. Service providers that seek to meet the needs of their customers are required to modify their service provision to match the specific preferences of each customer. For example, hotel guests interact with hotel staff and with hotel equipment. The quality of their experience is dependent not only on the performance of the staff and the quality of the equipment but also on factors inherent to the customers themselves, such as prior knowledge, past experiences and mood at the time of the service encounter. Services are consumed while they are being produced, making them inseparable from the-the customer’s idiosyncrasies, which cannot be controlled by the service provider.

Intangibility

The intangible elements of service provision are dynamic, subjective and ephemeral—and difficult to quantify. Virtually all products and services contain both tangible and intangible characteristics. What differs are the relative proportions of each feature. Products tend to have more tangible features, whereas products with a service as their core tend to be more intangible (Shostack, 1977). Services typically have both tangible elements, such as the meal in a restaurant, the bed in a hotel or the magazines at the barber shop, and intangible elements, such as the taste of the food, the comfort of the bed or the friendliness of the barber. Few services are without any tangible elements, and few products are without a service component, for example, warranties, information provision and so on. Intangible factors associated with service quality are more important than the tangible ones because they tend to play a statistically more significant role in determining the overall customer satisfaction and the quality of the consumer’s experience (Kara, Lonial, Tarim, & Zaim, 2005).

Heterogeneity

The customer’s presence in the process of service provision, along with the level of intangibility inherent in service delivery, leads to heterogeneity in the quality of the services. The quality of services has a natural variability that is impossible to control fully. A service is always subject to some variation in performance, and developing realistic standards of performance is tough (Rushton & Carson, 1985). The problem of heterogeneity has spawned a lot of research as well as attempts to develop quantitative methods for measuring performance in services.

Service Quality

In production processes, a ‘widget’ is an abstract unit of production: an imaginary object of which the physical properties can be precisely specified. In the supply of products, customer satisfaction exists in matching the physical specifications of the product with the expectations of the customer. In the provision of services, the abstracted unit of production is not a widget but a so-called moment of truth. Every moment of truth is a human interaction: an opportunity for the expectations of the customer to be either confirmed or contradicted, resulting in either satisfaction or dissatisfaction (Grönroos, 1990). To allow for the specific issues in services provision, a body of theory around the construct of service quality has been developed over the past three decades.

Figure 1: Grönroos (1990) model of service quality.

Figure 1: Grönroos (1990) model of service quality.

Service quality is a model of how customers perceive the provision of services. It is “the consumer’s judgement about an entity’s overall excellence or superiority” (Parasuraman, Zeithaml, & Berry, 1988). Understanding how services are evaluated enables organisations to influence these evaluations and push them in the desired direction (Grönroos, 1990). Service quality is not only of academic importance but also of managerial concern, especially regarding its relationship to costs, profitability, customer satisfaction, customer retention and positive word of mouth. Service quality is widely considered as a driver of corporate marketing and financial performance (Buttle, 1996; Carrillat, Jaramillo, & Mulki, 2007). It is an elusive and abstract construct that is difficult to define and measure, although several methods have been proposed over the past three decades (Cronin & Taylor, 1994; Parasuraman et al., 1988).

In the model of service quality developed by Finnish marketing scholar Christian Grönroos (1990), service quality (total perceived quality) is the difference between the expected and the experienced quality (Figure 1). Both sides of the service-quality equation contain subjective parameters that cannot be expressed in numbers. Expected quality is influenced by the ‘image’ of the service provider, word of mouth, the individual customer’s needs, and how the customer has interpreted the service provider’s market communication. The experienced quality conditions by the technical quality of the actual service and the manner in which the service is provided. Experienced quality is also mediated by the ‘image’ of the service provider. The Grönroos model, illustrated below, shows the fundamentally subjective nature of service quality. The subjectivity of service quality lies not only in the experiential aspects of its provision but also in its foundational concepts, which are subjective at their very core.

To understand the concept of service quality, an understanding of its foundational concepts is required. These concepts—value, quality and satisfaction—are often treated as interchangeable, but each one needs to be understood separately in the management and measurement of services (Rust & Oliver, 1994). Putting satisfaction aside for the moment, it can be shown that value and quality are inherently complex concepts, which leaves the construct of service quality on uncertain foundations.

Value

Value, which is usually defined as the difference between benefits and cost, is strictly related to service quality. Definitions of the value construct can be divided into three paradigms: the subjectivist, objectivist and interactionalist schools (Holbrook, 1994; Walker & Johnson, 2009):

  1. In the subjectivist view, value is dependent upon the human experience. This interpretation of value is an ex-post consideration, as no value is considered to exist before the service encounter. In this view, value is entirely dependent upon and about the human experience. The value of a service is therefore determined by the perceived benefits and the perceived cost.

  2. The objectivist view holds that value is independent of human experience. Value is ex-ante to the service encounter and is inherent within the service encounter itself. In Objectivism, value is an intrinsic property of the object rather than of the subject.

  3. The interactionalist school of thought on value combines both views in that value is created in the interaction between the object and the subject. Value is, in this view, a personal and situational preference that characterises a consumer’s experience of a service.

Quality

A second key determinant of service quality is the construct of quality. Quality is easily visualised but notoriously hard to define or measure. Garvin (1988, p. 217) distinguishes between three definitions of quality, each based on different philosophical presuppositions:

  1. The transcendent view of quality is synonymous with innate excellence and is considered to be a sign of uncompromising standards and high achievement. In this paradigm, ‘quality’ service is one with a halo of excellence, for example, first-class air travel.

  2. The product-based approach sees quality as a precise and measurable variable. In this view, differences in quality reflect differences in measurable aspects of a product’s attributes. In this paradigm, the quality of, for example, air travel would be determined by the physical length of the leg-room or the width of the seats.

  3. User-based definitions start with the premise that quality can only be defined from the perspective of the customer. In these definitions, quality is equated with customer satisfaction, and quality is dependent upon the expectations of the customer. Using the example of air travel again, both a first-class traveller and an economy-class traveller could rate their experience as high quality.

Service Quality Measurement

These theoretical considerations on the nature of services and service quality influence the methodologies for measuring service delivery performance. Several formal methods have been proposed to measure the service quality construct, with the SERVQUAL (Service Quality) model (Parasuraman et al., 1988) as the most ubiquitous. Several alternatives have been proposed over the years, such as SERVPERF (Service Performance) (Cronin & Taylor, 1992) and industry-specific models based on these two previous methods, e.g. HEdPERF in higher education (Abdullah, 2005) and LibQUAL+ in libraries (Cook, Heath, & Thompson, 2002). The SERVQUAL instrument, which builds on the disconfirmation paradigm used in the Grönroos model discussed above, compares customer expectations with their perceptions of service delivery. The instrument is based on five salient dimensions of service quality, which are often identified with the acronym RATER: reliability, assurance, tangibles, empathy and responsiveness.

Figure 2: Determinants of Service Quality (Parasuraman et al. 1988).

Figure 2: Determinants of Service Quality (Parasuraman et al. 1988).

Reliability is the ability of the firm to perform the service dependably and accurately. The assurance dimension relates to the knowledge and courtesy of employees and their ability to convey trust and confidence. The tangibles dimension relates to the appearance of the physical facilities, equipment, and the personal and communication materials. The empathy dimension relates to the provision of individualised care and attention to customers. Lastly, responsiveness is the willingness to help the customer and to provide prompt service. This brief discussion of the most popular performance management methodology for service quality shows that the existing methods for the measurement of service quality are necessarily based on subjective dimensions such as assurance, tangibles and empathy. Reliability and responsiveness do allow for some quantitative measurement, but the SERVQUAL model focuses on the subjective interpretation of these by the customer.

Incompleteness Theorem for Service Quality Measurement

From the above discussion, it can be concluded that service quality is a subjective construct based on foundational concepts with wide-ranging interpretations. Therefore, when performance in services is measured through the construct of service quality, it is based on subjective dimensions. As service quality can be described from different perspectives, each based on a different paradigm, the most important question to be asked is whether there is an overarching principle that could render all other views obsolete. Should service quality be considered from an objectivist perspective, focusing on the aspects that can be adequately measured, such as responsiveness? Or should service quality be seen as an entirely subjective evaluation by customers?

Scottish philosopher Walter Gallie (1956) showed that it is impossible to conclusively define key concepts such as ‘social justice,’ ‘democracy’ or ‘art’. Gallie calls these ‘essentially contested concepts’ because they can never be conclusively defined. Although it is rational to discuss justifications for holding one interpretation over another, there will never be a final agreement on an overarching interpretation of essentially contested concepts. Clarification of such concepts does not involve the examination of causal relations, as in the case of scientific concepts; rather, consideration is given to how the concept has been used throughout its history. Gallie provides five conditions to test whether a concept is essentially contested. Two of the three foundational concepts of service quality, ‘quality’ and ‘value’, are essentially contested:

  1. Essentially contested concepts are evaluative and deliver value-judgements. This situation is the case for the concepts of ‘quality’ and ‘value’ because the concepts are used to evaluate different service offerings.

  2. Essentially contested concepts denote comprehensively evaluated entities that have an internally complex character. Both dimensions of service quality can be considered internally complex, given the numerous possible interpretations of their definition.

  3. The evaluation of ‘quality’ and ‘value’ are attributed to the internally complex entity as a whole.

  4. Descriptions of the concepts of ‘quality’ and ‘value’ are modified through the history of their use, without convergence towards a single understanding.

  5. Each school of thought recognises the fact that its use of ‘quality’ and ‘value’ is contested by other schools of thought, both aggressively and defensively.

The fact that the foundational concepts of service quality are essentially contested concepts, combined with the inherent subjectivity of services, leads to the incompleteness theorem of performance measurement in service delivery:

Any effectively generated system capable of measuring service quality cannot be both complete and consistent.

A performance measurement system is complete when it provides a full description of the provision of services, and it is consistent when it does not contain internal contradictions. Consistent measurement systems based on only one paradigm will be incomplete because each paradigm can be contested and replaced with another paradigm. A measurement system based on an objective understanding of quality excludes the customer’s perception, while a system based on an interactionist approach fails to recognise intrinsic quality. Conversely, complete systems for the measurement of service quality will contain competing paradigms, which leads to inconsistencies. Using both an objective and subjective understanding of quality leads to an inconsistent understanding of what it is the system measures.

Tap Water Case Study

The customer's perception of the level of service goes beyond measurement.

The customer’s perception of the level of service goes beyond measurement.

The measurement of performance in reticulated potable water, i.e. tap water, is dominated by engineering-based measurement of the physical parameters (Berg, 2010). Although these measurements can be physically verified, such systems are not complete because objective systems of performance measurement do not consider the subjective experience of customers. For example, the most salient parameter of any potable water system is water quality, the measurement of which involves the detection of contaminants. Although the water might be considered sufficiently purified from the objective perspective of an engineer, the customer’s experience of water quality mostly centres on the aesthetics and taste of the water. Methodologies have been developed to capture these subjective variables in an ostensibly objective manner, but the perception of customers is almost impossible to capture fully. The taste of water is dependent upon, among other things, the material of the container in which it is carried (Krishna & Morrin, 2008). Objective measurement systems for performance benchmarking of tap water services are thus substantially incomplete, because external aspects, such as the type of container, cannot be measured nor controlled by the service provider.

Objective performance measurement systems are enhanced by adding additional indicators that purport to measure the intangible aspects of service provision. Adding indicators usually leads to a plethora of unrelated performance measures, a trend that is currently occurring in the Australian state of Victoria concerning a proposal for a measuring system for tap water services (Essential Services Commission, 2012). In an attempt to achieve completeness, the proposal has introduced a raft of new performance measures and contains mutually exclusive parameters, which leads to inconsistencies in the system (Prevos, 2012).

The Epistemology of Service Quality

Too often we measure everything and understand nothing

This paper started by quoting Jack Welch and his view on measuring business performance. In the same book, Welch (2001) also provides some moderation to his insistence on measurement: “Too often we measure everything and understand nothing”. This intuition is supported by the incompleteness theorem of performance measurement in service delivery, which shows that objective quantitative systems are not able to describe the fullness of the service experience, but systems that come close to completeness are necessarily inconsistent.

For a complete understanding of the relationship between the customer and the service provider, both an in-depth qualitative understanding and a detailed quantitative determination are required.

Ontologically, service quality is not a thing in itself that can be independently analysed. Although the dimensions of service quality (RATER) are available for interrogation, the fullness of the service encounter cannot be fully grasped in quantitative methods. Reliance on quantitative measurement comes at a high price because it reduces the fullness of reality to statistics and loses the stories of the people that the research is actually about (Smith, Harré, & Van Langenhove, 1995).

Essentially contested phenomena such as service quality cannot be grasped by relying on quantitative methods alone. To develop a complete understanding of the service interactions, a performance-measurement system needs to be enhanced with qualitative assessments. According to German philosopher Wilhelm Dilthey, a qualitative approach assists in the Verstehen (understanding) of social phenomena, while naturalistic quantitative methods are essential to Erklären (explaining) phenomena (Astley, 1985). To be able to comprehend service provision fully, a manager must go beyond the facts and figures and use empathic observation to increase their understanding of the process of service delivery. Understanding cannot be taught but needs to be cultivated by, for example, experiencing the service first-hand or by performing casual observations of the interactions between customers and staff.

Conclusion

Measuring performance is an essential aspect of managing the delivery of services. Many organisations follow the Welshian axiom, “What you measure is what you get”. Quantitative measurement of service provision is a necessary condition for the complete understanding of services. It is undeniable that analysis is essential in managing service delivery. However, following the incompleteness theorem, although quantitative analysis is a necessary condition, it is not a sufficient condition for holistic performance measurement.

The message of this paper seems to be pessimistic towards the usefulness of performance measurement systems in service provision. However, the inherent incompleteness of service-delivery measurement does not render such systems useless, as facts and figures are required to explain issues in the service-provision processes; however, to understand the process, managers must take the responsibility of venturing beyond the data and dashboard displays.

References

Abdullah, F. (2005). HEdPERF versus SERVPERF: The quest for an ideal measuring instrument of service quality in higher education sector. Quality Assurance in Education, 13(4), 305–328. doi:10.1108/09684880510626584.

Astley, W. G. (1985). Administrative science as socially constructed truth. Administrative Science Quarterly, 30(4), 497–513.

Berg, S. V. (2010). Water Utility Benchmarking: Measurement, Methodologies and Performance Incentives. London: IWA Publishing.

Buttle, F. (1996). SERVQUAL: Review, critique, research agenda. European Journal of Marketing, 30(1), 8–32.

Carrillat, F. A., Jaramillo, F., & Mulki, J. P. (2007). The validity of the SERVQUAL and SERVPERF scales: A meta-analytic view of 17 years of research across five continents. International Journal of Service Industry Management, 18(5), 472–490. doi:10.1108/09564230710826250.

Cook, C., Heath, F., & Thompson, B. (2002). LibQUAL+ One instrument in the new measures toolbox. Journal of Library Administration, 35(4), 41–46. doi:10.1300/J111v35n04_09.

Cronin, J. J., & Taylor, S. A. (1992). Measuring service quality: A reexamination and extension. The Journal of Marketing, 56(3), 55–68.

Cronin, J. J., & Taylor, S. A. (1994). SERVPERF versus SERVQUAL. Reconciling performance-based and perception-minus-expectations measurement of service quality. Journal of Marketing, 58(1), 125–131.

Essential Services Commission (2012, April). Review of Water Performance Report Indicators – Staff Discussion Paper. Retrieved from http://www.esc.vic.gov.au/.

Gallie, W. B. (1956). Essentially contested concepts. Proceedings of the Aristotelian Society (Vol. 56, pp. 167–198).

Garvin, D. A. (1988). Managing Quality: The Strategic and Competitive Edge. New York: Free Press.

Grönroos, C. (1990). Service management and marketing: Managing the moments of truth in service competition. Lexington, Mass: Lexington Books.

Holbrook, M. B. (1994). The nature of customer value. In R. T. Rust & R. L. Oliver (Eds.), Service Quality: New Directions in Theory and Practice (pp. 21–71). Thousand Oaks, London & New Delhi: Sage Publications.

Kara, A., Lonial, S., Tarim, M., & Zaim, S. (2005). A paradox of service quality in Turkey: The seemingly contradictory relative importance of tangible and intangible determinants of service quality. European Business Review, 17(1), 5–20. doi:10.1108/09555340510576230.

Krishna, A., & Morrin, M. (2008). Does touch affect taste? The perceptual transfer of product container haptic cues. Journal of Consumer Research, 34(6), 807–818.

Lovelock, C. H. (1983). Classifying services to gain strategic marketing insights. Journal of Marketing, 47, 9–20.

Lovelock, C. H., Patterson, P. G., & Walker, R. H. (2007). Services Marketing: An Asia-Pacific and Australian perspective (4th ed.). Frenchs Forest NSW: Pearson Education Australia.

Parasuraman, A., Zeithaml, V., & Berry, L. L. (1988). SERVQUAL: A multiple-item scale for measuring consumer perceptions of service quality. Journal of Retailing, 64(1), 12–40.

Prevos, P. (2012). The Invisible Water Corporation. Presented at the VicWater Conference, Melbourne.

Rushton, A. M., & Carson, D. J. (1985). The marketing of services: Managing the intangibles. European Journal of Marketing, 19(3), 19–40. doi:10.1108/EUM0000000004748.

Rust, R. T., & Oliver, R. L. (Eds.). (1994). Service Quality: New Directions in Theory and Practice. Thousand Oaks, London & New Delhi: Sage Publications.

Schmenner, R. W. (1986). How can service businesses survive and prosper? Sloan Management Review, 27(3), 21–32.

Shostack, G. L. (1977). Breaking free from product marketing. Journal of Marketing, 41(2), 73.

Smith, J., Harré, R., & Van Langenhove, L. (Eds.). (1995). Rethinking methods in psychology. London; Thousand Oaks CA: Sage Publications.

Walker, R. H., & Johnson, L. W. (2009). Signalling intrinsic service quality and value via accreditation and certification. Managing Service Quality, 19(1), 85–105.

Welch, J. (2001). Straight from the gut. Warmer Books.

Williams, M. (2006). Measuring performance in a military organisation: The Royal New Zealand Navy. Presented at the World Congress for Total Quality Management, Wellington.


  1. This paper is based on a post on my management blog, The Lucid Manager

Leave a Reply

Your email address will not be published. Required fields are marked *