School Work

A multidimensional performance model for consolidating Balanced Scorecards

A multidimensional performance model for consolidating Balanced Scorecards
of 11
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
  3rd International Workshop on Software and Performance, Rome, Italy, July 24-27, 2002 1 A Multidimensional Performance Model for Consolidating Balanced Scorecards A LAIN A BRAN  L UIGI B UGLIONE    École de Technologie Supérieure (ETS) Université du Québec à Montréal 1180 Notre-Dame Ouest,  Montréal, Québec, Canada H3C 1K3   Software Engineering Management Research Laboratory Université du Québec à Montréal C.P. 8888, Succ. Centre-Ville  Montréal, Québec, Canada H3C 3P8 E-mail: Tel: +1 (514) 396.8632 Fax: +1 (514) 396.8684   E-mail: Tel: (39) 338.95.46.917 Fax: (39) 06 41.536.385 Index 1. Introduction  – 1.1 Context – 1.2 Generic performance management and measurement frameworks  – 1.2.1 Balanced Scorecard (BSC) – 1.2.2. Intangible Asset Monitor (IAM) – 1.2.3. Skandia Navigator – 1.3. The Balanced Scorecard tailored for the ICT field – 1.4. An issue: determining the overall value in a BSC – 1.5. Requirements   and evaluation criteria  – 2. Candidate consolidation techniques – 3. QEST: a multidimensional model for Software Performance Measurement  – 3.1. The 3-dimensional  basic model – 3.2. The n-dimensional extension    – 4. Integration of QEST and BSC  – 4.1. First option: BSC(QEST nD) – 4.2. Second option: QEST nD(BSC) – 4.3. Integration of the options: BSC(QEST nD) + QEST nD(BSC)    – 5. Conclusions – References  Abstract  A Balanced Scorecard (BSC) presents the quantitative goals selected from multiple perspectives for implementing the organizational strategy and vision. However, in most current BSC frameworks, including those developed for the  Information and Communication Technology (ICT) field, each perspective is handled separately. None of these  perspectives is integrated automatically into a consolidated view, and so these frameworks do not tackle, either in relative or in absolute terms, the contribution of each goal to the whole BSC. Here, this issue is highlighted, candidate consolidation techniques are reviewed and the preferred technique, the QEST model, is selected; more specifically, three options are presented for incorporating the QEST model into a BSC framework. Key words – Software Performance Measurement; Balanced Scorecard; QEST model; Indicators; Strategy.  3rd International Workshop on Software and Performance, Rome, Italy, July 24-27, 2002 2 1. Introduction 1.1. Context Evaluating the IT function remains a challenge: well-known financial measures such as “return on investment” (ROI), “internal rate of return” (IRR), “net present value” (NPV) and “payback” time (PB) have been demonstrated to be inadequate, both in explaining IT investment decisions and in assessing them. In assessing IT investments, it is crucial to understand how organizational and strategic goals are achieved, and how IT investments contribute to these goals. Since the financial dimension alone is not sufficient and hides the relations (e.g. cause and effect) among processes, an improvement step has been suggested which would introduce additional dimensions (or perspectives) of analysis. Performance Management is defined by the United States Office of Personnel Government as “ the systematic process by which an agency (organization) involves its employees (resources), […], in improving organizational effectiveness in the accomplishment of agency (organization) mission and goals ” [OPM – ]. The following five phases are defined for managing organizational effectiveness: • Planning  work and setting expectations   • Continually monitoring  performance   • Developing  the capacity to perform   • Periodically rating  performance in a summary fashion   • Rewarding  good performance   1.2. Generic performance management and measurement frameworks Over the years, several frameworks have been developed to address the management of organizational assets, both tangible and intangible. The three main ones are the Balanced Scorecard (BSC), the Intangible Asset Monitor (IAM) and the Skandia Navigator. 1.2.1. B ALANCED S CORECARD (BSC) [KAPL96] In 1993, Robert S. Kaplan of the Harvard School of Business, and consultant David Norton developed the Balanced Scorecard (BSC), an evolution of the concepts in the Tableau de Bord which emerged in France at the turn of the 20 th  century [EPST97]. The aim of the Tableau had been to translate each company's unitary vision and mission into a set of objectives, through the identification of Key Success Factors and Key Performance Indicators. Kaplan and Norton defined the Balanced Scorecard (BSC) as a multidimensional framework for describing, implementing and managing strategy at all levels of an enterprise by linking, through a logical structure, objectives, initiatives and measures to an organization’s strategy. The resulting scorecard provides the details of an enterprise view of an organization’s overall  performance: it complements the financial measures with other key performance indicators around customer  perspectives and internal business processes, and around organizational growth, learning and innovation. It must be noted that the BSC is not a static list of measures, but rather a logical framework for implementing and aligning complex programs of change, and, indeed, for managing strategy-focused organizations. In summary, a scorecard is to  be used to facilitate the translation of strategy into action. The four basic perspectives of the srcinal BSC are: Financial, Customer, Internal Processes, Learning & Growth (Figure 1). ü Financial: this perspective typically relates to profitability – it is measured, for instance, by the ROI (Return On Investment), ROCE (Return On Capital Employed) and EVA (Economic Value Added). ü Customer: this perspective includes several core or generic measures of successful outcomes from the company strategy, like, for instance, customer satisfaction, customer retention, and market share in targeted segments. ü Internal processes: this perspective focuses on the internal processes that will have the greatest impact on customer satisfaction and on achieving an organization's financial objectives. ü Learning and growth: this perspective identifies the infrastructure the organization has to build and manage to create long-term growth and improvement through people, systems and organizational procedures. The execution of this strategy is then monitored through an internal performance measurement framework with a set of goals, drivers and indicators (both lag and lead types) grouped into each of the four perspectives. The other key objective of a BSC is to tell the story of the organization’s strategy. Three criteria help in determining whether or not this objective has been achieved: • Cause-and-effect relationship : every measure selected should be part of a cause-and-effect relationship (causal relationship chain) which represents the strategy.   • Performance drivers : the drivers of performance (lead indicators) tend to be unique since they reflect what is different about the strategy of a company.   • Links to financial indicators : the various strategic goals (such as quality, customer satisfaction and innovation) must also be translated into measures which are ultimately linked to financial measures.    3rd International Workshop on Software and Performance, Rome, Italy, July 24-27, 2002 3 Figure 1 – Balanced Scorecard: srcinal  perspectives [KAPL96]  1.2.2. I NTANGIBLE A SSET M ONITOR (IAM) [SVEI97][SVEI98] Developed by Karl Sveiby, this framework classifies the intangible part of a balance sheet into a so-called  family of three thematic areas, each of them composed from four categories (growth, renewal, efficiency, stability/risk): ü Internal structure: consists of a wide range of concepts, models, computers and administrative systems. ü External structure: relationships with customers and suppliers, brand names, trademarks and image. ü Individual competence: people's capacity to act in different situations, including skill, education, experience, values and social skills. The basic tenet of the IAM framework is that people are the organization's only profit generators, since every human action is converted into both tangible and intangible knowledge structures, in other words a " knowledge perspective ". 1.2.3. S KANDIA N AVIGATOR [EDVI97][SKAN98]   Developed by Edvinsson and Malone, this framework combines the two previous framework structures, applying the BSC presentation format to the family of the three IAM   categorizations and using them under the  Intellectual Capital   label in a supplement to the Skandia Annual Report. Also, the Navigator uses the same four perspectives ("focus") as the BSC 1 , with a central and pervasive "Human Focus". 1.3. The Balanced Scorecard tailored for the ICT field A few attempts to adapt the BSC for software intensive organizations (SIOs 2 ) have been reported, such as: • the Balanced IT Scorecard (BITS) proposed by the European Software Institute (ESI) [IBAÑ98] [REO99a] [REO99b]: provides a new version of the four srcinal perspectives (financial, customer, internal process, infrastructure and innovation) and adds a fifth one, the People perspective. • the BSC of Advanced Information Services Inc. [FERG99]: considers the "employee" element as a distinct  perspective, thereby expanding the analysis to five perspectives (financial, customer, employee, internal business  process, learning and growth). The five distinct perspectives derived from the srcinal scorecard and adapted to the ICT field are: • Financial Perspective : How do our software processes and software process improvement initiatives add value to the organization? • Customer Perspective : How do we know that our customers (internal and external) are delighted with our  product? • Process Perspective : Are our software development processes performing at sufficiently high levels to meet customer expectations? • People Perspective : Do our people have the necessary skills to perform their jobs and are they happy doing so? • Infrastructure & Innovation Perspective : Are process improvement, technology and organizational infrastructure issues being addressed with a view to implementing a sustainable improvement program? Table 1 summarizes how both tangible and intangible concepts are being addressed in each of the generic frameworks  presented, as well as for those tailored to ICT organizations. 1   The fourth perspective is called "  Renewal and Development  " instead of "  Learning and Growth ". 2   SIOs are organizations whose main objective is software development and sales, departments or organizations which develop software as an integral  part of their end-products, and organizations which develop software for internal use to achieve better business results or whose software department can be described as an independent organizational unit (European Software Institute definition, 1997).    3rd International Workshop on Software and Performance, Rome, Italy, July 24-27, 2002 4 IAM BSC N AVIGATOR   ESI BITS AIS BSC T ANGIBLE A SSETS   Tangible Net Book Value Financial Perspective Shareholders' Equity Financial Perspective Financial Internal Structure Internal Process Perspective Organizational Capital Process Perspective Internal Process External Structure Customer Perspective Customer Capital Customer Perspective Customer Infrastructure & Innovation Perspective Learning & Growth   I NTANGIBLE A SSETS   Individual Competence Learning & Growth Perspective Human Capital People Employee Table 1 - Comparison of   Performance and Measurement  Frameworks 1.4. An issue: determining the overall value in a BSC Of these generic frameworks, Kaplan and Norton's BSC framework has the largest market penetration and tackles  performance at several levels, from the organizational level to the small business unit, and to the individual level. However, while providing a vast array of individual quantitative indicators, it does not provide for consolidated  performance values, either for the individual perspectives or for their consolidation. Although in a BSC the logical (i.e. causal) links are described textually, there is no provision in a BSC for a quantitative consolidation for each perspective to handle separately its various numbers of indicators representing the various elements of each business strategy. Even though it is structured logically to represent the series of causal links between the goals and drivers, a BSC framework does not provide the support necessary to represent quantitatively how much each perspective contributes, either in relative or in absolute terms. In practice, the consolidation has to be carried out intuitively by the users of the BSC; this consolidation is, of course, subject to significant variability of assessment and interpretation depending on the expertise and background of the users. Current BSC frameworks function as “dashboards”, in that they are made up of sets of individual indicators (like the various gauges on an automobile dashboard) which provide the elements necessary for monitoring the deployment of a (business) strategy. Knowing the business links (causal relationships) across the various indicators, the business executives must then, each time, figure out a consolidated assessment of current organizational performance. Since BSC frameworks do not include techniques for consolidating individual perspectives, this has to be done subjectively, without the benefit of a formal quantitative representation. Therefore, consolidated assessments which can be communicated unambiguously and consistently to those with a knowledge of the consolidation rules are not possible. Using a BSC, a manager can typically gain access to detailed information about the goal, driver and indicator (GDI) elements for each BSC perspective and about the follow-up actions to take when results do not meet established itemized thresholds. But, as with any automobile dashboard or control panel,  an overall value of that perspective is not defined. It is therefore challenging to answer concerns at aggregate levels above that of the details included in the BSC, whether for internal or external purposes. This issue can be restated in the following way: How can the consolidated management view of performance, and the data collection view , be expressed? How, for instance, can a question on   process performance  be tackled, a question which, when translated into BSC terms corresponds to: What is the overall value expressed by a set of indicators chosen in your organization for the Internal Process perspective?  This paper addresses the consolidation issue and illustrates the way in which a software performance measurement model, namely the QEST model, can be used for a BSC for information and communication technology (ICT) organizations (referred to as “ICT BSC” in this paper). 1.5. Requirements and evaluation criteria To tackle this issue, we have determined that the following mandatory high-level capabilities have to be added to the BSC framework and integrated into a performance measurement system (PMS): • A model providing a multiperspective approach [FINK96]; • A model which expresses these multiple viewpoints in a single, consolidated value; • Dimensional principle: The « dimensional principle » refers to the direct relationship between the number of elements to be represented and the number of dimensions used for their representation. A proper representation of an entity must align the number of elements to be represented and the number of dimensions used to represent them. If this principle is not respected, the danger is a loss of information. A well-known visual example is the way in which the ancient Egyptians painted human subjects. They used a 2D representational style instead of a three dimensional (3D) format. This misalignment – translated from paintings to operational data management – can cause similar misconceptions when data is analyzed for the decision-making process. • A model with a formal and consistent way to express the outcomes. In section 2, candidate consolidation techniques are identified and analyzed against the above evaluation criteria. In section 3, the QEST multidimensional performance measurement model is presented, with an emphasis on its consolidation features; both the 3D srcinal version and its nD extension of the QEST model are presented. In section 4,  3rd International Workshop on Software and Performance, Rome, Italy, July 24-27, 2002 5 three options are presented for combining the QEST model in a BSC framework, thereby consolidating the ICT BSC final outcomes into a single overall value. The summary is presented in section 5. 2. Candidate consolidation techniques Some PMSs proposed in the literature are reviewed and analyzed against these consolidation requirements. • The Performance Pyramid [LYNC95] approach, by Lynch and Cross, uses a pyramid-shaped “map” for understanding and defining the relevant objectives and measures for each level of the business organization (four levels are identified: business, core business processes, department and teamwork, individuals). • The Performance Prism [NEEL], developed jointly by Accenture and the Cranfield School of Management (UK), allows two groups of stakeholders (investors and customers) and up to five groups of stakeholders (employees, suppliers, intermediaries, regulators and communities) to be handled in the performance management measurement scheme through a 3D shape (a  prism  with 5 facets). This approach could be positioned midway between a value chain like in the EFQM Excellence Model [EFQM99] and the BSC. For the purpose stated in this study, the following constraints have been identified: the calculation matrix does not use the geometry to derive the  performance results, and it is limited to handling five perspectives. • The General Framework for Performance Measurement  [ROUS97], developed by Rouse, Putterill and Ryan at the University of Auckland (NZ), uses DEA (Data Envelopment Analysis) models to represent performance graphically with a 3D pyramid having a square base where each of the four faces corresponds to the srcinal four BSC perspectives. Measures are linked to the critical success factors (CSF) and to the underlying processes or cost drivers. Table 2 presents a high-level comparison of these three performance measurement approaches and the BSC framework, including the following evaluation criteria: the number of perspectives taken into account, whether or not the approaches have a fixed structure or a dynamic one allowing for a variable number of perspectives, the representational format used and the provision of a performance value for each perspective and a consolidated value for all perspectives. Perspectives handled Model Type Dimensions handled Dimensional Principle respected Performance value for each perspective Consolidated performance value Reference: Balanced Scorecard  – BSC (  D .  Kaplan & D.Norton ) 1. Financial 2. Customer 3. Internal process 4. Learning & growth Closed N/A (Map)  N/A No No Candidate Performance Measurement Systems (PMS)   • Performance Pyramid  (  R .  Lynch & K.Cross ) 1. Business 2. Core business processes 3. Departments and workteams) 4. Individuals Closed N/A (Map)  N/A N/A N/A • Performance Prism (  A. Neely & C.Adams )   1. Stakeholder satisfaction 2. Strategies 3. Processes 4. Capabilities 5. Stakeholder contribution Closed G3  N No Yes • General Framework for Performance Measurement  (  P  .  Rouse, M.Putterill and S.Ryan ) Can be used with any 4-perspective set (typically the ones in the Kaplan & Norton’s BSC) Closed N/A (Map)  N/A Yes Yes Table 2 - Comparison of     Performance Management-Measurement Models   From Table 2, it can be observed that •  None of these three PSMs provides for the derivation and presentation of a consolidated performance value; • A geometrical representation is used only to present the distinct results visually. • The details are not consolidated into a single outcome •  No geometrical formula is associated with the particular shape used for deriving performance indicators, and the dimensional principle  is not respected.   3. QEST: a multidimensional model for Software Performance Measurement In addition to the above PMSs proposed for the generic measurement of performance, a measurement model srcinally designed for measuring the quality and performance of ICT projects, the QEST model, has been proposed by Buglione and Abran. It is to be noted that this model is generic and not restricted to the quality domain: it could be generalized to the generic measurement of performance. We will see that this QEST model does indeed meet all the requirements and evaluation criteria for a consolidated representation of performance. 3.1. The 3-dimensional basic model The initial QEST (Quality factor + Economic, Social and Technical dimensions) model proposed a geometrical representation of quality performance with the number of dimensions corresponding to the number of viewpoints considered; in addition, this number of dimensions can vary, depending on the organization's choice of the number of
Similar documents
View more...
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks