Development of user-satisfaction-based knowledge management performance measurement system

Abstract

In knowledge-based economy, organizations face critical issues of survival and competence. By implementing knowledge management, organizations can increase the capability of managing and utilizing their knowledge, and ultimately achieve superior performance. Despite active commitment toward better knowledge management, organizations still wonder how well knowledge management works in practice. In order to justify the effectiveness of knowledge management, organizations have to evaluate it by knowledge management performance measurement methods. In the arena of performance measurement, various knowledge management performance measurement methods have been developed. Some of the methods focus on organizations’ financial and intellectual capital outcomes. However, the financial measure is not truly to report the results of organizations’ knowledge management efforts since knowledge is intangible and knowledge management projects may sometimes not generate immediate return on investment; the definitions of intellectual capital are discordant and its measurements are too company-specific and difficult for benchmarking. Other performance measurement methods focus on organizations’ overall performances by underlining the importance of a mix of quantitative and qualitative measures. They fail to demonstrate direct impact of knowledge management on an organization’s performance due to extremely large number of exogenous factors affecting the organizational performance, such as competition or monopoly in the marketplace. To address these problems, there is a need of a better approach for measuring knowledge management in the organization. Knowledge management involves internal changes in operational practices. The knowledge users (K-users) are the final arbiters of knowledge management activities. Only when the K-users are satisfied with implemented knowledge management strategies, they will voluntarily participate in creating and sharing knowledge in their organizations. Thus, user satisfaction with knowledge management strategy and system is recognized as the effective indicator for direct measurement of knowledge management performance. However, the existing knowledge management performance measurement methods overlook this issue. Therefore, this research proposes a new and innovative approach for performance measurement of knowledge management for this perspective, named user-satisfaction-based approach. The user-satisfaction-based approach provides the concept for evaluating how good the overall integration and consistency of knowledge management enablers and knowledge management results with alignment of goal to satisfy K-user needs and expectations. The user-satisfaction-based performance measurement is developed in respect of Knowledge Management Enablers (KME) and Knowledge Management Results (KMR). In addition, a new knowledge management performance measurement system, named User-satisfaction-based Knowledge Management Performance Measurement System (USBS) is developed for common uses by organizations in various industries. The aim of this research is to propose the USBS to tie knowledge management criteria into a common set of metrics that can be made for evaluating knowledge management performances in order to assist organizations to pinpoint its strengths and eliminate the gaps, as well as guide organizations to focus on improvement opportunities and drive continuous learning in knowledge management. In order to achieve the aim, three objectives are developed by means of IDEF0, Analytic Hierarchy Process and Evidential Reasoning : (i) developing the user-satisfaction-based knowledge management performance measurement model - identify user-satisfaction-based core values of knowledge management; - identify user-satisfaction-based critical success factors of knowledge management; and - identify user-satisfaction-based knowledge management processes (ii) developing a roadmap for implementation of the USBS (iii) applying and validating the USBS The result of objective 1 indicates that a user-satisfaction-based knowledge management performance measurement model embodied with KME and KMR derived from seven core values, eight critical success factors and five-phase knowledge management process is developed through literature review and expert judgments with the aid of IDEF0. Eight Assessment Categories and nineteen Assessment Factors are identified as KME, while five Assessment Categories are termed as KMR. The KME covers the knowledge management activities of an organization, what it does and how it does it. The KMR refers to what an organization has achieved in the past, is currently achieving and plans to achieve in the future. KMR are caused by KME, which in turn, KME are affected and improved using feedback from KMR achieved. This cause and effect logic gives USBS more dynamic and enhanced continuous learning. Moreover, evaluation is guided by a standardized scoring system by means of three evaluation dimensions, namely Approach, Deployment and Result. The Approach and Deployment are used when assessing KME, while Result is used for assessing KMR. The USBS enables organizations to discern clearly its strengths and areas in which improvement can be conducted. Following this process of evaluation, improvement plans are launched and monitored for progress. Organizations carry out this cycle of “evaluating and taking actions” repeatedly so that they can achieve continuous learning in the journey to excellence. The result of objective 2 indicates that a roadmap comprising seven phases, namely (i) structuring analytical hierarchy for knowledge management performance measurement; (ii) prioritizing weights from expert interviews; (iii) developing knowledge management performance measurement scheme; (iv) designing Assessment Items for performance measurement; (v) designing scoring guidelines, (vi) designing Adoption Status; and (vii) designing assessment software tool for performance measurement, is developed with the aid of expert interviews, Analytic Hierarchy Process, Expect Choice software, Evidential Reasoning and Intelligent Decision System software to provide standard guidance for implementing USBS. Just as benefits for knowledge management in helping business organizations support innovation and efficiency, the benefits of knowledge management in primary schools are analogous that enhance school leaders to improve problem-solving capabilities and abilities for school improvement and support teachers to improve classroom teaching. Owing to Education Reform in Hong Kong, there is a need of pedagogical knowledge in using information technology in education. Although school leaders recognize that effective management of knowledge can improve teaching and school administration, practices of knowledge management implemented in primary schools are lagging behind the expectations. Therefore, USBS is applied for case studies in the knowledge management environment in primary schools in Hong Kong in objective 3. The weights of Assessment Categories and Assessment Factors are generated collectively from experts of primary schools. The point values are determined from the normalized weights obtained from the Analytic Hierarchy Process analysis and taken together to calculate the overall Adoption Status with the aid of Intelligent Decision System. The findings reveal that Primary School A and B are recognized as “Achievers”, while Primary School C is recognized as “Improver”. In addition, the positive applicability of USBS is validated by three primary schools and the research findings have been confirmed by two independent experts. To conclude, organizations using the USBS can be benefited in (i) providing a highly structured, fact based approach to identify and assess an organization’s strengths and areas for improvements in knowledge management; (ii) integrating the various improvement initiatives of knowledge management into normal operations; (iii) facilitating benchmarking with knowledge management of other organizations by using a set of comprehensive Assessment Criteria and allowing for sharing best practices for accelerating learning; and (iv) delivering ever-improving values to K-users.