Measuring the Performance of Information Systems: A Functional Scorecard

Abstract

This study develops an instrument that may be used as an information systems (IS) functional scorecard (ISFS). It is based on a theoretical input-output model of the IS function's role in supporting business process effectiveness and organizational performance. The research model consists of three system output dimensions--systems performance, information effectiveness, and service performance. The "updated paradigm" for instrument development was followed to develop and validate the ISFS instrument. Construct validation of the instrument was conducted using responses from 346 systems users in 149 organizations by a combination of exploratory factor analysis and structural equation modeling using LISREL. The process resulted in an instrument that measures 18 unidimensional factors within the three ISFS dimensions. Moreover, a sample of 120 matched-paired responses of separate CIO and user responses was used for nomological validation. The results showed that the ISFS measure reflected by the instrument was positively related to improvements in business processes effectiveness and organizational performance. Consequently, the instrument may be used for assessing IS performance, for guiding information technology investment and sourcing decisions, and as a basis for further research and instrument development.