A framework for socially meaningful evaluations of technology based social innovations

Abstract

The social innovation space that tries to marry the pursuit of profits and social impact is at interesting cross-roads in developing countries like India. Among other things, the euphoria and subsequent deflation around microfinance, points to the need for a continuous critical examination of not only how well a sector, or organizations within it, are doing on different operational criteria, but also socially locating the evaluative criteria. This implies asking questions of the relevance of metrics being used to evaluate the social problem being targeted, the manner in which the problem is being defined and implications for the relationship between the state and citizens being impacted by private social interventions. To this end we suggest and test a framework adapted from Fischer (Fischer 1995) that makes a useful distinction between technical verification (performance on indicators relevant to the innovation) and societal validation (relevance of indicators to societal problem being addressed, nature of democratic society promoted etc.) as different levels of evaluation. While most impact reporting standards restrict their scope to technical verification, it is the level of societal validation where most conflicts and controversies lie. We develop on Fischer’s framework in the paper, elaborating on what the different levels of evaluations imply for social enterprises. We elucidate various aspects of the framework with ample examples, drawn from organizations that have garnered attention for their potential to create social impact via the use of technology. Although the scope of the framework is not limited to them, we focus on technology based social innovations in the paper. We use the framework to both organise the discourse evaluating the impact of the One Laptop per Child project (OLPC), which aims to equip every child in the world with a laptop of his/her own. The purpose of our case study is not to evaluate the OLPC itself, but instead organise the discourse around the impact of OLPC. Therefore, the case study serves to illustrate how the evaluation framework can be used, the manner in which the levels are interrelated, questions that emerge in using the framework and the importance of deliberating on the normative.