(How) are Outliers Used for Theory Building in Management and Organization Science? The Link Between Rigor and Impact in Deviant Case Analysis

Theory has been called the “currency” of management and organization science (Corley and Gioa, 2011: 12), and article impact has long been used to evaluate the “strength” of this currency, i.e. the effects of one scientist’s work on another (Cole and Cole, 1972; Judge, Cable, Colbert, and Rynes, 2007). There are myriad ways to build theory (with no less than three special issues in Academy of Management Review focusing on the topic, see 1989 issue 4, 1999 issue 4, and 2011 issue 1). This proposal focuses on one, specific theory-building device: outliers, in particular those with large model residuals, i.e., “deviant cases”. These present us with a valuable opportunity to improve model correspondence with empirical realities in organization and management science, i.e. help us in theory building. Our preliminary work so far suggests that deviant cases as a theory building device have been underutilized so far: few empirical field studies mention outliers, let alone analyze them rigorously. This is in stark contrast to other disciplines, where outlier analysis constitutes an impactful theory-building device in areas as diverse as biology, comparative politics, health care, law, and criminology. The purpose of the present proposal, then, is to examine if and how outliers are currently used for theory building in management and organization science field studies (qualitative as well as quantitative), as well as why they are not used. We link the methodological rigor with which deviant cases are analyzed with their impact, defined as summed counts of references in the scientific literature to a cited research article. To this end, we aim to (1) rigorously and reliably identify those outliers with the greatest theoretical potential (phase 1), (2) assess the methodological sophistication of outlier-analysis strategies used in exploiting this theoretical potential (phase 2), and (3) explore reasons, motivations, and constraints experienced by authors who report outliers in the final write up, but then forego their theoretical potential by not analyzing them further (phase 3). The potential contribution in the academic community is twofold: rehabilitating outlying observations as serviceable levers for theory-building in management, as well as improving the methodological sophistication in analyzing outliers. Beyond the immediate academic community, the research bears substantial implications for policy makers, funding agencies, and other stakeholders aiming at making most of invested funds and fostering rigorous research practice. The idea for this project emerged when collecting the data for the SNSF grant 100018_134523 (2012 January-2014 January) and the Docmobility grant P1TIP1_158602 (2015 February-2016 February). The former grant enabled us to further develop the initial characterization of methodological rigor in small-N research initially proposed by Gibbert et al. 2008 in Strategic Management Journal. During the course of grant 100018_134523, we identified various articles which did not fit neatly into the coding approach we used, since they discussed cases which deviated from theoretical expectations (so-called deviant cases, or outliers). The subsequent Docmobility grant meanwhile helped us to bridge the qualitative and quantitative worlds in that we explored transparency in both qualitative and quantitative articles as a prerequisite for assessing an article’s rigor. During this study, we noticed that few if any papers mentioned outliers, let alone analyzed them in management and organization science, both in qualitative and quantitative field studies. This discovery provided the impetus for a more systematic engagement with outliers and as such constitutes the basis and rationale for the present proposal.