10.1.1.167.1159

METHODOLOGICAL FIT IN MANAGEMENT FIELD RESEARCH AMY C. EDMONDSON Harvard Business School STACY E. MCMANUS Monitor Executive Development Methodological fit, an implicitly valued attribute of high-quality field research in organizations, has received little attention in the management literature. Fit refers to internal consistency among elements of a research project—research question, prior work, research design, and theoretical contribution. We introduce a contingency framework that relates prior work to the design of a research project, paying particular attention to the question of when to mix qualitative and quantitative data in a single research paper. We discuss implications of the framework for educating new field researchers. To advance management theory, a growing number of scholars are engaging in field re- search, studying real people, real problems, and real organizations. Although the potential rele- vance of field research is motivating, the re- search journey can be messy and inefficient, fraught with logistical hurdles and unexpected events. Researchers manage complex relation- ships with sites, cope with constraints on sam- ple selection and timing of data collection, and often confront mid-project changes to planned research designs. With these additional chal- lenges, the logic of a research design and how it supports the development of a specific theoreti- cal contribution can be obscured or altered along the way in field research. Compared to experimental studies, analyses of published data sets, or computer simulations, achieving fit between the type of data collected in and the theoretical contribution of a given field research project is a dynamic and challenging process. This article introduces a framework for as- sessing and promoting methodological fit as an overarching criterion for ensuring quality field research. We define methodological fit as inter- nal consistency among elements of a research project (see Table 1 for four key elements of field research). Although articles based on field re- search in leading academic journals usually ex- hibit a high degree of methodological fit, guide- lines for ensuring it are not readily available. Beyond the observation that qualitative data are appropriate for studying phenomena that are not well understood (e.g., Barley, 1990; Bouchard, 1976; Eisenhardt, 1989a), the relationship be- tween types of theoretical contributions and types of field research has received little ex- plicit attention. In particular, the conditions un- der which hybrid methods that mix qualitative and quantitative data are most helpful in field research—a central focus of this paper—are not widely recognized. We define field research in management as systematic studies that rely on the collection of original data—qualitative or quantitative—in real organizations. The ideas in this paper are not intended to generalize to all types of man- agement research but, rather, to help guide the design and development of research projects that centrally involve collecting data in field sites. We offer a framework that relates the stage of prior theory to research questions, type of data collected and analyzed, and theoretical contributions—the elements shown in Table 1. We thank David Ager, Jim Detert, Robin Ely, Richard Hackman, Connie Hadley, Bertrand Moingeon, Wendy Smith, students in four years of the Design of Field Research Methods course at Harvard, seminar participants at the Uni- versity of Texas McCombs School, the MIT Organization Studies group, and the Kurt Lewin Institute in Amsterdam for valuable feedback in the development of these ideas. We are particularly grateful to Terrence Mitchell and the AMR reviewers for suggestions that improved the paper im- mensely. Harvard Business School Division of Research pro- vided the funding for this project. � Academy of Management Review 2007, Vol. 32, No. 4, 1155–1179. 1155 Copyright of the Academy of Management, all rights reserved. Contents may not be copied, emailed, posted to a listserv, or otherwise transmitted without the copyright holder’s express written permission. Users may print, download, or email articles for individual use only. In well-integrated field research the key ele- ments are congruent and mutually reinforcing. The framework we present is unlikely to call for changes in how accomplished field research- ers go about their work. Indeed, experienced researchers regularly implement the alignment we describe. However, new organizational re- searchers, or even accomplished experimental- ists or modelers who are new to field research, should benefit from an explicit discussion of the mutually reinforcing relationships that promote methodological fit. The primary aim of this article, thus, is to provide guidelines for helping new field re- searchers develop and hone their ability to align theory and methods in field research. Because a key aspect of this is the ability to anticipate and detect problems that emerge when fit is low, our discussion explores and categorizes such problems. A second aim is to suggest that methodological fit in field re- search is created through an iterative learning process that requires a mindset in which feed- back, rethinking, and revising are embraced as valued activities, and to discuss the impli- cations of this for educating new field re- searchers. To begin, in the next section we situate our efforts in the broader methodolog- ical literature and describe the sources that inform our ideas. BACKGROUND Prior Work on Methodological Fit The notion of methodological fit has deep roots in organizational research (e.g., Bouchard, 1976; Campbell, Daft, & Hulin, 1982; Lee, Mitch- ell, & Sablynski, 1999; McGrath, 1964). Years ago, McGrath (1964) noted that the state of prior knowledge is a key determinant of appropriate research methodology. Pointing to a full spec- trum of research settings, ranging from field re- search to experimental simulations, laboratory experiments, and computer simulations, he pre- sented field studies as appropriate for explor- atory endeavors to stimulate new theoretical ideas and for cross-validation to assess whether an established theory holds up in the real world. The other, non-field-based research settings were presented as appropriate for advancing theory. Understandably, given the era, McGrath did not dig deeply into the full range of methods that have since been used within field research alone. Subsequently, Bouchard, focusing on how to implement research techniques such as inter- TABLE 1 Four Key Elements of a Field Research Project Element Description Research question ● Focuses a study ● Narrows the topic area to a meaningful, manageable size ● Addresses issues of theoretical and practical significance ● Points toward a viable research project—that is, the question can be answered Prior work ● The state of the literature ● Existing theoretical and empirical research papers that pertain to the topic of the current study ● An aid in identifying unanswered questions, unexplored areas, relevant constructs, and areas of low agreement Research design ● Type of data to be collected ● Data collection tools and procedures ● Type of analysis planned ● Finding/selection of sites for collecting data Contribution to literature ● The theory developed as an outcome of the study ● New ideas that contest conventional wisdom, challenge prior assumptions, integrate prior streams of research to produce a new model, or refine understanding of a phenomenon ● Any practical insights drawn from the findings that may be suggested by the researcher 1156 OctoberAcademy of Management Review views, questionnaires, and observation, noted, “The key to good research lies not in choosing the right method, but rather in asking the right question and picking the most powerful method for answering that particular question” (1976: 402). Others have issued cautions against as- suming the unilateral rightness of a method— wielding a hammer and treating everything as nails (e.g., Campbell et al., 1982). Yet all re- searchers are vulnerable to preferring those hammers that we have learned to use well. Thus, we benefit from reminders that not all tools are appropriate for all situations. At the same time, exactly how to determine the right method for a given research question—particu- larly in the field—has not been as well speci- fied. More recently, Lee et al. (1999: 163) tackled the challenges of research in “natural settings” to explicate strategies for effective qualitative or- ganizational and vocational research. Using ex- emplars, these authors showed that qualitative data are useful for theory generation, elabora- tion, and even testing, in an effort to “inspire [other researchers] to seek opportunities to ex- pand their thinking and research” and to help them “learn from this larger and collective ex- perience and avoid misdirection” (1999: 161). In advocating the benefits of qualitative work for organizational researchers, these authors pro- vide a helpful foundation for the present paper. We build on this work by distinguishing among purely qualitative, purely quantitative, and hy- brid designs, as well as by including a fuller range of field research methods in a single framework. The categories we develop allow a more fine-grained analysis of field research op- tions than offered previously. A recent body of work debates the appropri- ateness of combining qualitative and quanti- tative methods within a single research project. Issues addressed in this debate in- clude whether qualitative and quantitative methods investigate the same phenomena, are philosophically consistent, and are paradigms that can reasonably be integrated within a study (e.g., Greene, Caracelli, & Graham, 1989; Morgan & Smircich, 1980; Sale, Lohfeld, & Bra- zil, 2002; Yauch & Steudel, 2003). Consistent with Yauch and Steudel (2003), who provide a brief review of the current thinking on this topic, we propose that the two methods can be combined successfully in cases where the goal is to increase validity of new measures through triangulation1 and/or to generate greater understanding of the mechanisms un- derlying quantitative results in at least par- tially new territory. This paper complements prior work on hybrid methods by addressing how the state of current theory and literature influences not only when hybrid research strategies are appropriate but also when other methodological decisions are appropriate and how different elements of research projects fit together to form coherent wholes. Sources for Understanding Fit in Field Research Several sources have informed the ideas pre- sented in this paper. A long-standing interest in teaching field research methods fueled exten- sive note taking, reflection, and iterative model building over the past decade. In this reflective process we drew first from the many high- quality papers reporting on field research pub- lished in prominent journals; we use a few of these articles as exemplars to highlight and ex- plain our framework. Second, we drew from our own experiences conducting field research, complete with missteps, feedback, and exten- sive refinement. Third, the first author’s experi- ence reviewing dozens of manuscripts reporting on field research submitted to academic jour- nals provided additional insight into both the presence and absence of methodological fit.2 Unlike reading polished published articles, re- viewing offers the advantage of being able to observe part of the research journey. Moreover, a reviewer’s reward is the opportunity to see how other anonymous reviewers have evalu- ated the same manuscript—constituting an in- formal index of agreement among expert judges. Papers rejected or returned for extensive revi- sion because a poor match among prior work, 1 Triangulation is a process by which the same phenom- enon is assessed with different methods to determine whether convergence across methods exists. See Jick (1979) for a thoughtful discussion. 2 These reviewing experiences were important inputs into the framework in this paper; however, the confidentiality of the review process precluded using these cases as exam- ples. To illustrate poor fit and attempts to improve fit later in the article, we resorted to drawing on our second primary source—our own field research projects. 2007 1157Edmondson and McManus research questions, and methods helped inform our framework; agreement among expert re- viewers strengthens our confidence in these ideas. This agreement is not explained by explicit instruction. A glance at the current Academy of Management Journal and Administrative Sci- ence Quarterly checklists for reviewers reveals an emphasis on the quality of the individual elements of a submission—for example, “tech- nical adequacy”—without a formal criterion for evaluating fit among elements. Yet researchers may employ a particular method exceptionally well, without it being an effective approach to studying the stated research question. This hap- pens, in part, because field research is often spurred by unexpected data collection opportu- nities. Responding to requests from contacts at companies, researchers may collect data driven by company interests but not well matched to initial research questions. For example, surveys may be distributed that help the site but that have limited connection to the researcher’s the- oretical goals. Similarly, interview data from a consulting project may be reanalyzed for re- search, focusing on an area of theory not well suited to purely qualitative research. The oppor- tunistic aspect of field research is not in itself a weakness but may increase the chances of poor methodological fit when data collected for one reason are used without careful thought for an- other. The experience of reviewing also highlights that a lack of methodological fit is easier to discern in others’ field research than in one’s own. This motivated us to develop a formal framework to help researchers uncover areas of poor fit in their own field research earlier in the research journey, without waiting for external review. Drawing on the above sources, we inductively derived the framework presented in this paper, revising it along the way, driven by each other and by colleagues and reviewers both close and distant. In exploring methodological fit, we are particularly focused on how the state of current theory shapes other elements of a field research project. For clarity of illustration and compari- sons across diverse methods, we limit the sub- stantive topic of the research projects discussed to one area—organizational work teams. In the next section we show that producing methodological fit depends on the state of rele- vant theory at the time the research is designed and executed. We use the state of prior theory as the starting point in achieving methodological fit in field research because it serves as a given, reasonably fixed context in which new research is developed: it is the one element over which the researcher has no control (i.e., the state of extant theoretical development cannot be mod- ified to fit the current research project). A CONTINGENCY FRAMEWORK FOR MANAGEMENT FIELD RESEARCH The State of Prior Theory We suggest that theory in management re- search falls along a continuum, from mature to nascent.Mature theory presents well-developed constructs and models that have been studied over time with increasing precision by a variety of scholars, resulting in a body of work consist- ing of points of broad agreement that represent cumulative knowledge gained. Nascent theory, in contrast, proposes tentative answers to novel questions of how and why, often merely sug- gesting new connections among phenomena. In- termediate theory, positioned between mature and nascent, presents provisional explanations of phenomena, often introducing a new con- struct and proposing relationships between it and established constructs. Although the re- search questions may allow the development of testable hypotheses, similar to mature theory research, one or more of the constructs involved is often still tentative, similar to nascent theory research. This continuum is perhaps best understood as a social construction that allows the develop- ment of archetypes. Consequently, it is not al- ways easy to determine the extent of theory de- velopment informing a potential research question.3 We propose a continuum rather than 3 We thank an anonymous reviewer for pointing this out and Terry Mitchell for suggesting howwemight address this issue. To gain insight into raters’ agreement on this catego- rization approach, we prepared short descriptions of four- teen research questions that each began with a brief sum- mary of the state of prior work on the topic. The fourteen cases included the articles described in this paper, along with a few additional field research studies. We then asked four organizational researchers to categorize them accord- ing to definitions of the three stages of theory we provided. The average overall agreement with our intended classifi- 1158 OctoberAcademy of Management Review clear stages to acknowledge that the categories we suggest are not obvious or inviolable and to recognize the potential for debate on the status of prior work related to a given research ques- tion. In short, our aim is to help field researchers think about methodological fit in a more ex- plicit, systematic way, using exemplars from the organizational literature to illustrate how the state of current theory informs methodological decisions. Developing Sensible Connections to Prior Work In a given field study, the four elements in Table 1 should be influenced by the stage of development of the current literature at the time of the research. In general, the less known about a specific topic, the more open-ended the re- search questions, requiring methods that allow data collected in the field to strongly shape the researcher’s developing understanding of the phenomenon (e.g., Barley, 1990). In contrast, when a topic of interest has been studied exten- sively, researchers can use prior literature to identify critical independent, dependent, and control variables and to explain general mech- anisms underlying the phenomenon. Leverag- ing prior work allows a new study to address issues that refine the field’s knowledge, such as identifying moderators or mediators that affect a documented causal relationship. Finally, when theory is in an intermediate stage of de- velopment—by nature a period of transition—a new study can test hypotheses and simulta- neously allow openness to unexpected insights from qualitative data. Broadly, patterns of fit among research components can be summa- rized as in Table 2. We begin our more detailed exploration of fit between theory and method with a discussion of mature theory, because it conforms to tradi- tional models of research methodology and so serves as a conceptual base with which to com- pare the other two categories. By drawing pri- marily on the topic of work teams, we demon- strate that the state of prior knowledge for specific research questions within one broad topic can vary from mature to nascent. Mature Theory Research Mature theory encompasses precise models, supported by extensive research on a set of re- lated questions in varied settings. Maturity stimulates research that leads to further refine- ments within a growing body of interrelated the- ories. The research is often elegant, complex, and logically rigorous, addressing issues that other researchers would agree from the outset are worthy of study. Research questions tend to focus on elaborating, clarifying, or challenging specific aspects of existing theories. A re- searcher might, for example, test a theory in a new setting, identify or clarify the boundaries of a theory,