The Leadership Quarterly 18 (2007) 435 – 446 www.elsevier.com/locate/leaqua The typical leadership study: Assumptions, implications, and potential remedies Samuel T. Hunter a,⁎, Katrina E. Bedell-Avers b , Michael D. Mumford b b a 421 Moore Building, Penn State University, State College, PA 16802, USA The University of Oklahoma, Center for Applied Social Research2 Partner's Place3100 Monitor, Suite 100Norman, OK 73072, USA Abstract Since the turn of the century, the area of leadership has seen notable growth in the amount of research conducted. As such, it now seems appropriate to evaluate how most leadership research is conducted, considering in particular the assumptions that are made when conducting the typical leadership study. Specifically, we explored the assumptions made with regard to (a) subordinates, (b) leaders, (c) context, and (d) the processes involved in leadership. Consideration of these assumptions reveals a number of problems ranging from simple methodological issues to more substantive theory-based concerns. Potential remedies are presented, along with a consideration of the long-term impact associated with the typical leadership study approach. © 2007 Elsevier Inc. All rights reserved. Keywords: Leadership; Leaders; Subordinates; Research methods; Theory; Multi-level; Method bias The field of leadership has enjoyed a recent pique in interest — evident in the introduction of new publication outlets and the increased number of graduate training and leadership development programs (Bandelli, Rivas, & Ottinot, 2006; Hunt, 2005). Not surprisingly, since the turn of the century we have seen substantial growth in the number of leadership studies conducted. In fact, Hunt (2005) referred to the recent research increase as an “explosion of the leadership field” (p. 1). In light of this increase, it appears relevant to consider how such research is conducted. Specifically, we might now reflect on the assumptions made when conducting the typical leadership study and what these assumptions tell us about the developing paths of future leadership research. More importantly, consideration of these assumptions may help us develop alternative paths if, indeed, our current trajectory is viewed as problematic. The above being stated, it seems best to begin with a description of a typical or average leadership study. Although studies vary in numerous subtle ways, most leadership studies generally begin with the distribution of a self-report survey or questionnaire (Lowe & Gardner, 2000). These surveys are often in paper-and-pencil format, although more recently web-based surveys have been used with increasing frequency (Cycyota & Harrison, 2006; Tourangeau, 2004). The typical survey often involves the use of a pre-developed, behaviorally-based leadership assessment tool. These questionnaires tend to be given to individuals working in an organization and ask for a self-report assessment of their immediate supervisor's behavior. At times, these surveys are distributed to the manager or supervisor directly, asking ⁎ Corresponding author. Tel.: +1 814 865 0107. E-mail address: samhunter@psu.edu (S.T. Hunter). 1048-9843/$ - see front matter © 2007 Elsevier Inc. All rights reserved. doi:10.1016/j.leaqua.2007.07.001 436 S.T. Hunter et al. / The Leadership Quarterly 18 (2007) 435–446 the manager to provide self-report estimates of their own behavior (e.g., Rosete & Ciarocchi, 2005). Generally, the results of these surveys are then correlated with self-report outcomes of interest such as organizational performance or commitment — with final conclusions being drawn in turn. The literature is replete with examples of the above study description, though for illustrative purposes it appears useful to explore a few of them explicitly. In a study purported to be examining the effect of transformational leadership on followers' influence strategies, Krishnan (2003) distributed the multifaceted leadership questionnaire (MLQ; Bass & Avolio, 1990) and a measure of leader–member exchange (LMX) to 281 employees, asking them to rate the behavior of their immediate supervisor. In addition, employees were asked to provide self-report estimates of their personal influence strategies (e.g., friendliness). Based on the results of the study, the researcher concluded that “transformational leadership enhances the use of friendliness, while LMX enhances the use of reasoning and reduces the use of higher authority” (p. 70). In a second illustration, Rosete & Ciarocchi (2005) attempted to examine the relationship between emotional intelligence (EI) and transformational leadership by providing 41 managers with a measure of EI (Mayer, Salovey, & Caruso, 2002) and other personality variables. Additionally, the supervisors of the managers were asked to provide estimates of the managers' organizational performance. Through correlational analysis, the researchers concluded that “executives higher on EI are more likely to achieve business outcomes and be considered effective leaders by their subordinates and direct manager” (p. 396). A final example is provided by Lee (2004) who examined LMX and organizational commitment by providing 220 engineers and scientists with measures of LMX and the MLQ (Bass & Avolio, 1990). Organizational commitment was assessed using a self-report instrument and served as the study's single dependent variable. Using correlational and regression analysis, Lee (2004) concluded that “R&D organizations should endeavor to select and nurture transformational leadership qualities among leaders for potential increased performance and effectiveness of followers” (p. 669). Although more examples exist, the above should suffice to provide illustration of the typical or average leadership study. As may be inferred from these examples, the typical approach to investigating leadership is not without its flaws. More precisely, by conducting a study that focuses on subordinate perceptions and uses pre-developed measures and approaches to leadership, we are making a number of assumptions — assumptions that appear, in many cases, misguided. For example, the studies mentioned earlier required subordinates to report on the behavior of their supervisor and thus the assumption is made that they actually witnessed the leadership behaviors in question — an assumption that may not prove warranted. In light of examples such as these, we will explore the assumptions made when conducting a typical leadership study as well as consider the implications and problems arising from such assumptions. Because leadership involves at a minimum: a leader, a collection of subordinates, the exchanges among them (and other constituencies), and the context in which exchanges take place, the following framework will be used to guide our discussion: (a) subordinate-based assumptions, (b) leader-oriented assumptions, (c) assumptions regarding the context, and (d) assumptions associated with the leadership process itself. Finally, we conclude with a discussion of the potential impact of the above issues, considering the long-term implications of the typical approach to leadership. 1. Assumptions about subordinates As noted above, many studies of leadership ask subordinates to report on the perceived behaviors of their supervisor with study hypotheses most often following the rationale that leader behaviors impact subordinate actions or perceptions, ultimately resulting in some type of desired outcome. Fundamental to this rationale, however, is the assumption that the employees sampled innately need or desire leadership. Although some studies have addressed this issue directly via leadership theories such as leader substitutes (Kerr & Jermier, 1978), path-goal theory (House, 1971) and contingency theory (Fiedler, 1967), the simple act of asking about leadership behaviors implies that leadership behaviors are critically relevant to the situation at hand. Moreover, and potentially more problematic, by applying the same surveys across all types of subordinates (i.e., in differing studies), we are assuming that this need, or impact of perceived leader behaviors operates in the same fashion across varying samples and employment domains. Running counter to this assumption is work by de Vries, Roe & Tallieu (2002) who examined need for leadership as a moderator in 15 different predictor/criterion relationships. Although only a third of the interactions tested supported the moderator hypothesis, the results are strong enough to suggest that need for leadership may not operate in the same fashion across all conditions. Thus, it is possible that by failing to use relevant comparison or control groups we may be drawing inappropriate conclusions with regard to potential boundary conditions. S.T. Hunter et al. / The Leadership Quarterly 18 (2007) 435–446 437 Even if we were to concede, however, that all employees require leadership and that leadership impacts each of them equally – troublesome points, it should be noted – there are still several problematic assumptions made with regard to using subordinates as the source of information in leadership research. The first, as alluded to earlier, is that a subordinate has witnessed the leadership behavior(s) in question. In point of fact, recent research suggests that there are a number of leadership activities that are not likely witnessed by a subordinate. These activities may include meetings with other leaders, meetings with clients, and more cognitively-based actions such as planning (Mumford, Bedell-Avers, & Hunter, in press) or strategy development (e.g., Bourgeois, 1985; Hambrick & Fukutomi, 1991). Moreover, even when behaviors in question are witnessed by subordinates, it is assumed that the ratings provided are always accurate estimates of leader actions — an assumption called into question when examining work by Wexley & Youtz (1985) who found that subordinates rated their supervisor less accurately when they held certain beliefs about them (e.g., that the supervisor was not trustworthy or altruistic). Finally, a critical prerequisite to using survey research is that participants are not familiar with the scale or items being used in that familiarity, or lack of naivity, may result in potential biasing effects (Martell & Evans, 2005; Morrow, Eastman & McElroy, 1991). Despite this need for participant naivety, it appears that managers themselves are often being used as “subordinate samples” (e.g., Rosete & Ciarocchi, 2005). It seems reasonable to suggest that managers have likely had either formalized or organizational-sponsored leadership training, which would call into question their naivety as participants. The point also holds for many “true” subordinate samples in that it is likely that a substantial number of employees sampled have also taken leadership courses or been exposed to the very dimensions they are being asked to report on. 1.1. Remedies for assumptions about subordinates The first and most straightforward remedy for dealing with potentially false assumptions is to stop relying so heavily on subordinates as a primary source of information. Granted, it is the existence of a subordinate that, in a sense, allows a leader to actually be a leader vis-à-vis their relationship with that individual. However, there appears to be a number of relevant leader activities (e.g., strategic planning, environmental scanning) that are lost from primary research investigation because they are simply not witnessed by subordinates. Thus, a balance must be struck among sources, with avenues of information deriving from multiple and varied perspectives. On a related note, we must also refrain from over-relying on convenience samples to explore our research questions. Though pertinent information may be gleaned from survey-based research and large sample sizes are, indeed, often desirable — the lack of relevant comparison or control groups limits the generalizability of the obtained results. Even worse, it appears assumed that obtained relationships operate in the same fashion across all types of subordinates. If, instead of making this assumption, we obtain carefully chosen comparison samples or employ the use of control groups in conjunction with convenience samples, far more may be gathered about the generalizability of results as well as determining and assessing potential boundary conditions. A final set of recommendations deal broadly with those assumptions being made. Namely, it may be necessary to empirically examine and consider specific assumptions such as participant naivety, accuracy of subordinate ratings, and need for leadership rather than simply assuming them to be true. By directly assessing such potential biases, they may be controlled for or even dismissed altogether if, indeed, justified. Initial attempts at directly assessing these variables have been particularly fruitful in obtaining a clearer picture of the role subordinates play in leadership (e.g., de Vries et al., 2002). Thus, it is critical that we operationalize and explore these assumptions directly, addressing them accordingly rather than sweeping them under the proverbial research “rug”. 2. Assumptions about leaders With even a cursory investigation of typical leadership studies it is clear that there is no shortage of assumptions made with regard to leaders themselves. The first, and somewhat most obvious, is the assumption that all managers are leaders. By sampling subordinates and asking them about their supervisors, it is implied that those individuals being rated are, in actuality, serving as “leaders” in the organization. Although beyond the scope of this discussion to provide a definition of leadership distinct from a manager (cf. Bedian & Hunt, 2006), this basic assumption – one that may not be met – is noteworthy. A second set of assumptions is tied to the heroic conceptualization of leadership. With the exception of a few (e.g., Conger, 1998; Hogan & Hogan, 2001; Hogan & Kaiser, 2005) most leadership studies have explored only the positive 438 S.T. Hunter et al. / The Leadership Quarterly 18 (2007) 435–446 relationships and outcomes of leader actions, ignoring those behaviors that may be harmful to subordinates and organizations. This occurs even in light of a history replete with examples of “bad” or “evil” leaders. On a related note, there appears to be an assumption that only positive leader actions are critical to the success of an organization. More precisely, there has been little investigation of leader errors and how those errors impact organizational success or failure, thereby implying that such actions are not noteworthy phenomena. Recent high-profile leadership cases such as those occurring during hurricane Katrina would, however, seem to highlight the importance and need for such investigation. A final collection of assumptions are bound to the tools used to assess the behaviors and actions of leaders themselves. An assumption fundamental to the use of surveys and questionnaires is that those instruments are psychometrically sound with clear and agreed upon factor structures. Lack of agreement precludes consistency in instrument interpretation across studies and samples. It seems, however, that general agreement with regard to psychometric soundness and factor structure has not been reached. Although some evidence exists for the psychometric soundness of common leadership instruments (e.g., Antonakis, Avolio, & Sivasubramaniam, 2003), there is at least equal if not more evidence suggesting relative disagreement regarding their appropriateness (Carless, 1998; Heinitz, Liepmann & Felfe, 2005; Rafferty & Griffin, 2004; Tejeda, Scandura, & Pillai, 2001). Moreover, researchers have recently begun to question approaches used in assessing the psychometric quality of leadership measures, suggesting that the field of leadership lags severely behind other research areas with regard to psychometric assessment methodology (Scherbaum, Finlinson, Barden & Tamanini, 2006). This, again, is somewhat beyond the scope of this discussion, yet is noteworthy given the frequency and widespread use of common leadership assessment tools. If we were to concede that common leadership instruments have acceptable psychometric properties, an assumption potentially met, we must next consider an assumption bound to their widespread implementation. More precisely, by frequently using the same leadership instruments, it is assumed that such instruments capture the most critical and essential leadership behaviors. As noted in the discussion of subordinate assumptions, there appear to be several critical leader behaviors that would not be captured with such instruments (e.g., strategic planning, meetings with customers). Moreover, the overuse of pre-developed instruments reveals the assumption that all typologies of leadership are captured within them. Recent evidence suggests the existence of alternative pathways to leadership that would not have been discovered using typical leadership measures or with the implementation of a typical leadership study (Mumford, 2006). 2.1. Remedies for assumptions about leaders To begin to address the problems arising from potentially false assumptions when conducting a typical leadership study, researchers must first be more explicit in their operationalizations and justification for what a leader is and why, precisely, a given sample represents “leaders”. For example, in many cases managers may in actuality be acting as leaders. However, without addressing the operationalization issue directly, we may be drawing false conclusions about leadership and leadership behaviors. Through explicit operationalization, however, observed outcomes may be compared and contrasted with studies considering leadership in differing ways — with general and specific boundary conditions being more clearly determined. A second remedy for issues deriving from the application of the typical leadership study involves a greater and more direct investigation of negative leadership behaviors. Granted, some research has been done exploring the “dark” side of leadership (e.g., Hogan & Hogan, 2001), yet even these studies note that more work needs to be done in these areas. We must also re-shape our thinking about leadership, specifically with regard to leader errors. Although positive leader actions clearly have an impact on organizational performance, it is also clear that mistakes can result in notable and important outcomes as well. Leaders are not infallible and must be viewed as imperfect for the full picture of leadership to be gained. Finally, with regard to leadership measures and their use, it is clear that we must be careful about the widespread implementation of pre-developed measures employed in the typical leadership study. It should be noted that it is not the widespread implementation of these measures, per se, that is wholly problematic. Rather, it is the singular implementation of these measures that is potentially dangerous. By failing to pair current leadership tools with additional approaches and considerations of leadership, it is implied that all relevant behaviors are assessed in commonly used measures. By pairing such tools or using alternative approaches altogether, however, a more complete view of leadership may be gleaned. Additionally, if we are to continue to place emphasis on the conceptualizations of leadership captured S.T. Hunter et al. / The Leadership Quarterly 18 (2007) 435–446 439 by the dominant leadership measures, it would be useful to perform more experiments (i.e., use fewer surveys) exploring these behaviors in a more direct and causal manner. 3. Assumptions about the leadership context Noticeably absent from the typical leadership study is a consideration of the context in which leader behaviors are occurring as well as the extraneous variables that may be operating within that context (Yukl, 2006). Put another way, the typical leadership study often fails to take into account the situation, either through the lack of examining potential moderators or by failing to measure and subsequently control for potentially biasing effects. The broad assumption made with this lack of moderator and boundary condition exploration, then, is that such variables are not particularly important to leadership research. Despite the fact that there exist prevalent and well-established theories of leadership that have specifically taken into account the situation, (e.g., Hersey Blanchard model, path-goal theory), the typical leadership study appears to ignore the context altogether, assuming that the situation is not of notable relevance. Not surprisingly, a number of studies call this assumption into question. For example, de Hoogh et al. found that environmental dynamism moderated the relationship between charismatic leadership and performance (de Hoogh, den Hartog, & Koopman, 2004) as well as personality and leadership behaviors (de Hoogh, den Hartog, & Koopman, 2005). Additionally, Elenkov & Manev (2005) found that sociocultural context moderated the relationship between leader behavior and organizational innovation. Finally, situational context has been shown to serve as a moderator in several meta-analyses (e.g., Fuller, Patterson, Hester, & Stringer, 1996; Van Engen & Willemsen, 2004) — again, illustrating the important role context may play in understanding leadership. Flowing from the assumption that the context does not play a substantive role in leadership is the assumption that method bias is not a concern when conducting the typical leadership study. This assumption is evident in the lack of control variables applied as well as the overemphasis on survey methodology — often making use of same-source variable assessment (e.g., using subordinates for both independent and dependent variable obtainment). Although there is some debate regarding the exact nature and magnitude method bias has on observed relationships (e.g., Cote & Buckley, 1987; Crampton & Wagner, 1994; Podsakoff, MacKenzie, Lee, & Podsakoff, 2003; Spector, 1994, 2006), most would contend that using the same source and same method for both independent and dependent variable assessment can result in an inflated relationship. These inflated relationships have been observed in several meta-analysis of charismatic and transformational leadership, where larger effects were observed for same-source relationships (de Groot, Kiker, & Cross, 2000; Fuller et al., 1996; Lowe, Kroeck, & Sivasubramaniam, 1996). It is also important to note that same-source method bias is not the only form of method bias. Discussed in detail by Podsakoff, MacKenzie, Lee, & Podsakoff (2003), other forms of method bias may include item characteristics effects, item context effects, and measurement context effects. Again, what is noteworthy with regard to the current discussion is that the typical or common leadership study fails to consider these potential biases, seemingly implying that they are not relevant to most leadership investigations. One particularly noteworthy biasing variable was recently examined by Brown & Keeping (2005). Exploring the role of liking on ratings of leaders, the researchers found that a sizable amount of variance (around 32% on average) could be accounted by how much a subordinate liked (i.e., had positive affect toward) their supervisor. Similar results with peer ratings were also found by Antonioni & Park (2001a,b) who discovered that ratings of consideration, communication and self-management as well as overall performance were substantially influenced by liking. These results are not wholly surprising when considering previous research on leader-member exchange, where liking has been shown to predict the perception of an exchange up to six months later (Liden, Wayne, & Stilwell, 1993). Despite this evidence and similar evidence found in other studies (e.g., Antonioni & Park, 2001b; Engle & Lord, 1997; Wayne & Ferris, 1990) most researchers do not account for liking either as a moderator or control variable — again, revealing the assumption that obtained relationships operate independent of how much subordinates like their leaders. 3.1. Remedies for assumptions about context The most straightforward remedy for dealing with the assumptions made regarding the leadership context is to simply consider and measure variables related to the context in which leadership is occurring. This would reduce the error in obtained data allowing for a better understanding of potential moderators and boundary conditions as well as enhancing the clarity of observed results. What must be done before control variables may be measured, however, is the appropriate identification of those controls most relevant to answering the research question(s) at hand. Thus, 440 S.T. Hunter et al. / The Leadership Quarterly 18 (2007) 435–446 substantial pre-planning and consideration is required before a study is conducted. This requisite time-consuming preparatory work is likely the reason so few studies have employed relevant controls and is an issue that will hamper future efforts. Fortunately, research has revealed a few notable controls to assist in the required preparatory design work. For example, it appears that liking, in particular, is a relevant biasing variable to consider when conducting leadership research. Others may include implicit leadership schemas, where leaders who fit – or do not fit – our stereotype for a “good” leader may be rated in biased fashion (e.g., Gioia & Sims, 1985; Phillips & Lord, 1986; Yukl, 2006). These variables, specifically, provide the researcher with direction for control variable investigation. The discovery of these and related biasing constructs, however, suggests that other biasing variables may also exist and thus it is critical that they too be explored. The evidence obtained by Brown & Keeping (2005), and others (e.g., Liden et al., 1993) suggests that such biasing effects are not trivial and may be driving a number of conclusions when the typical leadership study methods are employed (de Groot et al., 2000; de Hoogh et al., 2004; Fuller et al., 1996; Lowe et al., 1996). Once extraneous variables have been identified and measured, there are a number of ways to handle their examination. It is possible to statistically control for such variables, removing their influence from obtained results. This method, however, can be problematic if study-relevant variance is shared with the biasing variable (e.g., liking may actually serve as a legitimate motivator to subordinates). Other methods include variations on structural equation modeling or the multitrait multimethod (MTMM) approaches, where models are considered that account for the role and impact of biasing variables. It should also be noted that although it is desirable to explicitly measure biasing variables, there do exist more sophisticated methods for handling common method bias even if the biasing variable is not measured directly (e.g., Podsakoff et al., 2003). These and the aforementioned methods have been used with some success in a number of published studies (e.g., Carmeli & Schaubroek, 2006; Conger, Kanungo, & Menon, 2000), suggesting that it is possible to address the method bias issue in leadership research. Finally, the most direct and potentially best way to handle method bias is to avoid, or at least limit, the use of data collection methods that allow for the influence of potential biases. Avoiding same-source data collection, for example, can substantially limit possible method variance problems. Though such advice is generally well-known by most researchers, it appears that leadership research is particularly susceptible to these biasing issues, with many studies seemingly ignoring the problematic impact they may have on obtained results. Granted, at times self-report surveys are essential tools for gathering data obtainable only through the use of such methodology (Spector, 2006), however, the continued over-emphasis on the methods used in the typical leadership study may result in wide-spread, substantive problems in leadership research. 4. Assumptions about the leadership process Although leaders may at times be characterized by singular events, leadership is rarely, if ever, the result of a sole action or behavior. Rather, leadership is a process, a series of activities and exchanges engaged in over time and under varied circumstances. The typical distribution of a single survey to subordinates, however, provides researchers with only a snapshot of what is occurring as leaders interact with subordinates and other stakeholders. Lost, as a result, is information regarding leadership processes. More accurately stated, the cognitive and behavioral activities that occur as leadership takes place are, for the most part, ignored from research investigation when the typical leadership study is conducted. Because such variables are omitted, the assumption appears to be that the processes involved in leadership are unimportant or irrelevant, a criticism noted by a number of researchers (e.g., Hunt, 1999; Hunt & Ropo, 2003; Yukl, 2006). As noted earlier, the typical leadership study employs the use of pre-developed leadership measures that most often ask a subordinate to provide behavioral estimates about a leader. These measures typically operate under the assumption that the subordinate has a unique relationship with the supervisor, one that is independent of other subordinates, additional supervisors, customers or other organizational stakeholders. Researchers also typically do not explore team and group level process variables, reiterating the unique, independent leader/subordinate perspective taken when conducting the typical leadership study. Thus, it appears assumed that leadership is a wholly dyadic phenomenon — a point also criticized by a number of researchers (e.g., Hunt, 1999; Yukl, 1999a,b, 2006). This emphasis on dyads, or one-to-one relationships between leaders and subordinates, has created a notable gap in knowledge with regard to multi-level leadership investigation. A handful of researchers are attempting to fill this gap vis-à-vis special issues in major publication outlets (e.g., Dansereau, Yammarino, & Markham, 1995; Yammarino & S.T. Hunter et al. / The Leadership Quarterly 18 (2007) 435–446 441 Dansereau, 2006) as well as books dedicated to the topic (e.g., Dansereau & Yammarino, 1998) — yet are currently in the minority with regard to volume of research conducted. The research that has been done, however, suggests that multi-level investigation is critical to understanding leadership and absolutely must be explored in greater detail (e.g., Anit, 2003; Ostroff, Kinicki, & Clark, 2002; Yammarino, Dionne, & Chun, 2005). Another process-related assumption that must also be explored with greater rigor is that of causality. The methods applied in the typical leadership study lend themselves particularly well to correlational analyses or some variation thereof (e.g., regression). Though such analyses provide valuable information (Hatfield, Faunce, & Soares Job, 2006) their interpretation typically operates under the assumption that causality has been solidly established in leadership research — an assumption that clearly must be explored further. For example, early work by Greene (1975) examined the reciprocal relationship of leader and subordinate interactions — a dynamic relationship that is also fundamental to some versions of leader–member-exchange theory (e.g., Graen & Cashman, 1975; Graen & Scandura, 1987; Graen & Uhl-Bien, 1991). Moreover, researchers have called for more complex considerations of relationships rather than accepting the basic contention that leader behaviors simply cause subordinate outcomes (Hunt & Ropo, 2003; Marion & Uhl-Bien, 2001). In short, causality has not been fully established in leadership research and assuming it as such is likely to result in, at best, a narrowed conceptualization of leadership processes or worse, an inaccurate view of how leaders think and operate. Causality is not the only process variable that has been avoided with potentially damaging frequency. The emphasis on one-time administered surveys has also resulted in a narrowed exploration of time. The lack of longitudinal research also reveals an assumption similar to the causality assumption, yet one that is potentially even more problematic. Specifically, the lack of longitudinal exploration suggests the assumption that observed relationships are not time-contingent or timerelated. This has occurred despite the fact that several researchers have called for the direct consideration of time-relevant variables. For example, Atwater, Dionne, Avolio, Camobreco, & Lau (1999) suggested that time is essential to identifying leadership potential. Similarly, Fulmer & Vicere (1996) called for a consideration of time with regard to leadership development programs. Moreover, the results of longitudinal efforts reveal that changes over time are critical to understanding leadership. To illustrate, Day, Sin & Chen (2004) tracked role changes in leadership over six years revealing a non-linear trend in performance that would not have been obtained using a single, one-time assessment. Gathering data over a five-year period, Keller (2006) found that a number of leadership behaviors (e.g., transformational, initiating structure, and leader substitutes) predicted performance in R&D units. Keller's work is particularly noteworthy from a longitudinal perspective considering the lengthy time-frame required for new products and processes to be deemed useful and relevant to an organization (Mumford & Hunter, 2005). Although more examples exist, the above appears sufficient to demonstrate the critical need for longitudinal investigation and the frequent insufficiency of one-time administered surveys (Hunt & Ropo, 2003; Ployhartz, Holtz, & Bliese, 2002). A final assumption made regarding the leadership process is that the model tested in a given study is the best approach to answer the research question at hand. Put simply, there is a lack of alternative model testing occurring in leadership research that perpetuates similar considerations of observed relationships, thereby limiting the range of leadership conceptualizations explored. This issue is tied to both causality and longitudinal investigation, where a reconsideration of causality may produce models that better account for the observed relationships and increased longitudinal investigation may reveal the existence of more complete, if not wholly unique, better-fitting models. 4.1. Remedies for assumptions about leadership process The first remedy for addressing the problematic assumptions relating to the leadership process is to simply explore process variables with greater frequency and increased detail. Not surprisingly, this recommendation runs contradictory to the typical leadership study approach, which has largely ignored process variables. However, until we understand what occurs during leadership, we will be left with only a narrowed and incomplete understanding of leadership. The second remedy is similar to the first in that we must explore multi-level leadership more frequently and with greater emphasis. Leadership is an inherently multi-level phenomenon with relationships occurring between leaders and subordinates, leaders and teams, leaders and other organizational leaders, as well as leaders and leaders of other organizations. Moreover, as some research on multi-level relationships has revealed there may exist cross-level paradoxes, where what is necessary for performance at one level may run counter to what is necessary for performance at another level (Mumford & Hunter, 2005). Such findings would help researchers and practitioners who may be making potentially misleading recommendations based on results of one level of analysis (e.g., individual-level results) 442 S.T. Hunter et al. / The Leadership Quarterly 18 (2007) 435–446 when, in fact, they actually seek performance – and therefore a different set of recommendations – at a different level of analysis (e.g., organizational performance). Similarly, it is also critical that more longitudinal research be conducted. Longitudinal research is somewhat more conducive to survey methodology and in many cases, can help aid in problematic issues arising from the use of surveys. Researchers must be selective, however, in the length of time-frames chosen. Time-frames that are too short may suffer from the biases inherent in the typical survey approach. Leadership is a complex phenomenon — the results of which may not be visible for a substantial period of time. Moreover, longitudinal research that is both qualitative and quantitative in nature can provide clues to the process of leadership, as well as allowing for examination of unique trends (e.g., non-linear relationships) in the data. This blend of qualitative and quantitative methodology should, moreover, not be limited to longitudinal investigations and has substantial utility with other methodological approaches as well. With regard to issues of causality, it is clear that an increase in the number of experiments would aid in causal interpretation. The issue, however, runs somewhat deeper than an increase or change in certain methodologies. Rather, we must view leadership in a different way – realizing and admitting that leadership is a complex phenomenon – with the possibility of reciprocal, recursive relationships occurring among multiple constituencies and levels of analysis. If we think of leadership in this way, better methods and approaches are likely to follow in that they will be necessary to answer the research questions derived from these new perspectives. Finally, it is important to test and consider alternative models of leadership. An increase in longitudinal investigation and better causal exploration are essential first steps to solving problems in this area. However, it is critical that we first admit that alternative models may exist and second, that current conceptualizations may not be wholly comprehensive, or at least not the only approaches to explain observed relationships. It should be noted that we agree with Vandenberg (2006) in that alternative model examination should not involve testing “straw-man” models, where hypothesized models are considered against purposely designed poor-fitting models solely for the sake of having tested an alternative approach. Rather, it is our contention that if plausible alternative models exist, they should be tested with a reasonable degree of rigor and inquiry. If, on the other hand, the hypothesized model is viewed by the authors as the best suited to answer the research question, a justification and discussion of why such a model is appropriate should prove sufficient. Unfortunately, neither alternative model testing nor appropriate justification occurs with requisite frequency in the typical leadership study. 5. Discussion Before turning to the broader implications of the present effort, a few limitations should be noted. First, this discussion intentionally paints a negative view of leadership research — one that focuses on the most frequently occurring, typical leadership studies. As such, a number of well-done studies were purposely omitted from discussion. This focus on the typical leadership study may have misled the reader into thinking that there has been a lack of welldone studies conducted in the area of leadership. This, simply stated, is false. There have been a great number of appropriately conducted studies, many of which are responsible for lifting the field from the brink of failure to a thriving area of research (Hunt, 1999). Without these well-done studies, moreover, a discussion such as this one would not be relevant or even possible. Thus, it is not our intent to “throw the baby out with the bath-water”, and we concede fully the existence of an array of substantive and relevant leadership research investigations. Further, for nearly every criticism made regarding the typical leadership study we admit that there exists a well-done investigation that has appropriately dealt with or completely circumvented the potential criticism outright. Thus, it should be reiterated that this discussion focuses on leadership as a whole, not on studies that have been conducted with care, foresight, and scientific rigor. Second, the assumptions and related problems discussed in the present effort emphasized the potential for inflated results. In point of fact, there exists the possibility that some biasing variables can reduce, or suppress, observed relationships. This emphasis was intentional, we felt that the biases most likely to occur in leadership research would result in inflated rather than suppressed results. The fact remains, however, that not all biases have inflationary outcomes. Thus, some caution is warranted in our discussion of the impact of potential biases, particularly common method bias. Finally, the list of assumptions addressed in the present effort is not exhaustive. An attempt was made to discuss the most prominent and relevant assumptions. In no manner, however, do we feel that all assumptions were considered. A S.T. Hunter et al. / The Leadership Quarterly 18 (2007) 435–446 443 few of those notable to the present effort include assumptions tied to specific types of method bias (e.g., item-context method bias). It was our contention that such detail would have overburdened the reader and hindered the conveying of our fundamental points. Still, it must be noted that a number of problematic assumptions were not addressed and because of this, the present discussion stands as limited. Having addressed the above limitations, we feel that the present discussion brings to fore a number of important points. First, the application of the typical leadership study brings with it a number of assumptions — assumptions that, in many cases, are likely to prove unjustified. Second, the number of assumptions made when conducting the typical leadership study is substantial, suggesting that the problems that arise are not localized to only one or two areas of concern. Rather, criticisms and questionable assumptions appear applicable to several areas (e.g., subordinate, leader, context, and process). With the two above points noted, an important question comes to fore: What is the end result of these potentially false assumptions? Put another way, what will happen if we continue to employ the typical approach to studying leadership? The answer, it seems, is two-fold. First, application of the typical leadership study may include false assumptions that may, in turn, lead to inappropriate conclusions for a specific study. For example, a researcher who concludes that leader behaviors cause a given outcome for all organizations without applying requisite controls may be falsely concluding generalizability of results. Similarly, a survey-based study that does not include manipulations or examination of longitudinal effects may result in misattributions of causality. Second, and potentially more problematic, these results will then feed into streams of research where future investigators will base study design decisions on the outcome of such efforts. If the same methods, approaches, and assumptions are applied to subsequent studies, resulting hypotheses are likely to be confirmed. What may occur, ultimately, is a cyclical reaffirmation of similar constructs, relationships, and methodologies. This self-perpetuating cycle can have particularly dangerous effects. Witness the recent examination of leader-substitutes theory by Dionne, Yammarino, Atwater, & James (2002) who used a sample of 49 organizations, 940 subordinates, and 156 leaders. The authors concluded that “prior significant effects in substitutes literature may be merely a statistical artifact, resulting from common-source bias” (p. 454) — in essence suggesting that there was no support for a leadership theory considered by many to be a mainstay of the leadership arena. Granted, most of the assumptions discussed in the present effort are not likely to result in such dramatic outcomes as the neutralization of an entire leadership theory. Instead, the self-confirming cycle that occurs is problematic in how it limits exploration of leadership variables. For example, if a given theory suggests that a set of leader behaviors is related to an outcome of interest and the typical leadership study approach is taken — the obtained results are likely to be supportive of this theory. In fact, with even a kernel of truth and a minimal amount of “true” variance obtained, potential biases could inflate results such that researchers are less, or not at all, motivated to explore alternative behaviors, contexts, or relationships. Thus, the real problem resulting from the application of a typical leadership study is not actually finding an inflated relationship — it is in the failure to obtain negligible effects which may, in turn, push researchers into exploring alternative conceptualizations. Though less dramatic, this self-perpetuating cycle may be more dangerous than overturning or negating previous results. How are we to resolve these issues, then, when the problem is paradoxically tied to finding supportive results for our theories and studies? The resolution, at least in part, lies in stronger research methods such as a greater number of experiments and applying a larger number of study-relevant control variables. Better methods, unfortunately, have limited utility and are best-suited for resolving some of the more superficial issues hampering substantive growth in leadership. The real resolution lies in a reevaluation of leadership research and how we go about investigating associated constructs. For example, if we concede that leadership is a complex phenomenon, then simple correlational analysis will simply not answer our research questions and we will be forced to apply more sophisticated analytical approaches (e.g., hierarchical linear modeling). If we concede that leadership is a reciprocal, dynamic phenomenon, then a greater number of time-series investigations will be necessary. And if we admit that current leadership measures only capture a set of relevant leader behaviors — we will be forced to develop new methods for exploring alternative leadership conceptualizations. If, on the other hand, we continue to conduct typical leadership studies we are likely to find supportive results — but results that lack true meaning, substance, and practical utility. It is our hope that the present effort will help guide researchers in the former direction, rather than the latter. Acknowledgement We would like to thank Melissa Hunter and Jazmine Espejo for their help with the present effort. 444 S.T. Hunter et al. / The Leadership Quarterly 18 (2007) 435–446 References Anit, S. (2003). Relationships of participate leadership with relational demography variables: A multi-level perspective. Journal of Organizational Behavior, 24, 1003−1018. Antonakis, J., Avolio, B. J., & Sivasubramaniam, N. (2003). Context and leadership: An examination of the nine-factor full-range leadership theory using the Multifactor Leadership Questionnaire. Leadership Quarterly, 14, 261−296. Antonioni, D., & Park, H. (2001). The effects of personality similarity on peer ratings of contextual work behaviors. Personnel Psychology, 54, 331−360. Antonioni, D., & Park, H. (2001). The relationship between rater affect and three sources of 360-degree feedback. Journal of Management, 27, 479−495. Atwater, L. E., Dionne, S. D., Avolio, B., Camobreco, J. F., & Lau, A. W. (1999). A longitudinal study of the leadership development process: Individual differences predicting leader effectiveness. Human Relations, 52, 1543−1562. Bandelli, A. C., Lopez, G. E., & Ottinot, C. (2006). TIP-topics: Discussion on leadership. The Industrial and Organizational Psychologist, 43, 73−81. Bass, B. M., & Avolio, B. J. (1990). Multifactor leadership questionnaire. Palo Alto, CA: Consulting Psychologists Press. Bedian, A. G., & Hunt, J. G. (2006). Academic amnesia and vestigial assumptions of our forefathers. Leadership Quarterly, 17, 190−205. Bourgeois, L. J. (1985). Strategic goals, perceived uncertainty, and economic performance in volatile environments. Academy of Management Journal, 3, 548−573. Brown, D., & Keeping, L. M. (2005). Elaborating the construct of transformational leadership: The role of affect. Leadership Quarterly, 16, 245−272. Carless, S. A. (1998). Assessing the discriminant validity of transformational leadership behavior as measured by the MLQ. Journal of Occupational and Organizational Psychology, 71, 353−358. Carmeli, A., & Shaubroek, J. (2006). Top management team behavioral integration, decision quality and organizational decline. Leadership Quarterly, 17. Conger, J. A. (1998). The dark side of leadership. In G. R. Hickman (Ed.), Leading organizations: Perspectives for a new era (pp. 25−260). Thousand Oaks, CA: Sage. Conger, J. A., Kanungo, R. N., & Menon, S. T. (2000). Charismatic leadership and follower effects. Journal of Organizational Behavior, 21, 747−768. Cote, J. A., & Buckley, R. (1987). Estimating trait, method, and error variance: Generalizing across 70 construct validation studies. Journal of Marketing Research, 24, 315−318. Crampton, S., & Wagner, J. (1994). Percept–percept inflation in microorganizational research: An investigation of prevalence and effect. Journal of Applied Psychology, 79, 67−76. Cycyota, S., & Harrison, D. A. (2006). What (not) to expect when surveying executives: A meta-analysis of top manager response rates and techniques over time. Organizational Research Methods, 9, 133−160. Dansereau, F., & Yammarino, F. J. (1998). Leadership: The multi-level approaches: Classical and new wave. Oxford, England: Elsevier Science/JAI Press. Dansereau, F., Yammarino, F. J., & Markham, S. E. (1995). Leadership: The multi-level approaches. Leadership Quarterly, 6, 97−102. Day, D. V., Sin, H., & Chen, T. T. (2004). Assessing the burdens of leadership: Effects of formal leadership roles on individual performance over time. Personnel Psychology, 57, 573−605. de Groot, T., Kiker, S. D., & Cross, T. C. (2000). A meta-analysis to review organizational outcomes related to charismatic leadership. Canadian Journal of Administrative Sciences, 17, 356−371. de Hoogh, A. H. B., den Hartog, D. N., & Koopman, P. L. (2004). Charismatic leadership, environmental dynamism, and performance. European Journal of Work and Organizational Psychology, 13, 447−471. de Hoogh, A. H. B., den Hartog, D. N., & Koopman, P. L. (2005). Linking the big five-factors of personality to charismatic and transactional leadership: Perceived dynamic work environment as a moderator. Journal of Organizational Behavior, 26, 839−865. de Vries, R. E., Roe, R. A., & Tallieu, T. C. B. (2002). Need for leadership as a moderator of the relationships between leadership and individual outcomes. Leadership Quarterly, 13, 121−137. Dionne, S. D., Yammarino, F. J., Atwater, L. E., & James, L. (2002). Neutralizing substitutes for leadership theory: Leadership effects and commonsource bias. Journal of Applied Psychology, 87, 454−464. Elenkov, D. S., & Manev, I. M. (2005). Top management leadership and influence on innovation: The role of sociocultural context. Journal of Management, 31, 381−402. Engle, E. M., & Lord, R. G. (1997). Implicit theories, self-schemas, and leader-member exchange. Academy of Management Journal, 40, 988−1011. Fiedler, F. E. (1967). A theory of leadership effectiveness. New York: McGraw-Hill. Fuller, B. J., Patterson, C. E. P., Hester, K., & Stringer, D. Y. (1996). A quantitative review of research on charismatic leadership. Psychological Reports, 78, 271−289. Fulmer, R. M., & Vicere, A. (1996). Strategic leadership development: Crafting competitiveness. Oxford: Capstone. Gioia, D. A., & Sims, H. P. (1985). On avoiding the influence of implicit leadership theories in leader behavior descriptions. Educational and Psychological Measurement, 45, 217−232. Graen, G. B., & Cashman, J. F. (1975). A role making model of leadership in formal organizations: A developmental approach. In J. G. Hunt, & L. L. Larson (Eds.), Leadership Frontiers (pp. 143−165). Kent, OH: Kent State University Press. Graen, G. B., & Scandura, T. (1987). Toward a psychology of dyadic organizing. Research in Organizational Behavior, 9, 175−208. Graen, G. B., & Uhl-Bien, M. (1991). The transformation of work group professionals into self-managing and partially self-designing contributors: Toward a theory of leadership-making. Journal of Management Systems, 3, 33−48. S.T. Hunter et al. / The Leadership Quarterly 18 (2007) 435–446 445 Greene, C. N. (1975). The reciprocal nature of influence between leader and subordinate. Journal of Applied Psychology, 60, 187−193. Hambrick, D. C., & Fukutomi, G. D. (1991). The season's of a CEO's tenure. Academy of Management Review, 16, 719−742. Hatfield, J., Faunce, G. J., & Soames Job, R. F. (2006). Avoiding confusion surrounding the phrase “correlation does not imply causation”. Teaching of Psychology, 33, 49−51. Heinitz, K., Liepmann, D., & Felfe, J. (2005). Examining the factor structure of the MLQ: Recommendations for a reduced set of factors. Psychological Assessment, 21, 182−190. Hogan, R., & Hogan, J. (2001). Assessing leadership: A view from the dark side. International Journal of Selection and Assessment, 9, 40−51. Hogan, R., & Kaiser, R. B. (2005). What we know about leadership. Review of General Psychology, 9, 169−180. House, R. J. (1971). A path-goal theory of leader effectiveness. Administrative Science Quarterly, 16, 321−339. Hunt, J. G. (1999). Transformation/charismatic leadership's transformation of the field: an historical essay. Leadership Quarterly, 10, 129−144. Hunt, J. G. (2005). Explosion of the leadership field and LQ's changing of the editorial guard. The Leadership Quarterly, 16, 1−8. Hunt, J. G., & Ropo, A. (2003). Longitudinal organizational research and the third scientific discipline. Group and Organization Management, 28, 315−340. Keller, R. T. (2006). Transformational leadership, initiating structure, and substitutes for leadership: A longitudinal study of research and development project team performance. Journal of Applied Psychology, 91, 202−210. Kerr, S., & Jermier, J. M. (1978). Substitutes for leadership: Their meaning and measurement. Organizational Behavior and Human Performance, 22, 375−403. Krishnan, V. R. (2003). Impact of leadership on follower's influence strategies. The Leadership & Organization Development Journal, 25, 58−72. Lee, J. (2004). Effects of leadership and leader-member exchange on commitment. Leadership & Organization Development Journal, 26, 655−672. Liden, R. C., Wayne, S. J., & Stilwell, D. (1993). A longitudinal study on the early development of leader-member exchanges. Journal of Applied Psychology, 78, 662−674. Lowe, K. B., & Gardner, W. L. (2000). Ten years of the leadership quarterly: Contributions and challenges for the future. Leadership Quarterly, 11, 459−514. Lowe, K. B., Kroeck, K. G., & Sivasubramaniam, N. (1996). Effectiveness correlates of transformation and transactional leadership: A meta-analytic review of the MLQ literature. The Leadership Quarterly, 7, 385−425. Marion, R., & Uhl-Bien, M. (2001). Leadership in complex organizations. Leadership Quarterly, 12, 389−418. Martell, R. F., & Evans, D. P. (2005). Source-monitoring training: Toward reducing rater expectancy effects in behavioral measurement. Journal of Applied Psychology, 90, 956−963. Mayer, J. D., Salovey, P., & Caruso, D. R. (2002). Mayer-Salovey-Caruso emotional intelligence test. Toronto: Multi-Health Systems. Morrow, P. C., Eastman, K., & McElroy, J. C. (1991). Concept redundancy and rater naivety in organizational research. Journal of Applied Social Psychology, 21, 219−232. Mumford, M. D. (2006). Pathways to outstanding leadership: A comparative analysis of charismatic, ideological, and pragmatic leadership. Mahwah, NJ: Erlbaum Press. Mumford, M. D., & Hunter, S. T. (2005). Innovation in organizations: A multi-level perspective on creativity. In F. J. Yammarino, & F. Dansereau (Eds.), Research in multi-level issues: Volume IV Oxford, England: Elsevier. Mumford, M.D., Bedell-Avers, K.E., & Hunter, S.T., (in press). Planning for innovation: A multi-level perspective. In M.D. Mumford, S.T. Hunter, & K.E. Bedell-Avers (Eds.), Research in Multi-level Issues: Vol. VII. Oxford, England: Elsevier. Ostroff, C., Kinicki, A. J., & Clark, M. A. (2002). Substantive and operational issues of response bias across levels of analysis: An example of climate-satisfaction relationships. Journal of Applied Psychology, 87, 355−368. Phillips, J. S., & Lord, R. G. (1986). Notes on the practical and theoretical consequences of implicit leadership theories for the future of leadership measurement. Journal of Management, 12, 31−41. Ployhartz, R. E., Holtz, B. C., & Bliese, P. D. (2002). Longitudinal data analysis applications of random coefficient modeling to leadership research. Leadership Quarterly, 13, 455−486. Podsakoff, P. M., MacKenzie, S. B., Lee, J., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88, 879−903. Rafferty, A. E., & Griffin, M. A. (2004). Dimensions of transformational leadership: Conceptual and empirical extensions. Leadership Quarterly, 15, 329−354. Rosete, D., & Ciarocchi, J. (2005). Emotional intelligence and its relationship to workplace performance outcomes of leadership effectiveness. Leadership & Organization Development Journal, 26, 388−399. Scherbaum, C. A., Finlinson, S., Barden, K., & Tamanini, K. (2006). Applications of item response theory to measurement issues in leadership research. Leadership Quarterly, 17. Spector, P. E. (1994). Using self-report questionnaires in OB research: A comment on the use of a controversial method. Journal of Organizational Behavior, 15, 385−392. Spector, P. E. (2006). Method variance in organizational research: Truth or urban legend? Organizational research methods, 9, 221−232. Tejeda, M. J., Scandura, T. A., & Pillai, R. (2001). The MLQ revisited: Psychometric properties and recommendations. Leadership Quarterly, 12, 31−52. Tourangeau, R. (2004). Survey research and societal change. Annual Review of Psychology, 55, 775−801. Vandenberg, R. J. (2006). Statistical and methodological myths and urban legends: Where, pray tell, did they get this idea? Organizational Research Methods, 9, 194−201. Van Engen, M. L., & Willemsen, T. M. (2004). Sex and leadership styles: A meta-analysis of research published in the 1990's. Psychological Reports, 94, 3−18. 446 S.T. Hunter et al. / The Leadership Quarterly 18 (2007) 435–446 Wayne, S. J., & Ferris, G. R. (1990). Influence tactics, affect, and exchange quality in supervisor-subordinate interactions: A laboratory experiment and field study. Journal of Applied Psychology, 75, 487−489. Wexley, K., & Youtz, M. A. (1985). Rater beliefs about others: Their effects on rating errors and rating accuracy. Journal of Occupational Psychology, 58, 265−275. Yammarino, F. J., & Dansereau, F. (2006). The leadership quarterly special issue on multi-level approaches to leadership. The Leadership Quarterly, 17, 109–109. Yammarino, F. J., Dionne, S. D., & Chun, J. U. (2005). Call for papers — Leadership and levels of analysis: A state-of-the-science review. Leadership Quarterly, 16, 879−919. Yukl, G. (1999). An evaluative essay on current conceptions of effective leadership. European Journal of Work and Organizational Psychology, 8, 33−48. Yukl, G. (1999). An evaluation of conceptual weaknesses in transformational and charismatic leadership theories. Leadership Quarterly, 10, 285−305. Yukl, G. (2006). Leadership in organizations, 6th ed. Upper Saddle River, NJ: Prentice Hall.