ID model construction and validation: a multiple intelligences case

Education Tech Research Dev (2007) 55:369–390 DOI 10.1007/s11423-006-9015-4 DEVELOPMENT ARTICLE ID model construction and validation: a multiple intelligences case Monica W. Tracey Æ Rita C. Richey Published online: 16 September 2006
Association for Educational Communications and Technology 2006 Abstract This is a report of a developmental research study that aimed to construct and validate an instructional design (ID) model that incorporates the theory and practice of multiple intelligences (MI). The study consisted of three phases. In phase one, the theoretical foundations of multiple Intelligences and ID were examined to guide the development of such model. In phase two the model components were determined and an initial model was constructed. In phase three, the model was reviewed and validated by experts in the field of ID through a three-round Delphi study. The result was a revised and validated Multiple Intelligences Design Model. This paper presents the decision-making processes and procedures used in model development, and provides a framework for the internal validation of ID models using expert review procedures. Keywords Design and development research Æ Model construction and validation Æ Instructional design Numerous models exist in the field of instructional design (ID) that, assist designers working in a variety of settings (Gustafson & Branch, 2002). In addition, as new understandings of learning and instruction become available and accepted, existing ID models are refined and enhanced to take into account such developments and changes (Dick, Carey, & Carey, 2001; Morrison, Ross, & Kemp, 2004). In recent years, there has been an increased focus on systematically studying the processes involved in the construction, M. W. Tracey (&) Oakland University, 435D Pawley Hall, Rochester, MI 48309, USA e-mail: tracey@oakland.edu R. C. Richey Wayne State University, 381 Education, Detroit, MI 48202, USA e-mail: rrichey@wayne.edu 123 370 M.W. Tracey, R.C. Richey validation, and implementation of ID models (Richey, 1998; Richey, Klein, & Nelson, 2004; Seels, 1994). This research focus, usually referred to as developmental research, draws on and contributes to the literature in areas such as the nature and practice of ID, designer decision-making, the role of theory in instructional design, and the relationships between theory and practice in ID. This article reports on one such developmental research study conducted to construct and internally validate an ID model that incorporates the theory and practice of multiple intelligences. Approaches to model construction and validation In spite of the widespread use of models in the field of ID there is a paucity of literature, let alone research, on model formation. Dick (1997) describes the initial formation of the influential Dick and Carey model as a process of applying a diverse body of research and thinking of the times to the task of creating instructional products. It was a logical process of synthesis. The validation of the model came through repeated use rather than empirical study. Today, it is likely that most new ID models are constructed in a similar fashion. This is supported by much of the model-oriented developmental research reports. For example, Tessmer, McCann, and Ludvigsen (1999) describe a model for the reassessment of the need for existing training and report on the initial validation of the model. Weston, McAlpine and Bordonaor (1995) also describe a model directed toward understanding the formative evaluation process. Both models were developed by analyzing formative evaluation theory and research. Some model construction procedures have been suggested. Reigeluth and Frick (1999) propose using formative research methodologies, a type of developmental research. This approach involves creating a case to help generate the model, and then entering a repeated process of collecting and analyzing formative data on the case and revising it as warranted. They tend to equate design theories and design models. There is also research that specifically describes model construction. Jones and Richey’s (2000) study, for example, resulted in a revised rapid prototyping ID model. This model was based upon interview data describing designer tasks performed while using rapid prototyping techniques, the concurrent processing of those tasks and the nature of customer involvement. In contrast to the gaps in the model construction literature, there is more literature focused on the implementation and systematic validation of ID models. Richey (2005) describes five different approaches to validation and cites examples of their use in the literature. These include expert review, usability documentation, component investigation, field evaluation, and controlled testing. The Weston, McAlpine, and Bordonaor model (1995) was validated by a type of expert review. They systematically reviewed 11 ID and evaluation texts to determine the model’s level of support. Tessmer et al. (1999) model was validated twice through field evaluation techniques in two 123 MI design model 371 settings. Some, such as Seel (1997), however, question whether many ID models are confirmable without confirmation of their underlying theories. This suggests another more rigorous form of model validation. In spite of this literature on ID models, there is still a weak knowledge base on the construction and validation of such models. This paper aims to address this gap in the literature by describing the processes used here to construct and validate a multiple intelligences ID model. In particular, this study sought to explore the following questions: (1) what are the theoretical foundations of ID and multiple intelligences (MI) and is it possible to synthesize these theories into a practical design model? (2) What are the components of a design model that are oriented toward addressing the nature of multiple intelligences? (3) What processes and procedures are involved in the conceptualization, revision and validation of such a model? Theoretical foundations and model applications It was first necessary to determine if it was logical and feasible to synthesize the theories underlying ID and multiple intelligences into one useable ID model. ID is the process used to construct instructional products, programs, and delivery systems. Multiple intelligences theory is built on the premise that learners acquire knowledge based on learning potential and that people learn in at least seven different ways. While the theory of multiple intelligences and its implications for learning and instruction have been available for over two decades (Gardner, 1983), until now there has been no systematic process for guiding the application of this knowledge into ID. While some curriculum models have been developed with the goal of incorporating multiple intelligences into the design of instruction, such models have a narrow focus in terms of the types of learners and learning outcomes for which they have been intended. In addition, such models are generally unknown in the professional ID community. A comprehensive MI design model could provide instructional designers a systematic way of creating products that introduce instructional material to learners in at least seven different ways. Such a model also could provide an approach to creating environments that allow learners to construct their own understandings of knowledge. The need for such a model is based upon the validity of both multiple intelligence theory and instructional design theory. A review of the literature on these two knowledge bases was conducted to verify these premises and to determine what elements might be incorporated into a proposed model. Instructional design theories and models Instructional design is defined as an arrangement of resources and procedures used to promote learning (Gagne, Wager, Golas, & Keller, 2005). Instructional design models are visual representations of the ID process and are used to guide design in many settings and for many purposes (Seels & 123 372 M.W. Tracey, R.C. Richey Glasgow, 1998). They are typically a result of the combination of abstract principles of General Systems Theory and analyses of practitioner experience (Banathy & Jenlink, 2004). In 1981, Walter Dick suggested that these models serve as the theory of the designer field. In essence, we are taking this position here. Seven ID models were selected and reviewed for this study. The seven ID models were chosen based on their contributions to the discipline of ID, including their historical impact on the field, their applicability in a variety of environments, their level of detail, or their theoretical focus. The models reviewed were: 1. 2. 3. 4. 5. 6. 7. Instructional Development Institute (IDI) Model (Gustafson & Branch, 1997), chosen for its historical significance and its use in teacher training; Interservice Procedures for Instructional Systems Development (IPISD) Model (Branson, 1978), chosen for its level of detail and its use in military settings; Seels and Glasgow Model II: For Practitioners (Seels & Glasgow, 1998), chosen for its focus on project management; Smith and Ragan Model (Smith & Ragan, 1999), chosen for its focus on cognitive psychology; Morrison, Ross and Kemp Model (Morrisonet et al. 2004), chosen for its non-linear orientation; Dick and Carey Model (Dick et al. 2001), chosen for its historical significance and its wide-spread use; and ARCS Model (Keller, 1987), chosen for its motivation emphasis and because it exemplified a model that combines ISD with another orientation. Multiple intelligences theories and models Learning is the acquisition of the knowledge of a skill, art, or trade, by study, and/or experiences (Lindvall, 1995). Learners are ‘‘constructors’’ of knowledge when they take an active role in forming new understandings. It is generally agreed that learners construct understanding for themselves in ways that differ, sometimes quite sharply, from other learners (Winn, 2004). Multiple intelligences can be thought of as the learners’ tools that facilitate knowledge construction. Howard Gardner states, ‘‘I have posited that all human beings are capable of at least seven different ways of knowing the world.’’ Gardner (1983, p. xi). Rooted in cognitive brain research, Gardner proposes that people learn in a variety of ways and have diverse strengths and abilities which, if recognized, can be developed to enable learners to reach their potential. He defines intelligence as ‘‘the capacity to solve problems or to fashion products that are valued in one or more cultural settings’’ (Gardner & Hatch, 1989, p. 4). Gardner’s seven ways of knowing the world or intelligences are: verbal-linguistic, logical-mathematical, musical-rhythmic, visual-spatial, bodily-kinesthetic; and two forms of personal intelligences, one directed toward other persons, interpersonal intelligence, and one directed toward 123 MI design model 373 oneself, intra-personal intelligence. The characteristics of these intelligences and their associated behaviors are provided in Table 1. Gardner’s theory has been tested in a variety of research studies. Kelly and Tangey (2002) tested it in the construction of an intelligent tutoring system. They found that the most effective systems used strategies that encourage the learner to use as many of the identified intelligences as possible. Similarly, Martin (2002) tested the theory in instruction for business students. Here the MI instruction was also most effective, resulting in students who were more likely to recognize diversity in the workplace. Finally, Rauscher and Zupan (2000) found that music students’ problem solving capacity was improved after instructional activities that built upon musical/rhythmic intelligence. Recently, the theory of multiple intelligences has been applied to adult learners (Kallenbach & Viens, 2002), although the greatest use of the theory has been in elementary school curricula. Table 1. Types of Intelligences Type of Intelligence Characteristics A person with this well-developed intelligence... Verbal-Linguistic Uses language to construct and/or acquire information. LogicalMathematical Musical-Rhythmic Visual-Spatial Bodily-Kinesthetic Interpersonal Intrapersonal Speech, writing, narratives, poetry, and other forms of communication. Ordering and reordering objects; in assessing quantity, learners gain initial/ fundamental knowledge of the world. Relates to the ability to perceive and replicate rhythm, pitch, or melody and qualities of a tone. Capacity to recreate one’s visual experience, even in the absence of relevant physical stimuli. The ability to use one’s body in highly differential and skilled ways, for expressive and goal-oriented purposes. Capacity to read the intentions and desires of other individuals, even when these have been hidden. Capacity to detect and to symbolize a complex and highly differentiated set of feelings. Uses pattern ability, symbolic mastery; understanding of the relationship between objects to acquire information. Uses the different functions of rhythm, pitch, tone and/or melody to acquire information. Possesses abilities in art, architecture, and use visual imagery to construct and/or acquire information. Uses the different physical functions of the body to construct and/or acquire information. The ability to make distinctions among other’s temperaments, motivations, and intentions to acquire information. Understanding of oneself and the knowledge of his or her control of their own learning to construct and/or acquire information.* * Since this study was conducted, Gardner has theorized that there are at least two more intelligences (Gardner, 1999)]. However, since the theory incorporated only seven intelligences at the time of this study, the study reported here incorporates only the original seven intelligences. 123 374 M.W. Tracey, R.C. Richey The literature on multiple intelligences also was reviewed to determine what curriculum models, if any, had been developed to guide instructors in the use of multiple intelligences in instruction. The review produced only six curriculum models, varying widely in levels of detail and breath of application. The models identified were: 1. 2. 3. 4. 5. 6. Problem-Based Learning Model (Fogarty, 1997), a model that focuses on authentic problems as the impetus for learning; Year-Long Curriculum Model (Lazear, 2000), a model that emphasizes multiple intelligences in a K-12 curriculum; Thematic Learning Model (Fogarty, 1997), a multi-disciplinary model that focuses on themes as a method to connect learning activities with subject matter; Developing Mindful Learners Model (Fluellen, 1996), a model that focuses on increasing knowledge test scores; Model of Learning Preferences (Munro, 1994), a model that combines two theories; and Performance Learning Model (Fogarty, 1997), a model that focuses on hands-on learning and learner performance. Summary of models reviewed The 13 models in this study were analyzed in terms of the four major ID activities as defined by Gustafson and Branch (1997), and the six core elements of ID defined by Richey (1986). The Gustafson and Branch activities are: (1) Analysis (of learner needs and setting); (2) Design (including specifications for an effective, efficient, and relevant learning environment); (3) Development (including all learner and management materials); and (4) Evaluation. The six core elements defined by Richey are: (1) Determine Learner Needs (including problems identification, occupational analysis, and competence or training requirements); (2) Determine Goals and Objectives (including formulation and sequencing of broad goals and detailed sub goals ); (3) Construct assessment procedures; (4) Design/Select delivery approaches; (5) Try-out instructional system; and (6) Install and maintain system. Figure 1 illustrates a summary matrix of the elements of the models reviewed. Each of the reviewed 13 models included the major design elements as indicated in Fig. 1, but the ID models possessed a greater level of detail. All ID models addressed learner assessment and problem analysis, but only three of them included a needs assessment. All ID models identified and formulated objectives, but only four included the step of developing assessments based on those objectives. As a group, all seven of the ID models spoke to formulating and selecting instructional strategies. Six models also provided steps for trying out the materials developed, and installing and maintaining the instruction. The seven ID models reviewed possess many of the core 123 MI design model 375 Fig. 1 Summary of Models elements identified by Richey (1986) each imbedded in a slightly different way in the model steps. The ARCS model was the only model reviewed that was used as an overlay with the standard ID model. This model emphasized both instructional materials (similar to ID models) and the learner (similar to MI models). The ARCS model, however, does not focus on evaluation, nor does it emphasize courseware development. The models incorporating multiple intelligences into instruction were not consistent in terms of their goals or the steps included. One model exhibited just four of the identified core elements, three of which were in the evaluation 123 376 M.W. Tracey, R.C. Richey phase. The other five models included steps in analyzing learner needs with five of them focusing on assessing the need. All formulated goals and detailed sub-goals. Overall, the MI models focused more on learner needs than on the subsequent instruction. Only one focused on media selection. Only two focused on courseware development and try out of material, and none included the revision step. Evaluation was the least frequently identified step in the MI models. Burton, Moore, and Magliaro (2004) in their review of three behavioral design models found that each of the models places the responsibility for successful instruction on the teacher (as indicated by the emphasis on validation and revision of materials). This was also the case in the review of the seven ID models chosen for this study. The six MI Models reviewed, on the other hand, focus more on the learner. The primary emphasis is on learner needs and problem identification, and there is little emphasis on the development and validation of the instruction. Each of the models reviewed—both ID and MI—include a step on formulating instructional strategies. This is the only commonality in both types of models. Determining the components of a MI design model The ID and multiple intelligences theories and models describe a plethora of procedures that speak to their separate concerns, but there are no models that combine the two theoretical orientations. The MI curriculum models available cannot adequately direct the design of instruction. Nor do the ID models speak directly to the various innate capabilities of learners. There also appears to be no integration or cross-pollination of the ID and multiple intelligences literature. Consequently, there seems to be an opportunity to add to the ID knowledge base through the development of a MI design model. The review of literature provided inspiration for the initial MI Design model developed. Specifically, the literature pointed to ways in which the analysis, design and assessment phases of the ID process could be amplified to reflect MI principles. The first iteration of The Multiple Intelligence Design Model was structured around Gustafson and Branch’s (1997) four stages in ID (Analysis, Design, Develop, and Evaluate). This initial MI Design Model was a five page bulleted and narrative list of behaviors, characteristics, and examples of how to implement multiple intelligences into ID. The analysis stage included analyzing the learner behaviors, characteristics, and MI capabilities. Examples of multiple intelligence learner behavior characteristics were included to provide the designer with a snapshot of how each intelligence may be displayed, along with capacities that these learners may exhibit while learning. Understanding learner capacities aids the designer in determining if the desired performance naturally lends itself to one or more of the multiple intelligences. Following analysis of the learner, the environment and the desired performance, the behavior characteristics 123 MI design model 377 identified are provided to assist in writing the behavioral objectives incorporating multiple intelligences. In the design stage, the learner behaviors, characteristics and capacities previously identified in the analysis stage are then used to generate and/or select potential MI strategies and activities from the listed examples. Instructional strategy examples gathered in the literature review of multiple intelligence model research were used to incorporate the use of different intelligences. The designer is instructed to use the selected strategies/activities while developing the materials and to incorporate at least one strategy or activity for each MI. Learner behaviors and capacities identified in the analysis step provided the foundation for assessing the instructional objectives in a manner that incorporates multiple intelligences. Revisions and internal model validation After initial model development and refinement, an internal validation study was conducted that focused on verifying the components and processes suggested in the newly developed Multiple Intelligences (MI) Design Model. This was accomplished using a three round Delphi study. A panel of four subject matter experts was selected. The members of the panel, recommended by university professors, were chosen due to their specific backgrounds and expertise in model development, instructional self-regulation, learner differences, motivational design of instruction, and ID in general. Three of the panel members were from academic settings conducting research in the area of ID and one member was an ID practitioner. An expert in multiple intelligences was not chosen for the validation study because the validation focus was on components of an ID models and their potential use by practicing designers. Round one: procedures In phase one of the Delphi study, a packet of information was emailed to each of the reviewers. The packet included an introductory letter with the schedule and directions for the Delphi Study, the MI Design Model, a set of five open– ended questions to be answered in written format, a description of the purpose of the study, and a brief review of ID models and multiple intelligences to assist in an understanding of the study. The SMEs were asked to respond within one week to the following five questions: 1. 2. 3. How would you amend/clarify the four stages in the model? How do you think a novice and an expert would work with the model? What changes would help them? How would you amend the learner behaviors, characteristics and capacities section of the analysis stage? 123 378 4. 5. M.W. Tracey, R.C. Richey How would you amend the multiple intelligences strategies section in the design stage? Which area(s) do you feel demand the most revisions? Their written responses were reviewed and grouped together based on the questions asked, the area of the model addressed and miscellaneous feedback. These responses were then summarized by category. Round one: factors identified The factors that emerged from the original five questions were categorized into the following topics: (1) amending the four stages; (2) model usability by novices and experts; (3) changing the learner behaviors, characteristics and capacities section of analysis stage, (4) design stage, and (5) areas for revisions. Amending the four stages All of the SMEs indicated that they would require more guidance and elaboration on how to incorporate multiple intelligences into the design of instruction using the proposed model. In response to the first question regarding amending the stages in the model, SME 1 wanted more guidance in the analysis phase: What do I begin with? You say to begin with an analysis of learner behaviors, but I am not given any guidance on which learner behaviors I am supposed to examine. Also, I am wondering if I should look at learner behaviors or the results of a job/task analysis or some other list of k/s/a/’s that the learner is supposed to acquire to be able to meet some performance requirement? In addition, SME 1 acknowledged, ‘‘From a practical standpoint, I am not likely to go through the entire list of MIs in relation to each behavior identified in the learner analysis. I would require some more guidance as to how to use the MI list in a feasible way when I am designing a whole module or course.’’SME 2 suggested adding more guidance to the preparation of objectives incorporating multiple intelligences. She wrote: How does one select the MI objectives? Do I first derive the learning objective, e.g., write a coherent, grammatically correct paragraph of 3–5 sentences, and then scan each dimension to match? Do I choose from among 2 or more applicable dimensions, e.g., Visual-Spatial may be irrelevant, if they can negotiate the software to write a paragraph. SME 3 also referred to the need to clarify how analysis would influence the writing of instructional objectives. SME 4 recommended more detail/elaboration in the actual ID portion of the model. He noted, ‘‘At this point, the phases offer a very broad and rather vague visualization of the overall process. 123 MI design model 379 I think a second focus for my suggestions involves the need for the practical integration of MI into the ‘traditional’ process.’’ Model usability by novices and experts Several SMEs suggested the need for designers of all levels to have more guidance in using the model. For example, SME 1 suggested that the model would be useful as a source of ideas and a crosscheck on whether the designer was addressing the different intelligences and believed that designers would use the model as a heuristic to do a high level review of their learning requirements and strategies. However, he reiterated his previous comments related to the need for more guidance in the analysis phase. SME 2 stated that she imagined the expert could contrive a strategy for applying the model or deriving a cognitive process for analyzing and incorporating the multiple intelligences into instruction, but believed that a novice designer would be less likely to accomplish this task. SME 3 believed that the complexity of the model could be a problem for novice designers, and also believed that more detail was needed: How would you integrate the strategies in your design prescription with the design of information and practice activities that are appropriate for the instructional objectives? For novices, this model might over-complicate matters. SME 4’s central concern was whether designers could translate this model into a workable, practical application for everyday training problems where time and other resources are often scarce. He believed both the expert and the novice would probably want more direction. Changing the learner behaviors, characteristics and capacities within analysis SME 1 explained that if the learner behaviors, characteristics, and capacity list in the model were the ‘‘official’’ list of attributes, it should not be changed. He did, however, express concern for those not familiar with all of the technical vocabulary and concepts in the list and recommended a column of examples. SME 2 observed that the analysis of the learner step might be less feasible to measure than to just assume the learners are an average group. She approved of the list adding, ‘‘Why guess when you can compose the instructional objective and then use the strategy list to select a strategy?’’ SME 3 indicated that the distinction between learner behaviors and characteristics was not clear. SME 4 questioned the need for more guidance on how the MI relate to other characteristics: What do the results of your analysis suggest as far as next steps? For instance, does this suggest possible alterations in the learning environment of the preparation of objectives? Also, I think it might help to address the connectivity between intelligences and other learner characteristics such as motivation, prerequisite learning and other attitudes. 123 380 M.W. Tracey, R.C. Richey Design considerations Addressing the design step, SME 1 noted, ‘‘...it seems to me that the list of strategies includes many of the things that designers and teachers do anyway. I would want some additional guidance on which ones to select and when I would want a richer collection of possibilities.’’ His final suggestion was to ‘‘see some concrete examples of its application.’’ SME 2 found the list of strategies very useful. She added: ...the most important revision is on converting the MI model into, well, let me use a metaphor. Make it like a supercharger on an engine. The engine would be any instructional-objective or learning-goal design model such as constructivism or Dick and Carey and the MI would be rebuilt as a cross-model compatible supercharger. SME 3 suggested the need to provide information on how to integrate MI within the general design of the events of instruction, while SME 4 questioned the inclusion of those intelligences that have no apparent connection with the desired performance. SME 3 further recommended that the model reflect strategies composed of three different aspects as defined by Reigeluth (as cited in Smith & Ragan, 1999). This includes strategies related to: 1. 2. 3. Organization (what content will be presented, how it will be presented, and how it will be sequenced); Delivery (what media will be used and how learners will be grouped); and Management (how schedules and resources should be allocated). Areas for revisions In addressing areas of revision, SME 1 recommended ...to provide considerably more information on how to actually use it and some convincing evidence about the benefits of using this model, not just at the conceptual level, but also in terms of creating a better instructional environment, one which is more appealing, and more effective, or something. SME 1 also recommended including ‘‘some examples of its application and actual products that a designer would produce at each step.’’ SME 2 asked for ‘‘more information on how this model could be used in conjunction with a traditional ID.’’ SME 4 indicated that he believed the areas most in need of revision were analysis and evaluation followed very closely by design. His suggestion for evaluation was ‘‘to provide more guidance for connecting MI learning outcome assessments with the specific learning outcome assessments for a particular instructional event and the objectives of the sponsoring organization. 123 MI design model 381 Revisions to model based on round one feedback In general, the panel of experts recommended the following revisions in the MI model: • • provide more guidance in model use, especially with respect to conducting learner analysis and writing objectives that incorporate multiple intelligences; and use an ‘‘overlay’’ approach in the model that employs the existing graphical layout of the Dick and Carey. In response, the Dick and Carey Model was incorporated into the physical layout of the MI Design Model (see Fig. 2). The MI model components and the Dick and Carey ID Model were merged into a one-page flowchart and the model was supported by a one-page list of MI behaviors and examples. This merger involved combining the ‘‘analyzing learner environments’’ step in the original MI model and the ‘‘analyzing learner and contexts’’ in the Dick and Carey model. The ‘‘development of assessment instruments’’ included directions to match the MI behaviors, objectives, and environmental considerations in the design of assessment instruments. The ‘‘instructional strategy’’ step was expanded to include directions in determining the identified MI behaviors, the extent of learner control during the instruction, and the structure of each objective. Finally, organization, delivery and management strategies were incorporated into the model. The MI list of learner behaviors, characteristics and capacities from the first round model became a list of statements identifying each MI with examples of how to teach a learner with the identified MI behavior. Guidelines on when and how to use the list of MI behaviors and examples were associated with the appropriate phases in the revised model. The model changes made after the Delphi Round One basically simplified the process of incorporating multiple intelligences into the larger process. Furthermore, they emphasized the benefits of such an approach and minimized the burdens of extra steps. Round two: procedures Round Two of the Delphi consisted of another packet electronically sent to the four SMEs which included a summary of the feedback from round one, the revised MI design model, and a questionnaire to be filled out while reviewing the model. This questionnaire pertained to: • • • • • learner analysis; environmental analysis; assessment instruments; instructional strategies and examples; and model use. 123 382 Fig. 2 MI Design Model, Round Three Revise Instruction Assess Needs to Identify Goal(s) Example: Goal: teams solve design process problems by comingt o consensus. Analyze Learners and Environment Learners Use learner characteristics from analysis to determine present MI behaviors. *Refer to next page forg uidelines. Example: Cognitive characteristicso f learners indicates high-level of visuall iteracy; incorporate VS strategies. Learning Environment Determine if practice of desired performance and matchedM I behaviors can take place in the learning environment, if not make alterations. Example: To practice performance, the environmentm ust encourage verbal and written communication (VL), have materials for problem-solving (LM), imagining (VS),t eam learning and discussion (ITE),a nd quiet selfreflection for consensus (ITA). Write Performance Objectives Develop Assessment Instruments Develop Instructional Strategy Match identifiedM I behaviors, appropriate performance objectives, and environment considerations when designing assessment instruments. Use information developed during analysis to make instructional strategy decisions. Using the performance objectives and identified MI behaviors, design Instruments including: ••Type, Type,Form: Form:observation, observation, simulation, simulation,pencil-paper, pencil-paper, etc. etc. ••WWhere here in in instructional instructional strategy? ••AA match with identified objective: objective:conditions conditions presented presentedand and performance performancerequired. required. Example: Assessments can include: written clarification of problem, observation of verbal debate and report of process and solution, documentation of logical rationale for solutions, assessment of diagrams, flowchart developmento f process and reflection of thought process. Develop and Select Instructional Materials Design and Conduct Formative Evaluation of Instruction *Refer to next page for guidelines. Example: Organizational Strategies: • VL:D ebating, reading, and journal keeping • LM: Thinking formulas and strategies, problem-solving, complex lines of reasoning, strategy, and reasoning games • VS: Visualizing concepts through mind-mapping • ITE: Discussing, creating, and maintaining synergyi n teams, cooperative learning, and consensus building • ITA: Valuing clarifications, self-reflections, questionnaires, and surveys Delivery Strategies: • VL:W ritten and verbal strategies • LM: Computer problem-solving software programs • VS: Computer software programs • ITE: Group work • ITA: Independent assessments and studies Management Strategies: • VL: Tables, microphones • LM/VS: Computer stations Design and Conduct Summative Evaluation Legend Verbal-Linguistic (VL) Logical-Mathematical (LM) Visual-Spatial (VS) Bodily-Kinesthetic (BK) Musical-Rhythmic (MR) Interpersonal (ITE) Intrapersonal (ITA) M.W. Tracey, R.C. Richey The SMEs were once again given one week to respond to the questionnaire sent to them. Their responses in each category per question were tabulated from the questionnaires with written comments from each question 123 Conduct Instructional Analysis MI design model 383 Fig. 2 continued categorized by respondent name. The data were once again summarized by question. Round two: factors identified The Round Two response data related to five components of the revised MI Model and its usability. 123 384 M.W. Tracey, R.C. Richey Learner analysis Two survey questions related to the appropriateness and completeness of the learner analysis phase of the model were asked. These questions generated additional recommendations for further model revision. SME 3 noted that ‘‘a simpler model of critical learner, characteristics variables would be more generic, more valid, and easier to use.’’ SME 2 was concerned that ‘‘the relationship between regular and MI learner analysis is still unclear.’’ SME 3 said that she would like to see ‘‘step-by-step guidelines and instruments’’, while SME 4 felt that the adequacy and completeness of the model still depended on whether a novice or an expert were using the model. In summary, all four of the SMEs questioned the adequacy of the guidelines given in the learner analysis phase for novice designers of instruction. Environmental analysis SMEs responded to three survey questions related to the appropriateness and completeness of the environmental analysis phase of the model. There was no unanimity among responses relating to the environmental analysis part of the model. SME 1 and SME 2 were unsure what steps in the model were being referred to in this section, but SME 3 considered the environmental analysis presented to be an appropriate step in the model. SME 2 also suggested adding specific examples on how to make alterations to the environment. Assessment instruments In response to the two questions related to the appropriateness and clarity of the assessment instruments, SME 1 stated that, ‘‘it is clear how the steps of this model interface with ID, but not so clear as to how to actually apply this MI design part’’. SME 3 indicated that the assessment instruments presented ‘‘may be appropriate in terms of the format of the assessment rather than the content, as the content is determined by the instructional objectives’’. SME 4 did not believe ‘‘the development of actual instruments is clearly explained. In summary, none of the SMEs expressed satisfaction with the explanation of the assessment instruments in the model. Instructional strategies and examples All of the SMEs agreed that the list of instructional strategies appropriate for the different intelligences was a very useful element of the model. Representative of the responses, SME 2 simply said, ‘‘I like this idea very much.’’ SME 4 also suggested explaining ‘‘the relationship between organizational delivery and management strategies more carefully, particularly regarding MI.’’ All SMEs also agreed that the new examples were useful in the revised model. However, SME 1 and SME 4 felt that their physical size and color need to be changed to make them easier to see. 123 MI design model 385 Model use The SMEs responded to four questions related to the clarity of model use. SME 1 stated that he understood the overall concept, but believed work was still needed on illustrating how one would apply it systematically in a way that integrates the MI strategies with the instructional strategies, rather than just adding activities. SME 1 summarized his feelings by saying, ‘‘It’s getting close but I am not sure it is there yet. It is an operational model in principle, but probably still needs some work to make it practical.’’ SME 2 agreed. She also felt that it was getting close, but still questioned how to combine and integrate MI with instructional objectives. She asked: ‘‘If one has the objectives and assessment clear, do we need to bring up MI in the other steps? It would seem if they are incorporated in objectives and assessment, the other steps would be designed as usual.’’ While not responding directly to the question relating to model clarity, SME 3 indicated that her understanding was that ‘‘the designer rates the learners on the MIs and adapts instruction to match some, or all, of them’’. SME 4 continued to express concern that novice designers might not find the model easy to use. Revisions to model based on round two feedback In the second round of feedback, the experts generally expressed greater understanding of the goal of incorporating multiple intelligences in designing instruction, but still expressed dissatisfaction with the adequacy of the guidelines presented for learner analysis and development of assessment instruments. Again, the second round of feedback emphasized the challenges involved in developing an operational model with an adequate level of detail for instructional designer practitioners, but particularly for novice designers. However, the panel of experts continued to support the belief that specialized models of ID can be developed that can be efficiently used in conjunction with existing ID models, rather than as standalone models. As a result of this second round of feedback, more detailed step-by-step guidelines were developed and included in the model, and more clearly identified examples related to instructional strategy development and assessment were included. Specifically, the model was revised to provide a list of learner characteristics and MI behaviors. These were related to aspects of the training environment (see Fig. 2). These changes addressed the relationships between each component of the model. They also provided step-by-step guidelines and instruments, as well as more examples. The model was also revised by listing examples of how to incorporate multiple intelligences into the development of assessment instruments and the selection of instructional strategies. The revisions in each of these steps also included guidelines on what to do, and when to do it. The relationship between organizational, delivery and management strategies was more clearly illustrated in this revision with step-by-step guidelines for instruc- 123 386 M.W. Tracey, R.C. Richey tional strategy development. Multiple intelligence strategy examples were also added to help the designer in strategy selection and creation. The newly revised model now included specific guidelines, steps and examples to facilitate its use. See Fig. 2. Round three: procedures and feedback Round Three consisted of one final packet sent via e-mail to each of the SMEs which included a summary of the feedback provided by the panel in Round Two, the revised MI Design model, and a letter with a question. The panel determined that the goal of the third round was to gain agreement on the overall model. Thus, only a single question was posed. The question asked: ‘‘Can you agree with this model?’’ Each SME responded ‘‘yes’’ and Round Three was complete; the Delphi study ended. No further changes were made to the MI Design Model presented in Fig. 2. Discussion The purpose of this study was to translate theory into practice through the construction and validation of an ID model that incorporates the knowledge of multiple intelligences. The procedures and findings of the study have implications not only for the use of multiple intelligences in ID, but also for the processes involved in validating ID models. Incorporating multiple intelligences into instructional design Both multiple intelligences and ID are rich in their theoretical bases and are implemented with the goal of building learner knowledge. There are two fundamental multiple intelligence assumptions that are also important to ID: 1) One learns information best when it is presented in a rich context; and 2) It is difficult to secure transfer from separate courses or isolated definitions and skills to the kinds of problems that arise unexpectedly in the course of schoolwork or life (Gardner, 1993). In addition, the theory of multiple intelligences advocates problem-solving, context-rich instruction by using alternative contexts for practice thus promoting transfer. The advantage of the MI Design Model is its focus on the recognition of multiple intelligences in every step of the ID process; thus it has a continuous learner focus. This new MI ID Model, however, has benefits that go beyond the added value given to an instructional intervention. It demonstrates an approach to ID model enhancement. This is the ‘‘overlay’’ approach that involves taking an existing general ID model and embedding an additional layer of design procedures that address special concerns. The ARCS Model of Motivation Design (Keller, 1987) is the most common example of this approach to building ID models. This study replicates this approach and provides data supporting its usefulness. 123 MI design model 387 The advantages of this overlay approach of model construction are twofold. First, such an approach makes it feasible to complete the difficult task of developing a new operational ID model with the appropriate level of detail by allowing the model developer to focus only on the unique aspects of the new model. Second, the resulting design model typically can be easily mastered by both novices and expert designers because of their familiarity with traditional ID models. Thus, the new model is only new in part. One need not make radical changes in existing design habits to expand one’s repertoire of design skills. This study resulted in a validated model that should be useable by designers regardless of context, content, or learners. Furthermore, this new model should be useable by all instructional designers, novice or expert. These assumptions, however, are yet to be tested. Internal validation in instructional design ID model validation has been viewed as either internal or external. Internal validation is a confirmation of the components and processes of an ID model; external validation, on the other hand, is a validation of the impact of the products of model use (Richey, 2005). This study demonstrated validation procedures involving expert review, one of the three common internal validation techniques. Expert review is a process whereby ID experts critique a given model in terms of its components, overall structure and future use. It is the most expeditious of the internal validation methods. Essentially, this is a cyclical process of model review and critiquing based upon pre-specified criteria, and subsequent model revision based upon the data. Validation procedures of this type can also be viewed as a type of formative evaluation. The validation process in this study used the Delphi technique as a framework for four ID experts to critique and come to consensus on the components and overall structure of the MI Design Model. There were two aspects of this Delphi process that proved invaluable. First, this technique proved successful in part due to the qualifications of the reviewers. The reviewer panel had expertise not only in ID, but also in model construction and use. Selecting these experts was a critical part of the internal model validation process. In addition, the use of electronic communication proved to be an excellent method for receiving feedback. The expert reviewers were given a one-week window to review and reflect on the model in each round, answering five open-ended questions in the first round. This resulted in the most significant model revisions. It provided each reviewer with the opportunity to reflect and comment in a somewhat flexible timeframe. As a consequence extensive and important data were gathered which led to subsequent model revisions. This study can serve as a model of validation research as well as an application of the theory of multiple intelligences. There is a need for more empirical studies that explicate the processes involved in the construction or refinement of ID models. Moreover, validation should become a natural part of the model development process. The presence of this body of research could clarify the processes involved in ID model 123 388 M.W. Tracey, R.C. Richey construction and refinement. However, they may also lead to a greater understanding of the ID process itself. Conclusions It is clear that further research is required to externally validate the MI Design Model. In particular, there is a need to study the use of the MI Design Model by instructional designers with various levels of expertise, working in a variety of contexts. In addition, it is also necessary to determine the effects of using the MI Design model on the instruction designed and the learning experiences of a wide variety of students, including young people in traditional school settings and adults in employee training settings. This study, however, was more than an attempt to apply MI theory. It was an attempt to systematically construct and internally validate an ID model. It sought to gather empirical support for the components of this new model rather than relying primarily on personal advocacy as a basis for recommending its use. This study may serve as a framework for others involved in ID model construction and validation research. References Banathy, B., & Jenlink, P. M. (2004). Systems inquiry and its application in education. In D. H. Jonassen (Ed.), Handbook of research for educational communications and technology (2nd edn) (pp. 74–92). New York: Simon & Schuster Macmillan. Branson, R. K. (1978). Handbook of procedures for instructional systems development. Educational Technology, 18(3), 11–14. Burton, J. K., Moore, D. M., & Magliaro, S. G. (2004). Behaviorism and instructional technology. In D. H. Jonassen (Ed.), Handbook of research for educational communications and technology (2nd edn) (pp. 3–36). New York: Simon & Schuster Macmillan. Dick, W. (1997). A model for the systematic design of instruction. In R. D. Tennyson, F. Schott, N. M. Seel, & S. Dijkstra’s (Eds.), Instructional design: International perspectives: Volume 1–Theory, research and methods (pp. 361–369). Mahwah, NJ: Lawrence Erlbaum. Dick, W., Carey, L., & Carey, J. O. (2001). The systematic design of instruction (5th edn). New York: Addison-Wesley Educational Publishers. Fluellen, J. (1996). Developing mindful learners model: A 21st century ecological approach. Paper presented at the 8th World Future Society General Assembly (Washington, DC, July 13–19). Fogarty, R. (1997). Problem-based learning and other curriculum models for the multiple intelligences classroom. Arlington Heights, Illinois: Skylight Training and Publishing. Gagne, R. M., Wager, W. W., Goals, K. C., & Keller, J. M. (2005). Principles of instructional Design. (5th edn). CA: Wadsworth/Thomson Learning, Publishers. Gardner, H. (1983). Frames of mind. The theory of multiple intelligences. New York: Basic Books. Gardner, H. (1993). Multiple intelligences. The theory in practice. New York: Basic Books. Gardner, H. (1999). Intelligence reframed: Multiple intelligences for the 21st century. New York: Basic Books. Gardner, H., & Hatch, T. (1989). Multiple intelligences go to school: Educational implications of the theory of multiple intelligences. Educational Researcher, 18(8), 4–9. Gustafson, K. L., & Branch, R. M. (1997). Survey of Instructional Development Models (3rd edn). Syracuse, NY: ERIC Clearinghouse on Information and Technology. 123 MI design model 389 Gustafson, K. L., & Branch, R. M. (2002). Survey of Instructional Development Models (4th edn). Syracuse, NY: ERIC Clearinghouse on Information and Technology. Jones, T., & Richey, R. C. (2000). Rapid prototyping methodology in action: A developmental study. Educational Technology Research & Development, 48(2), 63–80. Kallenbach, S. & Viens, J. (2002). Open in interpretation: Multiple intelligences theory in adult literacy education. Retrieved September 2, 2002, from the National Center for the Study of Adult Learning and Literacy, NCSALL Reports #21. Keller, J. M., (1987). Development and use of the ARCS model of motivational design. Journal of Instructional Development, 10(3), 2–10. Kelly, D. & Tangey, B. (2002). Incorporating learning characteristics into an intelligent tutor. In Proceedings of the Sixth International Conference on ITSs, ITS2002. Lazear, D. (2000). The intelligent curriculum: Using MI to develop your students’ full potential. Tucson, Arizona: Zephyr Press. Lindvall, R. (1995). Addressing multiple intelligences and learning styles: Creating active learners. Unpublished doctoral dissertation, IL: Saint Xavier University. Martin, J. (2002) Multiple intelligences and business diversity. Journal of Career Assessment, 11(2), 187–204. Morrison, G. R., Ross, S. M., Kemp, J. E. (2004). Designing effective instruction (4th edn). New York: John Wiley & Sons, Inc. Munro, J. (1994). Multiple intelligences and mathematics teaching. Paper presented at the Annual Conference of the Australian Remedial Mathematics Association. University of Melbourne. Rauscher, F. H., & Zupan, M. A., (2000). Classroom keyboard instruction. Early Childhood Research Quarterly, 15(2), 215–228. Reigeluth, C. M., & Frick, T. W. (1999). Formative research: A methodology for creating and improving design theories. In C. M. Reigeluth (Ed.), Instructional design theories and models, Volume II: A new paradigm of instructional theory (pp. 633–651). Mahwah, NJ: Lawrence Erlbaum Associates Publishers. Richey, R. C. (1986). The theoretical and conceptual bases of instructional design. New York: Nicols Publishing. Richey, R. C. (1998). The pursuit of useable knowledge in instructional technology. Technology Research and Development, 46(4), 7–22. Richey, R. C. (2005). Validating instructional design and development models. In J. M. Spector & D. A. Wiley (Eds.), Innovations in Instructional Technology: Essays in Honor of M. David Merrill (pp. 171–185). Mahwah, NJ: Lawrence Erlbaum Associates, Publishers. Richey, R. C., & Klein, J. D. & Nelson W. A. (2004). Developmental research: Studies of instructional design and development. In D. H. Jonnassen (Eds.), Handbook of research for educational communications and technology (2nd edn) (pp. 1099–1130). New York: Simon & Schuster Macmillan. Seel, N. M. (1997). Models of instructional design: Introduction and overview. In R. D. Tennyson, F. Schott, N. M. Seel, & S. Dijkstra (Eds.), Instructional design: International perspectives: Volume 1–Theory, research and methods (pp. 355–359). Mahwah, NJ: Lawrence Erlbaum Associate, Publishers. Seels, B. (1994). An advisor’s view: Lessons learned from developmental research dissertations. Paper presented at the 1994 Annual Meeting of the Association for Educational Communications and Technology. Seels, B., Glascow, Z. (1998). Making instructional design decisions (2nd edn). Upper Saddle River, NJ: Prentice-Hall, Inc. Smith, P. L., & Ragan, T. J. (1999). Instructional design (2nd edn). Upper Saddle River, NJ: Prentice-Hall, Inc. Tessmer, M., McCann, D., & Ludvigsen, M. (1999). Reassessing training programs: A model for identifying training excesses and deficiencies. Educational Technology Research and Development, 47(2), 86–99. Weston, C., McAlpine, L., & Bordonaro, T. (1995). A model for understanding formative evaluation in instructional design. Educational Technology Research and Development, 43(3), 29–48. 123 390 M.W. Tracey, R.C. Richey Winn, W. (2004). Cognitive perspectives in psychology. In D. H. Jonnassen (Eds), Handbook of research for educational communications and technology (2nd edn). New York: Simon & Schuster Macmillan. Monica W. Tracey is an Assistant Professor in the Human Resource Development Department at Oakland University. Rica C. Richey is a Professor and the Program Coordinator of the Instructional Technology program at Wayne State University. 123