Tpac4

Computers & Education 71 (2014) 206–221 Contents lists available at ScienceDirect Computers & Education journal homepage: www.elsevier.com/locate/compedu Investigating the impact of an integrated approach to the development of preservice teachers’ technological pedagogical content knowledge (TPACK) Chrystalla Mouza*, Rachel Karchmer-Klein, Ratna Nandakumar, Sule Yilmaz Ozden, Likun Hu School of Education, University of Delaware, 219 D Willard Hall, Newark, DE 19702, USA a r t i c l e i n f o a b s t r a c t Article history: Received 1 April 2013 Received in revised form 20 September 2013 Accepted 23 September 2013 The purpose of this study is to describe an integrated pedagogical approach, aimed at advancing preservice teachers’ learning on the use of technology and investigate its impact on participants’ knowledge (i.e., TPACK) and practice. The integrated approach juxtaposes an educational technology course with methods courses and field experience through careful instructional design. Both quantitative and qualitative data were collected. Quantitative data were collected through a pre-post administration of the Survey of Preservice Teachers’ Knowledge of Teaching and Technology. Qualitative data were collected through open-ended survey responses and preservice teacher case narratives reporting on the design and implementation of technology-integrated lessons in a field placement. Finding revealed that participants experienced significant gains in all TPACK constructs. Further, findings indicated that participants applied their knowledge in practice though there was variability in the ways in which knowledge domains were represented in participants’ narratives. Findings have implications for teacher education programs and for researchers interested in the development and assessment of preservice teacher knowledge of teaching with technology. Ó 2013 Elsevier Ltd. All rights reserved. Keywords: Elementary education Improving classroom teaching Pedagogical issues Teaching/learning strategies 1. Introduction Common Core Standards, now adopted by 45 states across the U.S., place increased attention to the use of new technologies as a way to acquire knowledge and skills in core subject matter areas (Common Core State Standards Initiative, 2010). The standards are not only designed to promote benchmarks of student achievement in literacy and mathematics, but they are also intended to reflect the knowledge and skills students need to succeed in college, the workplace, and life in a technological society (Common Core State Standards Initiative, 2010). This shift makes it necessary for new teachers to enter the classroom with the knowledge and skills required to design and implement rigorous standards-based lessons that emphasize strategic use of technology in support of curricular goals (Lawless & Pellegrino, 2007). Existing research indicates that a critical factor influencing new teachers’ adoption of technology is the quantity and quality of technology experiences included in their teacher education program (Agyei & Voogt, 2011; Tondeur et al., 2012). Although this generation of preservice teachers is more technologically savvy and actively engaged with digital media, knowledge and skills alone are not sufficient conditions for curricular use of technology in support of rigorous standards (Lei, 2009; Margaryan, Littlejohn, & Vojt, 2011). Niess (2008), in particular, argues that since this generation of future teachers has not traditionally experienced their own content learning with digital technologies, they need specialized instruction on how to teach their core content with technology while simultaneously guiding students in learning about new forms of technology. Recent calls have indicated that to prepare prospective teachers for effective use of technology, teacher education programs must help them build knowledge of content, good pedagogical practices, and technical skills as well as an understanding of how these constructs interactively relate to one another (Koehler & Mishra, 2008). In fact, the interactions among content, pedagogy and technology form the core * Corresponding author. Tel.: þ1 302 831 3108. E-mail addresses: cmouza@udel.edu (C. Mouza), karchmer@udel.edu (R. Karchmer-Klein), nandakum@udel.edu (R. Nandakumar), shule@udel.edu (S. Yilmaz Ozden), lhun@udel.edu (L. Hu). 0360-1315/$ – see front matter Ó 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.compedu.2013.09.020 C. Mouza et al. / Computers & Education 71 (2014) 206–221 207 of what has been called technological pedagogical content knowledge (TPACK), a distinct type of flexible knowledge required for effective use of technology in classroom teaching (Angeli & Valanides, 2009; Mishra & Koehler, 2006; Niess, 2005). While researchers and teacher educators have embraced the TPACK framework with excitement (Graham, 2011), recommendations on how to develop and assess TPACK vary widely in the literature (Koehler, Shin, & Mishra, 2012; Polly, Mims, Shepherd, & Inan, 2010; Tondeur et al., 2012). Further, limited research exists to date examining the actual technology topics addressed across teacher education institutions, the impact of these topics on teaching practice, and the empirical basis for the inclusion of these topics within the teacher education curriculum (Hew & Brush, 2007; Lawless & Pellegrino, 2007; Ottenbreit-Leftwich et al., 2012). 2. Purpose and significance of the study The purpose of this study is to describe a pedagogical approach, aimed at advancing preservice teachers’ learning on the use of technology and investigate its impact on participants’ knowledge (i.e., TPACK) and practice. The approach discussed in the study consisted of an educational technology course with a specific set of technology topics offered in conjunction with methods courses and field experience. This integrated approach aimed at fostering both technological knowledge as well as knowledge related to the intersections of content, pedagogy and technology. Existing literature indicates that the stand alone educational technology course can benefit when connected to methods courses and field experiences that allow prospective teachers to teach particular content with technology (Niess, 2005, 2012). Yet, few studies articulate how such integration can be accomplished to effectively leverage preservice teachers’ knowledge bases (e.g., Harrington, 2008). Using a combination of self-reported survey and performance assessment measures we answer the following research question: How and to what extent does participation in an educational technology course, offered in conjunction with methods courses and field experience, influence preservice teachers' TPACK development and practice? In our work, we acknowledge that simply juxtaposing the educational technology course with methods courses and field experience without careful instructional design may not necessarily ensure TPACK development. Thus, in our analysis we articulate how that integration was accomplished and the ways in which it leveraged preservice teachers’ learning. Findings from this work help improve our understanding of effective pedagogical models, instructional topics, and strategies that help advance preservice teachers’ learning on the use of technology as well as ways in which we can measure such learning. As Tondeur et al. (2012) note, “it is of great importance to examine how teacher education programs influence preservice teachers to use technology in their future classrooms” (p. 135). 3. Conceptual framework This work is grounded in the theoretical framework of TPACK, which is used to describe the knowledge base required for effective use of technology in teaching and learning (see Fig. 1; Mishra & Koehler, 2006). Beginning with Pierson’s (2001) initial articulation of the idea, TPACK has emerged as a key construct in teacher preparation to emphasize a content-specific orientation to technology integration (Angeli & Valanides, 2005; Koehler & Mishra, 2005; Margerum-Leys & Marx, 2004; Niess, 2005). Building upon Shulman’s (1986) conception of pedagogical content knowledge (PCK), the TPACK framework includes three core knowledge domains namely, content knowledge (CK), pedagogical knowledge (PK), and technological knowledge (TK). It also includes the interactions among these three core domains, which form four additional knowledge domains namely, pedagogical content knowledge (PCK), technological pedagogical knowledge (TPK), technological content knowledge (TCK), and technological pedagogical content knowledge (TPACK). Contextual knowledge is also included as a part of the model (Kelly, 2008). Each type of teacher knowledge represented in the framework is shown in Fig. 1. Fig. 1. TPACK framework: http://tpack.org. 208 C. Mouza et al. / Computers & Education 71 (2014) 206–221 Despite the wide acceptance of the framework, there are different conceptions of TPACK in the literature. In a recent review of the TPACK literature, Voogt, Fisser, Roblin, Tondeur, and van Braak (2012) uncovered three common views of TPACK that have developed over time: (a) TPACK as extended PCK (see Niess, 2005); (b) TPACK as a new and distinct body of knowledge (see Angeli & Valanides, 2009); and (c) TPACK as the interplay among three domains of knowledge and their intersections (see Mishra & Koehler, 2006). In this work we employ a combination of the latter two views. Although we acknowledge TPACK as a distinct body of knowledge that is more than the sum of its parts, we also recognize that it is rooted in the TPACK sub-domains described in the framework (i.e., CK, TK, PK, PK, PCK, TCK, and TPK). As a result, when examining participants’ knowledge development over time we measure both TPACK as a distinct entity and its individual subdomains. It is important to clarify, however, that TPACK is not static or fixed, but a dynamic and flexible body of knowledge influenced by both rapid changes in technology and the bidirectional relationship between knowledge and practice (Cox & Graham, 2009; Doering, Veletsianos, Scharber, & Miller, 2009; Mouza, 2009). As such, TPACK takes many forms and variations in practice and can be further developed through iterative cycles of application and reflection (Kinuthia, Brantley-Dias, & Clarke, 2010). In fact, through extensive observations and data collection with in-service teachers Niess, Lee, and Sadri (2007) identified five levels of TPACK development that included recognizing (knowledge), accepting (persuasion), adapting (decision), exploring (implementation), and advancing (confirmation). Using these levels as a guide, Niess et al. (2009) identified descriptors that characterize teachers’ work in relation to four key themes: curriculum and assessment, learning, teaching, and access. We believe that this developmental view of TPACK is particularly appropriate for preservice teachers who lack teaching experience and have not yet had an opportunity to go through iterative cycles of application and reflection. 4. Background literature 4.1. Approaches to the development of preservice teachers’ TPACK A common strategy used to advance preservice teachers’ TPACK is the delivery of a technology course, which has been consistently offered since the early 1990s (Niess, 2012; Polly et al., 2010). Specifically, a survey of 1439 institutions with teacher education programs in the U.S. revealed that 85% of them offer an educational technology course (Kleiner, Thomas, & Lewis, 2007). This course has frequently focused on learning about different technologies (e.g., word processors, presentation software, the Internet) along with their affordances and constraints. Although there are many benefits associated with this approach including the ability to improve preservice teachers’ selfefficacy, provide a good overview of the use of technology in teaching, and develop a strong foundation of technology skills, it has not resulted in preservice teachers’ implementation of digital technologies in their teaching (Kay, 2006; Niess, 2012). As a result, Mishra, Koehler, Shin, Wolf, and DeSchryver (2010), suggest retaining the technology course but reconsidering its curriculum. They propose engaging teachers with rich pedagogical, technological and content problems that maintain the inter-relationships through approaches such as learning-by-design. Learning-by-design allows teachers to work in collaborative groups developing technological solutions to authentic pedagogical problems (Koehler & Mishra, 2005). In one of the earlier efforts to advance TPACK development through learning-by-design, Koehler and Mishra (2005) studied faculty and master students working collaboratively to transform a face-to-face course into an online course. Similarly, using a project-based approach in the context of an educational technology course, Wetzel, Foulger, and Williams (2009) engaged small groups of preservice teachers working together to master specific technology tools. Subsequently, participants were responsible for teaching other classmates about their selected tool, provide a picture of its use in a future classroom, and begin to use the new tool to accomplish learning goals. Other researchers have identified content and technology-based approaches, such as instructional modeling and application (Niess, 2005), TPACK-based case development (Mouza & Wong, 2009), instructional design through technology mapping (Angeli & Valanides, 2009), action research, inquiry and self-reflection (Cavin, 2008; Pierson, 2008). While the above approaches have provided needed insights into the development of TPACK, Niess (2012) acknowledges that we must continue investigating methods and learning trajectories for integrating the development of TPACK within entire teacher preparation programs. In this study we examine the impact of an integrative approach to TPACK development that carefully juxtaposes the educational technology course with methods courses and field experience. We also identify trajectories of preservice teacher learning using a developmental approach to TPACK, as described by Niess et al. (2009). 4.2. Approaches to measuring TPACK As researchers began to empirically investigate the impact of their instructional approaches, the issue of how to capture levels of understanding in TPACK became prominent (Koehler et al., 2012). To date, however, there is no widely accepted and generally applicable instrument for measuring teachers’ TPACK (Albion, Jamieson-Proctor, & Finger, 2010). In a recent review of the literature that involved 66 studies, Koehler et al. (2012) identified five commonly used techniques: self-report measures, performance assessments, open-ended questionnaires, interviews, and observations. Self-report measures such as surveys are one of the most frequently used methods to measure TPACK (Koehler et al., 2012). The most widely used survey instrument has been developed by Schmidt et al. (2009), and is directly linked to the TPACK framework. The survey has been constructed specifically for use with preservice teachers. Construct validity suggests that the instrument is reliable and could be used with confidence among preservice teachers majoring in elementary education (Albion et al., 2010). In a similar effort, Archambault and Crippen (2009) created a survey instrument linked to the TPACK framework specific to teaching online while Lee and Tsai (2010) proposed a survey instrument focusing specifically on web technology. Performance assessments evaluate TPACK by directly examining participants’ performance on given tasks that often are designed to represent complex, authentic, real-life tasks (Koehler et al., 2012). The earliest attempt to measure TPACK through performance assessment has been described by Koehler and Mishra (2005) who analyzed conversations between student-teachers and faculty who had to design online courses through the period of one semester. Their analysis looked at how TPACK elements were represented in team conversations at C. Mouza et al. / Computers & Education 71 (2014) 206–221 209 two different points in time. The study indicated that over time the initially separate topics of content, pedagogy, and technology became more strongly interconnected. Similarly, Mouza and Wong (2009) looked for evidence of TPACK constructs (i.e., PK, CK, PCK, TK, TPK, & TCK) in teacher artifacts such as case narratives, to judge the existence of TPACK while Angeli and Valanides (2009) and Graham, Borup, and Smith (2012) looked for evidence of TPACK through a design task assigned to preservice teachers. Specifically, Angeli and Valanides (2009) scored participants’ design tasks on five criteria that included identification of suitable topics, appropriate representations to transform content, identification of teaching strategies difficult to implement by traditional means, selection of appropriate tools and their pedagogical uses, and identification of appropriate integration strategies. Along the same lines, Graham et al. (2012) used TPACK as a lens for understanding how preservice teachers make decisions about the use of technology in their teaching in three content teaching design tasks during a pre-and posttreatment assessment. Researchers identified themes from preservice teachers’ rationales that mapped to the TPACK constructs. In their analysis, Koehler et al. (2012), found that open-ended questionnaires, interviews and observations were used less frequently in efforts to measure TPACK. For example, So and Kim (2009) asked participants to write brief responses to prompts which were later coded using the TPACK frame as a guide while Ozgun-Koca (2009) interviewed teachers on the advantages and disadvantages of calculator usage and the effects on teaching and learning. Similarly, studies involving observations typically used field notes or video-tapes which were later transcribed and coded following a TPACK-based coding scheme (e.g., Suharwoto, 2006). Despite the vibrant body of research, Koehler et al. (2012) pointed to a significant concern namely the short shrift given to issues of reliability and validity. In particular, 90% of the studies examined provided no evidence of validity while 69% of the studies examined provided no evidence of reliability. These findings echo concerns identified by Kay (2006) in an early effort to evaluate strategies used to incorporate technology into preservice teacher education. Specifically, Kay also found that the research methods utilized in many studies to gauge effectiveness left a lot to be desired, often employing a small or non-existent sample size or surveys without reliability and validity estimates. In this study, we use a combination of self-reported survey data and performance assessment measures to report on the effectiveness of an educational technology course offered in conjunction with methods courses and field experience. We include important contextual information on the teacher education program, as well as the course itself, and use a survey instrument (Schmidt et al., 2009) that has been found reliable for our sample (see Methods). 5. Rationale and description of the integrated approach to TPACK development 5.1. Rationale for an integrated approach In recent years, a number of teacher preparation programs strive to reinforce their stand alone educational technology course by infusing technology in methods coursework and field experience, providing a more integrated approach to the development of preservice teachers TPACK (e.g., Gronseth et al., 2010). Niess and colleagues (e.g., Niess, 2005; Niess & Gillow-Wilson, 2013; Suharwoto & Niess, 2006), in particular, conducted a number of studies examining preservice teachers’ development of TPACK in integrated teacher preparation programs for mathematics and science teachers. Results indicated that those programs were effective in developing participants’ TPACK as well as transfer of learning to classroom educational experiences. Although these efforts are critical, they were mostly conducted in the context of subject-specific teacher preparation programs for mathematics and science teachers. As a result, little is known about how such integrated approaches can support TPACK development among preservice elementary teachers who will be teaching a variety of subject areas or middle school teachers prepared outside subject-specific programs. In this section we provide information on the teacher education program in this study and describe the methods courses, field experience and educational technology course. Subsequently we explain how we brought those elements together into an integrated approach to support preservice teachers’ learning. 5.2. Teacher education program The Elementary Teacher Education (ETE) program in this study is a four-year undergraduate program accredited by the National Council for the Accreditation of Teacher Education (NCATE). Graduates of the program are eligible for elementary (K–5) and middle school (6–8) teacher certification. The program curriculum is divided in three areas in terms of coursework; the general studies courses which develop preservice teachers’ content knowledge, the professional studies courses (e.g., methods) which prepare preservice teachers for their future classroom, and the concentration courses which help preservice teachers’ develop expertise in one of six concentration areas including middle school English, mathematics, science, social studies, and special education. Consistent with contemporary research that advocates the necessity of field experiences in teacher education, the program curriculum is designed to provide preservice teachers with a range of field experiences in a variety of classroom settings beginning with freshman year. 5.3. Methods courses Helping preservice teachers develop knowledge and skill in how to teach is a critical component of teacher preparation (NCATE, 2010). Effective teachers not only know their subject matter but they are also able to create a stimulating learning environment and apply pedagogical strategies, including technology-rich strategies, that engage students while helping them improve their achievement (NCATE). The ETE program requires a number of methods courses in literacy, social studies, mathematics, and science. Those courses are divided in three blocks that include the elementary block, middle school block, and special education block. All methods courses focus on curriculum and appropriate methods for teaching subject matter concepts to elementary or middle school students. Activities include review, development and evaluation of curriculum materials, teaching strategies including strategies that utilize technology, as well as research assessment on student learning. Methods faculty model a variety of ways in which technology can be used in conjunction with specific content (e.g., literacy, mathematics, science, social studies) and pedagogical strategies. 210 C. Mouza et al. / Computers & Education 71 (2014) 206–221 5.4. Field experience Clinical preparation or field experience is a key component of teacher preparation. In fact, the Blue Ribbon Panel Report (2010) commissioned by NCATE placed clinical practice at the center of transforming teacher education. Acknowledging the importance of clinical practice, the ETE program employs a variety of field experiences including early field experiences, methods field experiences, and student teaching. Further, efforts are made to place preservice teachers in classrooms that demonstrate effective use of technology. Early field experiences begin in the freshman year and provide preservice teachers the opportunity to observe experienced teachers and learn about the classroom environment and how to interact with children. Methods field experiences are taken in conjunction with methods courses designed to develop preservice teachers’ teaching skills and provide participants with the opportunity to teach lessons to an entire class, including lessons that integrate technology. During that time, preservice teachers are in the field for three full weeks each semester allowing them to observe and experience the daily routines of a classroom teacher including their use of technology, see growth in student learning, and develop strong relationships with cooperating teachers. Early and methods field experiences are designed to prepare preservice teachers for student teaching where they gradually take over classroom responsibilities for a period of one semester. 5.5. Educational technology course and description of the integrated approach The educational technology course, titled Integrating Technology in Education, runs for 15 weeks every semester and is required for all preservice teachers within the ETE program. The purpose of the educational technology course is to (a) introduce prospective teachers to technologies available for use in classroom content areas (e.g., concept mapping software, interactive manipulatives, Internet resources, and Web 2.0 tools); (b) pedagogical considerations with these technologies; and (c) teaching and learning practices that effectively combine these technologies with content and pedagogy. Course instructors model a variety of ways in which these technologies can be used to support student learning. Table 1 provides an overview of key course content and activities. Our pedagogical approach to helping preservice teachers acquire a deeper understanding of the interactions among technology, content and pedagogy involved simultaneous participation in the educational technology course, methods courses and field experience. When designed carefully, this model is consistent with key markers of effective teacher preparation in the use of technology recently described in the research literature (Tondeur et al., 2012). These markers include: (a) theory to practice connections, (b) opportunities for instructional design, (c) modeling, (d) authentic experiences, and (e) opportunities for enactment and reflection. Below we articulate how the integration of educational technology coursework, methods courses and field experience was designed to meet these markers of effective teacher preparation (see also Table 1). We also discuss the ways in which such integration leveraged changes in preservice teachers’ learning. 5.5.1. Theory to practice connections One of our key goals in juxtaposing the educational technology course with methods courses and field experience was to facilitate a theory to practice connection by allowing preservice teachers to connect what they learn from the university classroom to the school classroom and vice versa. We have developed recursive communication with the methods faculty where we inform each other of the tools, strategies and ideas emphasized in the courses. This approach allows us to provide unified content that supports learning in important ways. Table 1 Description of educational technology course in relation to markers of effective teacher preparation. Marker Activities/assignments Instructional design Lesson critique Authentic experiences Reflection Theory to practice Role models Description Participants identify a technology-integrated lesson in a content area of their choice from a lessonplan portal called Thinkfinity and prepare a critique discussing the instructional objectives of the lesson, the strengths and weaknesses of the learning activities, the content and technology standards addressed, and the role of technology in relation to the lesson’s objective. Lesson development Concept mapping. Participants practice using concept mapping software by writing a compare and contrast essay. In small groups, they reflect on their experiences and generate ideas that integrate concept mapping in a content area of their interest. Inquiry-based. Participants learn how to design inquiry-based activities around web-based resources in order to reinforce or teach a literacy, social studies, mathematics or science concept. First, participants decide the content area and specific standards/skills they want students to practice (e.g., what should students know, understand or be able to do). Second, they search for web-based resources that support students as they learn/practice these skills. Third, they design inquiryoriented activities that engage students with the web resources as they learn content-specific concepts. Finally, they describe the mechanisms in which student work will be assessed. Web 2.0 tools. In small groups, participants learn about web a 2.0 tool such as blogs, wikis, or podcasts. Subsequently, they generate a lesson idea that focuses on the integration of the selected tool in a classroom setting and prepare sample work using the selected tool (e.g., blog or wiki entry, podcast, etc.). Finally, in jigsaw groups participants share their work with their peers. Technology inventory Participants construct an inventory of technological resources available in their field placement. The inventory needs to identify both hardware and software resources to help participants understand the kinds of technologies typically available in K–12 settings and where they are located (e.g., computer lab, classroom, library, etc.). Classroom implementation In consultation with their cooperating teacher, participants implement a technology-integrated lesson developed in the course (e.g., concept mapping, inquiry) into their field placement classroom. Case development Upon completing the implementation of a technology-integrated lesson in their field placement, participants develop a reflective case that discusses their experience following specific writing and reflection prompts. Educational technology course integrated with methods courses and field experience to facilitate learning and implementation of technology with content and pedagogy. Educational technology faculty, methods faculty and cooperating teachers in the field model uses of technology. C. Mouza et al. / Computers & Education 71 (2014) 206–221 211 For example, during their methods courses preservice teachers learn lesson-planning strategies for teaching particular content (PK & PCK) while during their educational technology course they learn how to use concept mapping software (TK). Afterward, as part of their educational technology course, they are required to develop a lesson using concept mapping software in a content area of their choice considering the characteristics of the students in their field placement (TCK & TPK). Finally, after consulting with their cooperating teacher they are required to implement a technology-integrated lesson in their field placement and prepare a reflective narrative documenting their experience and lessons learned (TPACK). 5.5.2. Instructional design TPACK is a framework that emphasizes the importance of preparing teachers to make effective choices in their uses of technology when teaching specific content to a particular student population (Tondeur et al., 2012). Toward this end, preservice teachers need repeated opportunities to examine instructional design, practice planning, and prepare materials that integrate technology tools. The educational technology course takes a two-step approach to developing knowledge of instructional design: lesson critique and lesson development. Initially, participants are asked to identify a technology-integrated lesson in a content area of their choice from a lesson-plan portal called Thinkfinity. Subsequently, they prepare a critique which discusses the instructional objectives of the lesson, the strengths and weaknesses of the learning activities presented in the lesson, the content and technology standards addressed, and the role of technology in relation to the lesson’s objective. This is a challenging assignment because it requires preservice teachers to think deeply about content, pedagogy and technology as well as how these three constructs combine to develop effective instruction. They must draw upon knowledge gleaned in the educational technology course in conjunction with the content and pedagogy learned in their content and methods courses. After spending considerable time critiquing published lessons, preservice teachers are given repeated opportunities to design their own for their methods field experience classrooms. Utilizing a variety of technology tools and applications, including interactive whiteboards, concept mapping, Internet inquiry, and collaborative tools, the plans must reflect an understanding of the TPACK framework. For instance, when learning about the pedagogical uses of concept maps, preservice teachers are asked to generate a lesson idea that utilizes electronic concept mapping software to support a learning goal within a content area of their choice. Additionally, they are required to create a sample concept map that would help students in their field placement understand the task. 5.5.3. Role models Research indicates that preservice teachers who had opportunities to observe the implementation of technology-rich units into methods courses reported greater technological skills and more ideas on how to use technology with students (Polly et al., 2010). As the instructors for the educational technology course, we provide the most relative models of technology integration since we choose the tools and applications preservice teachers will explore in our class. Therefore, for each tool and application introduced, we model its use and implement an activity requiring preservice teachers to learn about it and then apply it to an educational context. For example, in preparing lecture materials delivered and posted through our course management system we model the use of the interactive tool, Voicethread, which allows users to embed video, audio and text to deliver presentations. Later in the semester, preservice teachers are asked to open a free VoiceThread account, watch several video tutorials, and explore public VoiceThreads. These basic steps expose preservice teachers to the tool and allow them to practice the technical skills necessary to effectively engage with it (TK). Subsequently, while completing a unit on the characteristics of digital texts, preservice teachers read a related article and then present their ideas in Voicethread following a set of guiding questions where they use text, audio or video. This process provides them with an explicit use of the technology tool within the context of learning about reading and writing digital texts. By juxtaposing the educational technology course with methods courses and field experience, we seek to provide preservice teachers with models beyond our own. Research indicates that preservice teachers greatly benefit from experiencing effective technology integration in the field and tend to disregard practices taught at the university if they are not used in real classroom settings (e.g., Korthagen & Kessels, 1999). Specific examples of the ways in which methods faculty and cooperating teachers modeled the use of technology are presented in the Findings section. 5.5.4. Authentic experiences According to Tondeur et al. (2012) many teacher education programs emphasize the importance of providing preservice teachers with authentic technology experiences, including increased opportunities for hands-on work. In our course, preservice teachers have repeated opportunities to engage with technology including hands-on activities utilizing graphic organizers, Internet inquiry, and web 2.0 tools. Additionally, preservice teachers apply their learning into practice by implementing technology-integrated lessons designed in our course into their field placement classroom. This exercise enables them to witness use of technology first hand with their own students. This is another example of how the integration of the educational technology course with field experience proves beneficial. 5.5.5. Reflection To support the development of TPACK several researchers emphasize the importance of designing, implementing and reflecting on teaching experiences that incorporate appropriate technologies. Mishra et al. (2010) propose spiraling stages of more complex instructional design activities where TPACK reflection is completed at the end of the process. This type of metacognitive reflection allows preservice teachers to consider technology, content and pedagogy and their inter-relationship when considering specific instructional problems. During their participation in the educational technology course, we make it possible for preservice teachers to reflect on their ideas, beliefs and experiences with respect to technology through a case development process (Mouza & Karchmer-Klein, 2013). The process of case development progresses incrementally throughout the semester and is divided in various stages where preservice teachers design or identify a curricular-based technology-integrated lesson, enact the lesson in their field practicum, and write a reflective case on the implementation and outcomes of the lesson. To facilitate the process of reflection, a series of writing and reflection prompts are provided that help preservice teachers engage in systematic and detailed analysis of their practice. The process of case development heavily depends on the juxtaposition of educational technology coursework, with methods courses, and field experience as it allows preservice teachers to bring together the knowledge bases of technology, content and pedagogy and provide particular contexts for enactment and reflection in real classrooms. 212 C. Mouza et al. / Computers & Education 71 (2014) 206–221 6. Methods 6.1. Participants Participants included 88 preservice teachers enrolled in 4 sections of the required educational technology course offered in conjunction with methods courses and field experience during one semester. All sections utilized the same syllabus, which instructors developed and refined collaboratively. Further, instructors worked together throughout the duration of the study to ensure fidelity of implementation. One section was taught by the first author, two sections were taught by the second author and a fourth section was taught by a third instructor. All participants majored in elementary teacher education (i.e., ETE majors) and were typical undergraduate students in their early 20s. Further, all participants concentrated in a middle school content area such as English (14.8%), mathematics (9.9%), science (16%), social studies (9.9%), or special education (58%). Of the 88 students 95% were female and 5% were male. More than half were juniors (58%) while the rest were seniors (42%). 6.2. Data collection 6.2.1. TPACK survey Data were collected through the Survey of Preservice Teachers’ Knowledge of Teaching and Technology, a survey instrument developed around the TPACK framework (Schmidt et al., 2009). This instrument is the most universally used survey designed specifically for preservice teachers majoring in elementary education and focuses on the content areas that they would be teaching in their future classrooms. The survey emphasizes teachers’ self-assessed levels of knowledge in each of the TPACK domains. The survey was administered to all preservice teachers twice; at the beginning (first class) and end (last class) of the course. It was administered online and response rates were 100% for the initial survey and 91% for the follow-up survey. The survey consists of 8 items designed to collect demographic data and 47 items focusing on preservice teacher knowledge (TK, CK, TCK, PK, PCK, TPK, and TPACK).1 These items use a 5-point Likert-scale response format that ranges from Strongly Disagree to Strongly Agree with additional space for open-ended comments. Additionally, 3 open-ended questions ask preservice teachers to describe specific episodes where university faculty, cooperating teachers in their field placement, or themselves effectively combined content, technologies and teaching approaches in a classroom lesson. Participants are asked to include in their description what content was being taught, what technology was used, and what teaching approaches were implemented. Although the survey instrument was found to be reliable by earlier studies (Schmidt et al., 2009), we also assessed the reliability for our sample. The Cronbach alpha of a scale should be greater than 0.70 for items to be used together as a scale (Nunnally, 1978). Results of our analyses of Likert-scale responses revealed that all subscales and the instrument as a whole achieved alpha levels exceeding 0.70 indicating that the survey was reliable (see Table 2). 6.2.2. Performance assessment: case reports To triangulate survey data and gain a better understanding of preservice teachers’ TPACK development we have also examined qualitative data collected through performance assessments completed in the educational technology course. A key assignment of the course engages preservice teachers in a case development project, which progresses through stages that allow participants to design, enact and reflect on their own technology-integrated experiences in their field placement. To culminate the project, participants write a reflective case report following a series of writing and reflection prompts that scaffold inquiry and reflection. Each case report is divided in two sections; a case narrative and a case reflection and is approximately 1000 words. For the purpose of this study, we have collected all case reports and associated artifacts in order to examine the way in which preservice teachers’ recognized and discussed interactions among technology, content and pedagogy in their own practice. Although cases are also selfreported, they provide important insights into preservice teachers’ strategic thinking in the process of planning, organizing and implementing their lessons (Leatham, 2008). Such data are essential for considering participants’ knowledge development as well as application of knowledge into practice. Given the volume of the data, a stratified random sample of 22 case reports (25% of the data) was selected that represented approximately 5 cases from each section of the educational technology taught. This selection ensured that our sample was representative of the larger population of preservice teacher participants. 6.3. Data analysis 6.3.1. TPACK surveys Survey data were analyzed using both quantitative and qualitative methodologies. Quantitative analyses included descriptive statistics, ttests and reliability analyses. Likert-scale items were initially scored based on guidelines provided by Schmidt et al. (2009) and subsequently were exported into Excel and the Statistical Package for the Social Sciences (SPSS) where means and standard deviations were calculated for each knowledge component and for TPACK as a whole. To test for the significance of the gain score (post measure-pre measure), a repeated measures t-test was conducted on each of the scales. Open-ended responses describing specific episodes where university faculty, cooperating teachers in their field placement, or preservice teachers themselves effectively combined content, technologies and teaching approaches in a classroom lesson were analyzed using inductive approaches (Glaser & Strauss, 1967). Specifically, initially all preservice teachers’ responses to the three open-ended questions in both the pre and the post administration of the survey were read repeatedly by two researchers to generate a starting list of episodes that demonstrated specific uses of technology with content and pedagogy. Subsequently, using the constant comparative method (Miles & 1 The original survey includes 55 items. However, 8 items were not relevant to our work and although we have administered the survey as a whole, analysis for those items is not reported in this manuscript. Those items were considered when calculating the reliability of the instrument as whole. 213 C. Mouza et al. / Computers & Education 71 (2014) 206–221 Table 2 Reliability of the survey scales. Scale Number of items Cronbach’s alpha TK CK PK PCK TCK TPK TPACK Total items 7 12 7 4 4 5 8 55 0.885 0.820 0.838 0.849 0.856 0.726 0.884 0.932 Huberman, 1994), redundant themes or themes that fit together conceptually were combined and less frequently represented themes were eliminated. This approach, helped identify common themes that cut across participants’ responses. Specifically, the analysis revealed that participants discussed content and pedagogy primarily in relation to three types of technologies: (a) digital content (e.g., videos, websites, and online content); (b) presentation technologies (e.g., PowerPoint, LCD projectors, document cameras, interactive whiteboards); and (c) content-specific software. 6.3.2. Case reports To identify evidence of TPACK domains in student planning and decision making, we turned to the 22 selected case reports. Specifically, following an a priori coding scheme guided by existing literature and refined through earlier studies (see Table 3; Mouza, 2009, 2011), we looked to identify evidence of TK, TPK and TPACK in participants’ case reports. Those constructs are of particular importance to researchers in educational technology because they mark a distinctive move from general technology integration (TPK) to content-specific technology integration (TPACK; Graham, 2011). Using this coding scheme, two researchers independently coded all case narratives. Inter-rater reliability was calculated at 90%. All differences were discussed and resolved through consensus. Once all data were coded, percentages were calculated in order to quantify emergent patterns. Additionally, we have made a qualitative assessment on the level of TPACK in relation to teaching represented in preservice teachers’ case reports utilizing descriptors provided by Niess et al. (2009) that included: (a) recognizing: technology is used for rote activities; (b) accepting: technology is peripheral to classroom instruction; (c) adapting: technology is used to reinforce student learning; (d) exploring: technology supports exploration of concepts and high-level thinking; and (e) advancing: a variety of technology tools are used consistently to facilitate deep understanding of content. 7. Findings 7.1. Preservice teachers’ knowledge development Scores on each of the scales associated with the Survey of Preservice Teachers Knowledge, TK, CK, PK, PCK, TCK, TPK, TPACK, and the TOTAL (total score on all items) were computed for each participant at the beginning of the course (pre measure) and at the end of the course (post measure). To test for the significance of the gain score (post measure-pre measure), a repeated measures t-test was conducted on each of the Table 3 Coding scheme representing TPACK. Technological knowledge (TK) Evidence: Operating computer hardware Using standard software tools (e.g., MS Word, PowerPoint, Internet browsers, email) Installing and removing peripheral devices (e.g., USB drives, microphones) & software Troubleshooting equipment Using appropriate vocabulary (e.g., technology terms) Technological pedagogical knowledge (TPK) Evidence: Motivating students through technology Differentiating instruction when technology is used Ability to organize collaborative work with technology Holding students accountable for equipment used Developing strategies for assessing student work with technology Knowing about the existence of a variety of tools for particular tasks Knowing about the time required to teach with particular technologies Ability to envision potential student problems with particular technologies and plan relevant activities to support those students Generating alternatives in the event of technological failures Ability to explain a computer procedure to students (e.g., through modeling) Technological pedagogical content knowledge (TPACK) Evidence: Use of technology to facilitate subject-specific pedagogical methods (e.g., science inquiry, primary sources in social studies, etc.) Use of technology to facilitate content representation Use of technology to address learner content understanding (e.g., prior content knowledge, address misconceptions, improve content understanding) 214 C. Mouza et al. / Computers & Education 71 (2014) 206–221 Table 4 Preservice teachers’ TPACK. Subtests # of Items Mean for pre survey Mean for post survey Mean difference Std. deviation t df P sig. (2 tailed) Effect size TK CK PK PCK TCK TPK TPACK TOTAL 7 12 7 4 4 5 8 55 3.367 3.879 3.794 3.619 3.253 3.810 3.555 3.637 3.660 4.018 4.251 4.119 3.929 4.241 4.008 3.982 0.293 0.139 0.458 0.500 0.676 0.431 0.453 0.345 0.448 0.354 0.502 0.620 0.698 0.481 0.596 0.301 5.809 3.498 8.094 7.212 8.72 7.903 6.672 8.513 78 79 78 79 80 77 76 54 0.000 0.001 0.000 0.000 0.000 0.000 0.000 0.000 0.65 0.39 0.91 0.81 0.97 0.89 0.76 1.15 scales. These results are tabulated in Table 4. As seen in Table 4 there is a significant gain (P < 0.05) on all scales. The last column of Table 4 reports the effect size for all scales. The effect size denotes the increase in the mean score in standard deviation units. For example, for scale TK, the effect size is 0.65, which indicates that the average score on TK has improved by 0.65 standard deviation units. This is considered a large increase.2 For all scales that have shown significant improvement, the effect size is large except for CK where the effect size is medium. This finding is not surprising since neither the educational technology course nor the methods courses or field experience focused directly on CK. In contrast, improvements in PK and TK and their interactions with other knowledge domains (PCK, TCK, TPK and TPACK) can be largely attributed to the integrated approach we employed that allowed participants simultaneous enrollment in methods courses and accompanied field experience through carefully designed activities. As one participant noted: “My methods classes this semester have boosted my confidence in specific subject areas such as social studies and literacy.” Similarly, commenting on advances in her PK one participant noted: “From my experiences in the field, I have especially learned how to manage a classroom.” These knowledge bases are essential when thinking and planning technology-integrated lessons. 7.2. Modeling of TPACK by university faculty, cooperating teachers, and preservice teachers 7.2.1. Modeling of TPACK by university faculty When participants were asked to describe a specific episode where a university faculty effectively combined content, technologies and teaching approaches all participants were able to provide such descriptions. The majority of the examples presented described instances where methods faculty utilized digital content such as video clips and online resources (e.g., videos, website, etc.) related to the topic at hand. Describing an observed use of technology in social studies one participant explained: In my social studies the instructor demonstrated a way of using teaching artifacts. Each group was given an old object and was asked to make observations and predictions as to what it represented. After filling out a worksheet and discussing our ideas in groups, we were given the opportunity to investigate the artifact online. For example, some of the artifacts had the names of manufacturing companies printed on them so we could search those names online and see if we can get any hints as to the function of the item. This model was effective because we were able to participate in an investigation/inquiry, while learning about history at the same time. A sizable number of participants described instances where methods faculty used presentation technology, such as PowerPoint, to create a more interactive learning experience while a smaller number of participants described the use of content-specific software such as a spinner game used to teach probability, an interactive geometric board, the use of graphic organizers and electronic books in literacy, the use of Global Positioning System (GPS) and video conferencing software in social studies, and the use of scientific probeware in science. A participant explained: In my science class the professor modeled how to use probeware in middle school classrooms. We learned that during investigation about the human body and the circulatory system, the students could use a heart rate probe to learn how the heart works during various activities. 7.2.2. Modeling of TPACK by cooperating teachers Despite efforts to place preservice teachers in classrooms that model effective use of technology, our findings indicated challenges in this area. Findings from this work demonstrated a somehow narrow application of technology in teaching and learning. When asked to describe a specific episode where a cooperating teacher combined content, technologies and teaching approaches the majority of participants discussed uses of digital content. In a representative example on the use of digital content a participant explained: In a third grade classroom the students used the Internet to learn about the Titanic. First they were assigned a person who was on board the Titanic and researched that person on websites given to them by the teacher. Later, they used technology to watch a documentary about the Titanic. The students really enjoyed this lesson and wanted to learn more about the Titanic after the lesson was through. Further, a large number of participants reported on the use of presentation technologies including LCD projectors to display PowerPoint presentations, document cameras (e.g., Elmo), and interactive whiteboards. Content-specific uses of technology were more limited with examples focusing on the use of graphic calculators in mathematics, integrated learning software in literacy and mathematics such as Accelerated Reader (http://www.renlearn.com/ar/) and Accelerated Math (http://www.renlearn.com/am/), and the use of global positioning 2 Effect size < 0.3 is small, 0.3–0.5 is medium, and >0.5 is large (Cohen, 1988). C. Mouza et al. / Computers & Education 71 (2014) 206–221 215 systems (GPS) in social studies. Content-specific applications of technology were more prominent in the post survey responses. A possible explanation for this finding is that preservice teachers were more likely to notice content-specific uses of technology after their exposure to the TPACK framework. Overall, however, it is important to note that few episodes demonstrated seamless integration of technology, content and pedagogy as envisioned by the TPACK framework. One of the most successful examples was described as follows: My cooperative teacher used a lot of technology in the classroom. One lesson focused on the use of linear equations for a table of data values. Each student had a graphing calculator, tried to come up with an equation, and sent it to the Smartboard so all of the equations were graphed on the board at once. Then the class discussed which equations were correct and which ones were not. They also discussed how they could change them to correct them. 7.2.3. Modeling of TPACK by preservice teachers Findings indicated a fairly limited understanding of TPACK among preservice teachers at the beginning of the study. When asked to describe a teaching episode in which they effectively demonstrated combining content, technologies and teaching approaches at the beginning of the educational technology course, the majority of the open-ended survey responses (57%) revealed no opportunities to teach such a lesson. This percentage is consistent with the percentage of participants completing their first methods field experience (special education track) indicating that preservice teachers do not have opportunities to demonstrate those practices early in their teacher education program. Participants in the content area track were completing their second methods field experience and were able to describe how they combined content, technologies and teaching approaches. Those examples, however, focused primarily on the use of digital content such as online resources, presentation technologies such as the use of PowerPoint, and even more primitive tools such as overhead projectors. One participant reported characteristically: My teaching partner and I used a PowerPoint presentation for a social studies lesson on landforms. That is the closest I have come. Honestly, I had enough to worry about teaching my first-ever classroom lesson. I didn’t want to add another variable that might malfunction. In contrast, post survey responses paralleled significant improvements in preservice teachers’ TPACK documented in the survey data (see Table 4). When asked to describe a teaching episode in which they effectively demonstrated combining content, technologies and teaching approaches at the end of the course all participants provided such examples. The majority of the participants implemented one of the activities they designed in their educational technology course. The remaining participants implemented a technology-integrated lesson identified and modified from a lesson-plan portal (i.e., Thinkfinity). As discussed earlier, implementation of a technology-integrated lesson was required and so was reflection upon that implementation. The majority of the examples described focused on the use of digital content. Instead of simply describing the use of a particular website or video clip, however, many participants (35%) described a web-based inquiry activity designed during their participation in the course, which utilized digital resources with content and pedagogy. Describing the implementation of such activity a participant characteristically explained: I was able to implement the web-based inquiry activity developed in the technology class with students in seventh grade. The activity required students to visit a number of websites identified for them and learn more about plant and animal cells. Then they had to complete a worksheet where they defined each of the cell’s organelles and a venn diagram where they compared and contrasted the cells. In the end, they wrote a compare and contrast essay. Other participants described inquiry activities that placed students in the role of historians learning more about immigrant’s life and transition through Ellis Island or the role of scientists learning more about flower dissection through interactive online activities. A sizable number of participants described the use of presentation technologies including interactive whiteboards, and content-specific software such as the use of graphic organizers to write persuasive essays. Overall, the practices described by participants at the end of the course often mirrored the types of practices observed in their methods courses and field experience. Yet, they also revealed that participants were able to bring together different knowledge bases to design and implement lessons that integrated technology with specific content and sound pedagogical practices. 7.3. Representation of TPACK in preservice teachers’ practice To triangulate preservice teachers’ self-report survey data and gain a closer understanding of the ways in which participants recognized and demonstrated relationships among technology, content and pedagogy (TPACK) we turned to participants’ reflective cases where they reported on the design and implementation of a technology-integrated lesson in their field placement. In our analysis we looked to identify evidence of TK, TPK and TPACK in participants’ case reports – constructs that were of particular importance to the objectives of this study (see Fig. 2 and Table 5). Additionally, we looked to identify whether there were differences in how TPACK was discussed and represented in participants’ narratives and the ways in which the integrated approach influenced TPACK development. As shown in Fig. 2 and Table 5, our analysis indicated that case reports provided ample evidence of TK, TPK and TPACK. This finding is consistent with our survey results, which demonstrated significant improvements in participants’ knowledge development over the course of the study. Below we present the major themes that cut across participants’ responses related to TK, TPK and TPACK and their application in practice. 7.3.1. Role of TK Survey data indicated that preservice teachers experienced significant growth in their TK over the course of the study. Analysis of preservice teachers’ cases further indicated that all reports provided evidence of TK. This is important because one of the primary objectives of the educational technology course was to improve preservice teachers’ TK. When discussing TK, preservice teachers provided examples of both their own and their students’ knowledge with respect to technology (see Table 5). They also identified technical difficulties encountered during the implementation of their lesson and discussed issues of access by describing the technology resources available at 216 C. Mouza et al. / Computers & Education 71 (2014) 206–221 Fig. 2. Representation of TPACK in preservice teachers’ case reports. their field placement. Discussing his own TK, for example, one participant noted how he had to work his way around the use of concept mapping tool, Inspiration, introduced during the educational technology. He noted: “I had completed my concept mapping project a long time ago and thus forgot how to maneuver around Inspiration. Specifically, I could not remember how to draw the arrows and it took me a few seconds to get acquainted with the program again”. While discussing their students’ TK, preservice teachers often noted their students’ strengths and weaknesses in relation to technologies introduced in their lesson. One participant, for example, explained: “Because the students were both familiar with the Internet and the general operating system, it did not take much time to introduce the technology. Once we worked on the Internet, however, I helped them develop familiarity with the different features of the website”. The majority of the participants noted that no major technical difficulties were encountered during the implementation of their lesson, which was often surprising. For example, one preservice teacher noted: “There were no issues with the Internet working properly or the sound being turned on and loud enough on the computers. This allowed the lesson to run more smoothly”. Although it is natural to expect that preservice teachers will exhibit TK after their participation in an educational technology course, findings indicated that TK was not the dominant form of knowledge represented in case reports. Rather, technology was more likely to be represented in relation to pedagogy (TPK) as well as content and pedagogy (TPACK). This finding is important because it demonstrates that issues related to TK did not dominate preservice teachers’ case reports. Rather, these issues were more likely to be situated in the context of the TPACK framework introduced and modeled throughout our integrated approach. 7.3.2. Role of TPK Analysis of preservice teachers’ case reports indicated the prevalence of the TPK construct. This finding is consistent with survey results that demonstrated a large increase in participants’ TPK over the course of the study. As shown on Table 5, across the 22 case reports we coded 86 instances of TPK which represented 48% of all coded constructs (see also Fig. 2). These instances revealed that preservice teachers frequently modeled TPK practices observed and experienced in their educational technology course. For example, prior to introducing a new Table 5 Coding examples and number of coding instances. Construct Coding example Number of coding instances Mean SD TK The only difficulties I had with technology were at the end of the lesson when I tried to print what the students had created from the website. It asked if I would like to print it, but everything came out with blank questions when I tried. Although it may be tempting to step in and “show students how things work,” they need to be able to experiment (safely and appropriately) with technology. I wanted to make sure students completed the activity and had a chance to play the interactive game on the Internet because I knew they would enjoy it and I could help them answer questions if they were unsure about things. If I were to teach this again, I would break the lesson up over a few days and make sure to reserve a computer lab in advance. This would provide student with an opportunity to explore the website at their own pace and in more detail. I bookmarked the website that we were going to be using on all of the computers so it would be easy for the students to access. The major goals for this lesson were to help students identify different forms of energy and how these energy forms are transferred and transformed in energy phenomena. Students constructed a graphic organizer representing the flow of energy from one point to another (adapting level). Students used an inquiry-oriented activity on flowers and their functions. Using a collection of Internet resources, they learned the parts of the flower and the jobs of each part. Then, during the second part of the lesson, students applied the information gathered the first day to understand the process of pollination and identify at least one pollinator and its role in the process. These activities were completed in preparation of a flower dissection. The background knowledge allowed students to focus on the dissection and the interconnectedness of all the parts rather than worry about new terms (exploring level). 37 1.7 1.4 86 3.9 1.8 55 2.5 1.3 TPK TPACK C. Mouza et al. / Computers & Education 71 (2014) 206–221 217 technology to students, preservice teachers often modeled its use by creating a classroom product much like a practice experienced in their educational technology course. One participant explained: I knew that students would not be able to make a concept map by themselves right away, so I made one on the teacher computer and projected it on the screen in front of the classroom. After we read a book, I started with the main concept that included the word Animals and then asked the students if they could remember the five animal groups discussed in the book. I also asked them to name animals that fit with each category. As they provided the information, I filled out the bubbles on the graphic organizer. Interestingly, this activity parallels one conducted in the educational technology course when introducing the use of graphic organizers through the tool Inspiration. Similarly, other preservice teachers discussed how they often introduced their selected technology to students, provided instructions, and then let them explore while they simply walked around the room observing and helping as needed. This activity also parallels the ways in which educational technology faculty introduced new technologies in the course. Further, preservice teachers’ reports provided evidence that practices associated with TPK, were influenced by what they observed and experienced in their field placement context. For example, in trying to envision her students’ skills prior to the implementation of her lesson, one preservice teacher explained: My expectations of the student’s technology skills were not very high. In the three weeks that I had been in my field placement classroom, the only time I had ever seen my students use a computer was once a week during their computer lab special. My students had never used the computers in the back of the room, which I used for this lesson, while I was there. I expected them to be somewhat familiar with computers from their weekly special, but I also expected that they would need a lot of guidance. Other students, however, discussed the implementation of practices modeled by their cooperating teachers. One preservice teacher noted: “When I implemented my lesson, I started by going onto the main teacher computer and walking students through exactly what they were going to be doing. This is a technique that their computer teacher told me that she normally does with them when they have computer lab”. 7.3.3. Levels of TPACK Consistent with survey results that demonstrated large increase in preservice teachers’ TPACK over the course of the study (see Table 4), all 22 case reports exhibited instances of TPACK. As shown on Table 5, we have coded 55 TPACK instances representing 31% of all TPACK constructs. Despite significant growth in preservice teachers’ knowledge development, findings indicated that there was variability in the ways in which TPACK was represented in their case reports. This finding is not surprising given that preservice teachers acquire TPACK progressively and do not suddenly display this knowledge in their practice (Niess et al., 2007). Considering the five levels of TPACK discussed by Niess et al. (2007, 2009) in relation to participants’ teaching, our analysis also demonstrated growth in participants’ TPACK. Qualitative survey results, for example, indicated that at the beginning of the course many teachers (57%) had no opportunities to teach lessons that effectively combined technology with content and pedagogy. Although participants acknowledged the important role of technology they had difficulty using it to support student learning without prior experiences. Even when preservice teachers described TPACK instances from their practice, the examples provided focused primarily on uses of technology tools independent of the content and pedagogical strategies employed (e.g., I used PowerPoint). These findings indicated that as a group, preservice teachers recognized the value of technology but exhibited limited understandings and application of technology in classroom contexts in connection with content and pedagogy. Analysis of case reports indicated that all participants accepted the value of technology in teaching and learning as they incorporated it to address specific content in their field placement classrooms. Although in some cases activities were peripheral to instruction, preservice teachers acknowledged the importance of technology in reinforcing concepts covered in class and discussed future plans in relation to their use of technology. A preservice teacher who used computer games to help reinforce mathematics skills previously addressed in class, noted: I was surprised with how well the students did with the math. When at their desks in their classroom, they struggle to get the math done and are very distracted. In the computer lab, while playing fun and interactive games, students were able to solve problems much more easily and quickly. They made fewer mistakes than they normally do on their worksheets without computers. At the same time, the preservice teacher continued to discuss the use of technology in relation to content, noting: “The math games did not take away from classroom instruction because they allowed students to practice the same skills they normally do solely on paper”. Further, our analysis indicated that the majority of the participants were able to progress to the adapting level (n ¼ 17) but only a small number of participants progressed to the exploring level (n ¼ 5). Our data did not support analysis beyond the exploring level since case reports only presented one instance of technology-integrated practice. As such, we could not provide evidence consistent with the characteristics of advancing, such as active and consistent acceptance of technologies as tools for teaching and learning. Below we provide two examples that illustrate the TPACK knowledge at the adapting and exploring levels as illustrated in preservice teachers’ case reports. Briana’s case: adapting level of TPACK. Briana’s case was situated in a middle school in a sub-urban area. Briana implemented a lesson on the transformation of energy in science. This was a topic the students in her field placement were studying at the time. Her lesson, titled How Energy Moves in Liquids had two major objectives: (a) students will understand that if the temperature of a liquid rises, the amount of energy in the liquid will also rise; and (b) students will understand that the temperature of a liquid affects how quickly/slowly food coloring will disperse in that liquid. To accomplish her objectives Briana identified and used two pieces of technology, a document camera (Elmo) and an interactive whiteboard (i.e., SmartBoard) to demonstrate how energy moves in liquids. Briana noted that students were familiar with the use of the interactive whiteboard because they had used it in the past to create graphs and diagrams. To begin the activity, Briana used two beakers to place a drop of red food coloring in hot water and a drop of blue coloring in cold water. Using the document camera she allowed students to observe her actions on the interactive whiteboard while simultaneously a student volunteer used the interactive pen on the whiteboard to draw circles pointing to the areas in which the food color had spread. Additional volunteers performed the same activity 1 min and 2 min later to show change in the dispersion of the coloring and point out that the cold water still had huge areas of clear water compared to the hot water indicating that the food color was dispersing slower in cold water. 218 C. Mouza et al. / Computers & Education 71 (2014) 206–221 In this activity, Briana demonstrated knowledge of using the document camera and interactive whiteboard (TK) to demonstrate a science investigation and scaffold classroom discussion. In identifying appropriate tools, she also considered her students’ prior knowledge with technology and their prior familiarity with the use of the interactive whiteboard (TPK). Further, Briana demonstrated the ability to weave technology with content-specific activities required to facilitate student learning in her field placement (TPACK). Briana’s TPACK is consistent with the adapting level because she used technology to enhance or reinforce ideas that students had previously learned. Despite that, Briana’s instructional strategies were primarily teacher-directed. Students were asked to observe the teachers’ actions but did not have opportunities to engage with the technology. Kate’s case: exploring level of TPACK. Kate’s case was situated in a sub-urban elementary school. The major objective of the lesson was for students to understand the lives of pilgrims living in Plymouth during the first Thanksgiving. The students were asked to use an interactive Internet resource (http://www.scholastic.com/scholastic_thanksgiving/) that allowed them to assume the perspective of the Pilgrims traveling to the new world on the Mayflower as well as take a virtual tour of the Mayflower. They also conducted research to identify information related to the trip on the Mayflower, the ways in which Pilgrims interacted with the natives, and the events that transpired over the course of the Pilgrim’s first year living in Plymouth. Finally, students created timelines with major events of the Pilgrims first year in the New World and wrote a letter from the perspective of the Pilgrims living in Plymouth during the first Thanksgiving. In this activity, Kate demonstrated knowledge of using the Internet (TK) to facilitate student research. In doing so, she considered her students prior knowledge of both technology and content (TPK), such as their ability to use the Internet as a resource, navigate through a website, and use word processing programs as well as their understanding of what happened between the Pilgrims and the Native Americans after the Pilgrims arrived in America. Additionally, Kate demonstrated the ability to integrate interactive Internet resources with content-specific activities and representations required to facilitate student understanding (TPACK). Unlike Briana’s case, Kate engaged students in higher-level thinking activities that included inquiry and decision making. Kate, placed technology in the hands of her students allowing them to explore and engage with interactive web-based activities and resources that improved their understanding of content. As Kate noted, her students were surprised to find out about the length of the Pilgrim’s voyage, the different parts of the Mayflower, and the different living arrangements while traveling. Kate’s case acknowledged the importance of interactive online resources in engaging and supporting student learning. Although at this point, we have no evidence that Kate will continue exploring and incorporating technologies for numerous topics, we have beginning evidence to indicate that Kate has approached the exploring level. 8. Limitations Two limitations are evident in this work. First, data were collected from self-reported measures such as surveys and case reports. Although case reports provide rich accounts of preservice teachers’ thinking in the process of planning, organizing and implementing curricular uses of technologies they are still generated from the perspective of the participants. Nevertheless, we believe that the combination of those two sources provided a reliable measure of preservice teachers’ TPACK over time. Second, recent empirical work with the Survey of Preservice Teachers’ Knowledge of Teaching and Technology, has questioned its construct validity. Two studies that used a modified version of the survey indicated that the knowledge domains of TPACK could not be reproduced through exploratory factor analysis, indicating that teachers may have difficulty distinguishing among TPACK constructs (Archambault & Barnett, 2010; Koh, Chai, & Tsai, 2010). Similar results are reported by a recent study that examined the construct validity of the authentic version of the survey (Shinas, Yilmaz Ozden, Mouza, Karchmer-Klein, & Glutting, 2013). Using factor analysis and a large sample of preservice teachers in the U.S. this study also revealed that participants did not always make conceptual distinctions between the TPACK domains. Despite that, the Survey of Preservice Teachers’ Knowledge of Teaching and Technology is the most mature instrument developed to date that can be used to collect quantitative data among large samples of preservice teachers, and as such this study makes an important contribution to the field. 9. Discussion, conclusion and implications This study examined how and to what extent participation in an integrated approach that juxtaposed an educational technology course with methods courses and field experience through careful instructional design, influenced preservice teachers’ TPACK development and practice. Quantitative and qualitative data collected through the Survey of Preservice Teachers’ Knowledge of Teaching and Technology and case reports revealed that participants experienced significant gains in all TPACK constructs. Effect sizes were large in all areas except CK, which was not particularly targeted in this approach. Further, all participants were able to apply their learning to classroom practice as they implemented technology-integrated lessons in their field placement classrooms and reflected on their experience. Specifically, qualitative data indicated that preservice teachers progressed from recognizing to accepting levels of TPACK with the majority moving to the adapting level and few progressing to the exploring level. Our findings indicated that the educational technology course, methods courses and field experience collectively exposed preservice teachers to a variety of TPACK models that contributed to their learning and subsequent practice in their field placement classrooms. Nevertheless, evidence from participants’ case reports revealed important insights as to how such learning occurred. Specifically, it was apparent that the greatest source of TK was the educational technology course. Cooperating teachers were instrumental in exposing preservice teachers’ to interactive whiteboards such as the SmartBoard but for the most part, preservice teachers attributed their TK to the activities conducted in the educational technology course. This finding continues to reinforce the value of educational technology courses in developing preservice teachers comfort level with technology. Similarly, cooperating teachers appeared to contribute primarily toward PK. In many cases, preservice teachers discussed general pedagogical practices as well as pedagogical practices related to technology (TPK) observed in their field placement classroom (e.g., classroom management, monitoring technology use) and the ways in which they influenced their own application of technology in practice. Chai, Koh, and Tsai (2010) demonstrated that PK has the largest impact on technology use in the classroom and thus the field experience can foster growth in pedagogy that helps develop TPACK. Finally, methods courses contributed primarily toward the development of PCK as they provided strategies for teaching in one’s content area as well as TCK as they discussed and modeled interactions among content and C. Mouza et al. / Computers & Education 71 (2014) 206–221 219 technology. In their open-ended survey responses participants reported a number of content-specific uses of technology and the ways in which they influenced their own thinking in teaching within the specific discipline. The above findings support an existing body of literature indicating that educational technology courses can provide a strong foundation of technology skills and a good overview of the use of technology in teaching (Kay, 2006). Further, findings indicate that when taken in conjunction with methods courses and field experience stand alone educational technology courses can significantly influence preservice teachers’ ability to combine content, pedagogy and technology in the design and implementation of technology-integrated lessons (TPACK). Analyzing evaluation reports from a number of projects funded by the federally funded project Preparing Teachers to Teach with Technology (PT3), Polly et al. (2010) also found that without methods courses and field experiences, preservice teachers are left with technology skills but limited understanding about how to implement technology into their classroom. Yet, limited empirical evidence was provided to support this claim. In this work, we articulated ways in which such integration can leverage preservice teachers’ learning and practice. Two challenges were evident in our efforts to support preservice teacher learning through an integrative approach. The first challenge was related to the ways in which university faculty combined content, technologies and teaching approaches. Although findings indicated that participants had opportunities to observe practices where university faculty combined content, technologies and teaching approaches, the majority of practices observed focused on the use of digital content and presentation technologies. Further, in many instances participants reiterated the same observed models indicating that only a small number of faculty actually modeled how to combine content, technologies and teaching approaches effectively in their instruction. Opportunities to observe the implementation of technology-rich units into methods courses is important because they help preservice teachers develop positive attitude toward technology use in the classroom and acquire technology skills that are more strongly connected to use in K–12 settings (Pierson & Thompson, 2005; Tondeur et al., 2012). Niess (2012) urges teacher educators to re-think methods courses in ways that emphasize and provide learning experiences at the intersection of content, pedagogy and technology. Given the important role of modeling, administrators and teacher education programs might have to allocate more resources to methods faculty interested in modeling effective technology use, such as time for designing technology-rich instructional units, mentoring, and content-specific software or resources. The second challenge was related to field placements where frequently access to technology was limited and cooperating teachers provided few examples of effectively integrating technology, content and pedagogy. When participants were asked to describe practices where their cooperating teachers combined content, technology and teaching approaches in their field placement most responses concentrated again on the use of digital content and presentation technologies. This finding parallels similar accounts reported by Project Tomorrow (2011) on K–12 teachers’ use of technology. Specifically, in a national survey of K–12 educators Project Tomorrow found that teacher use of technology remained sporadic or lesson dependent. This finding is also consistent with existing literature pointing to difficulty in identifying technology-rich field experiences for preservice teachers and cooperating teachers with an understanding and facility in teaching their subject matter with appropriate technologies (e.g., Niess, 2012). Reasons for this difficulty often include lack of technological resources, school culture that does not value innovation, schools’ reluctance to participate in professional development projects, and a lack of alignment between technology use in teacher education programs and K–12 schools (Hammond et al., 2009; Polly et al., 2010). Studies documenting preservice teachers’ learning in technology-rich field placements, however, reported promising outcomes including positive attitudes toward technology, frequent use of technology, and more instances of preservice teaching using technology to support student learning (Strudler, Archambault, Bendixen, Anderson, & Weiss, 2003). As a result, teacher education programs must engage in more concerted efforts of identifying field placements rich in technology resources and models that integrate technology, content and pedagogy. A major strength of the integrated approach was the opportunity it provided preservice teachers to design, enact and reflect upon the implementation of a technology-integrated lesson in a real classroom. Although in many instances, preservice teachers described practices that paralleled those observed in their university and field placement classrooms, enactment and reflection was key to helping participants recognize and accept use of technology. Further, some preservice teachers were able to progress to higher levels of TPACK development, such as those associated with the adapting and exploring levels, and discuss future plans related to technology use. Voogt et al. (2012), found that preservice teachers are generally more likely to design rather than enact technology-rich lessons. Findings from this work indicate that teacher education programs should include opportunities for the enactment of technology-integrated lessons. Setting such expectations has proven effective in getting preservice teachers to use technology during clinical preparation and in developing the knowledge, skills and dispositions consistent with the TPACK framework (Dexter & Riedel, 2003; Niess, 2005). In conclusion, this work provided a detailed account of an integrated instructional approach that can help preservice teachers advance their TPACK, and the ways in which such integration can be used to leverage preservice teacher learning. This account includes descriptions of the actual technology topics and assignments addressed in the educational technology course and the types of technology uses observed in methods courses and field experiences. Our findings indicate that this approach holds promise for advancing preservice teachers’ TPACK as illustrated in both the quantitative and qualitative data. At the same time, it points to the need of providing preservice teachers with more diverse TPACK models that extent beyond the use of digital content and presentation technologies. Arguably this is difficult to be accomplished outside a subject-specific teacher preparation program like the one in this study (Niess, 2012). Findings from this work help establish an empirical basis related to best practices in TPACK development. References Agyei, D. D., & Voogt, J. M. (2011). Exploring the potential of the will, skill, tool model in Ghana: predicting prospective and practicing teachers’ use of technology. Computers & Education, 56, 91–100. Albion, P., Jamieson-Proctor, R., & Finger, G. (2010). Auditing the TPACK confidence of Australian pre-service teachers: the TPACK confidence survey (TCS). In D. Gibson, & B. Dodge (Eds.), Proceedings of Society for Information Technology & Teacher Education international conference 2010 (pp. 3772–3779). Chesapeake, VA: AACE. Angeli, C., & Valanides, N. (2005). Preservice teachers as ICT designers: an instructional design model based on an expanded view of pedagogical content knowledge. Journal of Computer-Assisted Learning, 21(4), 292–302. Angeli, C., & Valanides, N. (2009). Epistemological and methodological issues for the conceptualization, development, and assessment of ICT-TPCK: advances in technology and pedagogical content knowledge (TPCK). Computers & Education, 52, 154–168. Archambault, L. M., & Barnett, J. H. (2010). Revisiting technological pedagogical content knowledge: exploring the TPACK framework. Computers & Education, 55, 1656–1662. 220 C. Mouza et al. / Computers & Education 71 (2014) 206–221 Archambault, L., & Crippen, K. (2009). Examining TPACK among K–12 online distance educators in the United States. Contemporary Issues in Technology and Teacher Education, 9(1), 71–88. Blue Ribbon Panel Report on Clinical Preparation and Partnerships for Improved Student Learning. (2010). Transforming teacher education through clinical practice: A national strategy to prepare effective teachers. NCATE. Retrieved on 25.04.13 from http://www.ncate.org/LinkClick.aspx?fileticket¼zzeiB1OoqPk%3D&tabid¼715. Cavin, R. (2008). Developing technological pedagogical content knowledge in preservice teachers through microteaching lesson study. In K. McFerrin, et al. (Eds.), Proceedings of Society for Information Technology and Teacher Education international conference 2008 (pp. 5214–5220). Chesapeake, VA: AACE. Retrieved from http://www.editlib.org/ p/28106. Chai, C. S., Koh, J. H. L., & Tsai, C. C. (2010). Facilitating preservice teachers’ development of technological, pedagogical, and content knowledge (TPACK). Educational Technology & Society, 13(4), 63–73. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. Common Core State Standards Initiative. (2010). Common Core State Standards for English language arts & literacy in history/social studies, science, and technical subjects. Washington, DC: National Governors Association Center for Best Practices and the Council of Chief State School Officers. Cox, S., & Graham, C. R. (2009). Diagramming TPACK in practice: using an elaborated model of the TPACK framework to analyze and depict teacher knowledge. TechTrends: Linking Research & Practice to Improve Learning, 53(5), 60–69. Dexter, S., & Riedel, E. (2003). Why improving preservice teacher educational technology preparation must go beyond the college’s walls. Journal of Teacher Education, 54(4), 334–346. Doering, A., Veletsianos, G., Scharber, C., & Miller, C. (2009). Using the technological, pedagogical, and content knowledge framework to design online learning environments and professional development. Journal of Educational Computing Research, 41(3), 319–346. Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for qualitative research. Chicago, IL: Aldine Publishing Company. Graham, C. R. (2011). Theoretical considerations for understanding technological pedagogical content knowledge (TPACK). Computers & Education, 57, 1953–1960. Graham, C. R., Borup, J., & Smith, N. B. (2012). Using TPACK as a framework to understand teacher candidates’ technology integration decisions. Journal of Computer Assisted Learning, 28(6), 530–546. http://dx.doi.org/10.1111/j.1365-2729.2011.00472.x. Gronseth, S., Brush, T., Ottenbreit-Leftwich, A. T., Strycker, J., Serdar, A., Wylie, E., et al. (2010). Equipping the next generation of teachers: technology preparation and practice. Journal of Digital Learning in Teacher Education, 27(1), 30–36. Hammond, M., Crosson, S., Fragkouli, E., Ingram, J., Johnston-Wilder, P., Johnston-Wilder, S., et al. (2009). Why do some student teachers make very good use of ICT? An exploratory case study. Technology, Pedagogy and Education, 18(1), 59–73. Harrington, R. A. (2008). The development of pre-service teachers’ technology specific pedagogy (Unpublished doctoral dissertation). Corvallis, OR: Oregon State University. Hew, K. F., & Brush, T. (2007). Integrating technology into K–12 teaching and learning: current knowledge gaps and recommendations for future research. Educational Technology Research and Development, 55, 223–252. Kay, R. (2006). Evaluating strategies used to incorporate technology into preservice education: a review of the literature. Journal of Research on Technology in Education, 38(4), 383–408. Kelly, M. A. (2008). Bridging digital and cultural divides: TPCK for equity of access to technology. In AACTE Committee on Innovation and Technology (Ed.), The handbook of technological pedagogical content knowledge (TPCK) for educators (pp. 31–58). NY: American Association of Colleges of Teacher Education and Routledge. Kinuthia, W., Brantley-Dias, L., & Clarke, P. A. J. (2010). Development of pedagogical technology integration content knowledge in preparing mathematics preservice teachers: the role of instructional case analyses and reflection. Journal of Technology and Teacher Education, 18(4), 645–669. Kleiner, B., Thomas, N., & Lewis, L. (2007). Educational technology in teacher education programs for initial licensure (NCES 2008-040). Washington, DC: National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education. Koehler, M. J., & Mishra, P. (2005). What happens when teachers design educational technology? The development of technological pedagogical content knowledge. Journal of Educational Computing Research, 32, 131–152. Koehler, M., & Mishra, P. (2008). Introducing TPCK. In AACTE Committee on Innovation and Technology (Ed.), The handbook of technological pedagogical content knowledge (TPCK) for educators (pp. 3–29). New York: American Association of Colleges of Teacher Education and Routledge. Koehler, M., Shin, T. S., & Mishra, P. (2012). How do we measure TPACK: let me count the ways. In R. R. Ronau, C. R. rakes, & M. L. Niess (Eds.), Educational technology, teacher knowledge, and classroom impact: A research handbook on frameworks and approaches (pp. 16–31). Hershey, PA: IGI Global. Koh, J. H. L., Chai, C. S., & Tsai, C. C. (2010). Examining the technological pedagogical content knowledge of Singapore pre-service teachers with a large-scale survey. Journal of Computer Assisted Learning, 26, 563–573. Korthagen, F. A., & Kessels, J. P. A. M. (1999). Linking theory and practice: changing the pedagogy of teacher education. Educational Researcher, 28(4), 4–17. Lawless, K., & Pellegrino, J. W. (2007). Technology into teaching and learning: knowns, unknowns, and ways to pursue better questions and answers. Review of Educational Research, 77(4), 575–614. http://dx.doi.org/10.3102/0034654307309921. Leatham, K. (2008). The development of technological pedagogical content knowledge in “technology, pedagogy and mathematics” courses in the U.S. In K. McFerrin, et al. (Eds.), Proceedings of Society for Information Technology & Teacher Education international conference 2008 (pp. 5277–5283). Chesapeake, VA: AACE. Lee, M. H., & Tsai, C. C. (2010). Exploring teachers’ perceived self efficacy and technological pedagogical content knowledge with respect to educational use of the World Wide Web. Instructional Science, 38, 1–21. Lei, J. (2009). Digital natives as preservice teachers: what technology preparation is needed? Journal of Computing in Teacher Education, 25(3), 87–97. Margaryan, A., Littlejohn, A., & Vojt, G. (2011). Are digital natives a myth or reality? University students’ use of digital technologies. Computers and Education, 56(2), 429–440. Margerum-Leys, J., & Marz, R. W. (2004). Teacher knowledge of educational technology: a case study of student/mentor teacher pairs. Journal of Teacher Education, 26(4), 427–462. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis (2nd ed.). Thousand Oaks, CA: Sage Publications. Mishra, P., & Koehler, M. (2006). Technological pedagogical content knowledge: a framework for teacher knowledge. Teachers College Record, 108(6), 1017–1054. Mishra, P., Koehler, M. J., Shin, T. S., Wolf, L. G., & DeSchryver, M. (2010, March). Developing TPACK by design. In J. Voogt (Ed.), Developing TPACK. Symposium conducted at the annual meeting of the Society for Information Technology and Teacher Education (SITE). San Diego, CA. Mouza, C. (2009). Does research-based professional development make a difference? A longitudinal investigation of teacher learning in technology integration. Teachers College Record, 111(5), 1195–1241. Mouza, C. (2011). Promoting urban teachers’ understanding of technology, content, and pedagogy in the context of case development. Journal of Research on Technology in Education, 44(1), 1–29. Mouza, C., & Karchmer-Klein, R. (2013). Promoting and assessing pre-service teachers’ technological pedagogical content knowledge (TPACK) in the context of case development. Journal of Educational Computing Research, 48(2), 127–152. Mouza, C., & Wong, W. (2009). Studying classroom practice: case development for professional learning in technology integration. Journal of Technology and Teacher Education, 17(3), 175–292. NCATE. (2010). What makes a teacher effective. National Council for the Accreditation of Teacher Education. Retrieved 30.07.13 from http://www.ncate.org/LinkClick.aspx? fileticket¼JFRrmWqa1jU%3d&tabid¼361. Niess, M. L. (2005). Preparing teachers to teach science and mathematics with technology: developing a technology pedagogical content knowledge. Teaching and Teacher Education, 21, 509–523. Niess, M. L. (2008). Guiding preservice teachers in developing TPCK. In AACTE Committee on Innovation and Technology (Ed.), The handbook of technological pedagogical content knowledge (TPCK) for educators. New York: American Association of Colleges of Teacher Education and Routledge. Niess, M. L. (2012). Rethinking pre-service mathematics teachers’ preparation: technological, pedagogical and content knowledge (TPACK). In D. Polly, C. Mims, & K. Persichitte (Eds.), Developing technology-rich, teacher education programs: Key issues (pp. 316–336). Hershey, PA: IGI Global. Niess, M., & Gillow-Wilson, H. (2013). Advancing K–8 teachers’ STEM education for teaching interdisciplinary science and mathematics with technologies. Journal of Computers in Mathematics and Science Teaching, 32(2), 219–245. Niess, M. L., Lee, K., & Sadri, P. (2007, April). Dynamic spreadsheets as learning technology tools: developing teachers’ technology pedagogical content knowledge (TPCK). In Paper presentation for the American Education Research Association annual conference. Chicago, IL. Niess, M. L., Ronau, R. N., Shafer, K. G., Driskell, S. O., Harper, S. R., Johnston, C., et al. (2009). Mathematics teacher TPACK standards and development model. Contemporary Issues in Technology and Teacher Education, 9(1), 4–24. Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw Hill. Ottenbreit-Leftwich, A., Brush, T., Strycker, J., Gronseth, S., Roman, T., Abaci, S., et al. (2012). Preparation versus practice: how do teacher education programs and practicing teachers align in their use of technology to support teaching and learning? Computers & Education, 59, 399–411. C. Mouza et al. / Computers & Education 71 (2014) 206–221 221 Ozgun-Koca, S. A. (2009). The views of preservice teachers about the strengths and limitations of the use of graphing calculators in mathematics instruction. Journal of Technology and Teacher Education, 17, 203–227. Pierson, M. E. (2001). Technology integration practice as a function of pedagogical expertise. Journal of Research on Computing in Education, 33, 413–430. Pierson, M. (2008). Teacher candidates reflect together on their own development of TPCK: edited teaching videos as data for inquiry. In K. McFerrin, et al. (Eds.), Proceedings of Society for Information Technology and Teacher Education international conference 2008 (pp. 5305–5309). Chesapeake, VA: AACE. Retrieved from http://www.editlib.org/ p/28122. Pierson, M., & Thompson, M. (2005). The re-envisioned educational technology course: if addition isn’t possible, try division. Journal of Computing in Teacher Education, 22(1), 31–36. Polly, D., Mims, C., Shepherd, C., & Inan, F. (2010). Evidence of impact: transforming teacher education with preparing tomorrow’s teachers to teach with technology (PT3) grants. Teaching and Teacher Education, 26, 863–870. Project Tomorrow. (2011). The new 3 E’S of education: Enabled, engaged, empowered. Speak up 2010 national findings. Retrieved 28.05.12 from http://www.tomorrow.org/ speakup/pdfs/SU10_3EofEducation_Educators.pdf. Schmidt, D. A., Baran, E., Thompson, A. D., Mishra, P., Koehler, M. J., & Shin, T. S. (2009). Technology pedagogical content knowledge (TPACK): the development and validation of an assessment instrument for preservice teachers. Journal of Research on Technology in Education, 42(2), 123–149. Shinas, V. H., Yilmaz Ozden, S., Mouza, C., Karchmer-Klein, R., & Glutting, J. (2013). Examining domains of technological pedagogical content knowledge using factor analysis. Journal of Research on Technology in Education, 45(4), 339–360. Shulman, L. S. (1986). Those who understand: knowledge growth in teaching. Educational Researcher, 15(2), 4–14. So, H., & Kim, B. (2009). Learning about problem based learning: student teachers integrating technology, pedagogy and content knowledge. Australasian Journal of Educational Technology, 25, 101–116. Strudler, N., Archambault, L., Bendixen, L., Anderson, D., & Weiss, R. (2003). Project THREAD: technology helping restructure educational access and delivery. Educational Technology Research and Development, 51(3), 41–56. Suharwoto, G. (2006). Developing and implementing a technology pedagogical content knowledge (TPCK) for teaching mathematics with technology. In C. Crawford, D. Willis, R. Carlsen, I. Gibson, K. McFerrin, J. Price, & R. Wever (Eds.), Proceedings of Society for Information Technology and Teacher Education international conference 2006 (pp. 3824–3828). Chesapeake, VA: AACE. Suharwoto, G., & Niess, M. L. (2006, March). How do subject specific teacher preparation programs that integrate technology throughout the courses support the development of mathematics preservice teachers’ TPCK (technology pedagogical content knowledge)?. In Paper presentation for the Society of Information Technology and Teacher Education (SITE) annual conference. Orlando, Florida. Tondeur, J., van Braak, J., Sang, G., Voogt, J., Fisser, P., & Ottenbreit-Leftwich, A. (2012). Preparing pre-service teachers to integrate technology in education: a synthesis of qualitative evidence. Computers & Education, 59, 134–144. Voogt, J., Fisser, P., Roblin, N. P., Tondeur, J., & van Braak, J. (2012). Technological pedagogical content knowledge – a review of the literature. Journal of Computer Assisted Learning, 29(2), 109–121. http://dx.doi.org/10.1111/j.1365-2729.2012.00487. Wetzel, K., Foulger, T. S., & Williams, M. K. (2009). The evolution of the required educational technology course. Journal of Computing in Teacher Education, 25(2), 67.