Developmental evaluation

Tämän raportin tarkoituksena on kuvata perusopetuksen luokka-asteiden 6–9 oppilaanohjauksen tilaa syksyllä 2008 opetusministeriön rahoittaman oppilaanohjauksen kehittämisen käynnistyessä.

Oppilaanohjauksen kehittäjätahoille (f = 148) toteutettiin e-lomakkeella lähtötilannekartoitus, johon vastasi 442 henkilöä. Kyselyaineistoa täydennettiin kolmella kuntavierailulla.

Kolmasosa vastaajista tuli joko Lapin, Oulun tai Itä-Suomen läänistä, ja runsas kolmannes EteläSuomen läänistä. Vastaajista 71 % oli naisia, ja 38 % oli 46–55-vuotiaita. Runsaalla kolmanneksella ei ollut mitään oppilaan ja opinto-ohjauksen alaan liittyvää koulutusta, mutta 31 % oli suorittanut opinto-ohjaajan kelpoisuusopinnot. Puolet toimi hankkeen käytännön kehitystyössä, ja yhdeksän kymmenestä edusti kasvatus- ja opetusalaa. Noin 38 %:lla vastaajista oli aiempaa hankekokemusta.

Parhaiten toimiviksi ohjauskäytänteiksi arvioitiin seuraavat: työelämään tutustuminen, alakoulun oppilaiden yläkouluun tutustuttaminen, peruskoulun oppilaiden toisen asteen koulutukseen tutustuttaminen, luokkamuotoinen ohjaus ja ohjaukseen liittyvät infotilaisuudet. Huonoiten
toimivia olivat perusopetuksen päättävien jälkiohjauksen toimintamallit, oppilaanohjauksen käytäntöjen dokumentoiminen, toisen asteen koulutuksen ulkopuolelle jäävien ohjauskäytännöt, eri hallintokuntien keskinäinen vastuunjako ja sopimuskäytännöt sekä julkisten organisaatioiden ulkopuoliset ohjausjärjestelyt.

Tapauskunnista Tampereen kaupungin, Jyväskylän seudun ja Sallan kunnan ohjauskäytänteissä oli selviä eroja, koska niiden koko ja konteksti sekä niiden myötä ohjauksen kehittämishaasteet ovat varsin erilaiset. Kaikissa oli pyritty löytämään hyviä käytänteitä ja paikantamaan kehittämistä kaipaavia teemoja. Aineenopettajien ohjaus värittyi heidän opettamansa oppiaineelle ominaisista sisällöistä ja menetelmistä, ja haasteet liittyivät erityisesti työn- ja vastuunjakoon koulun eri
toimijoiden kesken. Oppilaat odottivat ohjaukselta erityisesti tietoa ja kannustusta ja huoltajat toimivaa kodin ja koulun välistä viestintää.

Kehittämistyötä ajatellen eniten kohdistui odotuksia ohjauksen näkyvyyden parantamiseen, ohjauksen arkikäytänteiden ja -työvälineiden karttumiseen sekä  erilaisen yhteistyön ja verkostoitumisen paranemiseen. Myös ohjaustaitojen ja henkilökohtaisen ohjauksen laadun kohenemista haluttiin saada aikaiseksi. Eniten epäiltiin ajan ja voimavarojen riittävyyttä, projektin tuloksellisuutta sekä toimijoiden aitoa sitoutumista. Näkökulma ohjauksen kehittämiseen oli selvästi aikuiskeskeinen, asioiden tarkastelu nuorten kannalta oli vähäistä.

Feminist approaches to evaluation seek to unpack the nature of gender and social inequalities; treat evaluation as a political activity, not as a value-free assessment; and use it as part of the change process. In response to the increased attention being given to evaluation, and the possible role of feminist evaluation in influencing policy that could lead to gender equality along different dimensions, the Institute of Social Studies Trust, based in New Delhi, designed a four-year program. The aim of program was to enhance capacity and understanding of feminist evaluations of various stakeholders, with a conviction that it would influence gender-transformative policy making. This chapter discusses the objectives and activities of this program, and critically analyzes the lessons learned. It also highlights the important takeaways that can make evaluations transformative as far as gender-based inequalities and power dynamics are concerned.

Learning Management System (LMS) has become an integral part of education technology in every institution. Almost all the Higher Education Institutions (HEIs) are using LMS to facilitate their teaching and learning processes. The study aims to evaluate the effectiveness of LMS and the basic perception of students on the usage of LMS and various plugins to support their learning process not only to ensure its quality but also to evaluate different strategies for its future enhancement. Hence, this study adopts a developmental evaluation framework to assess students' satisfaction and perception of using the (LMS) in their experiential Flipped Learning (FL) of English language modules. The methodology follows the pragmatic paradigm based on Dewey's theory to allow for mixed-method instruments to be employed for a better understanding of the LMS, exploring its challenges and opportunities. This small-scale evaluative study involved a survey of 70 respondents and a focus group of ten students in a private HEI in Oman to explore their FL experience in English modules. The depiction of this experience represented positive responses emphasizing how FL technology-enhanced individualized/experiential learning reinforced in lifelong learning skills suitable for career practice. Recommendations are offered for LMS to facilitate enhanced flipped teaching implementation, promoting a student-centered environment ideal for a broader approach of active learning in HEI contexts.

This report examines the effectiveness of a shared apprenticeship model (SAM) to address system-wide challenges in the delivery of apprenticeship training to electricians in the province of Ontario. It synthesizes findings from a developmental evaluation of a pilot program called Apprentice Sponsor Group (ASG), which was designed and implemented by Support Ontario Youth (SOY).
This report details the lessons learned during the implementation process and reviews the short-term outcomes of the ASG. We evaluate the extent to which a SAM is effective in improving the quality, diversity and administrative support for all stakeholders in the apprenticeship system for electrical trades in the three pilot locations in Ontario.

Since 2008, the government of South Australia has been using a Health in All Policies (HiAP) approach to achieve their strategic plan (South Australia Strategic Plan of 2004). In this commentary, we summarize some of the strengths and contributions of the innovative evaluation framework that was developed by an embedded team of academic researchers. To inform how the use of HiAP is evaluated more generally, we also describe several ideas for extending their approach, including: deeper integration of interdisciplinary theory (eg, public health sciences, policy and political sciences) to make use of existing knowledge and ideas about how and why HiAP works; including a focus on implementation outcomes and using developmental evaluation (DE) partnerships to strengthen the use of HiAP over time; use of systems theory to help understand the complexity of social systems and changing contexts involved in using HiAP; integrating economic considerations into HiAP evaluations to better understand the health, social and economic benefits and trade-offs of using HiAP.

In 2015-2016, the apartheid survivors' organisation Khulumani Support Group in the Western Cape (KSGWC) and its partner the Centre for the Study of Violence and Reconciliation (CSVR) set out to explore new ways for a victims' group and a nongovernmental organisation to work together on addressing socioeconomic exclusion in the context of political transition. Seeking to deepen their levels of collaboration, to foreground the knowledge and solutions of KSGWC members and to leave KSGWC with concrete outcomes beyond a research publication, the partners decided to use the participatory action research methodology in studying KSGWC members' understandings of how inequality and poverty drive violence in post-apartheid South Africa. They found that social transformation in the present requires redress for abuses in the past. This report outlines the process of designing and implementing the project, from project development and fundraising, to data collection and collaborative writing up of research findings, and finally to strategic planning, tailored trainings and KSGWC's development of a five-year plan and a new advocacy project. It reflects on the challenges and benefits of the participatory approach and offers some 'lessons learnt' for practitioners designing a similar project, particularly when working with members of social movements and victims' groups on complex and sensitive topics.

Underpinned by the nation-wide Early Learning STEM Australia
(ELSA) project, this practice illustration presents a design framework
to respond to the challenges of scaling and sustaining a large design-based research project. The framework, known as STEM Practices Framework, is informed by work within the Learning Sciences which suggests that the interplay between project innovation and the wider educational reform priorities are critical to the sustainability and scalability of projects. The ELSA project responded to this by developing processes of developmental evaluation to parallel the design based research of the project. Emerging from that process was a design proposition that the object of the project, and the entire STEM education
agenda, is not simply to improve educational practice, but to shift educational purpose. Specifically, the paper argues that STEM Practices represents a qualitative shift in purpose from the content bound traditions of science, technology, engineering and mathematics education towards developing a greater capacity to use practices in diverse STEM contexts. The STEM Practices Framework described here was developed to support educators and developers to implement the project innovations built on this understanding. The framework is in two parts: (1) an adaptation of Kemmis et al.’s (2014) practice architectures
approach and the practice architectures that support and constrain
those practices. (2) A heuristic for working with STEM
practices in large scale implementation.

Evaluation that supports strategic learning applies evaluation findings as well as non-evaluation data in order to improve a strategy. Tools to support the practice of evaluation for strategic learning have yet to be articulated, in part because learning itself can be so varied that the idea of common tools is hard to conceptualize. Yet there are tools that broadly apply to many strategic learning settings and that can become core to the success of a learning organization. These tools are familiar to the nonprofit and philanthropic sectors. Theories of change are commonly used for strategic planning and are often created by evaluators to clarify program logic and guide the evaluation process. In addition, evaluators often use periodic debriefs to provide program implementers with updated evaluation findings. However, using these tools specifically to support strategic learning requires some adaptation from how they are traditionally used.

This brief describes practical steps for designing and using theories of change and strategic learning debriefs as tools for creating the space and structure for strategic learning. It draws on multiple bodies of theory and research that can guide strategic learning practitioners in developing thoughtful and informed strategies for learning.

Social innovation has gained prominence as a way to address social problems and needs. Evaluators and social innovators are conceptualizing and implementing evaluation approaches for social innovation contexts; however, no systematic effort has yet been made to explore and assess the overlap between evaluation and social innovation based on the empirical knowledge base. We address this gap, drawing on 28 empirical studies of evaluation in social innovation contexts to describe what evaluation practices look like, what drives those practices, and how they affect social innovations. Findings indicate most had developmental purposes, emphasized collaborative approaches, and used multiple methods. Prominent drivers were a complexity perspective, a learning-oriented focus, and the need for responsiveness. Reported influences on social innovations included advancing strategies, improving delivery, balancing aggregate and local information needs, and reducing risk. Conflict resolution, the...

L’evaluation developpementale (ED), une approche concue par Patton pour soutenir le deploiement des innovations demeure peu experimentee en recherche. Cet article presente une experience d’application de l’ED dans le cadre d’une recherche sur l'implantation d’un Projet « Hopital promoteur de sante »  (HPS) dans un centre hospitalier universitaire. L’auteure met en evidence la double exigence de recherche et d’utilisation inherente a l’ED lorsque concue dans une perspective de recherche. Ensuite, les defis rencontres par la chercheure sont analyses. Enfin, nous concluons que l’ED peut etre envisagee en recherche a condition qu’une analyse prealable du contexte de l’evaluation soit conduite afin d’anticiper les difficultes liees a l’integration des exigences de recherche et d’utilisation