Assessment at Transition: Report

Assessment at Transition Report 2012 Research Team Professor Louise Hayward, Professor Ian Menter, Professor Vivienne Baumfield, Professor Richard Daugherty, Nasrin Akhtar, Dr Lesley Doyle, Dr Dely Elliot, Dr Moira Hulme, Carolyn Hutchinson, George MacBride, Dr Margaret McCulloch, Dr Fiona Patrick, Ernie Spencer, Dr Georgina Wardle, Harry Blee (until June 2011) and Liz Arthur 1 The research team would like to thank: • the Scottish Government for funding the project • staff and young people from local authorities and school clusters in North Ayrshire, North Lanarkshire, West Dunbartonshire and the Western Isles who contributed so much to this project • staff from Scottish Government, Learning and Teaching Scotland , HMIE, (from August, 2012, Education Scotland), ADES, Local Authorities, School Leaders Scotland, EIS and the University of Aberdeen who contributed to the project’s thinking through participation in seminars • Professor Wynne Harlen and Professor Mary James who offered such interesting insights from international research, policy and practice at the research seminar • Professor Paul Black and Professor Graham Donaldson who read drafts of the report and offered perceptive comments to help develop and refine it. • Colleagues attending two invited international assessment seminars in the University of Bergen and in the University of Oxford who offered insights into how some of the more intransigent assessment issues being faced by the project might begin to be addressed • and Jenni Thomson and Rachel Thomas for their invaluable administrative support. 2 Contents Page Executive Summary 4 Chapter One: Background and Context 21 Chapter Two: Methodology 25 Chapter Three: Perspectives from Research Literature 35 Chapter Four: Perspectives from Practice: Overview of the Case Studies 59 Chapter Five: Where do we go from here? 72 References 95 Appendix One: List of Journal Articles Reviewed 103 Appendix Two: Frameworks for Interviews 111 3 Executive Summary Introduction The Assessment at Transition project was funded by the Scottish Government and undertaken by the University of Glasgow. The project set out to explore how shared understandings of the purposes and potential of assessment at transition between primary and secondary might be developed most effectively. Over time this issue has remained difficult to realise in practice in a sustained and meaningful manner, ie, one which provides consistent progression in learning. In addition, there is strong evidence to suggest that even when policy innovations are highly regarded in their initial stages the process of holding true to initial policy aspirations as they become embedded in practice is complex. This research project was designed to explore how research might be used to support the better alignment of policy and practice. This summary describes the evidence base used to inform the project findings, presents four major questions the project sought to address, outlines the main findings from research and practice in response to these questions and finally presents a possible agenda for action to bring research, policy and practice into closer alignment. The Evidence Base for What We Say The project findings derive from the following sources of information: Evidence from Research • An extensive international research literature review (113 articles, books and websites were selected for analysis) which identified key principles and desirable practice in assessment and in effecting real change in the education system. Evidence from Policy • Review of Scottish Government and Local Authority policy documentation. • A policy perspective obtained through a series of 3 seminars attended by key policy representatives (Scottish Government, Education Scotland, ADES, LAs, SLS, EIS) and distinguished academics in the field of assessment research. 4 Evidence from Practice • Practice in assessment at transition in 4 local authority school clusters (range of contexts, 25 primary schools, 4 secondary schools). o Interviews with 28 P7 teachers, 30 S1 teachers, 18 primary HTs, 9 primary DHTs and 8 secondary DHTs; 4 secondary HTs and 4 LA officers also contributed. o Interviews with pupil focus groups (106 P7 pupils, 33 S1 pupils), including a ‘stars and wishes’ task in which pupils individually identified what they considered to be existing successful means by which they and their teachers knew what they were learning and what they thought could be done but was not presently part of practice. Key Findings: What did we find out from the literature? 1. What leads to successful progression in learning as young people move from primary to secondary school? • There is strong evidence that there are major challenges to having secondary teachers use information based on evidence from primary schools to support all pupils’ learning. This seems to be partly because information may often not be detailed enough to provide sufficiently specific guidance in different subject contexts and partly because of differing priorities among staff across sector boundaries. • Bridging projects have had mixed results. Pupils can feel that these interfere with their enjoyment of the sense of difference in secondary school. Some studies argue that it would be better to focus on shared teacher planning to build progression in practical skills and concept learning. Other evidence argues for the importance of meetings to build secondary teachers' understanding of the primary curriculum eg, in science. • Teachers developing relationships and spending time in one another's schools and classrooms are key factors in promoting communication and understanding. • There is a strong argument that what matters most is high quality pedagogy in both primary and secondary schools where learning is stimulating, challenging, safe and fun. Formative approaches to learning and assessment were highlighted as helpful ways of improving pedagogy. The importance of teachers talking with one another was 5 consistently highlighted as a key feature in improving pedagogy in both sectors, as was the need for protected time to allow such discussion. 2. What evidence is there to suggest that particular kinds of assessment arrangement support learning more effectively as young people move from primary to secondary school? • There was general consensus that assessment must be part of learning and teaching. At a national level this might involve appropriate national curricular guidance in which assessment approaches are designed as part of the curriculum development and clear criteria for success are defined – not lists of individual learning objectives but ‘rich’ criteria, building teachers' capacity in reflecting on and interpreting the curriculum. The design of appropriate courses requires teachers to reflect on, understand and discuss with colleagues what matters to enable pupils to achieve the intended learning. High quality tasks (including some interdisciplinary tasks) enable pupils to show breadth of learning, tackle challenge in learning and apply knowledge and skills in new and unfamiliar situations. Some countries use a design template for tasks and provide exemplification of tasks and learner responses for various specified stages. • One of the major challenges identified was how best to support teachers in summarising evidence to allow them to share information on progression in learning. Many studies highlighted the importance of having good evidence of learning from a range of welldesigned tasks, perhaps gathered in a portfolio, to ensure that there is assessment evidence about all the key aspects of the curriculum • There was evidence that typical coursework (certainly in mathematics) did not provide the full assessment evidence needed and that it was necessary to design portfolio tasks with clear assessment needs in mind, eg, ensuring a focus on what it means to be good at the subject or topic, discrimination among learners and opportunities to demonstrate breadth and depth of learning and application of knowledge and skills. Both advice and exemplification of possible ‘rich’ portfolio tasks were required. • In one study, the process of building teachers' ‘assessment literacy’ to a point where they could independently design and assess portfolio work (and moderate the assessment through intra-school and inter-school discussions) took approximately two years of 6 sustained and intensive activity, involving the teachers in working with people with significant assessment expertise. • A number of studies highlighted the importance of engaging learners in the process of sharing information on learning and assessment for a variety of reasons. These included the positive impact on learning of learners’ greater awareness of what mattered in the curriculum and why; and the value of peer-assessment in developing personal and social skills. There was clear evidence that pupils could be active partners in assessment and more generally in learning and that they could contribute valuably to informing and improving transition processes. • Engaging learners requires the development of their understanding of the goals of their learning, the criteria by which it is assessed and their ability to assess their own work. The evidence suggests that teachers need to develop and use strategies for encouraging self-regulation in learning and promoting positive interpersonal relationships. Through listening to what pupils say about their experiences as learners, teachers are able to gain new insights into the factors that make a difference to pupils’ learning and progress. • Learning conversations would involve teachers in making connections between previous learning and the curriculum, linking both backwards and forwards so that pupils can more readily appreciate what they have done before and will do in the future • There were suggestions that teachers would require focused professional learning to develop the skills required to support these purposeful learning conversations. • Research identified contradictions in systems which promoted learner autonomy, eg, through making learning explicit, but which retained a focus on assessing performance through testing. 3. What interpretations are there of the term standards? • International definitions of standards recognise that they are not merely a matter of a written description of expected knowledge and skills: each description needs to be supported by exemplification of work regarded as illustrating progression towards it and matching the expected knowledge and skills. Standards statements only become meaningful when they are interpreted by people, principally those responsible for making 7 decisions about what has been learned, ie, teachers (whether assessing school work or acting as examiners in a test or examination situation). • Primary and secondary schools may have different views about what should be included as standards, deriving from different cultural emphases, eg, ‘English’ may be interpreted as ‘literacy’ in primary and as response to literature in secondary schools. Such differences may contribute to secondary teachers not recognising the information received from primary schools as helpful in planning. • Common agreement on standards is much easier when agreement is reached on what is useful to pass on as samples of pupils’ work and examples of teacher assessment through primary-secondary teacher dialogue. • There is evidence to suggest that teachers often tend to understand ‘standards’ as marks or grades on externally set tests, which are used to categorise learners and to publicly characterise teachers’ competence. Implicit in this view of standards is the concept of learning as linear. • To dissociate the concept of standards as desirable expectations and aims of students’ work from standards as marks, scores or grades, research proposes that standards should be described in terms of expectation of desirable performance. This might involve the identification of indicators – the important curricular or behavioural aspects to be assessed – and the quality/value of performance or attributes in relation to these indicators. • In primary schools where there is no end-of-school test teachers are more likely to think of learning in terms of learners making progress from where they are towards shared expectations of their learning. • Where standards are very broad descriptors of expected achievement that required ‘unpacking’ by teachers in real classroom contexts, it was likely to take several assessment cycles to consolidate consistent judgements about pupils’ achievement. • The word ‘standard’ has many different meanings. It is crucial that each education system clearly defines its intended uses of the term and then uses these consistently in documentation. 4. What factors influence the extent to which professional judgements are trusted? 8 • Professional judgements are more likely to be trusted if they are evidence-based. Studies suggest that while teachers report using a number of tools to find out about learners’ needs they do not necessarily use the evidence in their everyday teaching. School leadership has a crucial role to play in promoting the use of evidence of learner achievement to make decisions likely to result in enhanced achievement, ie, to take evidence-informed action. The literature suggests that this will require professional learning on several fronts: understanding and skill in gathering and interpreting evidence; knowledge of the content to which the data refer; and how to apply the information gained from evidence. • There is consistent evidence emphasising the importance of collaboration; indeed collaboration is crucial to teachers’ trust in one another's judgements. Joint primary and secondary curriculum planning, working in classrooms in the other sector and cooperative teaching lead to enhanced sharing among teachers of their understanding of expectations of standards and developing the range of pedagogies and classroom organisation on which they draw. The link between building enduring personal relationships and enduring professional collaboration is evident; these require time to develop and maintain. • Intensive moderation is a key component in building trust in teachers’ professional judgements, both within and beyond the profession, and needs to be a structured process. • One study concluded that fundamental to professional judgement was trust on the part of the learner in their teacher as a model of expert practice in the knowledge and skills of the particular domain/discipline being studied. • It is important to be clear about the main purpose of and audience for professional judgement. If standards-based assessment decisions are high stakes for students and teachers (eg, qualifications for access to further study or the workplace), then there is clear evidence that dependability and consistency of judgements across schools are very important. If the main purpose is progression in learning then the evidence suggests that moderation as an opportunity for rich professional conversations about learning is key. Teachers, crucially, have to trust one another’s judgments. • The evidence suggests that different approaches to moderation are necessary to support different assessment purposes. When the principal purpose is progression in learning then 9 social moderation involves teachers in discussing and negotiating judgements made about learners’ work to reach common understanding of pupil progression and standards. This opens up opportunities for professional learning that can raise achievement. • Teachers’ trust in one another’s judgements about pupils’ progress and achievement is more likely where the purpose of assessment is formative. Where the emphasis is on gathering data for records, quality judgements that can be used as feedback to shape learning and practice are unlikely to be the result. • When the main purpose is to agree a level judgement then published standards are insufficient to account for how teachers ascribe a level to pupil work. There is clear evidence that in addition to rubrics (general statements of key indicators of reaching a particular level of achievement) there should be a number of examples of portfolios of pupil work annotated to illustrate how and to what extent they match the rubric. In addition, to embed ideas of ‘best fit’ in day-to-day practice teachers need to discuss with peers annotated portfolios of pupils’ work from their own classrooms with comments explaining their decision-making processes. • Evidence from analysing discussion in moderation meetings illustrates how teachers draw on a range of evidence and criteria, from their own experience as well as from within the range of material formally provided. It may be possible to resolve tensions between explicit knowledge, often provided in external documents, and tacit knowledge derived from teachers’ experience through the provision of a carefully structured framework in moderation which acknowledges the value of both types of knowledge and supports compatibility of judgements among teachers in different schools. • The role of an external person in guiding discussions was reported to be an important feature of effective moderation processes. Key Findings: What did we find out from practice? Current good practice There was much good practice already in place in local authorities and school clusters. These practices were supportive of the values and principles of Curriculum for Excellence and provided a strong basis for effective further developments. Examples included: 10 • teachers’ and pupils’ awareness of and engagement with various aspects of assessment for learning • teachers’ and schools’ strong commitment to do all they could to ensure for pupils a smooth transition into secondary school • the transfer of much valuable information relating to social and pastoral aspects of school life and to additional support needs • effective induction arrangements and very well developed local authority provision to ensure and support these • very positive teacher reaction to professional interaction with colleagues in the ‘other’ sector and local authority action to promote and support this • teachers’ awareness of the main lines of Curriculum for Excellence assessment policy, including, in some cases, the importance of involving pupils in assessment and obtaining their views. 1. What leads to successful progression in learning as young people move from primary to secondary school? • Primary and secondary staff considered that it was challenging to plan for secondary learning and teaching using both broad ‘levels’ information and detailed, contextualised information on individual progress; in practice, secondary teachers tended to use the former only for ‘setting’ or to give a general idea of the appropriate level of challenge. • Many secondary teachers considered that they would find useful: o curriculum coverage information o a portfolio of a pupils’ work o conversations with individual pupils about previous learning, eg, during induction visits and at the start of the S1 year. • Many of the pupils interviewed wanted more consultation with their teachers about their progress in learning to help them to identify successes and next steps. • Both primary and secondary staff valued professional interaction with colleagues in the ‘other’ sector in ‘protected time’ in relation to curriculum planning, teaching approaches and assessment. 11 2. What evidence is there to suggest that particular kinds of assessment arrangement support learning more effectively as young people move from primary to secondary school? • Teachers and pupils recognised the importance of assessment as part of the process of learning and were familiar with various aspects of Assessment for Learning. • Pupils often revealed significant understanding of the nature of learning, referring, for example, to the importance of depth, and suggesting that teacher expectations, a clear curriculum structure and interactive pedagogy could guarantee deep learning. • In relation to assessment of learning, there were significant variations (across clusters, within clusters and within secondary schools) in recording assessment information, in retaining work in a portfolio and in ways of reporting to parents on pupils’ learning, eg, levels judgements for all curricular areas or only for Literacy, Numeracy and Health and Wellbeing. • There was a need for greater clarity about the relationship between profiling and reporting. For example, there was an emphasis on the Profile and the Report as products. Some P7 teachers had doubts about the value of the P7 Profile for pupils’ learning or for giving information to the secondary school, if it consisted only of the pupil’s account of experiences and interests, without reflection on learning or future aims and goals. Some teachers saw the Profile as an unnecessary duplication of reporting and argued that pupils’ involvement in the reporting process could achieve the intentions of the Profile. • Teachers acknowledged the need for their LA and/or cluster to explore the potential of pupils recognising their achievement of a wide range of knowledge and skills, although awareness of the possible implications for using this information to plan learning was not consistent across the clusters. • There was an apparent overall need for staff to discuss how to proceed with and link together different strands of work in assessment, such as defining criteria, gathering evidence, making judgements, recording, reporting, profiling, and maintaining portfolios electronically or otherwise. 12 3. What interpretations are there of the term standards? • Teachers in all cases expressed uncertainty about how to make levels judgements and there was a great variety of approaches to this, including some continuing use of 5-14 levels as benchmarks. • Some teachers used an inappropriate ‘grading’ approach (grading each single task) rather than a ‘best fit’ judgement – this was in effect encouraged in some LAs by the requirement to record very frequently levels and ‘Developing, Consolidating, Secure’ within levels (for tracking individual progress), despite teachers’ expressed concerns that the information being recorded lacked validity and consistency across teachers and schools and was not helpful for planning future learning. • In the three clusters where levels judgements were required by the local authority, staff argued strongly that o the definition of levels standards is not part of their professionalism: rather, the field of their professionalism is effective pedagogy which enables pupils to achieve nationally agreed standards; o there should be national definition, explication and exemplification of standards, with provision for teachers to influence eventual outcomes. • Teachers in both sectors and all posts made many strongly worded requests for guidance on and opportunities to discuss the process of making a ‘best fit’ judgement of a body of evidence about pupils’ achievement for a level. o Teachers’ views on the support afforded at that time by the National Assessment Resource (NAR) were mixed: § those who had been directly involved in contributing to NAR or in discussing its content referred to positive CPD impacts § others expressed a number of concerns over accessing the NAR through GLOW, indexing and search arrangements and the lack of support on making levels judgements as required by the local authority. • There were indications from secondary teachers that new NQ arrangements would strongly influence patterns of assessment throughout the secondary stages. 13 4. What factors influence the extent to which professional judgements are trusted? • There was a high degree of consensus about the need for professional development based on clear guidance and exemplification and discussion in moderation meetings, involving all the primary and secondary teachers, not just those most immediately involved in transition arrangements. • While there was valuable current provision in each LA for planning and moderation meetings, this was probably insufficient to address the need for teachers to discuss curriculum planning, pedagogy and assessment standards in depth, even in a small number of aspects of school work, let alone across the whole curriculum. Current practice represents the early stages in a process that will take time to develop. An Agenda for Action Although the original focus was P7/S1 transition, the findings and action points relate to successful progression in learning at all stages of education. This agenda for action picks up important issues that emerged from the case studies, from the literature review and from the stakeholder conversations that took place throughout the project. Four clear priorities for action emerged to promote better alignment between the policy aspirations of Curriculum for Excellence and their realisation in practice: • Developing teacher professionalism in bringing together curriculum and assessment • Managing learning and progression at transitions • Building trust in professional judgement • Ensuring intelligent accountability in Curriculum for Excellence. 1. Developing teacher professionalism in bringing together curriculum and assessment • To support teachers in developing greater awareness of the complex interactions among all the factors that contribute to the overall process of learning, teaching and assessment, there should be a focus on validity. Clear understanding of what matters in the curriculum is the basis of establishing how much and how well pupils are learning and have learned, and for planning further learning. 14 • Thinking and discussion about assessment should be embedded in planning overall learning and teaching (for a sequence of lessons, a term, a year or a stage). They should begin from what matters in the curriculum. This is the essential basis for. developing good learning and assessment tasks; articulating relevant success criteria; involving pupils in planning and assessing their own learning; gathering classwork evidence and evaluating success; providing feedback and identifying next steps; summarising achievement and progress (including, when required, making a ‘level judgement’); and reporting information about pupils’ learning. • There should be (continued) provision of guidance on and exemplification of ways in which the statements of Experiences and Outcomes can be used to inform these processes. 2. Managing learning and progression at transitions • For teachers to be motivated to use information they receive from another teacher or school they must be involved in the design of the information gathering system. • The system needs to be manageable and focused on the transfer of information that will lead to changes in curriculum planning and/or in classroom practices for individual learners or for groups of learners; and will support conversations about learning between learners and teachers. Detailed analyses of each pupil’s progress in all areas across the curriculum are unlikely to be used. • At all points of transition teachers should receive information about prior curriculum coverage and have opportunities for learning conversations with pupils. These conversations can be informed by reference to relevant prior work in a portfolio. • Purposeful meetings of primary and secondary colleagues are essential, informed if possible by time spent in one another’s classrooms. These meetings need to be a permanent part of professional life. • The Scottish Government and Education Scotland should work with local authorities to promote these ideas and encourage and support communities of learning, including both primary and secondary staff. 15 3. Building trust in professional judgement • Building high quality teachers’ professional judgement is crucial to the success of Curriculum for Excellence, which promotes a range of learning that no external examination system could assess alone. • At points of transition within and between schools, what matters most is that teachers trust one another’s judgements about what pupils have learned. In the later years of secondary school, when assessment stakes are high, society must trust teachers’ professional judgements; these must be consistent with nationally specified standards for different qualifications. At all stages, trust will require close relationships among those involved – learners, parents, other teachers, other schools and society generally, as represented by local and national bodies responsible for education and by elected representatives. • The research review and case studies identified key action needed in relation to assessment for learning and assessment of learning. Enhancing teachers’ professionalism in assessment for learning • In addition to the range of existing assessment for learning practice, there should be emphasis on: o the importance of dialogue about progress in learning between teachers and pupils and amongst pupils o evidence-informed decision-making. The basis of such evidence about what has been learned and next steps (what to teach next and how) is clear understanding of the curriculum; of the kinds of learning and assessment tasks that will promote the learning embodied in that curriculum; and of what pupils need to do to demonstrate that they have learned it. This point is thus closely linked to ‘Developing teacher professionalism in bringing together curriculum and assessment’ (above) and to ‘Sharing information about pupils’ progress without reference to achievement of a level’ (below). • Teachers, researchers and curriculum developers should work together to build examples of using dialogue to promote and provide evidence of learning in different contexts. 16 Enhancing teachers’ professional judgement in assessment of learning Sharing information about pupils’ progress without reference to achievement of a level • Within clusters teachers should work with others to form and share an understanding of progression in different areas of the curriculum, through discussions of curriculum plans, learning and assessment tasks, criteria for success, samples of pupils’ work and teacher annotations of these. • Such discussion should focus on understanding the relationship between immediate ‘next steps’ related to current learning and the ‘big picture’ of progression through school in terms of key aspects of learning in a curriculum area. • A number of starting points within Curriculum for Excellence (eg, Principles and Practice papers) can be used to help articulate progression. Education Scotland, teachers, education authorities and researchers in collaboration should effectively use such material along with curriculum plans and samples of pupils’ work to provide examples of evidence-based decisions about what next steps might be the priority for an individual, a group or a class and how curriculum plans can be accordingly adapted. • These groups should develop and share examples of using this kind of information as the basis for reporting to parents and for passing key information to a subsequent teacher. • Consultation is needed about parents’ perspectives on different forms of communication about their children’s learning, including, for example, discussion of portfolios of work rather than detailed descriptive school reports. Making good decisions about achievement of levels • Curriculum for Excellence affords learners opportunities to explore concepts and learn in depth. To promote a focus on such learning and progression, to prevent the creation of sub-levels and to ensure validity of levels judgements, there is a strong case for reporting on level achievement only at the end of stages of school associated with likely achievement of a level by most pupils – P4, P7 and at the point of moving from broad general education into the senior phase. • To avoid fragmentation of assessment there is a need to develop clear statements of what matters to demonstrate the achievement of Experiences and Outcomes at a particular level. These statements should not comprise a list of detailed content; rather, they should 17 focus on key learning. Attempts to describe achievement of a level are only likely to be meaningful when accompanied by exemplification. A range of exemplification will be needed to show how what matters can be matched to several different kinds of pupil experience and types of work. These exemplars should include teachers' annotations explaining how the teacher has come to the decision that the profile of pupil’s work is a ‘best fit’ for the level. Exemplification should illustrate how learners have had opportunities to demonstrate breadth of coverage of Experiences and Outcomes, should provide evidence which reflects success in meeting an appropriate level of challenge and illustrate successful application of learning in different contexts. • ‘Best fit’ requires a number of pieces of work to be compared to a level; decisions are taken on the extent to which the whole body of work provides evidence that key learning has been achieved. Levels are meaningful only if they are related to a body of evidence of learning and cannot be assigned to individual pieces of work. • Developing exemplification representing the concepts of ‘developing, consolidating and secure’ should be avoided as it would in effect create separate sub-levels and risk labelling pupils, with consequent constraint of breadth and challenge in the learning of those working at the ‘lower’ sub-levels. • First draft descriptions of the qualities demonstrating level achievement and exemplification should be developed by teachers, researchers and local and experienced national support and challenge staff. There should be engagement and consultation with the profession about these materials, leading to final levels statements accompanied by annotated exemplification. Moderating teachers’ professional judgement against standards • Moderation requires teachers to come together to discuss examples of pupils’ work, compare them against agreed standards, using a ‘best fit’ approach, and discuss their judgements. • Moderation activities can provide feedback to those responsible for levels on the appropriateness of their expectations and thus inform periodic modification. • It must be recognised that moderation takes time and should be regarded as a key task in what it is to be a professional educator. Successful moderation will depend on the 18 development of high quality professional relationships amongst teachers and on in-depth learning conversations. It will take time to develop a common understanding of standards of achievement and skills in making dependable judgements against agreed standards among a group of teachers. The time needed to support teachers in moderation and sharing standards may have to be found by prioritising these activities and reducing time spent on other activities. • Moderation activities will be all the more effective when informed professional advisers work with school colleagues. 4. Intelligent accountability in Curriculum for Excellence A major threat to the assessment aspirations of Curriculum for Excellence comes from lack of alignment between these aspirations and accountability systems. • Effective accountability systems must be consistent with the aims of education rather than a diversion from or an obstruction to learning and teaching. For example, standardised tests do not provide valid information related to Curriculum for Excellence • A model of accountability in the context of Curriculum for Excellence should recognise that: education has multiple purposes; the education system is complex; education is concerned with learners both as individuals and as members of society; educational issues must be related to issues of social justice; and change must be based on building the expertise of the profession. Levels judgements cannot be the sole or main basis for such a model. • Evaluation systems need to be designed to focus on the impact of action by schools and teachers on learning within Curriculum for Excellence and on the extent to which their actions make a positive difference to children’s and young people’s learning. This requires consideration by all of such questions as: what evidence (from research, policy and other practice) has been used to inform the design of the innovation to promote its chance of success? how will success be judged? what evidence will be gathered to determine the extent of the success of the innovation? • It will be important for all those with policy responsibilities to consider how best to: o promote continuing development of self-evaluation and improvement planning based mainly on evidence about quality of learning and teaching and descriptions 19 of pupils’ progress, involving levels judgements only at key points, eg, P4, P7 and the end of broad general education. o discourage too frequent use of levels judgements for tracking individual pupils’ progress (on the grounds of the lack of validity when levels judgements are made on the basis of only small amounts of curricular coverage and pupil work). o consider, in consultation with local authorities, how future accountability systems might be developed in ways that will remain consistent with the aspirations of Curriculum for Excellence without the negative washback on classroom practice commonly associated with previous accountability systems. For example, there may be merit in considering how a sample survey such as the Scottish Survey of Literacy and Numeracy (SSLN) might be extended to monitor standards locally from time to time. The SSLN will provide information directly related to Curriculum for Excellence and, as an anonymous survey, is less likely to lead to the negative washback features associated with regular standardised testing and with the frequent and/or centralised gathering of information on levels (and potentially, sub-levels) achieved. • There is a duty on all involved to prevent the worst possible scenario, in which as a society and education system we become obsessed with measurement of progress against increasingly small and narrow targets and draw attention away from the broader aspirations of Curriculum for Excellence. 20 Chapter One: Background and Aims 1.1 Introduction This project was concerned to take forward thinking in two main areas. The first focus was to explore how shared understandings of the purposes and potential of assessment at transition between primary and secondary might be developed most effectively. Over time this issue has remained difficult to realise in practice in a sustained and meaningful manner, ie, one which provides consistent progression in learning. The second focus was to explore the potential of new partnerships and models of collaboration amongst research, policy and practice. This project, one of a series commissioned by the Scottish Government, was concerned to explore how research might be used to support the better alignment of policy and practice. Traditionally, in Scotland research evidence has been used to inform the development of policy. However, it had become clear in previous cycles of innovation that, as policy was interpreted in practice, gaps emerged between policy aspirations and practice. The complexity of the inter-relationship amongst research, policy and practice had at times been underestimated, with an over-simplified model dominating thinking, namely that the major challenge lay in the development of good policy. Putting policy into practice was simply an issue of communication, getting the right messages to schools and local authorities. Essentially, the model was hierarchical. Policy informed practice to be implemented in schools. More recent policy developments, for example, the Assessment is for Learning Programme, have begun to recognise the greater complexity of the research, policy, practice relationship, and new models have begun to emerge. More recent models of innovation have recognised the importance of shared aspirations, the realisation of which requires all communities to play equally important parts. The Curriculum for Excellence development in principle recognises the importance of these models. They imply more interactive processes of policy development, in which, though an initial policy framework may be developed, the process of bringing together policy and practice has potential to bring changes to both. In this context the role of research is different. It serves to 21 explore the spaces between the aspirations of both policy makers and practitioners and emerging practices in order to encourage greater alignment between policy and practice. The first chapter of this report provides information on the background to this assessment study and will identify the questions the study sought to address. The second chapter describes the methodology of the project designed to address both project strands. The third chapter provides an analysis of the research evidence in the area under investigation. The fourth chapter provides insights into the investigation undertaken in the school clusters in four local authorities. Finally, the fifth chapter draws together evidence from the various sources, identifies alignment challenges and proposes options to bring research, policy and practice into closer alignment. 1.2 Background to the Assessment Study The dream of using assessment to drive up standards (Black, 2001) has dominated the UK educational landscape for more than a decade but it appears to have had only limited success (Mons, 2009). The importance of managing the relationship between assessment and learning is universally accepted yet, in tackling some of the more intransigent challenges in the relationship, all too often there remains a tenuous relationship between research, policy and practice. The evidence from research on assessment (eg Baron et al, 2007) suggests that at the point of transition between primary and secondary school two key features are particularly difficult to achieve: firstly, the extent to which professional judgement is trusted and secondly, how relevant and useable the information is perceived to be by the receiving practitioners. In some respects exploring the inter-relationship amongst policy, practice and research in assessment is not contentious. Each community would express a desire to use assessment to offer children and young people better life chances. Equally uncontentious, at least in principle, is the desire to promote actions in schools and classrooms that are those most likely to enhance pupils’ chances of learning most effectively. However, beyond these broad general aspirations lie some of the most contentious issues in education. Assessment data are used for everything from providing feedback to individual learners to proclaiming the state of the nation in education. Newton (2010) identified 22 uses made of assessment data. It is within this deeply contentious 22 and complex arena that this project is set; an area where, perhaps more than any, the relationship amongst policy, practice and research communities matters if the aspiration of the Assessment Reform Group (Mansell et al, 2009) is to be realised and assessment is to become ‘Fit for Purpose’. From the earliest stages of the introduction and development of Curriculum for Excellence, the Scottish Government had made clear the importance of assessment to successful implementation within the classroom. Following a Ministerial Statement of the strategic vision and key principles for assessment within Curriculum for Excellence (Scottish Government, 2009), the Government developed and published, in consultation with the Management Board, a number of statements of authoritative advice and guidance on aspects of assessment. The key document is Curriculum for Excellence: Building the Curriculum 5, A Framework for Assessment, currently available in a slightly revised version (Scottish Government, 2011). This is supported by documentation, under the ‘BtC5’ label, on applying and sharing standards in assessment (Scottish Government, 2010a), on reporting (Scottish Government, 2010b), and on recognising achievement, profiling and reporting (Scottish Government, 2010c). These documents were widely distributed and publicised to practitioners. In June 2010, the Scottish Government commissioned the University of Glasgow to undertake a research project to explore how best to make progress in the development of shared understandings of the purposes and potential of assessment at transition between primary and secondary. It was recognised that this had been an issue of concern over time. Sharing information about learning across primary and secondary schools had proven to be consistently difficult to realise in practice in a sustained and meaningful manner. This was not a peculiarly Scottish problem. The effectiveness of the management of assessment at transitions has also been a challenge for schools beyond Scotland. This project began, from the perspectives of the potential users, to explore ideas of relevance and to work with schools, teachers and local authorities to develop approaches that were both useable and manageable. Although the focus of the Assessment at Transition project was primary secondary transition the implications were broader. How might assessment information be used to promote educational 23 experiences for learners that were coherent and progressive? Whilst the topic comes into sharp relief when learners move from primary to secondary school, the same issue exists as children move from class to class in primary school or from year to year in high school. Working with a cluster of associated primary and secondary schools in each of four education authorities in Scotland, the project sought to explore current arrangements for transition from primary to secondary school and how information about pupil learning might be shared in meaningful ways, including understandings of standards. It also aimed to investigate how ideas emerging from within clusters might be more widely shared with schools and local authorities in ways that go beyond the notion of ‘best practice’. The researchers suggested that in each cluster of schools the focus might be on four curricular areas (English and Literacy, Mathematics and Numeracy, Science and a fourth area identified by the school cluster). However, because of the project’s concern to remain consistent with current local priorities and interests, the range of subject areas varied: English/Literacy featured in all the four clusters, Mathematics/Numeracy in three and Science in two; other subject areas featuring in different clusters included Modern Languages and ICT. The project had seven aims: • to explore current arrangements for transition at the key stage primary to secondary • to explore understandings of how assessment supports learning before and after the transition between primary and secondary and how it could contribute to appropriate reporting and recording processes • to support teachers in developing knowledge and expertise in defining standards of achievement • within this process to explore how teachers and schools might best be able to share standards in areas of the curriculum which will be jointly identified as priorities • to work with teachers, schools and local authorities to develop approaches that will allow information to be shared in meaningful ways • to explore and evaluate the different sustainable models of dissemination adopted by those local authorities to support teachers’ development for the future • to share the outcomes and recommendations with other local authorities and stakeholders. 24 Chapter Two: Methodology The project methodology consisted of three inter-related strands of activity intended to explore issues through the lens of different communities: research, practice and policy. The first strand sought to establish the existing research evidence base and was undertaken as a review of literature. The second strand sought to explore the evidence base from the perspective of practice. This strand was undertaken by researchers working with the school clusters in four local education authorities. The third strand of the project sought to explore policy perspectives through a seminar series. The project was ethically approved by the University of Glasgow. 2.1 Evidence from Research and Policy The literature review was undertaken as a two part process. First, with permission from the SQA, this project built on evidence from relevant sections of a previous literature review commissioned by SQA (Spencer, 2009). Second, a further review specifically related to the aims of current project was undertaken. This second phase was based on the model developed as part of the review for the Donaldson Review of Teacher Education (Menter et al, 2010). The approach taken was to identify literature exploring the purpose and potential of assessment at points of transition. The research team drew on relevant peer-reviewed journal articles and reports of funded research written for research councils or other major funders. Government reports and policy documents were also reviewed and analysed according to their relationship with evidence from research. The analysis of research evidence was carried out using a ‘best evidence synthesis’ (Slavin, 1986) where criteria were developed for determining good quality research. Key terms and parameters of the review The first stage of the second part of this literature review involved the identification of key terms. These were identified using thesaurus information from ERIC and TLRP and discussed with the research team; additional search terms were included following these discussions. The resource parameters of the task led to the group focusing on particular areas of the curriculum in 25 the context of sharing standards although acknowledging that this might lead to some key studies being excluded from the search. Table One: Key Terms and Parameters Date Limitations 2000 - Geographical International Literature, Scotland, UK and Beyond Limitations Sector/Pupil Age Range Primary/ elementary; middle school; secondary/high school Language English Language publications Key Words Assessment Transfer Transition Liaison Primary-secondary Elementary-High School Progression Standards Profiles Authentic Assessment Reporting Personal Learning Planning Sharing standards Progression Sharing information primary to secondary Sharing standards in English Language Sharing standards in mathematics Sharing standards in science Sharing standards in mathematics 26 Primary secondary transfer Primary secondary transition Primary secondary transfer projects Assessment information transfer Progress in S1/2 Pupil progress 10-14 Managing change across primary and secondary schools Pupil centred assessment School clusters Broader Terms Professional learning in assessment Using assessment information Changing assessment culture Transitions within the health services Sharing of information within child protection and Reporter’s Administration, The small schools movement The role of pupil (young people’s) voice Excluded Not written in English Conducted before 2000 (unless seminal texts) Search, screening and categorisation A literature search was conducted using a number of commercial and other electronic databases, including the British Education Index, ERIC and the EPPI Centre library. In addition a hand search of key journals was conducted by members of the research team, eg, Assessment in Education, British Educational Research Journal, the Oxford Review of Education, the Cambridge Review of Education, Scottish Educational Review and the Curriculum Journal. Two members of the research team then screened the list of journal articles generated and graded each item to ensure that articles that were less relevant to the study or whose warrant was less strong were excluded from the list for analysis. 27 Synthesis 70 articles were identified from the above process. A list of the articles analysed is provided in Appendix One. A team of experienced reviewers then analysed the articles and reports focusing on key aspects of the enquiry. Each article was interrogated to identify evidence in response to a number of questions key to the investigation. Each question with linked to a small number of sub-questions. Table Two: The Framework for Analysis What leads to successful progression in What insights are there into what matters in learning as young people move from successful learning transition between primary to secondary school? primary and secondary schools? What are the biggest identified challenges? What evidence is there to suggest that To what extent are young people particular kinds of assessment arrangement themselves involved in these processes and support learning more effectively as young in what ways? people move from primary to secondary school? To what extent are parents involved in these processes and in what ways? What are teachers’ views of these processes? What interpretations are there of the term How have standards been defined? What standards? attempts have been made to share standards? What mattered in successful sharing of standards? What, if any relationship is there between ideas of standards and ideas of 28 progression? What factors influence the extent to which How have people attempted to build trust professional judgements are trusted? in professional judgements? If so, in what areas of the curriculum? What lay behind the success? Is there evidence of information on children’s learning being used either to influence the curriculum in secondary schools or to influence action in relation to children’s learning? The analysis of data obtained from the range of sources outlined above is provided in Chapter Three of this report. 2.2 Evidence from Practice The second strand sought to explore the evidence base from the perspective of practice. This involved teams of researchers working with school clusters – each involving a secondary school and its associated primary schools - in four local authorities. The clusters were chosen to reflect different circumstances, ie, one rural, one urban, two mixed urban and rural. Clusters included a range of primary school sizes, a range of socio-economic contexts and denominational and nondenominational schools. Discussions with representatives from local authorities sought to identify school clusters for which the research topic was an area of interest and thus the project could be part of their existing developments. In three clusters all of the primary schools participated in the project (9, 8 and 5 primary schools). The fourth cluster involved a large number of primary schools across a rural area (13), but for pragmatic reasons only 3 participated directly in the project. 29 The fourth cluster, located a significant distance from the central belt, played a slightly different role within the project. It had been intended originally to gather data from this cluster using the national GLOW intranet but a range of difficulties, including technical problems, made this impossible. In this cluster the number of teachers interviewed was more limited than in the others and travel constraints made it necessary to conduct interviews with teachers who could be available at a particular time. The constraints of time also meant that it was not possible to interview pupils in this cluster. Teacher and pupil interviews Information was gathered from teachers and, in three of the four clusters, from pupils in focus group interviews near the beginning of the project, using the frameworks which can be found in Appendix Two. A second set of meetings with teachers took place after the information obtained in the first set had been analysed and compared with both Curriculum for Excellence assessment guidance (in the Building the Curriculum 5 suite of documents, Scottish Government 2009) and key principles which had been drawn from the research review. At the second stage of interviewing the key findings from the literature and policy review and the analysis of the interviews were shared with the school clusters and discussions were held as to how the evidence obtained might influence future cluster assessment plans. The researchers also obtained additional information about current practice in these second stage interviews. In all, 28 Primary 7 (P7) teachers and 30 teachers who taught Secondary 1 (S1) pupils participated in the interviews. In addition, information was obtained from 18 Primary headteachers, 9 Primary depute headteachers and 8 secondary depute headteachers. The four secondary headteachers also attended meetings and contributed information or comments. In each local authority information about assessment and transition policies, support and practice was obtained in interviews with an authority officer and/or from documentary guidance to schools. 30 Pupil focus group interviews were conducted with P7 pupils in three of the primary schools in each cluster and with S1 pupils in each secondary school. In all 106 P7 pupils and 33 S1 pupils took part in interviews. They also individually completed an activity to rank different types of assessment activity in importance for aiding learning and wrote statements identifying assessment ‘Stars’ (current practices they thought were good/helpful) and ‘Wishes’ (assessment activities they would like to experience). Teacher and pupil interviews were recorded and transcripts were produced for analysis. Two researchers participated in each focus group meeting, one leading the discussion and the other making notes of significant points. The pattern of interaction between the researchers and the teachers and local authority staff varied in different clusters. For example, in one cluster, in addition to the interviews described above, the research team attended staff development events with teachers and primary/secondary staff meetings which were part of the transition process for the P7 pupils. In two other clusters they discussed with teachers examples of reports to parents and of the P7 Profile which was being developed in response to the expectation in the national assessment guidance that there should be such a Profile. Analysis of the information The information obtained in the teacher and pupils interviews was analysed in three stages. First stage The first stage was an NVivo analysis of the teacher and pupil interview transcripts across three clusters (omitting the one where interviews took place later than in the others, because of the technical difficulties with the intended data gathering via the GLOW intranet). This NVivo analysis, sought to identify recurring themes and issues across all three clusters and led to an account of these under the headings Use of Assessment Information; Reliability of Assessment Information; Digital Resources – GLOW and National Assessment Resources (NAR); Pupils’ Self-assessments; Assessment and Recording; Wider Achievement; and Learning Support. This 31 first stage of analysis also included categorisation by the research team of the types of assessment activity valued and/or desired by the pupils in their ranking exercise and in their ‘Two Stars and a Wish’ statements. Second stage The second stage of analysis was designed to enable the two researchers interacting with each of the different clusters to feed back information specific to each as the basis for discussion of future development plans/possibilities. At this stage data from all four clusters were analysed. Each researcher team worked through the transcripts for its cluster, the notes taken during the meetings and the other information obtained from the cluster (including that pertaining to local authority assessment and transition policy and practice) to produce a summary of key points. The structure of this analysis and the resulting summary was based on the teacher and pupil interview frameworks set out in Appendix Two, which, though they were adapted to the differing contexts of primary and secondary education, had sought to gather information on essentially the same topics. These topics were: How Assessment Contributes to Learning; Recording of Assessment Information; What Information is Transferred to the Secondary School? (How? Why? How Used? What is its Effectiveness as a Basis for Planning learning in S1?); What is Understood by ‘Sharing the Standard?’; What is Needed to Improve the Present Arrangements? What is/should be the Role of Learners in the Process? A case study for each cluster was produced describing the outcomes of this analysis. It included comparison and evaluation of the extent to which local policy and practice matched both broad national policy guidelines and key principles for high quality assessment drawn from the research review (see Chapter 3). The case studies thus explicitly compared and contrasted both national policy guidelines and practice in the cluster with research-informed principles for effective assessment. They formed the basis of the feedback to the clusters at the second stage of meetings. The case studies were adjusted/expanded as appropriate after further information was obtained during the second stage of meetings. 32 Third stage The third stage of analysis involved identification by the members of the research team of both recurring desirable practice and key issues/concerns across all four case studies and the evidence from the NVivo analysis. This process was undertaken by two team members separately before a common set of findings was agreed. It was structured using the project research questions 1. What are successful transition factors? 2. Which kinds of assessment arrangements support learning more effectively at P7/S1 transition? 3. Which interpretations of standards are being used? 4. What are the factors influencing trust in professional judgements? Chapter Four sets out the results of this third stage of the analysis carried out in the project. 2.3 Research and Policy in Practice The third strand of this project sought to how perspectives from research and practice interrelated with policy perspectives. This process was developed through an iterative series of seminar discussions. The first seminar held near to the beginning of the project focused on deepening understandings of the policy environment. Key stakeholders from national stakeholder groups came together with researchers to discuss the current policy position and to reflect on similar policy developments beyond Scotland. Professor Richard Daugherty contributed to this seminar, reflecting on issues emerging from recent policy developments in Wales. The second seminar brought together the same group and included two further members of the Assessment Reform Group, Professor Mary James and Professor Wynne Harlen. The focus of the second seminar was to reflect on emerging findings from the project and to begin to identify potentially tricky issues for policy, practice and research alignment. Originally it had been intended to hold a final seminar at the end of the project to explore how findings from the project might best be shared. However, evidence from the literature review and from discussion with members of policy and practice communities led to a change in strategy. 33 Rather than hold a final seminar it was agreed that members of the research team would attend a number of meetings with different communities, eg, government, Education Scotland, Local Authorities and School Leadership groups. Further it was agreed to develop a small number of short papers specifically targeted at key communities, government, Education Scotland, researchers, education authorities, school leaders and teachers. 34 Chapter Three: Perspectives from Research Literature This review of literature falls into two parts. The first part is a review undertaken for the Scottish Qualifications Authority (SQA) in December 2009 – Evaluation of Continuing Professional Development for ‘Assessment Literacy’, with a Particular Focus on Assessment of Learning in New Qualifications (Spencer 2009). This review of research and policy evidence (40 articles, books, policy documents, websites) described and evaluated key aspects of CPD for assessment literacy in a range of Scottish and international locations/organisations. It focused on CPD designed to develop teachers' abilities and confidence in undertaking assessment of learning and in-school assessment for certification. From the evidence reviewed, key principles of high quality assessment and high quality CPD were identified and used as touchstones for evaluating the CPD activities reviewed. Though the review focused principally on assessment for certification, some of the principles may be relevant to assessment of learning at other stages of school. Indeed, many of them have informed Scottish assessment policy for some time. The review was strongly influenced by a range of recent publications from members of the UK Assessment Reform Group. SQA has agreed that key findings from this review can be used to inform the work of the Assessment at Transition Project. The second part of the Assessment at Transition literature review built from the SQA work. Using the review methodology developed for the Scottish Government-funded Donaldson Review (Menter et al, 2010), an extensive search of journals (over the period 2000-2011) was undertaken using key terms related to the Assessment at Transition project. From the original search 70 articles were selected as potentially relevant to the themes of the project. A focused review of each article using the questions identified below was carried out by members of the project team. They were asked to identify and summarise key points relating to four questions central to the interests of the project: • What leads to successful progression in learning as young people move from primary to secondary school? (What insights are there into what matters in successful learning transition between primary and secondary schools? What are the biggest identified challenges?) 35 • What evidence is there to suggest that particular kinds of assessment arrangement support learning more effectively as young people move from primary to secondary school? (To what extent are young people themselves involved in these processes and in what ways? To what extent are parents involved in these processes and in what ways? What are teachers’ views of these processes?) • What interpretations are there of the term standards? (How have standards been defined? What attempts have been made to share standards? What mattered in successful sharing of standards? What, if any, relationship is there between ideas of standards and ideas of progression?) • What factors influence the extent to which professional judgements are trusted? (How have people attempted to build trust in professional judgements? In which areas of the curriculum? What lay behind the success? Is there evidence of information on children’s learning being used either to influence the curriculum in secondary schools or to influence action in relation to children’s learning?) Evidence from the individual articles was then clustered around the identified questions. This paper uses the four questions as the framework within which the key points emerging from both reviews are set out. The findings from this review are then related to the current policy advice in Building the Curriculum 5, the assessment policy statement related to Curriculum for Excellence. Key findings 1. What leads to successful progression in learning as young people move from primary to secondary school? The findings here derive from the journal articles search (the SQA paper did not address this question). The review found relatively little to help answer this question in relation to the whole curriculum or to most particular aspects of it. The articles reviewed dealt principally with transition issues in 36 science, with a small amount of information about English, mathematics and physical education. However, it is possible to summarise emerging points under 4 headings. Perceptions of discontinuity and inadequate information There is a common perception of discontinuity in curriculum, learning and pupils’ motivation (after a temporary high point at the beginning of secondary for the last). The existence of a clearly defined curriculum framework, such as the National Curriculum in England, did not improve the situation (Galton 2000). It is claimed that there is need for secondary teachers to know about prior achievement, but information passed from primary to secondary schools is inadequate. Braund (2007), on science, points out that while there has often been too little detail to make decisions about progression secondary teachers are not always prepared to work effectively with an enhanced level of information when they actually get it. This leads to mistrust and scepticism: time and low prioritisation of recording attainment/achievement are constraining factors (Capel et al 2008, on PE). Lack of meetings between primary and secondary staff and poor communication are factors; and even when records are passed on, there is no guarantee that appropriate secondary teachers get them, or use them (Gorwood 1991). Noyes (2006) argues that secondary teachers often guess prior attainment/achievement on the basis of sibling evidence, attitude to work, dress, appearance, etc – quite accurately in many cases, but with the obvious danger of serious bias. The review does not identify clearly ways in which transfer of information from primary to secondary schools could be improved with definite impact on learning and progression. Curricular continuity – bridging projects Several studies evaluate and comment on bridging work in science, with differing perceptions of their value. Galton (2002) argues that they do not constitute continuity of curriculum but rather just similar work to that done in the primary school, which is therefore familiar and comforting, but can be regarded by some pupils as repetitious and unchallenging (Davies and McMahon 2004). Indeed, an expectation of discontinuity rather than continuity was expressed by pupils themselves (Galton 2002, Braund and Driver 2005). This leads Braund and Driver to conclude that pupils continuing with a set of practical experiences in a similar context following transfer 37 could be counterproductive; rather teachers should plan work that is sufficiently different from the primary but recognises the level of practical skills and concept learning that have occurred before and moves pupils on from this. Scharf and Schibeci (1990) found that there was no significant difference in measured attitudes to secondary science of pupils who had and who had not taken part in planned science transition work. However, others identify advantages, especially where the secondary work complements and extends the earlier activities (Braund 2007) and reinforces primary work (Davies and McMahon 2004); and where the collaboration promotes among secondary teachers the importance of recognising and explicitly referring to pupils’ prior learning (Braund and Hames 2005). The collaboration, visits and joint planning involved in bridging work were regarded as important, in particular for improving secondary teachers’ awareness of primary science work (Davies and McMahon 2004). Jarman (1997) argues that there is a need to deconstruct the complex process of ‘building on’ or ‘taking account of’ prior learning and to provide ‘how to’ guidance and exemplars of successful practice in linking primary to secondary work (in science), as well as ensuring enough time to enable plans to be properly put into practice. Ethos and psychosocial factors Some articles argued that the most important factors for pupils at the transition stage do not include academic learning, but relationships/new friendships, experience of the new school prior to going there (Ashton 2008) and a sense of belonging and inclusion in its community (Humphrey and Ainscow 2006). Chedzoy and Burden (2007) dispute Galton’s claims about inevitable demotivation of pupils in early secondary school by highlighting the case of a school where this did not occur and arguing that the organisation and ethos of a school can ensure that pupils remain engaged and committed. Bryan and Treanor (2007), Graham and Hill (2003), West et al (2010) and Zeedyk et al (2003) consider issues of organisation and ethos in a Scottish transition context; in so doing, their focus is often on individual pupils or social groups who are in one way or another disadvantaged or vulnerable; all are clear that schools and schools groups can take actions to improve the transition experience for such young people. There is evidence in these studies that many young people, though not all, do make this transition without major psychological or social difficulty. With the exception of Bryan and Treanor (2007) they pay less explicit attention to curricular achievement than to social and psychological wellbeing, but it 38 would be reasonable to suppose that such improved wellbeing may be the basis of improved learning. Bryan and Treanor do in fact explicitly make this link (in different ways) in each of their three case studies. These points about ethos and psychosocial factors lead into the fourth category of findings. High quality pedagogy While recognising the benefits of an element of planning for curricular continuity, a number of articles argue (or imply) that what is needed for effective progression is high quality learning and teaching in both the primary and secondary sectors – stimulating and challenging yet ‘safe’ tasks that will not cause pupils to fear failure (Galton, 2002), and learning that is engaging and fun (Humphrey and Ainscow, 2006). Hodgen and Marshall, (2007) highlight high order thinking, explaining and justifying ideas, engagement with peers and peer evaluation as key factors in both primary and secondary work. Formative practice that uses feedback to engage pupils in developing metacognitive and learning skills was also seen as effective in promoting subjectspecific learning. (Some ideas about this type of practice are referred to later in this review under ‘What evidence is there to suggest that particular kinds of assessment arrangement support learning more effectively as young people move from primary to secondary school?’) Various authors argue that primary and secondary teachers are stimulated and encouraged towards this kind of high quality pedagogy when they have the opportunity to meet and reflect together on learning and teaching as part of their regular working life, rather than in one-off CPD events (Davies and McMahon 2004; Sato et al 2005; Hodgen and Marshall 2006; Braund 2007). The need for planned and protected time for sustained staff development is emphasised by, eg, Bryan and Treanor (2007), Ferguson (1996) and Sato et al (2005). 2. What evidence is there to suggest that particular kinds of assessment arrangement support learning more effectively as young people move from primary to secondary school? 39 The principles for effective assessment for national qualifications drawn from the range of research and local and international activities surveyed in the SQA review (Spencer 2009) relate to characteristics of courses, criteria and tasks. Ideally there should be: o Appropriate national curricular guidance with assessment approaches designed as part of the curriculum development. o Clear criteria for success – ‘rich’, rather than an extensive list of individual learning objectives and developed in a way that involves teachers in reflecting on and interpreting the curriculum and benefits from the input of experienced assessors/examiners. There is a need to promote teachers’ capacity to define success criteria for themselves, possibly using a strategic question like: ‘How will you know that Experience/Outcome has been learned/achieved?’ o Appropriate courses designed by teachers who have reflected on, understood and, if possible, discussed with colleagues what matters to enable pupils to achieve the intended learning. o Tasks (including some interdisciplinary tasks) that show breadth of learning, provide challenge within learning, require application of knowledge and skills in new and unfamiliar situations and enable pupils to show the key learning intended/implied by the curriculum specification. An approach (which is in use in several other countries) is the provision of a design template for tasks that assess key learning and of exemplars of such tasks for use by schools and/or to serve as models for teachers’ own tasks, for example, the Queensland Comparable Assessment Tasks (QCATs) for various specified stages of school. QCATs essentially form an ‘assessment bank’ of centrally designed tasks, each with teacher guidelines, a pupils’ booklet and a guide to assessment judgements (similar to grade-related criteria in Standard Grade for ‘C’ and ‘A’ awards). The teacher guidelines link the task and its context/theme to state curriculum statements (‘Essential Learnings’) for the relevant year group and offer teaching/learning suggestions (‘Ways of Working’). 40 Use of published QCATs is not, however, obligatory. Teachers can alternatively develop their own similar assessment tasks. There is a design brief that applies to both centrally designed and school-designed tasks. This specifies the Essential Learnings to be covered at the relevant stage, sets out the methodology for collecting evidence through a QCAT, describes the purpose of all the components (including the guide to making assessment judgements) and explains the design elements used to maximise validity and reliability. QCATs are intended to operate at stages/ level specified by the state, but the model of teacher support is applicable to assessment at any level. For summative assessment, evidence of learning from a range of such well designed tasks, perhaps gathered in a portfolio, is needed to ensure that there is assessment evidence about all the key aspects of the curriculum and to justify a judgement that they have been achieved. There is a case for inviting a range of teachers in each curriculum area to come up with their own ideas about the kind of evidence that will demonstrate the learning achieved and the amount of material to be required in a portfolio, to encourage real consideration of what is needed for valid assessment, achieve a consensus, promote ownership and, perhaps, reduce workload issues. A ‘best fit’ method of judging whether a portfolio of work matches criteria for a particular level should be adopted, taking account of all the evidence in it, rather than judging each piece separately. This approach to summative assessment is also advocated by Morrison and Busch (1994). MacPhail and Halbert (2010), referring specifically to physical education, adduce evidence that is consistent with this approach. Black et al (2010) describe an intervention project with mathematics and English teachers which helped them to survey their own current practices, design a portfolio system of gathering evidence and discuss standards in moderation meetings once the evidence had been gathered. The description of this project gives an idea about the complexity and the intensity of the support activities needed to enable teachers to carry out summative assessment effectively. A key factor 41 in developing valid assessments was a recurrent process of considering their own views/values about their subject in response to questions like: ‘What does it mean to be good at English/maths?’, ‘What will someone do who completes this task very well?’ It was found that ’normal’ pre-existing coursework (especially in mathematics) did not provide the assessment evidence needed and that it was necessary to design portfolio tasks with clear assessment needs in mind, ensuring, eg, discrimination among learners as well as opportunity to demonstrate breadth and depth of learning and application of knowledge and skills. The researchers provided the teachers with a good deal of advice and exemplification of possible ‘rich’ portfolio tasks, which challenged pupils to demonstrate that they really were good at English/mathematics – that they really did have command of the learning intended by the curriculum statements. The teachers concluded that 6 mathematics tasks were needed in the portfolio to cover an appropriate range of learning and 9 English ones (3 for each of Reading, Writing and Talking/Listening) and that these arrangements were manageable. Some tasks were undertaken by pupils in controlled conditions to strengthen reliability further. Black et al point out that these two groups of teachers had found it difficult to escape from reliance on national tests (or tests modelled on external tests) as the only conceivable means of assessing achievement and progress. The process of building their ‘assessment literacy’ and helping them to become capable themselves of effective design and assessment of the portfolio work (and of moderating the assessment through intraschool and inter-school discussions) took approximately ‘two years of sustained dedication’. It is important to keep in mind in considering apparent evidence of successful assessment of learning by teachers that success depends on high quality and intensive supportive interaction with teachers of the kind described by Black et al in this study. (in a paper published after this literature review was completed, Black et al. 2011 reinforced the conclusions from the study described here.) From the journal article search some ideas emerge about the role of formative assessment as part of the process of gathering evidence of learning. McGuinness (2005) argues that to find out what learners know and can do as they construct meaning and develop their understanding and metacognitive skills requires teachers to focus on learning processes, how learners know rather than what they know, explanations and justifications rather than right or wrong answers. This approach requires tasks, assessments and performances that allow learners to show what 42 they can do, and ways and means of mediating understandings to make them more visible to both learners and teachers. Formative assessment approaches offer this kind of experience to learners. This line of thinking is also found in Hodgen and Marshall (2006), as part of their advocacy of high quality pedagogy. Houlston et al (2009) write about the value of peer-assessment to the development of personal and social skills which are important among learning skills and Kirton et al (2007) also refer to the benefits of this kind of activity, so long as pupils are aware of clear objectives and assessment criteria. Katz (2000) argues that the principal aim of assessment/evaluation activities should be to develop learners’ capacity as evaluators, rather than to assess knowledge in some kind of ‘objective’ way. Judgement of success, he says, depends on processes of agreeing with learners what matters – what will count as success. Black and Wiliam (2005) and Remesal (2007) identify difficulties arising in practice in aligning formative and summative assessment, because of the major significance in educational culture of assessment by tests/examinations and associated accountability issues, particularly in secondary schools. Black and Wiliam argue that primary and secondary teachers need to develop a common understanding of learning across the whole curriculum, which would convince them of the need to use more complex assessment approaches than tests/examinations only. Daugherty et al (2009) recognise that what matters not only in the curriculum but also in pedagogy and assessment is multi-layered and diverse in different contexts and that development of such understanding by teachers is a complex process. Boyd-Batstone (2004) argues that the key means of developing such understanding by teachers is observing children in instructional settings while maintaining a standards-based focus. A key issue is that of readily managing and analysing anecdotal records to compare a learner’s performance to the standard expected. Harlen and Crick (2003) develop this theme in some detail. Teachers should share and emphasise with pupils learning goals, not performance goals, and provide feedback to students in relation to these goals. This requires the development of learners’ understanding of the goals of their learning, the criteria by which is it assessed and their ability to assess their own work. Teachers need to utilise strategies for encouraging self-regulation in learning and positive interpersonal 43 relationships. They also need to present assessment realistically, as a process which is inherently imprecise and reflexive, with results that have to be regarded as tentative and indicative rather than definitive. Some forms of assessment do not appear to support learning effectively across the transition. James and Pedder (2006) point up the contradictions between promoting learner autonomy through making learning explicit and a focus on performance in testing; the need for systemic integrity and avoiding a gap between values and practice is underlined. Hall et al (2004) point out that teaching strategies focused on test preparation drive out other forms of assessment – peer, self-evaluation, multi-mode assessment etc – and lead to a focus on outcomes (of assessment) rather than classroom process. It is important not to ignore the potential impact of assessment on teaching. Pupil voice Although not always directly related to transition issues, a number of research studies demonstrate ways in which pupils can, indeed should, be active partners in assessment and more generally in learning. These have implications for the management of transition procedures and the sharing of information. Cowie (2005) outlines how learners had sophisticated understandings of assessment, which they could draw on in appropriate circumstances: at times pupils described assessment as a joint teacher-pupil responsibility, at others as primarily a teacher responsibility. What underpinned their varying views was the recognition that assessment should promote learning. Learners drew on a range of types of criteria as they made assessments, including their own previous experience. For effective assessment and learning there required to be a culture of trust and mutual respect among learners and teachers: intellectual and social goals were mutually supportive. Flutter (2007) argues for the importance of pupil voice. One of the conclusions of this research into pupil voice lies in its power to challenge and support teachers to move outside their familiar routines of practice and thought. Through listening to what pupils say about their experiences as learners, teachers are able to gain new insights into the factors that make a difference to pupils’ learning and progress. These new understandings, in turn, provide a useful starting point for 44 improving practice at all levels: whether it is for teachers as individual practitioners in their own classrooms; in reviewing practice for departments within a school; or as a basis in school-wide professional development planning. A key finding from Ashton’s (2008) study is that children can be a very valuable resource in informing and improving transition processes. A wide range of pupils had provided through writing, talking or drawing their views on how the transition process could be improved. Jarman (1997) points out that this sort of activity requires teachers to learn (possibly a great deal) about how to talk purposefully to pupils about their earlier experiences. Related to this is Jarman’s (1997) argument that it would be advisable for teachers to make explicit links both forwards and backwards, so that pupils can more readily appreciate what they have done before and will do in the future and to recognise this as progression rather than repetition. Brookhart (2001) and Brookhart and Bronowicz (2003) also argue the importance of not underestimating the sophistication of learners’ approaches to assessment, whether for formative or summative purposes. Their research suggests that young people’s perceptions of specific assessment tasks have an impact on how they approach these and that this changes over time. It could be suggested that the assessment tasks which secondary schools give to pupils on their arrival at the school cannot reasonably be compared with assessments undertaken previously in the primary school and that pupil understanding of what these assessments are for, and how important (or not) they are, will impact on their approach to the assessments and to the effort they put into them. Blanchard (2008) provides a number of illustrations of the means by which learners at all stages of schooling have effectively contributed to the planning of their own and of the group’s learning. He sees the involvement of learners as one of the key factors in ensuring sustainability of changed approaches to learning. Blanchard concurs with the view of Black et al (2006) that two features of effective educational development can be identified: on the one hand, learners making choices, which makes clear the intention to promote learner autonomy; on the other hand, teachers ensuring development in accord with curricular guidance. This requires dialogue involving teachers and learners at all stages of learning: before, during and after each activity. While recognising the value of assessment for learning in promoting transparency, Blanchard 45 argues that this transparency associated with assessment for learning is in itself insufficient to ensure effective learning: the interactivity required for it is more fundamental than transparency. 3. What interpretations are there of the term standards? The SQA review (Spencer 2009) focused on assessment for certification and all or almost all the material reviewed explicitly worked to or assumed a definition of standards as publicly available statements of the nature, amount and quality of knowledge and skills expected of pupils, and often as such statements of expected performance at different levels or grades. In most of the cases considered the definition of the standard was not merely a matter of a written description of expected knowledge and skills: this description was supported by exemplification of work regarded as matching it. In some countries (including Scotland) the standards statements for public examinations incorporate grade-related criteria spelling out expected qualities of work in some detail. In addition examiners’ marking schemes applying these generic criteria in particular examination tasks are also publicly available. As part of the second stage of the Assessment at Transition review it emerged that Hodgen and Marshall (2005) describe this kind of conception of standards as features of ‘expert’ practice in a particular domain and practice community, to which ‘apprentice’ pupils aspire. The scaffolded experiences provided and structured by the teacher are designed to model and make explicit the kinds and qualities of knowledge, understanding and skills to be learned, which have been identified by subject experts. However, in primary and secondary schools there can be differences of view about what should be included as standards, deriving from different cultural emphases in the two sectors. Marshall and Brindley (1998) describe differing focus in English teaching in the two sectors – on ‘literacy’ in the primary and response to literature in the secondary. The result was that secondary teachers did not recognise the information they received from primary schools about standards of performance as helpful in planning their teaching and the tasks they could set for pupils. Agreement was reached in the study schools among the primary and the secondary teachers on what was useful to pass on as samples of pupils’ work and examples of teacher assessment. After this joint planning, reaching agreement on common standards was relatively easy. Marshall and Brindley argue that teacher to teacher dialogue is central to the resolution of these differences in primary and secondary teachers’ 46 understandings of curricular areas and of the consequent various kinds of discontinuity of experience across the primary/secondary transition. Black and Wiliam (2005) and Remesal (2007) raise a different kind of problem about ‘standards’ as understood by teachers. Their research suggests that teachers tend to understand ‘standards’ as marks or grades on externally set tests, which are used to categorise learners and in turn represent teachers’ competence. Implicit in this view of standards is the concept of learning as linear, with progression perceived as succeeding on a more difficult test with higher marks – reaching a higher ‘standard’. Harlen (in Gardner et al 2010) provides guidance on how to dissociate the concept of standards as desirable expectations and aims of students’ work from standards as marks, scores or grades. She describes a standard in terms of expectation of desirable performance as essentially qualitative statements reflecting value positions. They involve • ‘indicators’ – the curricular or behavioural aspects regarded as important and which we wish to assess, eg, in science, inquiry skills, application of knowledge and communication of results; • the quality/value of performance or attributes in relation to these indicators, ie, the application of appropriate knowledge (proposing a hypothesis and making a prediction based on it); how the inquiry skills are used (identifying a question that can be scientifically investigated; making, considering and checking relevant observations; testing the prediction against the findings and drawing guarded conclusions); and reporting on the activities logically, using scientific language appropriately. Harlen and her colleagues in Gardner et al (2010) also suggest that we should avoid too narrow a view of ‘standards’ as test results by thinking of them in another way, too – as indicators of the type and quality of action taken by various educational communities – teachers, school management, inspectors/advisers and policy makers – in relation to assessment generally, and to formative and summative uses of it. 47 Pollitt’s (2001) review essay outlines the complexity of the concept of ‘standards’, the dangers of adopting facile or simple definitions of the word and the frequent pressures or temptations to adopt such easy definitions. In carrying out research into moderation Hipkins and Robertson (2011) provide a definition of ‘standards’ which recognises the complexity of the concept and of realising it in practice: ‘A standard is a complex collective of: • the words used to describe the scope of the standard • a wide range of examples of tasks that could generate evidence of achievement in relation to this standard • student work that illustrates the full range we can expect for each task • an accumulating body of judgements made across this range of work, with particular attention to examples at the boundaries between standards’ gradations • an accumulating individual and collective awareness of all of these aspects within the profession.’ Standards describe the requirements and expectations for learners at particular levels. Within New Zealand, where standards are very broad descriptors of expected achievement and which therefore required unpacking by teachers in the real classroom context, it is acknowledged that it is likely to take several assessment cycles to consolidate consistent judgements about pupils’ achievement of them. Remesal (2007) argues that staff in primary schools where there is no end-of-school test are more likely to think of learning in terms of learners making progress from where they are towards shared expectations of their learning. Sato et al (2005) raise the question whether there is actually a need for shared understandings of standards. Their research underlines the importance of time, of individual engagement with the process of change and of professional discussion amongst colleagues which takes account of individual difference and approach. The success of any innovation depends on the individual teacher. Opportunities for collaborative approaches to looking at issues around assessment would seem to be crucial, but alongside a recognition that assessment practice may look different in different classrooms. How individual teachers interpret 48 what is important in learning is relevant and important. However, the sources reviewed do not necessarily provide specific information about how approaches to standard setting based on progress from current learning and teachers’ own and collaborative understanding of learning aims and relevant assessment approaches might be developed effectively across the curriculum and across different schools, or about how they might contribute to ensuring progression in learning across the primary/secondary transition. Hayward & Hutchinson (2012 in press) suggest that currently in Scotland the language of standards is confusing. The term is in common use but there is evidence of very different understandings of what is meant by standards in the context of Curriculum for Excellence. 4. What factors influence the extent to which professional judgements are trusted? Intensive moderation There are very well established approaches to strengthening teachers’ professional capacities as assessors of pupils’ work against standards which are clearly defined and exemplified. However, there are necessary conditions for the development of these approaches. Parr and Timperley (2008) found that while teachers reported that they used a number of tools in finding out about learners’ needs they did not necessarily employ the evidence derived from these in their everyday teaching. If the valued processes of teacher discussion, feedback and collegial processes were to be effective, school leadership had to understand and promote the principles of evidence-informed decision-making. This suggests that using evidence of learner achievement to make better decisions – decisions that are likely to result in enhanced achievement – involves more than preparedness in terms of valuing such evidence. To apply this knowledge to teaching practice requires considerable knowledge of the subject from the point of view of teaching it. Up-skilling of practitioners to participate in evidence-informed decision-making with respect to practice requires professional learning on two fronts: understanding and skill in gathering and interpreting evidence and knowledge of the content to which the data refer and how to teach this, in order to apply the information gained from evidence. 49 Two of the three Scottish case studies described by Bryan and Treanor (2007) provide evidence that collaboration – in terms of joint curricular planning, primary and secondary teachers working in classrooms in the other sector, and team teaching – leads to enhanced sharing among teachers of their understanding of expectations of standards. Further, such collaboration leads teachers in both sectors to develop the range of pedagogies and classroom organisation on which they draw. It was evident that this collaboration required resources of time to develop and maintain. The link between building enduring personal relationships and enduring professional collaboration was evident. (The third of the case studies reported by Bryan and Treanor did not involve cross-sector collaboration of this nature.) The SQA review (Spencer 2009) identifies essential actions to strengthen teachers’ professional capacities as assessors of pupils’ work against grade descriptors relating to learning outcomes and performance criteria which specify the expected standard of achievement for certificate courses: o a range of exemplification of standards at each level and grade (including annotated student work illustrating the features represented by the grade descriptors); o opportunities (protected time and effective use of it) for teachers to assess against the grade descriptors and discuss their judgements, using a ‘best fit’ approach to judging the quality of portfolios in collaboration with informed standardisers/professional advisers, school colleagues and colleagues elsewhere. Wyatt-Smith et al (2010) in analysing teacher talk in moderation meetings illustrate how teachers drew on a range of evidence and criteria, from outwith as well as from within the range of material formally provided (parallel to the ways in which learners in Cowie’s (2005) article drew on a range of types of criteria). They moved back and forward among the material supplied to them (eg, statements of standards and samples of learner responses, their own existing tacit knowledge of different types, and processes of dialogue and negotiation). The published standards, while important, were alone insufficient to account for how the teachers ascribed value and awarded a grade to pupil work. The authors conclude that the tension between explicit knowledge, often provided in authoritative externally produced documents, and tacit knowledge derived from teacher experience may be able to be resolved through the provision of a carefully 50 structured framework in moderation which acknowledges the value of both types of knowledge and which will support compatibility of judgement among teachers in different schools. Reid (2007) comes to similar conclusions. This analysis of assessment moderation of writing by teachers working across sectors (within the Scottish Assessment is for Learning Programme) concludes that the process of engaging in such moderation facilitated professional learning. Discussion was promoted and supported as teachers brought together explicit documents – such as pupil texts, performance criteria frameworks and curricular documents – along with contributions from teachers’ prior experiences and current competences in relation to both pedagogy and assessment. It was important that the meetings were marked by a culture of dialogue which afforded a context for these less tangible contributions to be articulated; the role of an external person (in this case a staff tutor) in guiding the discussions was important. The clear focus of the assessment task provided a structure to enable a professional discourse that drew upon both the explicit artefacts and the implicit understandings. It was notable that the practice of negotiating success criteria with pupils led to these teachers’ moderating discussions including pedagogical considerations as well as assessment considerations – ie, experiences as well as outcomes – which led to the sharing of pedagogical practices across the sector boundary. Reid, in her literature review, relates her work to Wenger’s social theory of learning and concept of communities of practice. Reid points out the importance of bearing in mind that any individual teacher may well be a member of several different communities of practice; reference to pupil texts and the classroom contexts in which they were produced was helpful in ‘brokering’ connections between these different communities. Hipkins and Robertson (2011) draw on a small number of empirical studies of moderation: three of these cases were located in Queensland; one was that reported from Scotland in Reid (2007); others were derived from practice in Canada, New Zealand and higher education in Australia. The study distinguishes two uses of moderation. In the first of these, when standards-based assessment decisions are high stakes for students and teachers (eg, qualifications), then dependability and consistency of judgements across schools are very important. In the second case, moderation can be seen as an opportunity for rich professional conversations and learning. It is this aspect of moderation which is the focus of the study. 51 This social moderation which involves teachers discussing and negotiating judgements made about learners’ work to reach common understanding of pupil work standards also opens up opportunities for professional learning that can raise achievement. Millwood (2007) argues that the development of professional learning communities where teachers come together to share ideas and support each other encourages and supports them in participating in processes of ‘deconstructing, reconstructing and co-constructing knowledge and skills’. Beyond the immediate focus on making appropriate moderation decisions, there is potential to build new knowledge about how to more effectively teach so that students have an improved chance of achieving the outcomes targeted by the standards.’ The authors employ Klenowski and Adie’s (2009) summary of three broad types of social moderation (these are also described in Maxwell, 2002 and 2010): • ‘The calibration model: a sample of students’ work is graded by teachers individually. The teachers then discuss their judgements with the aim of reaching a consensus and common understanding of the standards. • The conferencing model: students’ work is graded by an individual teacher. Samples of work that represent different levels of performance in relation to a standard are collaboratively selected by teachers and discussed. Again, the aim is to reach consensus and common understanding. • The expert model: teachers mark all of their work and then submit it to an expert. Teachers receive feedback on whether the standards have been interpreted and applied in the way in which they were intended (Queensland Studies Authority, 2007, as cited in Klenowski and Adie, 2009).’ Discussion of grades will be supported by specifications of standards, supportive guidelines and examples of learners’ work. These will be used in various ways to support the development of shared meanings which evolve through conversations. Teachers also draw on a range of social knowledge in moderation discussions. Hipkins and Robertson propose a threefold categorisation of this social knowledge: 52 • ‘knowledge and beliefs about assessment • knowledge and beliefs about students • knowledge and beliefs about the intended curriculum.’ Teachers will draw on all of these referents as they take part in moderation discussions which may need to carry out one or more of the following: • ‘make moderation decisions about individual pieces of student work • make an overall judgement in relation to multiple pieces of work • decide the relative weightings to give to different aspects of the same piece of work.’ Given the complex concept of standards and the richness of moderation discussions, the authors conclude that it takes several assessment cycles to achieve consistent judgements by teachers about grades in relation to standards. However effective moderation meetings may be in supporting such decisions, they do not necessarily lead to improved classroom practice. The authors conclude that there is a need to build teachers’ capacity to link assessment and learning theory and to build their pedagogical content knowledge. The study provides a tentative checklist which could be used to observe and reflect on the quality of moderation meetings. In relation to internal assessment for certification, the review originally undertaken for SQA proposed that for qualifications SQA should consider, in collaboration with the Scottish Government, arrangements to build into the system highly effective quality assurance (whether for portfolios or course assessment), along the lines of the Queensland upper secondary school model – this would require in-school training (eg, using the Welsh standards portfolio or a similar system); internal moderation, guaranteed annually by the school management; and external moderation of a sizable sample of work by ‘expert teachers’. The Queensland system provides advice on internal moderation using all three of the ‘calibration’, ‘conferencing’ and ‘expert feedback’ models described by Maxwell, 2002 and 2010, and Klenowski and Adie, 2009 (see above) in a moderation sequence. Masters and McBryde (1994) established that a correlation of 0.94 existed among assessors’ scores in the Queensland system for end of school qualifications, based on intensive consensus 53 moderation arrangements – much better than levels of inter-marker reliability in external examinations. Marshall (2004) argued that moderation through discussion and the interpretation of judgements is more accurate than carefully delineated assessment criteria. Wiliam (2000) found only a 0.8 inter-marker correlation in the English National Curriculum Tests and argued that only teacher assessment of classwork could lengthen the assessment process enough to reduce unreliability to acceptable levels. The Queensland Studies Authority 2009 report on the use of Queensland Curriculum and Assessment Tasks (QCATs) in the primary and early secondary sectors in 2008 found that QCATs assessed deeper levels of knowledge, understanding and skills than previous assessment approaches and produced a distribution of performance across the five available grades and ‘reasonably consistency’ of teacher judgements, comparable to inter-marker reliability among trained test markers. Teachers’ reactions were positive: they said QCATs were meaningful tasks that emphasised critical thinking, provided evidence of real learning and gave opportunities to the full range of pupils. QCATs are intended to operate at stages/ level specified by the state, but the model of teacher support is applicable to assessment at any level. Despite the evidence of successful internal assessment of learning for certification in the very comprehensive and intensive Queensland system of quality assurance of learning and assessment activities and moderation of assessment judgements, it is important to reiterate the warning referred to above in discussing Black et al’s study (2010) – success requires a great deal of high quality support and an infrastructure that maintains effective quality assurance and moderation practice continuously. (This issue was reinforced in a paper published after this literature review was completed – in the review of the OFQUAL reliability programme (Baird et al, 2011) points out that there is no substantial body of evidence to support any general claim that the reliability of internal assessment matches that of external tests). Local arrangements Some of the literature reviewed relates to the question whether there is a need to develop such intensive sharing the standard activities for the 3-15 stages of Curriculum for Excellence (as opposed to the later qualifications stages) and, in particular, for transition at the end of primary school. The answer depends on the purposes of the assessment. If the intention is to be able to 54 record/report on P7 pupils’ knowledge and skills in relation to a national standard (or in a common way across an education authority) then something of the sort is required. If the intention is, rather, to enable primary and secondary teachers in the same cluster of schools to trust one another’s assessment judgements and to use assessment information to plan progression in learning, in the best interests of their own pupils and focused on meeting their needs, more localised and less intensive arrangements might be more appropriate. Hodgen and Marshall’s (2005) study suggests that trust in professional judgements is primarily trust on the part of the learner in their teacher as a model of expert practice in the knowledge and skills of the particular domain/discipline being studied. It would be important for teachers and pupils in both sectors to share an understanding of what ‘guild knowledge’ and ‘expert’ practice in the subject/discipline looks like, and thus what pupils are aiming for. In this local model there is still a clear need for much professional interaction between primary and secondary teachers. Capel et al (2008) write that to establish trust and respect for one another as professional practitioners requires time to nurture and to build effective working relationships. This includes primary and secondary teachers both understanding what information is needed, in what format and how it will be used to inform practice. Moreover, primary and secondary teachers must have a shared understanding of the meaning of terminology and must use the same language to describe the same thing. Black and Wiliam (2005) and Remesal (2007) argue that this kind of valuable professional interchange and trust in one another’s judgements about pupils’ progress and achievement are more likely where the focus of particular assessments is on formative support for learning, rather than summative use of results to evaluate schools (and teachers), and the assessments/tests are clearly perceived as being helpful to both teachers and learners, as part of gathering evidence of understanding. Where the emphasis is on amassing data for records, quality judgements which can be used as feedback to shape learning and practice are unlikely to be the result, especially in high status subjects like English and mathematics. Sustainability Whichever model is used, highly intensive national or EA-organised moderation or local agreements, the time and cost implications of effective in-school assessment and the associated essential quality assurance and CPD activities need to be recognised. Daugherty (2010 and 2011) 55 argues very convincingly that effective teacher assessment to describe learning and progression against published performance standards in systematic, valid and reliable ways requires a supportive infrastructure that is designed into the system and is funded and sustained over an extended period. This infrastructure includes a number of critical stages, each of which needs to be clear, in a structured process of moving from curriculum specification and course work via the judgements teachers make to the drawing of inferences about pupils’ learning. These stages in the process call for some specification of: • learning/assessment task type (the opportunity to show what has been learned) • task conditions (the curriculum and classroom setting and context) • criteria against which student performance is to be judged • performance standards (descriptions of what successful learning in the domain looks like). (They thus reflect the attention to ‘Courses, Criteria and Tasks’ found to be crucial in various publications reported in Spencer, 2009.) Daugherty also argues for attention to further requirements: • explicit procedures for each stage • use of existing teacher expertise • on-going teacher training and support • quality assurance and control arrangements. Colbert, Wyatt-Smith and Klenowski (forthcoming, 2012), in their account of the evaluation of processes for ‘Building sustainable assessment cultures: moderation, quality task design and dependability of judgement’ in the context of the Queensland Curriculum, Assessment and Reporting (QCAR) Framework, similarly identify a range of important aspects of a necessary infrastructure if teachers’ assessment summarising pupils’ learning against published standards is to be effective: • clearly specified learning domain (ie, in the design brief for the development of assessment tasks) • resources (professional development and time) for planning learning and assessment tasks and teacher-generated criteria sheets 56 • system and/or local level endorsement of programmes and embedded assessment plans • consideration of the full range of standards evident in curriculum specifications, woven into work programmes, with the assessment plans taking account of the demands of tasks and the need for evidence of the intended skills and knowledge being assessed. • a key part of this process is careful specification of criteria and standards at the task level • inter- and intra-school moderation practices to ensure teacher judgements in different classrooms/settings align with each other for consistency of interpretation and state-wide implementation of criteria and standards • ongoing professional development in task development, moderation practices, including the social protocols necessary for effective moderation, and knowledge of the legitimacy or otherwise of the various resources that may be influential in judgement. The Scottish Government and local authorities need to be committed to provision of an infrastructure of support for professional practice in assessment similar to those described by Daugherty and Colbert et al and to sustaining appropriate arrangements over time. Such a commitment to sustainability is essential to guarantee the factors necessary for achieving real change and long term development which Hayward and Spencer (2010) identified: • clear educational integrity in the arrangements in that they manifestly promote what matters in learning. • personal and professional integrity: deep and sustained professional learning about assessment, ensuring understanding of assessment purposes and potential, not simply awareness of methods/techniques, for all teachers – not only for those involved in pilot work or for ‘trainers’; this professional development involves peer networking and professional development support; time to try out approaches, reflect, discuss, adapt; and guidance on how the time can be used most effectively. • systemic integrity: clear commitment to the arrangements by the whole system, ensuring that all key players – for example, national and local policy makers, researchers, advisers, providers of formal and informal professional development activities, managers at local authority and school levels and teachers – are all firmly and evidently committed to developing and sustaining teachers’ professionalism in relation to assessment as integral 57 to the continuous process of planning, implementing and evaluating curriculum and pedagogy to promote learning. Harlen and Hayward (in Gardner et al 2010) explore ideas of sustainability relating to the need to meet the requirements of different educational communities. They argue that there are two key features to which attention should be paid if assessment developments are to be sustainable, the scientific and the moral. The scientific aspect might involve sustaining the development of assessment practices, rooted in research evidence and contextualised in the everyday circumstances of working with different learners in different contexts. The moral aspect is to recognise that all are involved in the consequences of other people’s actions. For assessment to support learners and learning, attention must be paid to competing interests and values, eg, whilst it may be helpful to provide statistical data to elected representatives to assure them of educational quality, the kinds of data collected may have a negative impact on classroom learning. Gardner et al (2010) outline actions for different educational communities to consider as a framework to help reconcile the competing demands of assessment for formative, summative and accountability purposes 58 Chapter Four: Perspectives from Practice: Overview of the Case Studies The literature review investigated four areas that reflect the key questions to be addressed in the project: 1. What leads to successful progression in learning as young people move from primary to secondary school? 2. What evidence is there to suggest that particular kinds of assessment arrangements support learning more effectively as young people move from primary to secondary school? 3. What interpretations are there of the term standards? 4. What factors influence the extent to which professional judgements are trusted? Following the review of the literature, fieldwork was carried out in 4 regions in Scotland and case study reports were compiled and shared with the participants. Across the four clusters, 28 primary and 30 secondary teachers, based in 25 primary and 4 secondary schools, gave the project information on their practice and their ideas about desirable improvements. 18 primary and 4 secondary headteachers, along with 9 primary and 8 secondary depute headteachers, contributed to the data. 106 primary and 33 secondary pupils gave accounts of their experience and identified the kinds of assessment experience they would like to have. The methodology section in Chapter Two describes the 3-stage process of analysis of the data, which has resulted in this section of the report. In relation to each of the research questions addressed by the project we provide here some salient points from the research literature followed by a summary of findings/issues from the four case study reports. We introduce this summary with a description of a range of very positive factors, drawn from the interviews with teachers and pupils and from interaction with school management staff and local authority representatives in all four clusters, which currently benefit pupils’ transition from primary to secondary school. Current good practice: the basis for effective further developments 59 • The strong commitment of both primary and secondary teachers to do all they could to ensure that transition to secondary school is well planned, smooth and stress-free and that learning in S1 should build on pupils’ primary experience and achievements. • Awareness among teachers and pupils in both primary and secondary sectors of principles of Assessment for Learning and of AfL activities likely to promote learning, such as clarification/agreement of aims and success criteria, teacher-pupil dialogue about learning and self- and peer-assessment against the agreed criteria. • Pupils’ awareness about their own learning and their potential to be partners in the development of good practice. • Transfer of much valuable information relating to social and pastoral aspects of transition for all pupils and relating to detailed aspects of learning in the case of pupils with additional support needs. • Induction arrangements that give P7 pupils very positive experiences of the secondary school/teachers, enable secondary staff to begin to get to know the pupils as a group and individually and facilitate interaction between them and their primary colleagues. • Teachers’ thoughtfulness about the basis of good continuity of learning as pupils move from primary to secondary school and their awareness of the practical issues in ensuring the most effective use of helpful information. • Teachers’ conscientiousness and care in providing for parents and for colleagues the assessment/reporting information specified by their local authority, cluster or school. • Teachers’ awareness of the main lines of Curriculum for Excellence assessment policy set out in the Building the Curriculum 5 publications, including, in some cases, the importance of involving pupils themselves in assessment and obtaining their views to inform CPD. • Teachers’ very positive reactions to professional interaction with colleagues in the other sector – for example, in curricular planning and moderation activities involving discussion of pupils’ work and/or of the standard at a level and in well planned cooperative teaching – and their enthusiasm for more opportunities for these kinds of interaction. • Very well developed local authority arrangements to support/ensure effective transition in relation to social, pastoral and additional support for learning factors; and on-going action 60 to promote and facilitate professional learning and interaction among primary and secondary teachers relating more directly to continuity of pupils’ learning, through, for example, joint curricular planning and moderation meetings. To what extent have we answered the research questions? Successful transition factors The review of the literature highlighted the problem of discontinuity arising from inadequate information and poor communication across the phases. Where they are used, transition ‘bridging’ projects should be focused on continuity of learning and not be simply a common project across phases. Projects can support successful progression if they promote collaboration amongst teachers and provide opportunities for extensive professional dialogue. Due regard should be given to ethos and psychosocial factors for all pupils. Research highlights the key role of high quality pedagogy that promotes challenge and the active engagement of learners. Understanding of what constitutes high quality pedagogy is more likely to be shared across transition when there is opportunity for professional dialogue as part of the everyday work of teachers. Secondary teachers wanted to have curriculum coverage information. There are moves towards facilitating this through plans in some authorities to timetable meetings between primary and secondary staff to share information on coverage in all curriculum areas as part of transition arrangements; and plans to ensure consistency between primary teachers’ curriculum planners and pupil records, allowing sharing of both aspects with secondary colleagues in all curricular areas. Schools in some authorities are using systems for curriculum mapping – eg, a ‘learning wall’ in one authority and spreadsheet-based software in another. Primary and secondary staff recognised the difficulty of using detailed, contextualised information on individual achievement and progress, including the pupils’ own perceptions of learning, in planning secondary learning/teaching. The tendency is for most secondary teachers to use only ‘broad’ information – such as a statement of level of achievement, with or without the Developing/Consolidating/Secure categories – to set classes or to give a general idea of the 61 level of challenge they can present to pupils. Secondary staff involved directly in the transition team or those working regularly with pupils with additional support needs were more likely to make use of detailed information. In the case of pupils with additional support needs, there were indications that this more detailed information was shared with other staff and often taken into account by them. However, secondary teachers recognised the value of establishing a basis for learning through one-to-one interaction with individual pupils, eg, during induction visits and at the start of the S1 year. More one-to-one interaction with their teachers was also the clear and emphatic request of a large number of the pupils interviewed across the clusters involved in the project. They valued intrinsic markers of success, especially in discussions with their teacher, more highly than extrinsic rewards. Some secondary teachers suggested that good curriculum coverage information and one-to-one interaction with pupils are what is essentially needed for them to be able to build on pupils’ primary school experiences and learning. Some pupils indicated they would prefer a ‘clean break’ of curricular subject on entry to secondary school, rather than a continuation of primary work. There was support for a portfolio of a pupil’s work as useful information for secondary teachers. Current practice in respect of this in the four clusters varied from one piece of writing in each of English and the modern language studied in the primary school to a range of work in all curricular areas. It was suggested by some teachers that a portfolio of work should be accompanied by a teacher narrative or teacher comments. The availability of information is not the only important factor. Interest in it and ability to use it are also important and these require shared understanding of ideas about learning within and across schools. Such shared understanding arises in a close knit community of schools in which there are many opportunities for formal and informal dialogue. Staff felt that successful transition was achieved when the children already felt secure and well known as they started in S1 and that developing close relationships between staff and a shared common vision within a 62 ‘learning community’ was key to this. In one cluster, the transition programme involved a great deal of professional interaction between the primary and secondary staff across P6 and P7, including curriculum planning, moderation meetings and team teaching. These arrangements developed an ethos of flexibility and dialogue in the schools and encouraged them to contribute and debate ideas from their own perspective. Involving parents had also helped to develop shared expectations as a community in this cluster. Differences between the two sectors were the source of some difficulty – the complexity of secondary school organisation as compared to that of a primary school could be a barrier to providing opportunities for observing pupils, team teaching and professional discussion. The literature review highlights the importance for effective progression at transition of (a) high quality pedagogy in both sectors and (b) professional interaction/dialogue about this, involving all teachers, not just those involved in a cascade approach to sharing ideas and information. The teachers interviewed recognised the value of professional interaction – and in one cluster the involvement of pupils – in planning curriculum and learning/teaching approaches, eg, in the context of developing and implementing bridging projects or other joint primary/secondary activities. They were aware of the time and resource constraints, but argued that there was a need for protected time to meet together and participate in collaborative, professional dialogue, integral to the work throughout the year. Which kinds of assessment arrangements support learning more effectively at P7/S1 transition? Research suggests that effective support for learning as young people move from primary to secondary school requires assessment approaches to be integral to curriculum development. It requires provision for well-designed, appropriately challenging learning and assessment tasks matching experiences and outcomes to ensure validity. These will enable pupils to exemplify the application of learning in new contexts and compile a portfolio of work to provide evidence of achievement in all key aspects of the curriculum. The emphasis should be on learning processes and tasks should focus on developing the learner’s capacity to understand what it means to know something rather than on right and wrong answers. Teacher judgement should be holistic, taking account of all of the 63 evidence in order to arrive at a ‘best fit’ rather than assessing each individual piece of work against a level. To develop such judgement, teachers need to have a shared understanding of achievement across the curriculum. This is a complex, resourceintensive activity that is unlikely to be addressed through conventional CPD. Good Assessment for Learning was clearly recognised in both the literature review and interviews as crucial to effective learning (integral to planning curriculum and learning/teaching; focusing on understanding; engaging pupils in rich experiences and tasks and in application of learning; involving much teacher-pupil dialogue; getting pupils to think about their own learning). There were regular references in both teacher and pupil interviews to ideas about aspects of assessment for learning and indications of, eg, setting learning intentions and clear success criteria, both with and without learner input, and use of pupil log books, personal learning planning and self- and peer-assessment activities in various primary and secondary schools/classes. This familiarity with AfL showed that the teachers and pupils recognised the importance of assessment as part of the process of learning. Pupil responses suggest that, despite the many references to assessment for learning, they felt a need for more individual consultation with their teachers to help them to identify successes and next steps. Pupils often revealed a degree of understanding of the nature of learning, referring, for example, to the importance of depth, and suggesting that teacher expectations, a clear curriculum structure and interactive pedagogy could guarantee deep learning. However, pupils also said that they didn’t see many ways in which they could actively contribute to sharing information on their learning. In relation to Assessment of Learning, there were significant variations across the four clusters, within some clusters and within secondary schools in recording assessment information/retaining work in a portfolio and in reporting to parents on pupils’ learning. In three clusters teachers were expected to make and report levels judgements using the terms ‘Developing’, ‘Consolidating’, ‘Secure’ as a grading system, whereas in the fourth a decision had been taken not to use these categories (as yet, anyway). In some schools in the same cluster teachers reported on level achievement across all curricular areas, in others only in relation to Literacy/English, 64 Numeracy/Mathematics and Health and Wellbeing, with brief comment on, typically, an aspect of learning or interest in the other areas. In all cases, teachers were very uncertain about how to make levels judgements and in some clusters each P7 teacher simply found her or his own way to do it. Whilst there were examples of moderation within primary schools, teachers felt they lacked guidance on and opportunity to discuss the process of making the kind of best fit judgement of a body of evidence about pupils’ achievement recommended by the research literature, or even the idea that this approach is the most appropriate. In at least one cluster, in the absence of guidance, there was a tendency to use 5-14 levels as benchmarks. In three clusters the local authority was using a system of tracking pupil progress which required teachers to record levels judgements electronically, in some cases, several times a year. The frequency of this requirement tended to encourage teachers in using a grading approach to individual tasks rather than developing a ‘best-fit’ judgement based on a body of work. Given the current confusion in teachers’ minds about the appropriate basis for making levels judgements, they question the value of what is being tracked and about the extent to which any judgements about CfE levels achievement can at present contribute effectively to pupils’ learning. There are indications of some confusion between individual reporting and a perceived accountability agenda, with concerns about whether current action is in fact able to provide good accountability information. There is also the question of how best to help schools to implement the Building the Curriculum 5 policy that requires them to give parents an account of pupils’ learning across the curriculum – or perhaps, whether this is even feasible, at least at the present time. The schools and local authorities were only beginning to give thought to another BtC5 policy, the development of a P7 profile for every pupil. A few schools across the clusters had been involved in pilot work in this area. There was some doubt among P7 teachers in at least two clusters about the value of the Profile for pupils’ learning or for giving information to the secondary school, if, as in some of the exemplars so far developed, it consists of the pupil’s account of experiences and interests, without reflection on learning or future aims/goals. The BtC5 RAPR guidance that ‘Profiles are primarily aimed at learners and their parents and will provide a clear statement of progress and achievement at a particular point in time’ did not seem to be well known. The policy advice on Profiling further states that it will ‘support and inform 65 transition’. Some teachers saw the Profile as an unnecessary duplication of reporting and argued that pupils’ involvement in the reporting process would achieve the intentions of the Profile. Teachers recognised a need for the local authority and/or the cluster to explore the potential of ‘Recognising Achievement’ by pupils of a wide range of knowledge and skills, including those acquired in the community outside the school, although awareness of the extent of the implications of BtC5 for using this information to plan learning was not consistent across the clusters. There was an apparent overall need for staff to discuss how to proceed with and link together different strands of work in assessment such as defining criteria, gathering evidence, making judgements, recording, reporting, profiling, and maintaining portfolios electronically or otherwise. Interpretations of Standards The literature identifies a tension between standards understood as desirable expectations and aims of students’ work and standards conceived of as marks, scores or grades (particularly among secondary staff). The concept of standards as desirable expectations and aims is a key aspect of developing effective assessment that supports learning and provides valid evidence about it. This concept entails essentially qualitative statements of the curricular or behavioural aspects regarded as important and which, therefore, should be assessed; and of the expected characteristics of performance in relation to these important aspects of learning. To become meaningful these statements must be supported by annotated exemplification of what learners have achieved. Evidence of achievement should relate to what has been identified as important in curriculum planning. Joint planning and discussion by teachers (supported as appropriate by informed critical friends) of curriculum, pedagogy and assessment can promote understanding of what standards really mean in terms of pupils’ work. 66 There was much confusion and concern among teachers in all four clusters about uncertainty and inconsistency in defining and gathering evidence relating to achievement of CfE levels and the appropriate use of the terms Developing/Consolidating/Secure. There were strongly expressed requests for guidance and exemplification. Anxiety about the absence of common understanding of the levels was heightened by the requirement in three of the local authorities to record levels judgements to enable tracking of pupils’ progress. Teachers as yet felt unable to make these judgements accurately and suspected – or in some cases knew – that they and their colleagues in other schools were adopting very different approaches to the task. In the absence of clear definition of standards, some staff were turning to other bases of decision making about CfE levels, such as alignment with the former 5-14 levels. There was a clear tendency for teachers to use the Developing/Consolidating/Secure categories as grades for particular pieces of work, rather than to adopt a best fit approach to making levels judgements based on a wide range of work. Focusing on particular pieces of work risks not recognising the complexity of a pupil’s learning – eg, that they might be strong in some but not all aspects of the area (some of the pupils were aware of this and unhappy about it). Alternatively, some teachers tended to equate level with stage so that, for example, all S1 work was third level (with exceptions made for the ‘weakest’ pupils). In the three clusters where levels judgements were required by the local authority, staff argued strongly for national definition, explication and exemplification of standards (for levels, for Developing/Consolidating/Secure and for ‘breadth, challenge and application’). They argued that, whilst there should be provision for practising teachers to influence what was eventually agreed, definition of these standards is not part of their professionalism: rather, the field of their professionalism is effective pedagogy which enables pupils to achieve nationally agreed standards. Reaction to the National Assessment Resource (NAR) was mixed. In one cluster, secondary staff who had been involved in successfully developing quality marked materials for the NAR spoke positively about the impact of the experience on their professional learning about assessment. A small number of other teachers recognised a CPD value in discussion of some NAR materials 67 and one local authority was using the NAR model to encourage primary staff from different clusters to develop task exemplars. However, the large majority of teachers interviewed made negative comments. They spoke of access problems using the GLOW system, inadequate indexing and search arrangements and major difficulty in finding what they would consider to be valuable material. In particular they felt the resource did not help with what they considered to be the most pressing need – to make the levels judgements required by the local authority. All four local authorities were facilitating authority-wide and/or cluster action in joint planning or moderation meetings. The latter focused typically on Literacy and Numeracy and there was in some clusters discussion in cluster groups of CfE levels/standards in particular subject areas. There was some joint primary/secondary planning in other curricular areas, eg, science, ICT, modern languages, social subjects, but not typically with an assessment focus. Such joint activities were regarded by staff as very valuable and they would like to have more opportunities for them. There were some indications among secondary teachers that the eventual publication of new NQ arrangements by SQA would strongly influence patterns of assessment throughout the secondary stages. Some secondary teachers’ views on assessment of learning and progression in general were largely determined by SQA models of assessment 5. Factors influencing trust in professional judgements Research into the factors that influence the extent to which professional judgements are trusted highlights the need for sophisticated approaches to local moderation arrangements, based on a range of exemplifications of standards and interaction with experts in assessment. The intensity of activity required for success raises the question of the level at which such moderation should be executed; should it be at school, LA or national level? Central to any activity should be a conception of sustained, deep professional learning for teachers in conditions which provide sufficient time for reflection, opportunities for dialogue and permission to try out different ways of working. 68 Across all four case studies we can see all of these aspects from the research literature manifest in the teachers’ interview responses about what they felt was needed. There was a high degree of consensus about the need for professional development based on clear guidance and exemplification and discussion in moderation meetings involving all of the primary and secondary teachers, not just those most immediately involved in transition arrangements. There are significant cost/resource implications of this kind of professional learning, even allowing for the reasonable expectation that some primary/secondary professional interaction should be built into the normal work of a school year. As indicated above under Interpretations of Standards, each local authority was making some provision for teachers to meet in planning and moderation meetings. However, these arrangements are probably insufficient to address the need for teachers to discuss curriculum planning, pedagogy and assessment standards in depth even in a small number of aspects of school work, let alone across the whole curriculum. Current practice represents early stages in a process that will take time to develop. We need to recognise the complexity of these activities if we are to support teachers in making secure professional judgements that can be shared across the curriculum and across phases. Key points from the overview of the case studies • There may be a case for thinking about effective continuity of learning at transition in terms of clear information regarding curriculum coverage in the primary school for secondary teachers, plus o Opportunities for them to have one-to-one interaction with pupils (in visits to the primary school, on induction days or at the start of S1), possibly focused on a portfolio of the pupil’s work o Provision of time and opportunity for the development of a collaborative learning community of primary and secondary staff, pupils and parents, enabling teachers to get to know one another’s thinking about curriculum and learning/teaching in some depth as colleagues of equal status and to get to know pupils as individual learners. 69 • There is need to support teachers as they develop greater awareness of the complex interaction among factors, all of which contribute to the overall process of learning, teaching and assessment. These include: o Clear understanding and agreement on what matters in the curriculum – what the curriculum statements mean in terms of the knowledge, skills and attitudes pupils will learn o Appropriately challenging learning and assessment tasks enabling pupils to develop and demonstrate learning of what matters o Involving pupils as partners in planning and assessing their learning o Agreeing criteria for success based on understanding of what matters and what evidence will show it is being or has been learned o Gathering evidence from classwork o Making judgements about success in relation to the criteria o Feeding back to pupils and discussing next steps in one-to-one contact o Recording work and/or summarised assessment information o Making a ‘best fit’ judgement about achievement of a level, based on a range of evidence and understanding of the expected standard o Reporting information about pupils’ learning to parents • There is need for action at a local and/or national level on o Deepening understanding of the model of good learning implied by ‘breadth, challenge and application’ o Elucidation and exemplification of level descriptors (and consideration of the use of ‘Developing, Consolidating, Secure’) and guidance on the processes employed within and across schools to make judgements using a best fit model o Elucidation of the role and value of the P7 Profile in assessing, promoting and recording pupils’ learning o Recognition of Achievement and its contribution to planning future learning. • There is need to consider the feasibility of assessing/reporting learning and levels of achievement across the curriculum in Primary education; the advantages and 70 disadvantages of focusing solely on Literacy, Numeracy and Health and Wellbeing skills across learning for levels judgements; and the nature of assessment and reporting in the other curricular areas, if these three are the sole focus of levels reporting. • There is need to ensure that assessment information used for tracking progress or reporting for accountability purposes is of high quality. If it includes a recorded level, this should be based on a best fit judgement across a wide range of work and ensure that such best fit judgements are required only infrequently. • There is need to provide enough formal and informal opportunities for joint primary/secondary planning and moderation meetings as a normal part of school life to ensure that teachers get to know one another’s thinking about curriculum and learning/teaching in depth and collaborate as colleagues of equal status. 71 Chapter Five: Where do we go from here? Chapters 3 and 4 of this report provided evidence from research and practice in the context of current policy highlighting areas where potential gaps between these three areas were beginning to emerge. The summary at the end of Chapter 4 identified key points for action derived from both the research literature and the evidence from practice. A central aim of this project was to provide advice to improve the alignment of research, policy and practice. This is the focus of this final chapter. The original plans for Curriculum for Excellence were designed in a world very different from the one in which we now find ourselves. Severe constraints in public spending have had significant direct and indirect effects on the education system including the levels of resource available to support change. At such a time it is ever more important there is efficacy of action and that people are convinced that actions that they and others take are likely to be both efficient and effective. Difficult decisions have to be made about how, when and where to focus limited resources. Time is a finite resource and decisions taken will be concerned to identify how time is best used to enhance learning. This prioritisation will mean that time spent on other activities will be reduced or the activities will no longer take place. The focus of this project was assessment at the point of transition between primary and secondary school. However, it became clear that the issues arising applied more generally to any point of transition, as a child moved from one primary or secondary class to another or from one school to another. This report argues that in the context of assessment at transition, wherever that transition occurs, attention should be concentrated on that which is liable to lead to greatest impact on children’s and young people’s learning. This is entirely consistent with both Curriculum and Assessment policy and the practices being developed by the school clusters with which we worked. The findings suggest that the current strong emphasis on assessment in government, Education Scotland, SQA, local authorities and school clusters is timely. This chapter identifies some key issues that might underpin an agenda for further action to promote better alignment between the aspirations of Curriculum for Excellence, about which there was little disagreement, and their realisation in action (which is a more challenging concept). 72 Research evidence on change that is sustainable (Gardner et al, 2010) indicates strongly that education is a complicated business and that there are no single - or simple - solutions that can address all assessment issues in schools; every school community is different, every classroom is different, every local authority is different and so attempts to provide ready-made solutions to complex problems are doomed to failure. However, transcending the complexity, in every school cluster, building personal relationships was a key feature of enduring professional collaboration. Recognising the need to respond to this complexity, in this chapter we offer some discussion of the issues identified in Chapter 4 to stimulate discussion and consequently effective action amongst policy makers and practitioners. Feedback from schools and teachers suggested that whilst schools recognise the need to contextualise ideas in their own circumstances, discussions are often easier and more productive if teachers have ideas to discuss. What is offered here are not recommendations for action intended to be implemented in every school but rather ideas to stimulate discussion amongst schools, teachers and others. Crucially, each set of ideas and consequent actions proposed will require decisions to be taken and priorities to be established. These decisions and their prioritisation will influence the quality of children’s and young people’s learning and each will have costs, principally the time and effort required to put them into practice, and benefits, their potential positive impact on learning. These costs and benefits must be considered consciously and carefully to ensure the most effective prioritisation of limited time and resources. Perhaps this process might help address what can be the most challenging decision of all, what to stop doing even although it might be a desirable thing to do. There is a very positive context for such action. The research team was impressed by the level of dedication shown by teachers, schools and local authorities as they strove to offer young people the best possible experiences of transition between primary and secondary schools. Teachers cared, school management teams cared and local authorities cared. This commitment was appreciated by young people who almost unanimously spoke very positively about their experiences. In Chapter 4 many of the features of the high quality provision already in the school clusters involved in the project are listed. 73 The research team was also impressed by the lack of complacency amongst all those involved in the project, policy makers, researchers and practitioners; the ethos of continuing improvement appeared to be securely embedded across all three communities. Policy makers, researchers and practitioners shared a common aspiration to work together to improve research, policy and practice and to offer young people in Scotland the best possible educational opportunities as they moved from primary to secondary school. In an attempt to be consistent with the advice being offered to others in this report, the research team has identified key priorities arising from the evidence gathered from the study. Detailed information of a broader range of aspects related to assessment at transition can be found in earlier chapters. In this final chapter we seek to identify an agenda for action: key areas to address to encourage a strong relationship between the aspirations of Curriculum for Excellence and how these aspirations are realised in practice. Four key topics are identified: • Developing teacher professionalism in bringing together curriculum and assessment • Managing learning and progression at transitions • Building trust in professional judgement • Ensuring intelligent accountability in Curriculum for Excellence In each of these areas building trust and respectful personal relationships was a key feature of enduring professional collaboration. An Agenda for Action Topic One: Developing teacher professionalism in bringing together curriculum and assessment Curriculum for Excellence is Scotland’s current position on what matters in learning, ie, it sums up a conversation across generations of what it means to be an educated Scot. It makes no sense to separate curriculum and assessment when the focus is learning. Curriculum provides a broad statement of what is to be learned and the central purpose of assessment is to discern how much and how well learning identified as important in the curriculum 74 • is taking place (and how further progress might be made – assessment for formative purposes) and • has taken place (and can be usefully summarised – assessment for summative purposes). However, although the term ‘curriculum and assessment’ slips easily off the tongue, keeping both together is challenging. There are dangers when assessment becomes the main focus of attention that ideas such as judgements, grading and moderation dominate thinking and discussion at the expense of what it is that the assessment is evaluating – young people’s learning. What decisions do we have to take to ensure that we assess what matters manageably? Discussions about assessment should always begin from the curriculum. The whole set of key factors in good assessment processes are ultimately based on clear understanding of learning and the curriculum – good learning and assessment tasks and success criteria; involving pupils in planning and assessing their own learning; gathering classwork evidence and evaluating success; feedback and identification of next steps; summarising achievement and progress (including, when required, making a ‘level judgement’); and reporting information about pupils’ learning to parents. All of these depend ultimately on what we believe it is important that children and young people learn and how we will know that learning is taking place and has taken place. Thinking about assessment that begins from curriculum in this way keeps the focus on validity. Messick (1989) describes validity as an integrated evaluative judgement of the degree to which empirical evidence and theoretical rationales support the adequacy and appropriateness of inferences and actions based on test scores or other modes of assessment.’ (p.13). The implication is that assessment evidence must be such as to justify the inferences that its users - eg, secondary teachers receiving information about incoming S1 pupils - make about what has been learned. To make the evidence as good a basis as possible for teachers to make a value judgement about what it means in terms of pupils’ learning it is important to gather information on what matters, rather than concentrating only on what it is easy to assess or having too early a focus on reliability (ie, whether or not all teachers are making similar judgements about pupils’ work). Reliability is important, but the first crucial requirement is validity and this requires the design of learning and assessment tasks that challenge learners to demonstrate that they are learning or have learned all that has been 75 identified as important in the curriculum, ie, what matters. Planning learning across a term, a year or a stage requires clear understanding of what is important in the curriculum and recognition of what evidence is needed to show that pupils are learning or have indeed learned what matters – what they will need to say/write/make/do. Planning particular experiences and tasks requires a clear understanding of what is important in the areas of the curriculum that are the current focus for learning. The design process should incorporate specification of success criteria based on understanding of curriculum – the criteria relate to the particular evidence that will be accepted as proof that that learning is taking place. If the teacher (and also the pupils) engaged in the learning have a clear view of what the curriculum means, then the focus is kept on what matters in learning. All other assessment elements – feedback, identification of next steps, occasional summaries of achievement and progress and reporting to parents (or other teachers) flow from the decision about what matters in learning. The process of designing tasks that enable achievement of curricular outcomes and demonstrate that achievement has happened is, in effect, operationalising the Experiences and Outcomes described in the Curriculum for Excellence guidance. This is a dynamic process in which decisions about pedagogy and assessment are derived from curricular intentions and may also influence/modify the learning that actually occurs. There are clear indications in the research literature (see Chapter 3) that teachers need guidance on and exemplification of ways of operationalising curriculum statements through pedagogical and assessment action, What decisions do we have to take to ensure that we promote good learning? Commonly, the third part of the learning triangle involving curriculum and assessment is pedagogy. From this study it was clear that schools had been influenced by a wide range of innovations, many of them based on theories of socio-cultural learning and designed to support the development of better opportunities for learners to build knowledge within a context of working with others; these had a range of names. The names matter less than the ideas that underlie them. Learners in the study recognised the importance of working with one another as part of a community concerned with the learning of every child and young person. They reported 76 that they wanted more opportunities to discuss their learning with their teachers; and they recognised the value of such conversations. Putting learners at the centre is an idea that, internationally, people have recognised as crucial to effective learning but are only beginning to understand how to put into practice. Within curriculum guidance, it involves including learners in decisions about what is to be learned, what matters in a task, what a good performance would look like and how their performance or that of their peers relates to that. Learners ultimately control learning; their engagement is crucial. However, teachers need to be aware of what is important in socio-cultural theories of learning and to develop practical examples of what these key ideas might look like in practice. A primary teacher involved in another research project with the University of Glasgow team provided the following example illustrating how learners might be more actively involved in curriculum and assessment planning. One week before she began the class topic on farming with her 10 year old pupils she put up a large empty poster on the wall of her classroom and asked the pupils to make notes on the poster of the kinds of things that they would like to learn about farming. She also contributed to the collection of ideas. On the first day of the topic she then discussed and collated these aims with the class and together they agreed a plan for the project. This process provided an opportunity for learners to demonstrate what information they already had about farming. It also provided an opportunity for the teacher and learners to plan what further evidence they might collect to show the extent to which learning had taken place and how they would decide on the quality of that learning. The teacher reported that the levels of engagement and the quality of topic work from the class had been significantly better than had been her expectations. The NAR might be utilised as a useful source of such examples. Topic Two: Managing Learning and Progression at Transitions Learning and progression are at the heart of assessment at transition and the range and quality of what the research team saw in the transition arrangements made by schools was impressive. 77 However, recognising that there has to be a degree of prioritisation, the suggestions for consideration in this section not only offer proposals on what might be done but suggest possible areas where the scope of current activities might be reduced. What decisions do we have to take to ensure that information being passed from teacher to teacher or school to school is useable, used and involves learners? To improve the sharing of information on learning and progression there should be an emphasis on developing a system where information shared is both useable and used. It is likely that different school clusters will view the implications of these terms differently but one key feature of this process is undoubtedly the extent to which all of the people who are involved in using the information are also involved in the design of the information gathering system. Also key to the process is an honest appraisal by those involved of the likelihood of action flowing from the use of the information. There are two questions to be addressed: • What information will lead to changes in curriculum planning and/or in the adaptation of classroom practices for individual learners or for groups of learners? • What information will support conversations about learning between learners and teachers? Although without doubt it would be desirable to transfer an analysis of pupil’s progress in areas across the curriculum from one teacher to the next, the findings from the project question the extent to which this is the best use of the limited teacher time available at this stage of Curriculum for Excellence. There has been a long term educational aspiration to see progression in learning realised through the transfer of information across the curriculum from primary to secondary schools, but little evidence of this aspiration being realised apart from a few exceptional cases. Very few secondary teachers change what they do as a result of detailed teacher reports from primary school. The very different primary and secondary school structures militate against the use of detailed information from primary schools in the various secondary subject departments. Whilst at some future point this may become possible, at present it may be more fruitful to focus on what information is likely to be useful and useable. 78 A clear view emerged among some secondary teachers in the project that they could build effectively on pupils’ prior learning in the primary school if two conditions were fulfilled – they received clear and comprehensive information about curriculum coverage; and they had opportunities for one-to-one contact with pupils before or just after entry to S1. Secondary teachers also liked the idea that they could see and discuss with pupils at least some relevant aspects of their primary work in a folio. These suggestions would offer opportunities for teachers to use their professional subject expertise to make decisions about learning aims and activities and next steps for particular pupils/groups, contextualised in a broad understanding of progression within their subject area. The section below, ‘Can we use professional judgement at some stages to share useful information about pupils’ progress, without reference to achievement of a level?’ includes discussion of the role of and the need to develop professional awareness of what constitutes progression. Reports to parents commonly involve teachers in considerable amounts of time providing narrative reports on pupil progress. Although, as argued above, clear understanding of curriculum could make it easier to comment on what matters in reports and parents appreciate the effort expended by teachers, questions remain about the extent to which reports are sufficiently useful to justify the amount of teacher time spent in their construction. Further investigation is needed of parents’ perspectives on different forms of communication about their children’s learning, including discussion of portfolios of work. There has been a further concern over time about secondary school being a ‘fresh start’ which ignored earlier learning; to address this there have been significant attempts to move towards great continuity of experience for learners between primary and secondary schools. However, the evidence from this project suggested that that there may be a more sophisticated concept of a fresh start in many areas of the curriculum which recognises the value of the different cultures and structures of the two sectors. Recognising and building on learners’ previous learning is crucial. However, there was evidence that learners welcomed difference: they enjoyed moving from class to class, the variety of different subjects and teachers enthusiastic about their own subject area. Some at least saw no inconsistency between this recognition and appreciation of the values of their primary school in terms of security and greater opportunities for cross-curricular 79 learning. It was clear that learners saw the differences between primary and secondary schools as part of their rite of passage. Thus, in terms of prioritisation, it might be appropriate to focus less on attempts to make primary and secondary school experiences similar for learners and more on continuity of progression in developing skills and concepts. There is no better way of building understanding than purposeful meetings of primary and secondary colleagues, informed if possible by time spent in one another’s classrooms. Learning conversations amongst primary and secondary teachers need to be a permanent part of professional life. Topic Three: Building Trust in Professional Judgement In other areas of this final chapter we have attempted to identify where time spent on current practice might be reduced. Trusting professional judgement is an area where more time is needed for teachers to be supported to build expertise within the new context of Curriculum for Excellence. What is involved in trusting teachers’ professional judgement is complicated. In this section we identify four decisions that might help to prioritise action in this key area. Schools and teachers need to decide: • Who needs to trust teachers’ professional judgement? • What needs to be done to enhance teachers’ professional judgement in Curriculum for Excellence o in assessment for learning? o in assessment of learning? • How might we best moderate teachers’ professional judgement against nationally or locally agreed standards? Who most needs to trust teachers’ professional judgement? The quality of teachers’ professional judgement is crucial to the success of Curriculum for Excellence, which promotes a range of children’s and young people’s learning that no external examination system could assess alone. The validity of the assessment system in Scotland depends on an appropriate balance of internal and external assessment. Dependable teacher assessment will be a crucial element in a high quality system. This is an area likely to need considerable emphasis over the next three to five years. 80 There is a very clear indication from the international literature review that the factors involved in trusting teachers’ professional judgement may vary depending on the context and the assessment stakes. For example, in Scotland in the early years, it may matter most that parents trust teachers’ judgment. The level of trust will be related mainly to the quality of the relationship between teachers and parents. At points of transition, whether that transition is between nursery and primary teachers, between teachers within schools or between primary and secondary teachers, what matters most is that teachers trust one another’s judgement. This trust is most likely to be built by teachers and schools sharing ideas of progression in areas of the curriculum and agreeing their own interpretations of what statements about pupil progress mean in the context of their particular school cluster. In the later years of secondary school, when assessment stakes are high, especially for young people moving on into further or higher education or work, it may matter most that society trusts teachers’ professional judgements. There the concern will be to ensure that teachers’ professional judgements are consistent with nationally specified standards for different qualifications. It may be appropriate to recognise the different influences on what matters if teachers’ judgements are to be trusted and to prioritise different aspects of the assessment process at different stages. In any case, at whichever stage of education, trust is promoted most effectively by establishing close relationships among those involved – learners themselves, parents, other teachers, other schools or society generally, represented, for example, by local and national bodies responsible for education and by elected councillors and members of parliament. Although all these relationships matter at every stage of the education process, it may be helpful to recognise that at different stages the balance and prioritisation of factors affecting trust in teachers’ professional judgement may change. What needs to be done to enhance teachers’ professional judgement in Curriculum for Excellence in assessment for learning? Ideas of Assessment for Learning are part of the thinking of many teachers in schools across Scotland and in this project teachers and pupils clearly recognised the importance of assessment 81 as part of the process of learning. This is a sound basis from which to develop practice. We would suggest two priorities for action. • To promote greater opportunities for dialogue about learning between individual teachers and pupils and amongst pupils and to reflect on the kinds of professional learning opportunity that might be offered to teachers to support them in that process. • To encourage evidence-informed decision-making – using evidence of learner achievement to make better decisions about what to teach next and how; and about individual or group next steps for learners. The basis of such evidence about what has been learned and next steps is clear understanding of the curriculum; of the kinds of learning and assessment tasks that will promote the learning embodied in that curriculum; and of what pupils need to do to demonstrate that they have learned it. The research evidence offered in this report offers insights into how both of these priorities might best be addressed. However, becoming aware of evidence from research is only the first step. Consistent with the original design of Assessment is for Learning, groups of teachers and local authority representatives should work with researchers and curriculum developers to explore how ideas might be put into practice in a range of different contexts. What needs to be done to enhance teachers’ professional judgement in Curriculum for Excellence in assessment of learning? This was the area that school clusters involved in the project found most challenging and it is therefore the area where prioritisation is perhaps most important. It is also an area where currently there are significant levels of support available from Education Scotland and the SQA. Education Scotland has recently employed four full time development officers to support schools in moderation and the SQA academy has a wide range of exemplification to promote sharing standards. There are three decisions to be taken Can we use professional judgement at some stages to share useful information about pupils’ progress, without reference to achievement of a level? In earlier primary school stages and at P7/S1 transition it may be possible for teachers within the cluster to form and share an understanding of progression in different areas of the curriculum 82 through discussions of curriculum plans and samples of pupils’ work. Key issues for discussion would include coverage of the Experiences and Outcomes in the relevant work; the learning and assessment tasks and how they relate to the curriculum; the kinds of evidence needed to demonstrate that learning has been achieved; the criteria for success in tasks; judgements made about achievement of the criteria; and the kinds of summary of pupils’ progress in learning which could be made on the basis of the work considered. Such discussion could also enhance teachers’ sense of the nature of progression in curricular areas. Awareness of ‘next steps’ from current learning related to the ‘big picture’ of progression in knowledge, skills, understanding and application in the relevant area is a key professional skill. There is a need to support teachers in developing this sense of progression based on awareness of key learning in curricular areas. Black et al (2011) describe some work in progress to find ways of describing progression in terms of concept development and reasoning skills/argumentation. Some of the Curriculum for Excellence Principles and Practice papers, which offer guidance on concept development, might be a starting point to help articulate progression. They might usefully be used along with the curriculum plans and samples of pupils’ work mentioned in the preceding paragraph in discussions about pupils’ progress to encourage evidence-based decisions about what next steps might be the priority for an individual, a group or a class. These priorities would then lead to action, perhaps to adapting curriculum plans, or serve as an agenda for discussion with individuals or with groups of learners. These kinds of activity, supported appropriately, are reported in the research literature and in the data from schools as valuable in enabling primary and secondary teachers to become aware of curriculum, learning and teaching in the other sector. This approach to describing progression could be an effective basis for reporting to parents and for passing key information to a subsequent teacher. Schools/clusters need guidance on and exemplification of the processes involved and the types of information the processes might result in, eg, in reporting to parents and passing information from the primary to the secondary school. 83 How might we make good decisions about achievement of levels? The levels in Curriculum for Excellence are very broad, commonly covering three years of schooling. The policy intention is clear that this structure will afford children and young people opportunities to learn in depth and to apply their learning in a range of challenging applications. The structure affords learners opportunities to explore concepts and opportunities for personalisation and choice. Clearly there is no intention to regard these broad statements as yardsticks to be routinely and frequently reported on. They should be used for this purpose only occasionally. There is a strong case for reporting on level achievement only at the end of stages of school associated with likely achievement of a level by most pupils – ie, P4, P7 and at an appropriate point in early secondary. It makes no sense to assign a level to a particular piece of work, as level statements are too broad to be used as success criteria in this way. Further work needs to be undertaken to establish the kinds of information most likely to be helpful to parents about their child’s progress. The role of a statement of level achievement is likely to be one factor in considering this issue. Also potentially important is consideration of the kinds of information that can emerge from the process, described above, of identifying progress through discussion of pupils’ work, without reference to a level. Decisions about levels Decisions about levels should be made on a best fit basis – a number of pieces of work, which may be of several different kinds, are compared to a level rubric and decisions are taken on the extent to which the whole body of work provides evidence that the key learning indicated in the rubric has been achieved. A level rubric is essentially a description of what kinds of evidence pupils need to show to demonstrate that they have indeed achieved what matters in the relevant curriculum area. Learners will be expected to demonstrate breadth in terms of coverage of relevant Experiences and Outcomes and, where appropriate, in their learning within particular Experiences and Outcomes; to provide evidence of the quality of work which reflects success in meeting an appropriate level of challenge for the level; and of successful application of their learning in 84 different contexts. A Curriculum for Excellence statement of Experiences and Outcomes in a curricular area provides a broad basis for level rubrics, but does not in itself constitute one. There is need to apply the process of curricular understanding advocated above as the crucial basis of all aspects of effective assessment and so develop clear statements of what matters to demonstrate the achievement of Experiences and Outcomes. These statements should not comprise a list of detailed content, but should focus on key learning only. The development of a statement of what matters at a level is not enough. The level rubric also requires a range of exemplification of pupils’ work which shows how the statement of what matters has been matched to several different kinds of pupil experience and types of work. Exemplars should be accompanied by annotations explaining how they match the requirements of the level, fully or partially. It is important to recognise that a level rubric cannot be a detailed specification of prescribed knowledge or skills required of pupils: it identifies key learning and makes clear through exemplification that there are various ways of demonstrating that the key learning has occurred. If national policy continues to expect local teachers’ professionalism to be the basis of definition and exemplification of levels achievement, school clusters need to engage in this process of development of level rubrics drawing on their own collaborative understanding of what the Experiences and Outcomes for a level mean in terms of key learning. They also need to enable teachers within and across schools to meet to discuss their judgement of the match between the statement of key learning for a level in the rubric and both the published exemplars and, in due course, exemplars of pupils’ work from their own schools. However, to be able to do this, teachers need – and, in the project, asked urgently for – national and local support, including exemplar rubrics and associated annotated exemplars of pupil work. In practice, the central provision of such rubrics and exemplification might in effect create nationally agreed guidance on making judgements about achievement of levels. Many teachers in the project considered that there ought to be such national guidance. Comment on ‘Developing/Consolidating/Secure’ at a level 85 A major reason for the establishment of Curriculum for Excellence was the perception that Education 5-14 was too detailed and restricting. Curriculum for Excellence offers a very different model of learning. However, there is a danger that a developing practice of dividing each of the five levels into sub-levels could well lead to a more limited and limiting view of learning than that which Curriculum for Excellence would wish to encourage. This approach to identifying and labelling sublevels also risks weakening the importance of the major concepts of breadth, challenge and application. We do not believe it is possible to develop rubrics and exemplification representing the concepts of Developing, Consolidating and Secure without in effect creating three separate sub-levels. Such definition of sub-levels would bring a danger of narrower, less rich curricula and learning for each sub-level than Curriculum for Excellence intends. It would also encourage labelling of pupils, with consequent constraint of breadth and challenge in the learning of those working at the ‘lower’ sub-levels, and their likely demotivation. The disadvantages of labelling are particularly relevant in the Scottish education system, within which the OECD (2007) identified as the main issue requiring attention the learning and performance of the lowest achieving 20% of the population. The advice offered above in relation to making decisions about levels is, we believe, as far as it is validly possible to go within Curriculum for Excellence – ie, definition of the rubric for a level and exemplification of work representing full and partial achievement of it. It was clear that teachers require and wish further support in appropriately describing and reporting learners’ progress within a level, The advice offered above under ‘Can we use professional judgement at some stages to share useful information about pupils’ progress, without reference to achievement of a level?’ may be helpful. However, this is also an area requiring supportive action by national agencies and education authorities. How might we best moderate teachers’ professional judgement against nationally or locally agreed standards? When the process of moderation is concerned to make judgements against nationally agreed standards (or a commonly agreed standard within a local authority) a significant aspect of the process is essentially the same as that described above for decisions about levels – it requires a 86 rubric clearly describing key learning for a level and annotated exemplification of work matching this description. What is different is the need for the rubrics for different levels to be developed by informed representative groups – drawing on knowledge of pupils’ learning at different stages and preferably responding to consultative comment on their work from other teachers, eg, in moderation meetings, so that they are recognised by all teachers as universally applicable within the country or local authority. Social moderation is at the heart of the process – teachers come together to discuss examples of pupils’ work, compare them against agreed standards, using a best fit approach and discuss their judgements. There is also a potential for moderation activities to modify national or local authority levels rubrics through feedback to those responsible for the rubrics on the appropriateness of expectations. Moderation activities can involve decisions about the quality of individual pieces of pupils’ work – eg, fully or only partially successful – and the relative weightings to give to different aspects of the same piece of work in deciding on its quality. However, they should include also best fit overall judgements of multiple pieces of work in relation to rubric statements for levels – it is this process of matching a body of pupil work to the expected standard for a level that teachers in the project found the most daunting in the absence of performance descriptors and guidance about the nature of appropriate evidence. Moderation takes time. There is evidence that moderation activities are enhanced if undertaken in collaboration with informed professional advisers as well as school colleagues. Research evidence suggests that it may take two or more years to develop a common understanding of standards of achievement, and skill in making dependable judgements against them, in a group of teachers, but this is a key process to enhance teachers’ professional judgement and public confidence in teacher assessment. How might we prioritise? As indicated earlier in this chapter, time needs to be available for teachers to collaborate in the making of Curriculum for Excellence levels judgements, whether that be within a school or 87 cluster or on a local authority or national scale. Prioritisation, as also indicated earlier, might focus on reporting level achievement only at the end of stages of school associated with likely achievement of a level by most pupils – ie, P4, P7 and at an appropriate point in early secondary. It might lead to an initial focus on particular aspects of the overall curriculum – eg, Literacy, Numeracy and Health and Wellbeing across the curriculum. Consideration would need to be given to the advantages and disadvantages of such a decision. It would also be important to consider what would be the nature of assessment and reporting in the other curricular areas, if these three are the focus of levels reporting. Other possible prioritisation issues arise, for instance in relation to the P7 Profile and consideration of its role and relationship to reporting to parents and other teachers. Topic Four: Intelligent Accountability in Curriculum for Excellence There is clear evidence from research, from Scottish education’s own history and from within this project that there are major challenges in managing the relationship between assessment for learning and assessment for wider purposes of accountability. The evidence from the project suggests that the potential exists for old challenges to emerge in new forms. For example, some schools were uncomfortable at being asked to provide information on their judgement of pupils’ progress in terms of either levels or ‘sub-levels’ within Curriculum for Excellence when they felt unprepared to do so. The key question is that asked about assessment systems by the Assessment Reform Group. (2010). How might we best design a system that is Fit for Purpose? Building a system to reconcile Assessment for Learning and Assessment for Accountability is central to that process. In this section we reflect on possible implications from research evidence on ways to align policy aspirations in learning and assessment in Curriculum for Excellence with the uses of assessment for wider purposes of accountability. The levels of accountability felt necessary by a society traditionally have tended to relate to the level of trust in that society for the service under consideration. However, more recently the use 88 of assessment evidence for purposes of school accountability has become commonplace internationally, premised on a set of ideas for which there is relatively little evidence, of which the principal one is that measuring itself leads to improvement. This is reflected in the common uncritical use of the term ‘Raising Standards’, with an implication that this focus on readily defined external, statistically derived standards will lead directly to improvement in learning. There is evidence that such a focus on standards can lead to some changes in performance data but these changes may or may not be sound evidence of real general improvement in learning. For example, it is possible for schools to focus on the improvement of small groups of particular learners and/or on specific narrowly defined aspects of learning where changed performance will lead to statistical changes in the school’s performance. It is often difficult to argue that this is real improvement. Professor Mary James (at a policy seminar for this project) argued that people will always find ways of subverting accountability systems which they consider ill-planned. There is also evidence that schools and local authorities spend significant amounts of time collecting data for purposes of accountability. When data are used to judge the performance of individuals, schools or local authorities then the stakes are high. When assessment data are used for such high stakes purposes, unintended consequences are most likely to occur. Rather than focusing on raising standards as the stimulus for change and improvement, we should consider that raised standards are the evidence of successful change directed towards improvement of learning. Scottish Education is a sophisticated education system doing relatively well when we consider data from international surveys as evidence (OECD, 2010). Scotland, England and Northern Ireland perform similarly in PISA. The table below, summarising some key PISA 2009 results, suggests that there is no evidence of a crisis in Scottish Education, but neither is there room for complacency. England Northern Ireland Scotland Reading 494 - joint 27th of 67 499 - joint 19th of 67 500 - joint 15th of 67 Mathematics 493 - joint 28th of 67 492 - 30th of 67 499 - 21st of 67 89 Science 515 - 16th of 67 511 - joint 19th of 67 514 - 17th of 67 Wales 476 - joint 38th of 67 472 - 40th of 67 496 - joint 30th of 67 The OECD review of quality and equity of schooling in Scotland (OECD, 2007) argued that whilst Scottish education does well by 80% of the population, it does less well by the other 20%. ‘Little of the variation in student achievement in Scotland is associated with the ways in which schools differ. Most of it is connected with how children differ. Who you are in Scotland is far more important than what school you attend, so far as achievement differences on international tests are concerned. Socio-economic status is the most important difference between individuals.’ (OECD, 2007, p15) This is consistent with evidence from PISA. The central task of Scottish education is to move from a system doing relatively well for the majority of its young people to a system that does extremely well for all of its population. Yet over the last 20 years publishing data about examination performance has not changed the position of schools which are at the bottom of the league tables. So far, publishing performance has not driven change and there is no solid evidence from research or practice that investing in increasingly sophisticated measurement devices will drive change. This is an issue that concerns education but its implications go far beyond education. The evidence suggests that a more sophisticated and complex model of change is necessary. Such a model would recognise that: education has multiple purposes; the education system is complex; education is concerned with learners both as individuals and as members of society; educational issues must be considered within a broader front which includes issues of social justice, poverty, housing, health and education; and change is based on building the expertise of the profession (as is consistent with the findings of the review of teacher education (Donaldson, 2011). Effective accountability systems must be consistent with the aims of the system rather than a diversion from or an obstruction to it, taking time from or distorting learning and teaching. There is a need to consider how best to develop an accountability system consistent with the aspirations of Curriculum for Excellence. Given that levels judgements should be infrequent – perhaps on three occasions during a pupil’s career from 3 to 15 – such a system would not be based solely or even mainly on data about levels of achievement. Such data certainly contribute to the evidence 90 taken into account in school self-evaluation and in evaluation carried out by others such as education authorities and Education Scotland. However, the evaluation systems need to focus very clearly on the impact of action by schools and teachers on learning within Curriculum for Excellence, a great deal of which goes on in between the occasions when levels judgements make a broad summing up of where pupils are at a particular moment. Every school in Scotland is engaged in innovation, in taking action in the context of Curriculum for Excellence to improve the quality of learning, teaching and assessment. Schools should be held accountable for what matters - the extent to which their actions are making a positive difference to children’s/young people’s learning. This requires consideration of such questions as: • What evidence (from research, policy and other practice) has been used to inform the design of the innovation to promote its chance of success? • How will we judge success? • What evidence will be gathered to determine the extent of the success of the innovation? There exist within the Scottish education system bases for the development of intelligent accountability which is consistent with aims of Curriculum for Excellence. These include the fact that Scotland has a strong tradition of self-evaluation which holds schools accountable in ways that are likely to lead to improved practices and enhanced professionalism. The system provides monitoring of the processes of and the evidence from self-evaluation through Education Scotland and local authority personnel. Many local authorities (including some involved in this project) are using standardised tests. However, standardised tests cannot provide valid information related to Curriculum for Excellence. The Scottish Survey of Literacy and Numeracy, which is directly linked to the curricular statements of Curriculum for Excellence, will provide information to Scottish Government on national standards. This anonymous survey is unlikely to lead to the sorts of negative washback features noted above. However, if the survey itself were to be extended in consultation with local authorities and made available to them it could be used to monitor standards within local authority schools. Different levels of monitoring could be considered. For example, a local authority might choose to adopt a three year cycle, where every school in the 91 local authority would be surveyed once every three years. Feedback from survey data could be fed back to schools on their performance in relation to Curriculum for Excellence as part of the improvement process. Professor Mary James (at the seminar noted above) further argued that one should always identify the worst possible scenario and then plan taking that into consideration. In the current context perhaps the worst possible scenario would be that as a society and education system we become obsessed with measurement of progress against increasingly small and narrow targets and draw attention away from the broader aspirations of Curriculum for Excellence. We have a great deal of evidence to suggest that there should be a high level of trust for the people within our education system and for the system itself. We have processes both to monitor and to support. We have an opportunity to review our accountability practices to moderate their impact and to create time for practices more consistent with the aspirations of Curriculum for Excellence. Let us hope that in Scotland we have the courage to make the most of this opportunity. Final Thoughts The aspiration to align curriculum, pedagogy and assessment has been evident in the literature of Scottish Education since the 1970s. However, although the phrase slips easily off the tongue, its realisation is deceptively challenging. There seems to have been a pattern to educational developments, a pattern that has been mirrored internationally. In educational reform there is often little disagreement about the nature of the problem identified or about the ideals of the new curriculum designed to address the issues identified. However, as plans for assessment emerge, these become embroiled in tensions between what is believed to be desirable and what is perceived to be manageable. If manageability and desirability are seen as competing concepts then the danger is that manageability will always drive desirability and will encourage regression towards the status quo. 92 Previous experience in Scotland provides clear messages about what will happen in schools if the issues identified in this report are not addressed. Firstly the practices of assessment will quickly dominate thinking about the curriculum; there was some evidence in this project of this already happening. When that happens assessment is perceived to be onerous in terms of workload. Action is then taken to reduce the total volume of assessment, all too often leaving in place the easier to manage but less valuable types of assessment. This leaves a gap between the original curriculum aspirations and the experiences of learners. It can be argued that this happened with Standard Grade, with Higher Still and with Education 5-14. At this point in the process we have an opportunity to learn from our own history and not repeat past mistakes. We hope that this report might help in the construction of the agenda to manage the assessment process rather than leaving the assessment process to manage the curriculum. Even in the context of financial constraints there are some areas of comfort. Proposals for meeting the challenges presented in this report are supported by other changes on the policy horizon, eg, Donaldson, 2011. Previous experience in Scotland also provides clear messages about the complexity of change and the need for integrity: educational, systemic and professional. It is necessary, but not sufficient, that we have in Building the Curriculum 5 and associated documentation an assessment policy for schools that is consistent with the widely accepted educational aims of Curriculum for Excellence. It is further necessary that all aspects of assessment policy are consistent with this and, in particular, that rightful demands for accountability do not lead to practice in assessment which is inconsistent with this policy. Equally, practitioners can rightly expect the system to provide them with support in taking forward practice: through exemplification, supported moderation activities and staff development; this support will involve staff at all levels of the system in planning, providing and reviewing their practice. The responsibilities of teachers proposed are consistent with the changing models of professionalism as outlined in the Donaldson review of teacher education (Donaldson, 2011). The project team would advocate that to support schools more comprehensive cost and benefit analyses of action related to curriculum, pedagogy and assessment should be developed focusing 93 on the main decisions that policy makers, education authorities and school clusters are likely to have to make, making explicit the potential advantages and disadvantages of each. However, this is a task beyond the scope of this report. The Agenda for Action outlined in this chapter needs to be addressed. It is challenging but the potential rewards are too significant for learners, for teachers and professionals and for Scottish society as a whole for it to be ignored or dealt with half heartedly. There is too much at stake. 94 References Ashton, R., 2008. Children's views and transition – improving the transfer to secondary school: how every child's voice can matter. Support for Learning, 23: 4, 176-182. Assessment Reform Group, 2006. The role of teachers in assessing learning. ARG (Available from CPA Office, Institute of Education, University of London). Baird, J-A., Béguin, A., Black, P., Pollitt, A. and Stanley, G., 2011. The Reliability Programme: Final Report of the Technical Advisory Group. Coventry: OFQUAL. (Available for download on: http://www.ofqual.gov.uk/index.php?option=com_content&view=article&id=781 ). Baron, S., Hall, S., Martin, M., McCreath, D., Roebuck, M., Schad, D., Wilkinson, E., 2001. Interim Report: Evaluation of the Learning Communities Initiative. Glasgow: Faculty of Education University of Glasgow. Black, P., 2001. Dreams, strategies and systems: portraits of assessment past, present and future. Assessment in Education, 8: 1, 65–85. Black, P. and Wiliam, D., 2005. Lessons from around the world: how policies, politics and cultures constrain and afford assessment practices. The Curriculum Journal, 16: 2, 249-261. Black, P., Harrison, C., Hodgen, J., Marshall, B. and Serret, N., 2010. Validity in teachers’ summative assessments. Assessment in Education: Principles, Policy & Practice, 17: 2, 215-232. Black, P., Harrison, C., Hodgen, J., Marshall, M. and Serret, N., 2011. Can teachers’ summative assessments produce dependable results and also enhance classroom learning? Assessment in Education. 18(4), 451-469. Black, P., McCormick, R., James, M. and Pedder, D., 2006. Learning how to learn and Assessment for Learning: A theoretical enquiry. Research Papers in Education 21: 2 119–32. Black, P., Wilson, M. and Yao, Shih-Ying. 2011. Road Maps for Learning: A Guide to the Navigation of Learning Progressions. Measurement 9 (2-3), 71-123. Blanchard, J., 2008. Learning awareness: constructing formative assessment in the classroom, in the school and across schools. The Curriculum Journal, 19: 3, 137-150. Boyd, B., Hayward, L., MacBride, G. and Spencer, E., 2008. Just making them think. Scottish Qualifications Authority. (Available at http://www.sqa.org.uk) Boyd-Batstone, P., 2004. Focused anecdotal records assessment: a tool for standards-based, authentic assessment. The Reading Teacher, 58: 3, 230-239. Braund, M., 2007. ‘Bridging work and its role in improving progression and continuity: an example from science education’. British Educational Research Journal, 33: 6, 905-926. Braund, M. and Driver, M., 2005. Pupils’ perceptions of practical science in primary and secondary school: implications for improving progression and continuity of learning. Educational Research, 47: 1, 77-91. 95 Braund, M. and Hames, V., 2005. Improving progression and continuity from primary to secondary science: pupils’ reactions to bridging work. International Journal of Science Education, 7: 3, 781-801. Brookhart, S. M., 2001. Successful Students' Formative and Summative Uses of Assessment Information. Assessment in Education: Principles, Policy and Practice, 8: 2, 153-169. Brookhart, S.M. and Bronowicz, D.L., 2003. I don't like writing. It makes my fingers hurt: students talk about their classroom assessments. Assessment in Education: Principles, Policy and Practice, 10: 2, 221-242. Bryan, R. and Treanor, M. (MVA Consultancy), 2007. Evaluation of pilots to improve primary to secondary school transitions. Edinburgh: Scottish Executive Social Research Capel, S., Zwozdiak-Myers, P. and Lawrence, J. 2007. The transfer of pupils from primary to secondary school: a case study of a foundation subject: physical education. Research in Education, 77, 14-30. Chedzoy S. M. and Burden R. L., 2007. Marking time or moving on: student perceptions of school life in year 8 and their attributions for their success and failure in learning. Research in Education, 77, 31-45. Colbert, P., Wyatt-Smith, C. and Klenowski, V., 2012. A systems level approach to building sustainable assessment cultures: moderation, quality task design and dependability of judgment. Policy Futures in Education. 4: 2 (forthcoming).. Cowie, B., 2005. Pupil commentary on assessment for learning. The Curriculum Journal, 16: 2, 137-151. Cordingley, P., 2004. The impact of collaborative CPD on classroom learning and teaching. (Available http://eppi.ioe.ac.uk/EPPIWebContent/reel/review_groups/CPD/cpd_protocol2.pdf) Cromey, A. and Hanson, M., 2000. An exploratory analysis of school-based student assessment systems. North Central Regional Educational Laboratory (NCREL) (USA). (Available at http://www.eric.ed.gov/ERICWebPortal ) Cummings, J. and Maxwell, G., 2004. Assessment in Australian Schools: Current practice and trends. Assessment in Education, 11: 1, 89-108. Daugherty, R., Black, P., Ecclestone, K., James, M. and Newton, P., 2008. Alternative perspectives on learning outcomes: challenges for assessment. The Curriculum Journal, 19: 4, 243-254. Daugherty, R., 2010. Summative assessment by teachers. In Peterson, P., Baker, E. and McGaw, B. (eds.) International Encyclopedia of Education, Vol. 3, 384-391. Oxford: Elsevier. Daugherty. R., 2011. Designing and implementing a teacher-based assessment system: Where is the infrastructure? Paper presented at the seminar Teachers’ judgments within systems of summative assessment: strategies for enhancing consistency, Oxford University Centre for Educational Assessment, St. Anne’s College, Oxford. 96 Daugherty, R., Black, P., Ecclestone, K., James, M. and Newton, P., 2009. Assessment of Significant Learning Outcomes. In Berry, R. (ed) Assessment Reform and Educational Change. Dordrecht, Heidelberg, New York, London: Springer. Davies, D. and McMahon, K., 2004. A smooth trajectory: developing continuity and progression between primary and secondary science education through a jointly-planned projectiles project. International Journal of Science Education, 26: 8, 1009-1021. Donaldson, G., 2011. Teaching Scotland’s Future: Report of a review of teacher education in Scotland. Edinburgh: The Scottish Government Ebel, R., 1972a. Essentials of Educational Measurement. New York: McGraw Hill. Ebel, R., 1972b. Some limitations of criterion referenced measurement. In Bracht, G. (ed), Perspectives in Educational and Psychological Measurement. New Jersey: Prentice-Hall. Ferguson, P., 1996. Science and primary/secondary transition. Australian Primary and Junior Science Journal, 12: 4. Flutter, J., 2007. Teacher development and pupil voice. The Curriculum Journal, 18: 3, 343-354. Fullan, M., 2003. Change Forces with a Vengeance. London: Routledge Farmer. Galton, M., 2000. The National Curriculum balance sheet for Key Stage 2: a researcher’s view. The Curriculum Journal, 11: 3, 323-341. Galton, M., 2002. Continuity and Progression in Science Teaching at Key Stages 2 and 3. Cambridge Journal of Education, 32: 2, 249-265. Gardner, J., Harlen, W., Hayward, L., and Stobart, G. (Assessment Reform Group), 2008. Changing Assessment Practice – Process, Principles and Standards. ARG. Gardner, J., Harlen, W., Hayward, L. & Stobart, G with Montgomery, M.., 2010. Developing teacher assessment. Maidenhead/New York: Open University Press/McGraw-Hill. Gilmore, A., Crooks, T., Darr, C., Hattie, J., Smith, J. and Smith, L., 2009. Towards defining, assessing and reporting against national standards for literacy and numeracy in New Zealand, in Assessment Matters 1: 2009. Wellington: New Zealand Council for Educational Research Press. Gipps, C., 2005. Assessing English 21 – some frameworks. (Available at http://www.qca.org.uk//3008.html) Gorwood, B., 1991. Primary-Secondary Transfer after the National Curriculum. School Organisation, 11: 3, 283-290. Graham, C. and Hill, M., 2003. Negotiating the transition to secondary school. Spotlight 89. Edinburgh: Scottish Executive Education Department. Hall, K., Collins, J., Benjamin S., Nind, M. and Sheehy, K., 2004. SATurated models of pupildom: assessment and inclusion/exclusion. British Educational Research Journal, 30: 6, 802-818. 97 Harlen, W., 2004. A systematic review of the evidence of reliability and validity of assessment by teachers used for summative purposes. In Research Evidence in Education Library. EPPI-Centre, Social Science Research Unit, Institute of Education, University of London. Harlen, W. and Deakin Crick, R., 2003. Testing and Motivation for Learning. Assessment in Education: Principles, Policy and Practice, 10: 2, 169-207. Harlen, W. and Hayward, L., 2010. Embedding and sustaining developments in teacher assessment in Gardner, J., Harlen, W., Hayward, L. and Stobart, G., Developing Teacher Assessment. Maidenhead/New York: Open University Press/McGraw-Hill Hayward, L and Hutchinson, C., (2012, in press) Exactly what do we mean by standards and consistency? Assessment in Education Hayward, L., Simpson, M. and Spencer E., 2006. Assessment Is For Learning: Exploring Programme Success. Study of factors underpinning successful practice in formative assessment in the Scottish Assessment is for Learning Programme. (Available at http://www.ltscotland.org.uk ) Hayward, L. and Spencer, E., 2006. There is no alternative … to trusting teachers. In Sainsbury, M., Harrison, C. and Watts, A. (eds) Assessing Reading – from theories to classrooms. Slough: National Foundation for Educational Research. Hayward, L. and Spencer, E., 2010. The complexities of change: formative assessment in Scotland. Curriculum Journal, 21: 2, 161-177. Hipkins, R and Robertson, S., 2011. Moderation and teacher learning. What can research tell us about their interrelationships? Wellington: New Zealand Council for Educational Research. Hodgen, J. and Marshall, B., 2005. Assessment for learning in English and mathematics: a comparison. Curriculum Journal, 16: 2, 153-176. Houlston, C., Smith, P. K. and Jessel, J., 2009. Investigating the extent and use of peer support initiatives in English schools. Educational Psychology, 29: 3, 325-344. Humphrey, N. and Ainscow, M., 2006. Transition club: facilitating learning, participation and psychological adjustment during the transition to secondary school. European Journal of Psychology of Education, XXI: 3, 319-331. James, M. and Pedder, D., 2006. Beyond method: assessment and learning practices and values. Curriculum Journal, 17: 2, 109-138. Jarman, R., 1997. Fine in theory: a study of primary-secondary continuity in science, prior and subsequent to the introduction of the Northern Ireland Curriculum. Educational Research, 39: 3, 291-310. Jordan, P. and MacDonald, J., 2008. Externally moderated school-based assessment in Queensland: how do we know it works? Paper to Queensland Studies Authority Senior Schooling Conference. (Available at http://www.qsa.qld.edu.au) 98 Katz, S., 2000. Competency, epistemology and pedagogy: curriculum’s holy trinity. Curriculum Journal, 11: 2, 133-144. Kirton, A., Hallam, S., Peffers, J., Robertson, P. and Stobart, G., 2007. Revolution, evolution or a Trojan horse? Piloting assessment for learning in some Scottish primary schools. British Educational Research Journal, 33: 4, 605-627. Klenowski, V., 2006. Evaluation of the effectiveness of the consensus-based standards validation process. Brisbane. Queensland Department of Education, Training and the Arts. (Available at: http://education.qld.gov.au/corporate/newbasics/html/lce_eval.html) Klenowski, V., & Adie, L., 2009. Moderation as judgement practice: Reconciling system level accountability and local level practice. Curriculum Perspectives, 29: 1, 10–28. MacPhail, A. and Halbert, J., 2010. We had to do intelligent thinking during recent PE: students’ and teachers’' experiences of assessment for learning in post-primary physical education. Assessment in Education: Principles, Policy and Practice, 17: 1, 23-39. Mansell, W. and James, M. (Assessment Reform Group), 2009. Assessment in Schools: Fit for Purpose? A Commentary by the Teaching and Learning Research Programme. Swindon: Economic and Social Research Council, TLRP. Marshall, B. and Brindley, S., 1998. Cross-phase or just a lack of communication: models of English at key stages 2 and 3 and their possible effect on pupil transfer. Changing English: Studies in Reading and Culture, 5: 2, 123-134. Marshall, B., 2004. Goals or horizons – the conundrum of progression in English: or a possible way of understanding formative assessment in English. Curriculum Journal, 15: 2, 187-199. Masters, G. and McBryde, B., 1994. An Investigation of the Comparability of Teachers’ Assessments of Student Folio, Brisbane: Tertiary Entrance Procedures Authority. Maxwell, G., 2002. Moderation of Teacher Judgements in Student Assessment. Discussion paper for the Queensland School Curriculum Council. Maxwell, G., 2004. Progressive assessment for learning and certification: some lessons from school-based assessment in Queensland. Paper to 3rd Conference of the Association of Commonwealth Examining and Assessment Boards (ACEAB). Maxwell, G.S., 2010. Moderation of Student Work by Teachers. In Peterson, P., Baker, E. and McGaw, B. (eds) International Encyclopedia of Education, Vol. 3, 457–463. Oxford: Elsevier. McGuiness, C., 2005. Behind the acquisition metaphor: conceptions of learning and learning outcomes in TLRP school-based projects. Curriculum Journal, 16: 1, 31-47. Menter, I., Hulme, M., Elliot, D., Lewin, J., Baumfield, V., Britton, A., Carroll, M., Livingston, K., McCulloch, M., McQueen, I., Patrick, F., and Townsend, A., 2010 Literature Review on Teacher Education in the 21st Century. Edinburgh: The Scottish Government. Messick, S., 1989. Validity, in: R. L. Linn (Ed.), Educational measurement 3rd ed. (New York, 99 American Council on Education), 13-103. Millwood, K., 2007. What impact does cross-curricular, collaborative moderation of asTTle writing tests have on student achievement in science? Paper presented at the New Zealand Association for Research in Education conference, Christchurch. Mons N 2009. Background paper to the study ‘National Testing of Pupils in Europe Objectives, Organisation and Use of Results’. Brussels: EACEA; Eurydice) Morrison, H. G. and Busch, J. C., 2004. Setting reliable national curriculum standards: a guide to the Angoff procedure. Assessment in Education: Principles, Policy and Practice, 1: 2, 181-199. Newton, P., 2010. Educational Assessment – Concepts and Issues: The Multiple Purposes of Assessment in Baker, E., McGaw, B. and Peterson, P. (eds) International Encyclopedia of Education, Vol. 3. Oxford: Elsevier Noyes, A., 2006. School transfer and the diffraction of learning trajectories. Research Papers in Education, 21: 1, 43-62. OECD 2007. Reviews of National Policies for Education: Quality and Equity of Schooling in Scotland. Paris: OECD OECD 2010. PISA 2009 Results: What Students Know and Can Do: Student Performance in Reading, Mathematics and Science (Volume I) Paris: OECD Parr, J. M. and Timperley, H. S., 2008. Teachers, schools and using evidence: considerations of preparedness. Assessment in Education: Principles, Policy and Practice, 15: 1, 57-71 Pollitt, A., 2001. Educational Standards. Assessment in Education: Principles, Policy and Practice, 8: 2, 261-271. Reid, L., 2007. Teachers talking about writing assessment: valuable professional learning? Improving Schools, 10: 2, 132-149. Remesal, A., 2007. Educational reform and primary and secondary teachers’ conceptions of assessment: the Spanish instance, building upon Black and Wiliam (2005). The Curriculum Journal, 18: 1, 27-38. Sadler, D. R, 1986. Subjectivity, objectivity and teachers’ qualitative judgements. (Discussion paper 5). Brisbane: Assessment Unit, Board of Secondary School Studies. Sato, M., Coffey, J. and Moorthy, S., 2005. Two teachers making assessment for learning their own. The Curriculum Journal, 16: 2, 177-191. Scharf, P.F. and Schibeci, R.A., 1990. The influence of a `Transition Science' unit on student attitudes. Research in Science and Technological Education, 8: 1, 79-88. Scottish Government, 2009. Assessment for Curriculum for Excellence – Strategic Vision, Key Principles. Edinburgh: Scottish Government. 100 Scottish Government, 2010a. Curriculum for Excellence: Building the Curriculum 5, A Framework for Assessment: understanding, applying and sharing standards in assessment for curriculum for excellence: quality assurance and moderation. Edinburgh: Scottish Government. (Available at http://www.ltscotland.org.uk/Images/BtC5SharingStandards_tcm4-630057.pdf) Scottish Government, 2010b. Curriculum for Excellence: Building the Curriculum 5, A Framework for Assessment: Reporting. Edinburgh: Scottish Government. (Available at http://www.ltscotland.org.uk/Images/CfEReportingdocument_tcm4-612845.pdf) Scottish Government, 2010c. Curriculum for Excellence: Building the Curriculum 5, A Framework for Assessment: recognising achievement, profiling and reporting. Edinburgh: Scottish Government. (Available at http://www.ltscotland.org.uk/Images/BtC5RecognisingAchievement_tcm4-641217.pdf) Scottish Government, 2011. Curriculum for Excellence: Building the Curriculum 5, A Framework for Assessment (revised version). Edinburgh: Scottish Government. (Available at http://www.ltscotland.org.uk/Images/BtC5Framework_tcm4-653230.pdf) Senge, P. and Scharmer, O., 2001. Community action research: learning as a community of practitioners. In Reason, P. and Bradbury, H. (eds), Handbook of Action Research: Participative Inquiry and Practice. London: Sage. Slavin, R. E., 1986. Best-Evidence Synthesis: An Alternative to Meta-Analytic and Traditional Reviews Educational Researcher 15: 5-11 Spencer, E., 1981. Inter-Marker Unreliability in SCE ‘O’ Grade English Composition. Is Improvement Possible? Scottish Educational Review, 13: 1. Spencer, E., 2009. Evaluation of Continuing Professional Development for ‘Assessment Literacy’. Scottish Qualifications Authority. (Report to Scottish Qualifications Authority - in press). Sutton. R., 2009. Moderation: Experiences and Possibilities. (Presentation at Scottish Learning Conference). West, P., Sweeting, H. and Young, R., 2010. Transition matters: pupils’ experiences of the primary-secondary school transition in the West of Scotland and consequences for well-being and attainment. Research Papers in Education, 25: 1, 21-50. Wiliam, D., 2000. Reliability, validity, and all that jazz. Education 3-13, 29: 3, 9-13. Wyatt-Smith, C. 1999. Reading for Assessment: how teachers ascribe meaning and value to student writing. Assessment in Education, 6: 2, 195-223. Wyatt-Smith, C., Klenowski, V. and Gunn, S., 2010. The centrality of teachers' judgement practice in assessment: a study of standards in moderation. Assessment in Education: Principles, Policy and Practice, 17: 1, 59-75. 101 Zeedyk, M. S., Gallacher, J., Henderson, M., Hope, G., Husband, B. and Lindsay, K., 2003. Negotiating the Transition from Primary to Secondary School: Perceptions of Pupils, Parents and Teachers. School Psychology International, 24: 1, 67-79. 102 Appendix One: List of Journal Articles Reviewed Author Ashton, Rebecca Date 2008 Birenbaum, Menucha 2002 Black, Paul & William, Dylan 2005 Black, Paul, Harrison, Christine, Hodgen, Jeremy, Marshall, Bethan and Serret, Natasha May-10 Blanchard, John Sep-08 Boyd-Batstone, Paul Nov-04 Braund, Martin Dec-07 Title Children's views and Transition - Improving the transfer to secondary school: how every child's voice can matter Assessing Self-directed Active Learning in Primary Schools Lesson from around the world: how policies, politics and cultures constrain and afford assessment practices Validity in teachers' summative assessments Learning awareness: constructing formative assessment in the classroom, in the school and across schools Focused anecdotal records assessment: A tool for standards-based, authentic assessment Bridging work’ and its role in improving progression and continuity: an example from science education Pupils’ perceptions of practical science in primary and secondary school: implications for improving progression Braund, Martin and Driver, Mike Mar-05 103 Source Support for Learning, 23: 4 · Assessment in Education: Principles, Policy & Practice, 9: 1, 119 — 138 The Curriculum Journal, 16: 2, 249-261 Assessment in Education: Principles, Policy & Practice, 17: 2, 215 — 232 The Curriculum Journal, 19: 3, 137-150 The Reading Teacher, 58: 3 British Educational Research Journal, 33: 6, 905-926 Educational Research, 47: 1, 77-91 and continuity of learning Braund, Martin and Hames, Vicky Brookhart, Susan M. Jun-05 Improving progression and continuity from primary to secondary science: Pupils’ reactions to bridging work apparently Successful Students' 2001 Formative and Summative Uses of Assessment Information Brookhart, Susan M. and Bronowicz, Diane L. July 2003 I Don't Like Writing. It Makes My Fingers Hurt': Students talk about their classroom assessments Bryan, Ruth and Treanor, Morag /MVA Consultancy 2007 Buckridge, Margaret Jun-08 Busher, Hugh and Hodgkinson, Keith 1995 Busher, Hugh and Hodgkinson, Keith Feb-96 Chedzoy S M and Burden R L 2005 Chedzoy S M and Burden R L Jun-07 International Journal of Science Education, 7: 3, 781-801 Assessment in Education: Principles, Policy & Practice, 8: 2, 153-169 Assessment in Education: Principles, Policy & Practice, 10: 2, 221-242 Scottish Executive Social Research Evaluation of pilots to improve primary to secondary school transitions Teaching portfolios: their International role in teaching and Journal for learning policy Academic Development, 13: 2, 117-127 Managing Interschool School Networks: across the Organisation, primary/secondary 15: 3 divide Co-operation and tension Educational between autonomous Review,48: 1 schools: A study of interschool networking. , Making the move, Research in Assessing student Education, 74 attitudes to primarysecondary school transfer Marking time or moving Research in on, Student perceptions education, 77 of school life in year 8 104 and their attributions for their success and failure in learning The formative and summative uses of a Professional Development Portfolio: a Maltese case study Chetcuti, Deborah, Murphy, Patricia and Grima, Grace Mar-06 Cowie, Bronwen Jun-05 Pupil commentary on assessment for learning Crick, Ruth Deakin and McCombs, Barbara L. Oct-06 Daugherty, Richard, Black, Paul, Ecclestone, Kathryn, James, Mary and Newton, Paul Dec-08 Davies, Dan and McMahon, Kendra Jun-04 Ecclestone, Kathryn and Pryor, John Aug-03 Ferguson, Peter Nov-96 The Assessment of Learner-Centered Practices Surveys: An English case study Alternative perspectives on learning outcomes: challenges for assessment A smooth trajectory: developing continuity and progression between primary and secondary science education through a jointly-planned projectiles project Learning Careers' or 'Assessment Careers'? The Impact of Assessment Systems on Learning Science and primary/secondary transition Flutter, Julia Sep-07 Teacher development and pupil voice From the MCEETYA Performance Measurement and Reporting Taskforce Sep-04 Galton, Maurice Oct-00 The National Year 6 and Year 10 Civics and Citizenship Sample Assessment The National Curriculum balance sheet for Key 105 Assessment in Education: Principles, Policy & Practice, 13: 1, 97-112 The Curriculum Journal, 16: 2, 137-151 Educational Research and Evaluation, 12: 5, 423-444 The Curriculum Journal, 19: 4, 243-254 International Journal of Science Education,26: 8, 1009-1021 British Educational Research Journal,29: 4, 471-488 Australian Primary & Junior Science Journal, 12: 4 The Curriculum Journal, 18: 3, 343-354 Ethos, 12: 3 The Curriculum Journal, 11: 3, Galton, Maurice 2002 Gorwood, Brian 1991 Graham, Catherine & Hill, Malcolm 2003 Hall, Kathy Collins, Janet, Benjamin Shereen , Nind, Melanie and Sheehy Kieron Dec-04 Harland, Jennie 2007 Harlen, Wynne Jun-05 Harlen, Wynne and Crick, Ruth Deakin Jul-03 Hawe, Eleanor and Browne, Isabel 2010 Hayward, Louise and Spencer, Ernest 2010 Hipkins, R & Robertson, S 2011 Hodgen, Jeremy and Marshall, Bethan Jun-05 Stage 2: a researcher’s view Continuity and Progression in Science Teaching at Key Stages 2 and 3 Primary-Secondary Transfer after the National Curriculum Negotiating the transition to secondary school: Spotlight 89 SATurated models of pupildom: assessment and inclusion/exclusion The voice of young people Teachers’ summative practices and assessment for learning – tensions and synergies Testing and Motivation for Learning', National monitoring in social studies: the reliability of assessment interviews The complexities of change: formative assessment in Scotland Moderation and Teacher Learning: What can research tell us about their interrelationships? Assessment for learning in English and mathematics: a comparison Investigating the extent and use of peer support initiatives in Houlston, Catherine, Smith, Peter May-09 K, Jessel, John 106 323-341 Cambridge Journal of Education, 32: 2 School Organisation, 11: 3 Scottish Executive British Educational Research Journal, 30: 6 Educational Journal, 104 Curriculum Journal, 16: 2, 207-223 Assessment in Education: Principles, Policy & Practice, 10: 2, 169-207 Curriculum Journal, 21: 3, 281-297 Curriculum Journal, 21: 2, 161-177 New Zealand Council for Educational Research Curriculum Journal, 16: 2, 153-176 Education Psychology, 29: 3, 325-344 English schools Huggins, Mike and Knight, Peter, Nov-97 Humphrey, Neil and Ainscow, Mel 2006 Hutchinson, Carolyn and Hayward, Louise Jun-05 James, Mary and Pedder, David Jun-06 Jarman, Ruth 1997 Katz, Steven 2000 Kirton, Alison, Hallam, Susan, Peffers, Jack , Robertson, Pamela and Stobart, Gordon Aug-07 Klenowski, Val 2000 Lindsay, Geoff and Lewis, Ann 2003 Curriculum continuity and transfer from primary to secondary school: The case of history Transition club: Facilitating learning, participation and psychological adjustment during the transition to secondary school The journey so far: assessment for learning in Scotland Beyond method: assessment and learning practices and values Fine in theory: a study of primary-secondary continuity in science, prior and subsequent to the introduction of the Northern Ireland Curriculum Competency, epistemology and pedagogy: curriculum’s holy trinity Revolution, evolution or a Trojan horse? Piloting assessment for learning in some Scottish primary schools 'Portfolios: Promoting teaching', An Evaluation of the Use of Accredited Baseline Assessment Schemes in England 107 Educational Studies, 23: 3 European Journal of Psychology of Education, XXI: 3, 319-331 Curriculum Journal, 16: 2, 225-248 Curriculum Journal, 17: 2, 109-138 Educational Research, 39: 3 Curriculum Journal, 11: 2, 133-144 British Educational Research Journal, 33: 4, 605-627 Assessment in Education: Principles, Policy & Practice, 7: 2, 215-236 British Educational Research Journal, 29: 2 MacPhail, Ann and Halbert, John 2010 We had to do intelligent thinking during recent PE: students' and teachers' experiences of assessment for learning in post-primary physical education Goals or horizons—the conundrum of progression in English: or a possible way of understanding formative assessment in English Cross-phase or just a lack of communication: models of English at key stages 2 and 3 and their possible effect on pupil transfer. Behind the acquisition metaphor: conceptions of learning and learning outcomes in TLRP school-based projects Setting reliable national curriculum standards: A guide to the Angoff procedure Marshall, Bethan 2004 Marshall, Bethan and Brindley, Sue, Oct-98 McGuiness, Carol Mar-05 Morrison, Hugh G. and Busch, John Christian, Aug-04 Noyes, Andrew Mar-06 Parr, Judy M. and Timperley, Helen S. Mar-08 School transfer and the diffraction of learning trajectories Teachers, schools and using evidence: Considerations of preparedness' Pollitt, Alastair 2001 Educational Standards' 108 Assessment in Education: Principles, Policy & Practice, 17: 1, 23-39 Curriculum Journal, 15: 2 Changing English: Studies in Reading & Culture, 5: 2 Curriculum Journal, 16: 1, 31-47 Assessment in Education: Principles, Policy & Practice, 1: 2 Research Papers in Education, 21: 1, 43-62 Assessment in Education: Principles, Policy & Practice, 15: 1, 57-71 Assessment in Education: Principles, Policy & Practice, 8: 2, 261-271 Pustjens, Heidi , Van de Gaer, Eva, Van Damme, Jan, Onghena, Patrick and Van Landeghem, Georges Jun-07 The short-term and the long-term effect of primary schools and classes on mathematics and language achievement scores The way we do it: another practical approach to transition issues in LOTE The Relationship between Teacher Assessments and Pupil Attainments in Standard Test Tasks at Key Stage 2, 1996-98 Teachers talking about writing assessment: valuable professional learning? Educational reform and primary and secondary teachers’ conceptions of assessment: the Spanish instance, building upon Black and Wiliam (2005) Two teachers making assessment for learning their own The influence of a `Transition Science' unit on student attitudes Reddan, Francoise 1998 Reeves, David J, Boyle, William F and Christie, Thomas 2001 Reid, Lesley 2007 Remesal, Ana 2007 Sato, Mistilina, Coffey, Janet and Moorthy, Savitha Jun-05 Scharf, P.F and Schibeci, R.A May-90 Smith, Emma and Gorard, Stephen 2005 They don't give us our marks': The role of formative feedback in student progress' Storey, Anne 2004 Strand, Steve Apr-06 From performance management to capacitybuilding: an escape from the cul de sac? Comparing the predictive British validity of reasoning Educational tests and national end of Research 109 British Educational Research Journal, 33: 3, 419-440 Primary Educator, 4: 6 British Educational Research Journal, 27: 2 Improving Schools 2007 10: 132 Curriculum Journal, 18: 1, 27-38 Curriculum Journal, 16: 2, 177-191 Research in Science & Technological Education,8: 1 Assessment in Education: Principles, Policy & Practice, 12: 1, 21-38 Curriculum Journal, 15: 3 Key Stage 2 tests: which tests are the 'best'? Pupil mobility, attainment and progress in secondary school The transfer of pupils from primary to secondary school, A case study of a foundation subject: physical education Transition matters: pupils’ experiences of the primary–secondary school transition in the West of Scotland and consequences for wellbeing and attainment Teachers developing assessment for learning: impact on student achievement Strand, Steve and Demie, Feyisa Sep-07 Susan Capel, Paula ZwozdiakMyers and Julia Lawrence Jun-07 West, Patrick, Sweeting, Helen and Young, Robert Mar-10 Wiliam, Dylan , Lee, Clare , Harrison, Christine and Black, Paul Mar-04 Wyatt-Smith, Claire , Klenowski, Val and Gunn, Stephanie Feb-10 The centrality of teachers' judgement practice in assessment: a study of standards in moderation Zeedyk, M. Suzanne, Gallacher, Joanne, Henderson, Margie, Hope, Gillian, Husband, Bruce & Lindsay, Kenny 2003 Negotiating the Transition from Primary to Secondary School : Perceptions of Pupils, Parents and Teachers 110 Journal, 32: 2, 209-225 Educational Studies, 33: 3, 313-331 Research in Education, 77 Research Papers in Education, 25: 1, 21-50 Assessment in Education: Principles, Policy & Practice, 11: 1, 49-65 Assessment in Education: Principles, Policy & Practice, 17: 1, 59-75 School Psychology International 2003 24: 67 Appendix Two: Frameworks for Interviews Questions for Pupil Focus Groups: P7 Introduction Explain the context of what we are trying to find out ie that the focus is on what would be effective in helping young people’s learning at transition from primary to secondary rather than on ‘social’ or pastoral care information. 9 card sort Pupils should be divided into two small groups (3 pupils in each) and asked to carry out the card sort. Both sub-groups should be recorded and notes written up of any significant points observed. The materials are included in Annexes 1 and 2. The instructions are as much for our guidance as for that of the learners. Focused questions Continue in the two small groups. Introduce this discussion by reminding young people that learning takes place in classrooms in lots of ways, through lots of other school activities, through events like inter-school sports competitions, maths challenges, football leagues, chess tournaments, art competitions, and in lots of ways out of school (eg hobbies, youth organisations, online social networks, being with friends, looking after others, community organisations, environmental movements, faith groups). 1. Thinking about all of these possibilities, would you like to talk about a time in recent weeks when you and other young people did really good work which showed people making really good progress in their learning in (as decided, one of LIT, MNU, SCN, other): a. Tell us a bit about what people did. Was it easy to do? Was it challenging? b. What made you think that this was good learning? What do you think made it so good? c. Was this learning unusual in any way? eg in a new context, a new way of learning d. Who else knows about this learning? Do other people share your view that this was good learning? e. Did your success change your attitude towards LIT/MNU/SCI/other, or the way you went about your learning in the school/classroom? 111 2. Now I would like you to think back over P7 and the kinds of learning we have just spoken about in and out of school. If secondary teachers are to help you to learn really well when you go to secondary school, what do you think they should know about your learning, eg a. what should your English teacher know? b. what should your maths teacher know? c. what should your science teacher know? 3. What part do / could / should you play in helping teachers to understand your learning, both what you know and can do and how you go about learning things, so that they can best work with you to improve your learning? Open questions Pupils should now be brought together into one group. These questions are designed to generate base line data both on participants’ understandings and views on assessment and on their perceptions of the use of such information at transition. Therefore these questions are quite general and open and do not provide too much of a steer or guidance or indeed challenge to learners’ existing thinking. 4. What sorts of information about your learning do you think that your primary school gives to the high school? 5. As you go to high school, what do you think it would be good / useful / helpful for teachers to know about your learning – both what you know and can do and how you go about learning things? 6. Can you think of any way in which you have been involved in choosing this information about your learning and/or giving it to secondary teachers? 7. Are there other sorts of information about your learning which the primary school could / should give to the high school? Two stars and a wish Finally pupils should be given the two stars and a wish proforma to complete. This is included as Annex 3. 112 Questions for Pupil Focus Groups: S1 Introduction Explain the context of what we are trying to find out ie that the focus is on what would be effective in helping young people’s learning at transition from primary to secondary rather than on ‘social’ or pastoral care information. 9 card sort Pupils should be divided into two small groups (3 pupils in each) and asked to carry out the card sort. Both sub-groups should be recorded and notes written up of any significant points observed. The materials are included in Annexes 1 and 2. The instructions are as much for our guidance as for that of the learners. Focused questions Continue in the two small groups. Introduce this discussion by reminding young people that learning takes place in classrooms in lots of ways, through lots of other school activities, through events like inter-school sports competitions, maths challenges, football leagues, chess tournaments, art competitions, and in lots of ways out of school (eg hobbies, youth organisations, online social networks, being with friends, looking after others, community organisations, environmental movements, faith groups). 1. Thinking about all of these possibilities, would you like to talk about a time in recent weeks when you and other young people did really good work which showed people making really good progress in their learning in (as decided, one of LIT, MNU, SCN, other): a. Tell us a bit about what people did. Was it easy to do? Was it challenging? b. What made you think that this was good learning? What do you think made it so good? c. Was this learning unusual in any way? eg in a new context, a new way of learning d. Who else knows about this learning? Do other people share your view that this was good learning? e. Did your success change your attitude towards LIT/MNU/SCI/other, or the way you went about your learning in the school/classroom? 2. Now I would like you to think back over first year and the kinds of learning we have just spoken about in and out of school. If secondary teachers are to help you to learn really well, what do you think they should know about your learning, eg 113 a. what should your English teacher know? b. what should your maths teacher know? c. what should your science teacher know? 3. What part do / could / should you play in helping teachers to understand your learning, both what you know and can do and how you go about learning things, so that they can best work with you to improve your learning? Open questions Pupils should now be brought together into one group. These questions are designed to generate base line data both on participants’ understandings and views on assessment and on their perceptions of the use of such information at transition. Therefore these questions are quite general and open and do not provide too much of a steer or guidance or indeed challenge to learners’ existing thinking. 1. What sorts of information about your learning do you think that your primary school gave to the high school? 2. In what ways do you think your secondary school teachers used information about what you know and can do and about how you best learn things? 3. Can you think of any way in which you have been involved in choosing this information about your learning and/or giving it to secondary teachers? 4. Are there other sorts of information about your learning which the primary school could / should give to the high school? Two stars and a wish Finally pupils should be given the two stars and a wish proforma to complete. This is included as Annex 3. 114 Annex1 Assessment at Transition 9 card sort Instructions We would like you to think a bit about how you know you are making progress and doing well in your learning. 1. You group has been given 9 cards, each with a statement about how people in school might know this. 2. As a group you are asked to talk about these 9 statements and decide how to place the cards in a diamond pattern to show how important you think each of these ideas is. 3. Place the statement you think is most important at the top, then place the two next important features on line two, the three next important features on the middle line, then two on the second bottom line, ending up with what you think is the least important statement at the bottom. You will end with a pattern like this: 1st 2nd 3rd 2nd 3rd 4th 3rd 4th 5th 4. You have each been given 3 red spots. You should stick these on the card(s) which tell us about things which happen in school. You can stick three spots on one card if it happens a lot or one spot on each of three cards if they all happen a bit (or balance 2 to 1 if you think this is best). 5. We will take a photo of your pattern. 115 Annex 2 9 card sort statements (for info) 1 2 3 I can explain to other people what I have learned I can find the right answers to questions I am asked I compare my work with the work of my friends or peers 4 5 6 I achieve the success criteria set for our learning I use what I have learned to set my own targets and goals I get rewards and certificates and prizes for the work I have done 7 8 9 People (teacher, friends, parents) praise me for work which I have produced I get good marks in tests I work with others on deciding our success criteria and then use these to check how I have done 116 Annex 3 Two stars and a wish It is part of a teacher’s job to find out how well you have learned. There are many different ways of doing this and you have been talking about some of these today. Now we would like you to tell us which you think are the two best ways that teachers can find out what you learn, and tell us a way in which you think it could be done even better. I think that these are effective ways of finding out how well I have learned: 1. 2. I think that this would be a better way for the teacher to find out how well I have learned: School Code: Stage: 117 Teacher Focus Groups – General Primary Headteachers will be asked to inform the research team about the formal school (or cluster) policy and practice for transferring assessment information to the secondary school (in literacy, numeracy, science and one other identified curricular area). They might do this by providing documentation or in a brief discussion. At least two focus group meetings per cluster – primary and secondary – each lasting not more than 1h 30 mins. All the teachers of P7 pupils across the cluster will be invited to participate in a group discussion (not just those in the 3 schools where there will be pupil focus groups). Where there is a large number of primary schools there might be two primary focus groups. A joint primary-secondary meeting to be arranged later, to agree action on the messages from the initial focus group meetings about what is needed to further improve/develop current practice in transferring information to support learning across the transition. The research team will aim to ensure that the group discussion focuses on assessment for learning by indicating prior to and at the start of each meeting that we know that a lot of information may be passed on relating to, eg, pastoral care and additional support but we are not focusing on this in this project. 118 Primary 7 Teacher Focus Group Discussion Questions 1. How does assessment support/contribute to learning in your classroom? Could you give examples of effective use of assessment contributing to learning? Prompts Literacy Numeracy Science The other identified curricular area 2. How is information on learning recorded? What types and range of learning are assessed? What range of assessment information is recorded and available? Teacher records and/or pupils' own records? Prompts Curricular Areas? Interdisciplinary learning? Wider achievement? Other types? 3. What kind(s) of information about pupils’ learning is transferred from primary to secondary? Particular examples? 4 How is this assessment information transferred? Prompts Report? Meeting? Other? 119 Different in different curricular areas? Different for different pupils? 5. Why is the assessment information transferred? What are the purposes / intentions of transferring assessment information? Prompt Examples of intended purpose/intention in particular curricular areas. 6. How would you expect P7 assessment information to be used to support learning in S1? (Or, how is it used, if you know this?) Prompt Examples in particular curricular areas. 7. What do you think about the process of transferring the assessment information to the secondary school? Does it achieve the intended aim of supporting S1 learning effectively? Is it manageable? What opportunities exist for primary and secondary teachers to discuss assessment information for particular pupils? 8. When people use the term ‘sharing standards’ what does this mean to you? What steps is your school/cluster taking to define Curriculum for Excellence standards in literacy, numeracy, science and the other identified curricular area and to support you in understanding them? Prompts Use of Es and Os? Discussions on tasks and topics Moderation and quality assurance meetings? NAR? 120 CPD? Other arrangements? 9. In relation to assessment to support pupils' learning across the primary/secondary transition, what is needed to improve /develop existing arrangements? Prompts For individual teachers? School/cluster procedures? In particular curricular areas? In interdisciplinary learning? Wider achievement? Ensuring breadth, challenge and application of knowledge? Ensuring for pupils a process of developing knowledge/skills, consolidating and achieving secure grasp? Designing assessment tasks or class work that provide good assessment evidence? Identifying criteria of success in tasks? Getting an idea of CfE standards and what evidence would show achievement of them? Organising/taking part in moderation meetings? 10. What role do learners play in the process? What role might learners play in it? 121 Secondary 1 Teacher Focus Group Discussion Questions 1. How does assessment support/contribute to learning in your S1 class(es)? Could you give examples of effective use of assessment contributing to learning? Prompts In each subject area – 2. What range of information on learning do you receive for S1 pupils? In what format(s) do you receive it? Particular examples? Prompts Report? Meeting? Other? Different in different curricular areas? Different for different pupils? 3. To what extent is it possible to use the information you receive? Why? How does assessment in P7 support learning in S1? Can you give me an example of an occasion when you used information from a primary school to influence what you taught the class in general? Can you give an example of an occasion when you used information from a primary school about a particular child in your class to influence what or how you taught him/her? Prompt Examples of effective use of P7 information. 4. What do you think about the process of transferring the assessment information to the secondary school? Does it achieve the intended aim of supporting S1 learning effectively? 122 How useful is it? What opportunities exist for primary and secondary teachers to discuss assessment information for individual pupils? 5. What other assessment information would you like to have that would improve learning? What types of information would you be able you use in order to plan progression? Could assessment in P7 support learning in S1 more effectively? How? What would be needed? Prompts Maths, English, Numeracy, Literacy, Science, the other identified subject area? Interdisciplinary learning Wider achievement? 6. Should there be a single system of transferring assessment information for all curricular areas? Should there be a single system for all schools in a cluster? 7. When people use the term ‘sharing standards’ what does this mean to you? What steps is your school/cluster taking to define Curriculum for Excellence standards in literacy, numeracy, science and the other identified curricular area and to support you in understanding them? Prompts Use of Es and Os? Discussions on tasks and topics Moderation and quality assurance meetings? NAR? CPD? Other arrangements? 123 8. In relation to assessment to support pupils' learning in S1 and across the primary/secondary transition, what is needed to improve /develop existing arrangements? Prompts For individual teachers? School/cluster procedures? In particular curricular areas? In interdisciplinary learning? Wider achievement? Ensuring breadth, challenge and application of knowledge? Ensuring for pupils a process of developing knowledge/skills, consolidating and achieving secure grasp? Designing assessment tasks or class work that provide good assessment evidence? Identifying criteria of success in tasks? Getting an idea of CfE standards and what evidence would show achievement of them? Organising/taking part in moderation meetings? 9. What role do learners play in the process? What role might learners play in the process? 124