Please฀cite฀this฀paper฀as: Benseman,฀J.฀and฀J.฀Comings฀(2008),฀“Case฀Study:฀United฀ States”,฀in฀Teaching,฀Learning฀and฀Assessment฀for฀Adults:฀ Improving฀Foundation฀Skills,฀OECD฀Publishing. http://dx.doi.org/10.1787/172224183252 Teaching,฀Learning฀and฀Assessment฀฀ for฀Adults Improving฀Foundation฀Skills Case฀Study:฀฀ United฀States John฀Benseman฀and฀John฀Comings Centre฀for฀Educational฀Research฀and฀Innovation 002008091new.indd฀฀฀7 07-Feb-2008฀฀฀5:17:28฀PM Please cite this paper as: Benseman, J. and J. Comings (2008), “Case Study: United States”, in Teaching, Learning and Assessment for Adults: Improving Foundation Skills, OECD Publishing. http://dx.doi.org/10.1787/172224183252 TEACHING, LEARNING AND ASSESSMENT FOR ADULTS IMPROVING FOUNDATION SKILLS Case Study: United States John Benseman and John Comings TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 CASE STUDY – UNITED STATES – 3 Table of Contents Introduction ................................................................................................................... 5 The adult basic education system................................................................................. 6 The national reporting system ..................................................................................... 9 Formative assessment ................................................................................................. 11 Observations of teaching ............................................................................................... 11 The sites ........................................................................................................................ 12 Classroom settings ........................................................................................................ 15 The learners ................................................................................................................... 16 Learner outcomes .......................................................................................................... 18 The teachers .................................................................................................................. 18 Curriculum .................................................................................................................... 21 Broader contexts............................................................................................................ 22 Observation framework ................................................................................................. 24 Learning intentions and criteria for success .................................................................. 24 Activating students as owners of learning ..................................................................... 26 Teacher questioning and learner discussions ................................................................ 27 Use of feedback ............................................................................................................. 29 Students as resources for learning ................................................................................. 32 Conclusion.................................................................................................................... 33 References .................................................................................................................... 34 TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 CASE STUDY – UNITED STATES – 5 The United States has an extensive adult education and training system. In 1998, the federal Workforce Investment Act (WIA) consolidated more than 50 employment, training, and literacy programmes into three block grants for states to use for adult education and family literacy, disadvantaged youth, and adult employment and training services. WIA‟s primary focus is on preparing people for employment and on family literacy. While adults participating in WIA may also pursue other goals, such as citizenship or personal improvement, states are held accountable for outcomes related to learning gains as measured by standardized tests, acquisition of a high school diploma or equivalent such as the General Educational Development Test (GED), and employment. This study describes how eight programmes providing adult base education (ABE), adult secondary education (ASE), and English as a Second Langauge (ESL) have shaped their services to meet local needs. The programmes are in three states – California, Maryland and Massachusetts. California has almost 25% of the total national adult learner population. The California programme is also well-funded, with most of the funding coming from the state government, rather than the national government. Maryland is in the midst of a reform movement that is leading to greater funding and more comprehensive state management of programme services. Massachusetts began a reform movement 20 years ago that has led to well-resourced and well-managed programmes. The case study authors describe practice in these exemplary programmes. They find that while there is a strong focus on testing (summative assessment), instructors use a variety of strategies to individualise and contextualise learning (formative assessment). The case study also explores challenges involved in building a stable staff and finding financial resources and time for training and development in different states and programmes. Introduction As with many other OECD countries, the United States of America (U.S.) estimated the literacy skills of its adult population as part of the International Adult Literacy Survey (IALS). The U.S. published the results of its National Adult Literacy Survey (NALS), which employed what became the English version of the IALS test (National Center for Education Statistics 1993), and then published the results of a second estimate in 2006 after the completion of the National Assessment of Adult Literacy (NAAL), which employed a new version of the same test. The NAAL (National TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 6 – CASE STUDY – UNITED STATES Center for Education Statistics, 2006) changed the familiar five levels of skill identified in the IALS to four levels, labelled: below basic, basic, intermediate, and proficient. In addition, it eliminated several million adults from the sample who were unable to answer any of the test questions. The score range in the below basic level is slightly beneath that of IALS Level 1, while the score range in basic is slightly below that of IALS Level 2. Though the actual reporting of NAAL levels is different, any particular score on the NAAL is considered equivalent to that score on the IALS and NALS. Both the NALS, which collected data in 1992, and the NAAL, which collected data in 2003, found that a substantial portion of the adult population had skills in the two lowest levels. Table 1 summarizes the data for the NALS and NAAL. Table 1 Level Percentage of adult population* Score range NALS 1 NAAL Below Basic 22% 16% 0-225 0-209 NALS 2 NAAL Basic 24% 28% 226-275 210-264 NALS 3, 4 & 5 NAAL Intermediate & Proficient 54% 56% 276-500 265-500 * Adults 16 years of age and older The publication of the NALS began a public debate on how to address this issue. Out of that debate, came a new emphasis on improving the teaching of reading in the early grades of school and the provision of adult literacy services. At the time of the publication of the NAAL, the adult literacy system had been expanded and improved. This case study describes the national U.S. adult basic education system, three state systems (California, Maryland, and Massachusetts), and programmes in each state. The adult basic education system Government-funded adult basic education services began early in the history of the U.S. Sticht (2002)1 identifies the first national programme as 1 Sticht, T.G. (2002), “The Rise of the Adult Education and Literacy System in the United States: 1600-2000”, in J. Comings, B. Garner and C. Smith (eds.), The Annual Review of Adult Learning and Literacy, Volume 3, San Francisco, Jossey-Bass, http://www.ncsall.net/?id= 576 TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 CASE STUDY – UNITED STATES – 7 occurring in 1777, when General George Washington directed his chaplains to teach basic literacy skills to soldiers in the Continental Army at Valley Forge during the War of Independence. Sticht describes many different temporary programmes that came and went for most of the next 200 years. In 1966, the U.S. government established a permanent adult basic education system that grew to include many different funding programmes. In 1998, the government established the Workforce Investment Act (WIA) to consolidate more than 50 employment, training, and literacy programmes into three block grants for states to use for adult education and family literacy, disadvantaged youth, and adult employment and training services (National Institute for Literacy Policy Update, National Institute for Literacy, 1998). In addition to its specific authorization of adult education services, WIA encourages the co-ordination of efforts across employment, training, and adult basic education programmes. This co-ordination is fostered by “onestop” centres, local agencies within each state through which adults can gain access to an array of job training, education, and employment services. WIA‟s focus on preparing people for employment and on family literacy does not necessarily diminish the importance of services geared toward adults pursuing their education for other purposes, such as citizenship or personal improvement. However, these outcomes are considered secondary. The primary outcomes are: learning gains, measured by standardized tests, acquisition of a high school diploma or equivalent such as the General Education Development Test (GED), and employment. The basic skills component of WIA is funded through federal and state funds and administered by state agencies that fund programmes that provide services. The proportion of state funding varies. WIA requires each state to match the federal share with an equal amount of funding. Only ten states exceed this minimum, including states with large populations. Thus total state funding is almost three times the federal funding level. Total state and federal funding is around USD 2 billion per year. Some programmes follow a classroom format, while others use one-onone tutoring, and still others combine the two approaches to instruction. Many large programmes are able to offer classes at different skill levels, while smaller programmes can only offer a few classes that must accommodate a group of students with a range of abilities. Programmes offer classes that range from a few to 20 hours per week. Some programmes run in closed cycles of a few months to a year, while others have ongoing classes with open-entry admission that fills the seats of students who withdraw. Class sizes vary from small to large. Classes are held in a variety of venues, including community centres, social service agencies, workplaces, libraries, prisons, community colleges, churches, and schools. TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 8 – CASE STUDY – UNITED STATES Most of these programmes provide classes in basic skills (literacy and numeracy), English language, and high school equivalence. The first level of basic skills instruction is provided to students with very poor reading skills, often described as below the fifth-grade level, equivalent to IALS Level 1. Adults at this level may have learning disabilities (LD) that hinder their ability to decode the sounds of a word with the ease needed to read effectively. Given that a clinical LD diagnosis is expensive, programme staff often make informal LD assessments. However, some younger participants were identified (but not necessarily tested) for LD while in school. Instruction at this level requires a teacher who is well trained and a student who has the motivation and time to work on basic reading skills. Adults at this level usually need instruction in basic math as well. The second level of basic skills instruction is provided to students with neither severe learning disabilities nor significant problems with decoding. Adults at this level read at between fifth- and eighth-grade levels, equivalent to IALS Level 2. People at this level usually earn low scores on tests of oral vocabulary and background knowledge, as well as reading fluency (speed and accuracy). This situation requires students to engage in a good deal of practice in reading, writing, and math and direct instruction that builds vocabulary and fluency. English language instruction is usually provided at beginning, intermediate, and advanced levels, though some programmes have more than three levels of instruction. These levels are assessed by tests that measure Student Performance Levels (SPL),2 which range from 0 to 10. Beginning classes usually serve adults who are in the range of SPL 0 to 4. Intermediate classes usually serve adults who are in the range of SPL 5 to 6, and advanced classes usually serve adults who are in the range of SPL 7 to 10. This instruction begins with oral language development but eventually includes literacy and math in English as well. The General Educational Development Test (GED) – a set of five tests that measure writing skills, social studies, science, interpretation of literature and the arts, and math – is the most common way to earn a high school equivalence credential.3 To pass the test, an adult must achieve a minimum total score and a minimum score on each of the five subject tests. The GED 2 3 The 10 SPL levels are assessed by the Basic English Skills Test (BEST) is published by the Center for Applied Linguistics in Washington, D.C. In California, the acquisition of a GED does not allow students to earn a high school diploma. The GED is not always considered to be the equivalent of a high school diploma, although postsecondary institutions generally honour this equivalency. TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 CASE STUDY – UNITED STATES – 9 passing score is determined by testing high school graduates and setting the passing level at the point where 40% of high school graduates do not pass. However, states may set their own passing levels, so a test taker in one state may fail even if his or her test score is the same as someone who passes in another state. A newly revised version of the test was released in 2002. The new test has a more explicit emphasis on cross-disciplinary skills, such as information processing, problem solving, and communication, and also requires the student to write an essay. The math section demonstrates a greater emphasis on data analysis, statistics, and probability. Several other approaches to high school equivalence are available. According to data for Programme Year 2003-20044 on the Department of Education‟s Web site, 2 677 119 adults were provided with WIAsupported services. Of those, 1 172 579 (43.8%) received English language services, and 442 726 (16.5%) received preparatory services to take the GED, earn a high school diploma, or gain a high school equivalence in some other way. An additional 1 061 814 (39.7%) adults were provided with basic skills services, including instruction on reading, writing and mathematics that lasts until the student is ready to begin studying for the GED. National funding that year was USD 561 042 109, while states contributed an additional USD 1.5 billion, demonstrating that over the last ten years, total funding has doubled. The national reporting system WIA allows states the freedom to design services in any way. Programmes receiving federal funds are encouraged but not required to use instruction methods based on evidence from scientific research. Scientific research is defined as studies that use a rigorous methodology, particularly experimental or quasiexperimental, and have been peer reviewed. States must meet goals defined by the National Reporting System (NRS). The NRS sets out three core indicators of performance: 1. Demonstrated improvements in literacy skill levels in reading, writing and speaking English, numeracy, problem solving, English language acquisition, and other literacy skills. The benchmark is moving up one “NRS Level”. There are six NRS Levels for English for Speakers of Other Languages (ESOL) and six for literacy and numeracy, each equivalent to two grade-level equivalents. A student 4 http://www.ed.gov/about/reports/annual/ovae/2004aefla.pdf TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 10 – CASE STUDY – UNITED STATES might be one point away from moving up a level or might be two complete grade levels away from moving up a level, but both transitions are considered sufficient improvement. 2. Placement in, retention in, or completion of postsecondary education, training, unsubsidized employment, or career advancement; and 3. Receipt of a GED, high school diploma, or a recognized equivalent. These indicators are operationalized through five basic core measures: 1. Educational Gain—the percentage of adult learners in basic and English literacy programmes who acquire the basic or English language skills needed (validated through standardized assessment) to complete the educational functioning level in which they were initially enrolled. These gains may be measured by standardized tests approved by each state. To measure educational gain, the NRS established a hierarchy of six educational functioning levels, from beginning literacy through high school level completion, and six levels for English literacy, from beginning literacy level to high advanced, which represents skills considered sufficient to study for and pass the GED test. The levels are defined through reading, writing, numeracy and functional and workplace skills (and, for English literacy, speaking and listening skills) at each level. Included for each level is a corresponding set of benchmarks on commonly used standardized assessments, such as the Test of Adult Basic Education (TABE) and the Comprehensive Adult Student Assessment System (CASAS), as examples of how students functioning at each level would perform on these tests. In Massachusetts, the state Adult Performance Test (MAPT) is now used instead of the TABE to assess learner gains, although the TABE is still used as a placement tool. The Basic English Skills Test (BEST) Plus (for oral skills) and Arlington Education and Employment Program (REEP) (for writing) tests are used for ESOL. 2. High School Completion—the percentage of adult learners with a high school completion goal who earned a high school diploma or recognized equivalent. 3. Entered Postsecondary Education—the percentage of adult learners who establish a goal to continue their education at the postsecondary level and who entered postsecondary education or training after programme exit (within the fiscal reporting period). 4. Entered Employment—the percentage of unemployed adult learners (in the workforce) with an employment goal who obtained a job within one quarter after programme exit. TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 CASE STUDY – UNITED STATES – 11 5. Retained Employment—The percentage of adult learners with a job retention goal who (a) entered employment within one quarter after exiting and (b) were still employed in the third quarter after programme exit. States may identify additional performance indicators for adult education and literacy activities and incorporate these indicators, as well as corresponding annual levels of performance, in their state plans. Each state sets performance goals for the percentage of students who will make a one NRS level gain or pass the GED, and then the state is judged on whether or not it meets its goal. The goal then increases each year. Obviously, many states prefer to set their goals as low as they can in the beginning. States must also maintain their funding “level of effort.” If a state was matching the federal block grant dollar for dollar, then it must continue to do that. If for example, a state was contributing 5 dollars for every federal dollar, it must continue that level of match. If states add additional state dollars, they can use them to fund students who don‟t make the gains and are therefore not reported to the feds. Formative assessment As noted above, programmes receiving federal funds are encouraged but not required to use instruction methods based on evidence from scientific research. Since there is little rigorous scientific research on teaching basic skills to adults, programmes draw from research on children. This includes formative assessment, which has been demonstrated to have a positive impact for children in school settings. States and programmes must show sufficient learner gains as measured by NRS outcomes. Observations of teaching Most of the teaching observations reported here were done by John Benseman over a two week period in March and April, 2007. The sites for the observations were identified and arranged by John Comings who also participated in some of the observations. Interviews with learners and teachers were carried out wherever possible, either before or after the teaching sessions being observed. In addition, interviews were also carried out with programme administrators at most of the sites and state officials in all three states (California, Maryland and Massachusetts). California was chosen because it has the largest number of students, accounting for almost 25% of the total national population of students. The California programme is also well-funded, with most of the funding coming TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 12 – CASE STUDY – UNITED STATES from the state government, rather than the national government. Maryland was chosen because it is in the midst of a reform movement that is leading to greater funding and more comprehensive state management of programme services. Massachusetts was chosen because it began a reform movement 20 years ago that has led to well-resourced and well-managed programmes. As education is a state responsibility, some states have less well-developed systems, but no state has a significantly better-developed system than these three. Within each state, the system director was asked to identify programmes that were of high quality and that had some experience with formative assessment. The measure of quality was made by the directors through their personal observations and the data each programme provides to the state, often more extensive than that reported to the national government under the NRS. The sites A total of eight sites (14 classes) were visited for the observations:  California  Napa Valley Adult School (ESL, GED/ASE, ESL and a testing centre)  Florin Technical (ESL classes)  Winterstein Adult School (ABE)   Pacific Elementary School (family literacy) Maryland – Community College of Baltimore County  Eastpoint Workforce Development Center (ABE/GED)  Eastern Family Resource Center (run in association with Literacy Works Inc.)   CCBC Essex Campus (maths class for GED) Massachusetts  Quinsigamond Community College in Worcester All of these centres ran three main types of instructional programmes: Adult Basic Education (ABE), Adult Secondary Education (ASE), which is usually GED–oriented, and English as a Second Language (ESL). Each type was usually broken down into a number of sub-levels:5 ABE is the lowest 5 Which are usually related to a standardised score band such as the CASAS. TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 CASE STUDY – UNITED STATES – 13 level, with ASE being equivalent to secondary level; the ESL programme levels are ranked separately:  Adult Basic Education (ABE)  ABE Beginning Literacy  ABE Intermediate    ABE High Adult Secondary Education (ASE and Advanced ASE) ESL programmes (with up to four levels of classes). Daily classes are usually held in three periods – morning, afternoon and evening – to allow for students in paid work to attend. A typical timetable has students attending two to four hours per day, up to four days a week. Tuition is usually free of charge (although some programmes charge for learning resources) and there is no maximum number of hours students can attend provided they satisfy the attendance requirements. The number of annual instructional contact hours for students has been steadily increasing; in California hours of tuition per learner can range from 45 to 300+ hours; in Maryland, the average has risen to 70 hours over the past year, (the state goal is 120 hours by 2010), while in Massachusetts, it has now risen to an average of 150 hours per student. In 2003, the Department of Education reported that, on average, adults spend 113 hours6 in class in a 12-month period. This figure does not include adults who drop out before they complete 12 hours of instruction, which would lower the average significantly. In addition, this data was influenced by one state that reported an average persistence rate of 258 hours and enrolment at over 400 000 students, which is more than 15 percent of the national total. Only four states report more than 100 hours: California, Florida, Massachusetts, and North Carolina. Since 113 hours is an average, more than half of the students who stay at least 12 hours do not persist in their studies for 100 hours. Seven states reported an average persistence rate of less than 50 hours, and 36 states reported less than 80 hours. Class sizes also vary. In Maryland for example, while all classes are expected to have at least five students, beginning classes (both ABE and ESL) have a maximum of eight learners, intermediate classes a maximum of 10 and advanced classes a maximum of 12-15. Some of the classes observed have up to 30 students enrolled, although absences and testing usually mean that they are smaller than this number on any particular day. 6 134 hours for adults learning English, 103 hours for adults who are improving their literacy and math skills, and 87 hours for adults pursuing a high school equivalency. TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 14 – CASE STUDY – UNITED STATES While attendance at programmes is voluntary, most of the centres operate clear attendance policies and students are dropped from the programme if they do not meet the attendance requirements (usually about 80 percent attendance). Retention rates vary considerably, but the presence of long waiting-lists in most of the programmes undoubtedly exerts pressure to comply with these attendance requirements. In Worcester for example, there is currently a waiting-list of four to five years for students and in Maryland, there is a waiting list of 4 561 learners. The current waiting list in Massachusetts is 18 000. A recent survey of state systems found that 40 of the 43 states reporting confirmed students on waiting lists in their state; 917 (66%) of the 1 376 local programmes reporting confirmed waiting lists, and 90,000+ potential learners cannot access services (http://www.naepdc.org/January%202007.htm#Waiting_List_Survey_Surprises). Each of these sites catered for a diverse range of learners whose characteristics are summarised in the table below. Site/teacher Class type Total learners Male/female Ethnicities Ages ESL 21* 5/16 Mainly Hispanic Mainly 18-25 GED/ASE 9 7/2 Mixed Mainly 18-20 ESL 17* 10/7 Mainly Hispanic Diverse, most 25+ 20* 26 6/14 13/13 Asian/Hispanic Asian/Hispanic Diverse 18-40+ 19-78 Florin Technical - 3 Winterstein Adult Center ESL (high) ESL (medium) ESL (low) ABE 18 9 4/14 3/6 Diverse 18-40+ 20-40 Pacific Elementary FL 3/5 Eastpoint Workforce Development Center Eastern Family Resource Center Community College Baltimore County Essex Campus Quinsigamond Community College – 1 Quinsigamond Community College – 2 Quinsigamond Community College - 3 ABE/GED 8 adults/ 8 children 5 Asian/Hispanic Mainly African American Hispanic/Asian 20-40 ABE/GED 16 7/9 All African American Mixed Maths** 3 0/3 Mixed 18-30 ABE 7* 4/3 Mixed 18-30 Int. GED 9* 5/4 Mixed 18-50 Advanced maths 9 5/4 Mixed 20-40 Napa Valley Adult School – 1 Napa Valley Adult School 2 Napa Valley Adult School – 3 Florin Technical – 1 Florin Technical – 2 3/2 18-40 18-80 * class numbers normally bigger than this, with students out of the room being tested at the time of the observations; in some cases, up to 10 students. ** Not a normally scheduled class date – rescheduled so that it could be observed. Five students chose not to attend TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 CASE STUDY – UNITED STATES – 15 Classroom settings The observations took place in a diverse range of settings. In many cases, the adult education centres were situated in, or near to, a school. In two cases, the adult centres had „inherited‟ their buildings from schools when they were judged to be unsafe for children – but presumably safe for adults – a legislative requirement, but perhaps also indicative of the lesser status of the sector relative to the K-12 system. The great majority were rectangular rooms, with a whiteboard situated at the front of the room, around which most of the teaching was centred. All the rooms were well lit (some classes were occurring at night) and comfortable with ample space for individual or small group work. Most had multiple computers (laptops in several cases) available in the room (ranging from several to 10) or had access to a nearby computer suite. Only one class (family literacy) took place in a schoolroom used for children, although the teacher was quick to point out that it meant that they had ready access to learning resource material that they would not otherwise have. The rooms were typically arranged in three different ways: long rows (usually 2-3) of desks facing the whiteboard, groups of desks with up to six learners around them or desks placed in a U shape facing the whiteboard. The first arrangement was most common with the more formal classes where learners often worked independently. In all cases, the learners moved readily around the room or left the classroom as needed; the teachers were accepting of these movements, nonetheless ensuring that administrative requirements such as completing attendance registers were completed. In one case, a student left class 15 minutes early as he had just secured a new job and was anxious to be on time for his first day, while in another case a student wanting to leave early was told to stay and complete his work for the day. At Napa Valley Adult School, the site includes a specialist testing centre to carry out all of the CASAS7 testing for the school‟s students. The centre was a hive of activity with students coming and going even late into the evening. The students appeared to accept that visiting the centre was part of being in the programme and were clearly accustomed to the testing routine. 7 CASAS, or the Comprehensive Adult Student Assessment System, is a widely used system of standardised testing for assessing adult basic reading, math, listening, writing, and speaking skills of adults. All ESL tests are described in http://www.ncsall.net/?id= 521 CASAS and all other tests are described at http://www.ncsall.net/?id= 574 TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 16 – CASE STUDY – UNITED STATES The learners A total of 179 learners were observed in the course of this small study. The vast majority of these learners appeared highly motivated and actively involved in their classroom activities, which is not surprising given the selective nature of exemplary teachers in the sample for this study. One image perhaps exemplifies this commitment: that of a 78 year-old Chinese student running between his group of learners and a list of words on the whiteboard to complete a task and the obvious delight he showed not only in carrying out the task, but also when it was successfully completed. Despite high levels of motivation and involvement, it was also clear that many participants had considerable social and educational challenges to overcome. This was especially true of the ABE classes, where the teachers worked hard to ensure that their learners remained engaged and working at their level of need. For example, there was more one-to-one interaction between teachers and learners in these classes and less use of independent tasks. There was only one incident observed where a student was disruptive to the others in a GED class. The behaviour was tolerated for a short time and the student was then curtly told, “We don‟t have time for this sort of thing right now,” and the student resumed his tasks. This teacher was adamant in requesting that students show commitment to their studies or “go away until they are ready to be serious about it.” It is significant to note that there appeared to be a high degree of homogeneity in terms of learner levels in the classes observed compared to what would be found, for example, in comparable New Zealand classrooms. This degree of learner homogeneity meant that the teachers were able to pitch their teaching at a smaller span of learning needs, thereby enabling a high amount of whole-group teaching rather than a high degree of individualised work, which is what typically occurs when there is a high degree of learner heterogeneity. Achieving this homogeneity is likely due to the consistent use of placement tests and the high numbers of learners involved, giving administrators more latitude to group learners more accurately. Probably the class with the greatest heterogeneity in relation to learner levels was a community-based programme located in a hospital, although it should be noted that this was due in part to two classes working together as one of the teachers was absent. In this case, the presence of three volunteer teachers was invaluable in coping with the diversity and ensuring that individual learners were supported at their disparate levels. Several people also commented that small town and rural area programmes find it difficult to achieve this degree of homogeneity because of their small catchment numbers. TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 CASE STUDY – UNITED STATES – 17 Although many of the classes operated in a fairly formal way (conventional seating, formal timetable, programme regulations, externallyimposed curriculum8 and testing9), they all conveyed an appropriate adult atmosphere in keeping with the nature of the student clientele. This was as simple as using first names or allowing student choices in tasks, but mostly it was in the ways that the teachers related to the students on an equal basis in their interactions. In several of the learner interviews, the students expressed their appreciation of this difference from their school experience. One young student who had only recently left school said that “it‟s just so different from how I was treated at school,” while an older student said that his experience bore no resemblance at all to his schooling in the 1940s – “they‟ve already showed me how to do things that I just never knew before and that‟s great.” The characteristics of the learners observed varied considerably. In California, there was a clear dominance of ESL learners (especially Hispanic) which probably reflects this state‟s very high immigrant numbers. In the Baltimore programmes, state data show that ESL learners make up 39% of the 35 000 learners, while 44% are ABE and 17% are ASE. Most (78%) belong to a minority group, 97% are of working age, 47% are employed and 26% are unemployed. The ages of the learners varies according to the type of programme, with ESL classes containing quite varied age-groupings and the GED/ASE classes being mainly under 20 years of age. In terms of gender, there were more women than men overall, and the absence of older men was particularly evident. Recruitment of learners is done by a range of means such as public advertising, liaison with social agencies and close links with local high schools. Several of the programmes referred to their close ties with local schools to ensure the transitioning of students in danger of dropping out into the nearby adult centre programmes, to reduce the risk of dropping out of the system. 8 9 States have curriculum guidelines but, in general, each teacher develops their own curriculum, though since most follow a commercial text/workbook, the curriculum is mostly that source (Robinson-Geller, 2007). These tests were largely designed by the local provider or the teacher individually, although the more formal assessments such as the CASAS and ultimately the GED appeared to be the most prominent forms of assessment. TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 18 – CASE STUDY – UNITED STATES Learner outcomes While the GED is probably the most frequent formal outcome for programmes and the one that most students identify, there are also a number of other qualifications that have currency within states. For example, in Maryland (and in 12 other states) the Community College of Baltimore County offers an Adult External High School Diploma programme as an alternative to the GED that is aimed at mature-age students. This programme incorporates recognition of prior learning including both non-formal and informal learning such as home management experience, practical literacy and numeracy tasks and proof of a current occupation or trade. It is seen as particularly appropriate for self-directed, motivated students as it is selfpaced, conducted weekly through 1:1 appointments and not classroombased. The demand for the programme exceeds the resources available. In California, many students earn high school credits and take the California High School Exit Exam (CAHSEE) rather than the GED.10 With the GED and high school diploma goals so prominent in programmes, there is always a ready barometer of student outcomes available. For example, Maryland awarded a total of 6 380 high school diplomas over the past three years. With a current roll of 9 600 (including a large community education, vocational education and older adult programmes) the Napa Valley Adult School in California had helped 389 students toward their high school diploma or GED and 431 with their basic education in reading, writing and maths skills. Most of the programmes also ran specific tracking rubrics developed by the providers of their students‟ literacy competencies. For example, the Napa Valley Adult School records ESL students‟ competencies for listening, speaking, reading, writing and grammar on a three point scale (needs improvement/some mastery/good mastery) which are considered along with their CASAS scores in making decisions about promotion to the next class. The teachers Despite the large scale of adult education provision (California alone has in excess of 12 000 teachers in the adult school system), the status of the teachers involved remains marginal compared to their educational mainstream colleagues. Although there is reasonable continuity of employment, few are employed on a full-time basis (in Maryland, only 13% of teachers and 47% of administrators work full-time). In many cases, this arrangement suits the teachers who want part-time employment, but it 10 These credits and the CAHSEE are necessary to gain a diploma. TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 CASE STUDY – UNITED STATES – 19 appears that the main reason for the low numbers of full-time positions is the high cost of paying the additional benefits associated (up to a third of the salary costs) with them. Some teachers put together full-time equivalent positions by working for multiple centres. Administrators say that they would prefer to have more full-time teachers, but they find it difficult to achieve under the current constraints. It is also worth noting that some states have little or no paid time for professional development or even programme staff meetings. Others, such as Massachusetts, mandate specific paid hours and sometimes specific activities. Whether it is because of the low numbers of full-time positions or some other reason, there is a high turnover of teachers, especially in their first few years. In many cases (especially with ESL teachers), the teachers are often „poached‟ by the mainstream schooling system. State administrators in California interviewed for this project named this turnover of teaching staff as one of their top issues and are concerned about how they can be more involved in professional development programmes early in their careers, which will hopefully then better retain them in the workforce. Despite their predominant part-time status, teachers are required to have appropriate qualifications. In California, for example, teachers must have a bachelor degree, earn a teaching credential, and pass a state basic skills test.11 Teachers may also need some specialist qualifications, as in the case of ESL provision. There are similar requirements in Maryland and Massachusetts (apart from the basic skills test). The provision of professional development (PD) is becoming an important part of states‟ strategies to improve the quality of provision. In California, for example, there are a number of PD sources for teachers: CALPRO for providers, CASAS‟s training for summative assessment, the California Distance Learning Project aimed at non-traditional teaching settings and the Outreach and Technical Assistance Network (OTAN) aimed at integrating technology into the classroom. In addition, the STAR12 programme has recently been trialled in California in seven school districts 11 12 Requiring such a test of teachers may appear strange, but research in Britain (Coben et al., 2007) and New Zealand (Workbase, 2005) for example has shown teachers‟ lack of literacy skills to be an issue – especially in numeracy. The Star Achievement in Reading (STAR) is sponsored by the US Department of Education Office of Vocational and Adult Education (OVAE) Division of Adult Education and Literacy to infuse evidence-based practices into state delivery systems and ABE classrooms. The STAR toolkit is designed to materials that teachers can “use on Monday. “Its four main goals are to use individual learning plans, improve persistence, use diagnostic and standard reading assessment and increase the use of reading research. A big part of the STAR training includes how to use assessments of the components of reading for initial diagnosis and placement and later monitoring of learner progress. TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 20 – CASE STUDY – UNITED STATES and two state prisons along with five other states13 and appears to have been enthusiastically received by the practitioners. However the availability of PD appears to vary from centre to centre because the funding is dependent on access to supplemental sources such as the WIA, to which not all centres have access. In talking about the STAR programme for example, one centre principal said that “we would love to be involved in it, but there‟s just no way that we could afford it.”14 Access to PD may well be a case of „to those who have, more will come‟ as not only do the more successful centres have better funding for PD, but they also often have larger numbers of skilled and experienced staff to run their own in-house programmes as well as accessing PD activities run by other organisations such as CALPRO.15 Professional development is seen as integral to improving the quality of Adult Education and Literacy (AEL) provision in all three of the states visited. One principal commented that most of their expertise in the teaching of reading had historically come from elementary school reading specialists, but with the advent of STAR, they had started to develop their own adultspecific expertise in this area. He evoked the example of a teacher being observed and attributed the achievement of 100% attendance and the rigour of the programme in large part to her involvement in STAR. For teachers, the amount and type of PD on offer varies from centre to centre. In the larger centres, there is not only access to PD programmes run in the local area, but also often includes in-house PD programmes. The larger centres often have staff that run the area or state programmes and therefore can also run them in-house for their own centre staff. Napa Valley Adult School for example, ran a total of 45 PD events in 2004-2005 for their staff with topics ranging from teaching-specific topics (lesson planning, techniques for teacher evaluation, ESL retreat), through administration to general topics (such as cardiopulmonary resuscitation [CPR], and sexual harassment). PD can also include in-centre mentoring programmes for new teachers. Centres also run induction programmes for these new staff. It was noticeable that all of the teachers observed were very comfortable with someone coming into their classrooms to observe their teaching. This is probably due to the amount of observation done by programme administrators and as part of mentoring programmes, both of which appear to be reasonably commonplace. 13 14 15 One of the teachers observed for this study was a participant STAR costs vary across a number of services, but start at approx. USD 700 per teacher per annum, not including the costs of teachers attending the training. CALPRO run a wide range of professional development activities – see www.calpro-online.com TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 CASE STUDY – UNITED STATES – 21 There was considerable evidence that the teachers in this study are not only aware of research findings in relation to their teaching, but are also actively involved in trying to incorporate their findings into their teaching practices. Several of the teachers in California reported their involvement in action research projects and professional study circles, especially in attempts to improve their retention rates with learners and the teaching of reading. One teacher in Massachusetts had been a participant in a national research project. The teachers all felt that these projects had provided insight into their teaching skills and were generally enthused by the experience of doing the research. This emphasis on employing research in practice comes, in part, from the national government. Over the last five years, the U.S. Department of Education has championed the use of evidence-based practice, particularly evidence from scientific research. The WIA funding requires states to employ evidence-based practices. In addition, the National Institute for Literacy and the National Center for the Study of Adult Learning and Literacy have been disseminating research to the states and training teachers on how to access, judge and use research to improve instruction. California, Maryland, and Massachusetts have supported these initiatives, with California launching its own research-to-practice initiative. In addition, there have been workshops at state level run by NCSALL and state personnel (with funding provided by NIFL) to develop a common training programme for practitioners and state administrators to help instructors access, judge and draw upon research in their work. Curriculum The dominant driving force in all of the classrooms being observed in relation to curriculum is either the GED or the state high school diploma. While the GED covers a range of curriculum areas, the teachers pointed out that the great majority of the curriculum is still predominantly dependent on reading, writing, numeracy and critical thinking skills rather than specific knowledge about the disciplines involved. With such a strong influence from the long-term goals of the GED or high school diploma, it is perhaps inevitable that there is widespread use of student workbooks (especially in ASE and GED classes) that not only cover the curriculum content (all geared to the specific level of the learners), but also the „culture‟ of these exams. Several teachers being observed for example, spent some time explaining to their students that “when they ask you a question like this, what they (the GED examiners) are really looking for is answers that provide proof that you have understood …”. Another teacher told her students, “when you go in to do your GED, you need to hear my voice – if you can‟t hear it, then I haven‟t done my job.” In other words, they were TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 22 – CASE STUDY – UNITED STATES being taught the unwritten rules of exam participation, how to write the GED essay and test-taking strategies. One American study has shown that this component takes up more time than either the teaching of reading or maths (Scogins and Knell, 2001). Several teachers in this present study said that they consciously tried to avoid „teaching from the workbook‟ and devised their own lesson plans although it inevitably involved a lot more work on their part. They felt that this „tailor-made‟ planning was more directly related to their students‟ needs and gave more opportunity to localise the content. In this way, the instruction is more authentic and learner-related. While there were considerable variations in the format of the lessons observed, below is a reasonably typical session (in maths):     2-2.30 – individual review time. Students work on any skills they feel need reviewing with individual help from the teacher. Students use practice tests to determine where weaknesses remain. 2.30 - 3.30 – a group lesson where new skills are presented and the students have access to a range of resources such as workbooks, smart cart lessons (via the Internet) and teacher-directed board work. 3.30- 3.40 – short break 3.40 – 4.30 – individual practice of the new skills and another group lesson, followed by a wrap-up activity. Students often work in pairs in this session. Broader contexts The contexts in which the classes were held differed markedly across the three states. These contexts clearly influenced who attended the classes as well as some aspects of how they were organised and ran. In California for example, the state‟s large immigrant population was a highly visible component of most of the classes being observed. The sheer size of the immigrant population in California makes for a huge challenge for ESL provision in particular, making up 43% of overall provision for the state. For example, the state recently had an influx of 10,000 ethnic Hmong from refugee camps in Thailand, many of whom will invariably end up in ESL classes in the areas hosting them. The need is unlikely to diminish, unless immigration declines significantly. The local economy not only determines the makeup of the workforce in the community, but also how they are able to attend programmes. In the TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 CASE STUDY – UNITED STATES – 23 Napa Valley for example, while the environment appears affluent and wellheeled, the presence of the viticulture industry also signifies that a large, poorly-educated Hispanic immigrant community provides the workforce to grow, pick and process the grapes. Most of these people come from two of the poorest states of Mexico, meaning that they often have poor educational backgrounds in their original contexts, let alone English language needs. When enrolled in classes, their attendance waxes and wanes according to the demands of the viticulture calendar as well as various responsibilities in Mexico. Ensuring that learners are tested on the CASAS “before they disappear for picking” for example is not an uncommon call for the ESL teachers in this area. Baltimore on the surface appears to be an affluent area, with one of the wealthiest populations in the country and considerable re-generation of the inner city areas. Like many traditional industrial areas of this part of America however, there has been a large-scale loss of traditional heavy manufacturing industries, leading to high unemployment rates in some counties in the area served by the Community College of Baltimore County. There are also rising expectations of entry level qualifications (usually a high school diploma or equivalent) for even un- or semi-skilled jobs in developing industries, such as those centred on the Johns Hopkins University health services. Adults without these entry-level qualifications run high risks of permanent exclusion from the job market. While official statistics put the number of adults in the state without a high school diploma at about 20%, interviewees for this project reported that many areas in Baltimore have in excess of 50% without diplomas, pointing to an everincreasing demand for their services. The adult basic education services in Massachusetts address a specific weakness in the state‟s workforce. According to two studies (Comings et al., 2000; Nakosteen et al., 2003) by a non-partisan policy research group, MassINC, the size of the workforce would have declined over the last 15 years had foreign immigrants not filled in the gaps. In addition, though the state has little trouble attracting well-educated, highly-paid workers and poorly-educated, low-paid workers, it is losing educated mid-level workers. The workforce requirement for the latter has supported a sizable increase in state funding for services over the same 15 years to improve the English language abilities of immigrants and to help school dropouts to gain a GED and move on to postsecondary education or training and one of the states many two-year community colleges. One of the MassINC studies estimates that around 1.1 million of the state‟s 3.3 million workers could benefit from ESL, ABE, or GED services. TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 24 – CASE STUDY – UNITED STATES Observation framework As a framework for the observations, a series of headings have been used, based on a reading of the research literature on formative assessment. The writings of Black and Wiliam (Black, Harrison, Lee, Marshall and Wiliam, 2002, 2003; Black and Wiliam, 1998) and more recently the OECD (2005) have been given particular prominence, given their acknowledged expertise in this area. At the heart of formative assessment is what Wiliam (2007) terms „pedagogies of engagement‟ where teachers create an environment for all learners to actively participate. One of the key ways that teachers create this environment is through formative assessment, involving the use of assessment to inform and change how teachers teach. As Wiliam (2007) says: The central idea is that we should use assessment to influence learning and that the teaching should be contingent on what students have learnt, so that while we’re teaching, we collect evidence about where the students are to make adjustments to our teaching to better meet our students’ learning needs. He argues that it involves assessment for learning, rather than the dominant assessment of learning. There is no definitive list of formative assessment strategies, but Wiliam identifies the following strategies as key to formative assessment:      clarifying and understanding learning intentions and criteria for success engineering effective classroom discussions, questions and tasks that elicit evidence of learning providing feedback that moves learners forward activating students as instructional resources for each other activating students as owners of their own learning. Learning intentions and criteria for success All of the programmes visited for this study had counsellors, programme co-ordinators and assessment teams (who often do both diagnostic learning gain assessments) available for their students. Their role was seen as a cornerstone to running successful programmes by both administrators and the teachers. The counsellors are the first point of contact for new students TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 CASE STUDY – UNITED STATES – 25 where they undergo a series of formal and informal assessments in order to develop individual learning plans (including the students‟ own learning goals) and placement in the most appropriate class for their learning needs. The counsellors are also available to students at any time during the programme for counselling on broader issues. One teacher commented that it is important that students are challenged to keep their goals realistic and that they include short-term components Programme staff in one state commented that to a large degree it looks like they co-exist, but at times the students‟ individual goals are incorporated into the curriculum. For example, if a student wants to register to vote, s/he is given information and resources to be able to do so. If several students in a class share the same goal, then the teacher can incorporate a unit of instruction that addresses this goal and assists students to achieve it. Or if some students in a class want to learn about health, a health unit is taught –with guest speakers at times- to help the students work towards their goals in learning, as the unit becomes part of the class curriculum. . It is expected that the students‟ learning plans are reviewed periodically, that attained goals are recorded and new ones set in consultation with the student. While not all teachers use individual learning plans (ILPs), they are commonly promoted as „best practice‟ by groups such as CALPRO. Several of the observed GED classes provided very clear examples of what was expected of students in terms of progress towards their learning goals As a tracking mechanism, the teachers had written the GED components on one axis of a chart and the students‟ names along the other. After achievement of the various GED components by successfully completing a trial test, their name acquired a highly valued tick, and the next learning goal was identified and an accompanying work-plan was set by the teacher. Teachers also sat with the students after their exams, analysing their incorrect responses and suggesting remedial activities to help rectify the problems highlighted by the test. In one case, the whiteboard with these details was centre-stage in the classroom, serving as a constant reminder and prompt for seeing where students had progressed and what remained to be covered. The teacher reported that the students valued the chart for its clarity of goals and the deep satisfaction they felt as they progressed through the list. She also commented that the class was organised on the assumption that the students had the motivation and commitment to work with a high degree of independence, with the teacher providing individual help on an as-needed basis. Students are encouraged to „challenge the test‟ if they believe they are ready to take the practice test before completing the allocated workload. As this is an informal arrangement within the class, learners can re-take the test TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 26 – CASE STUDY – UNITED STATES if unsuccessful. It is presumed that they become more skilled in judging their rate of progress as a result and with on-going advice from the teacher. Activating students as owners of learning Probably the most obvious demonstration of students as owners of their learning was reflected in the process of setting student goals. Although the overall goals of the programmes were undoubtedly set by the requirements of the GED, students in all the programmes were expected to set their own goals for their participation. These goals were usually identified as part of the induction and initial assessment processes carried out by the programme counsellor. In some cases, they were written as open statements by the students, while in other cases the counsellor provided a sheet that the student could use as a prompt to identify their goals. A typical sheet could include:    Educational goals (e.g. improve reading skills, learn about US culture, enter college or advanced classes) Work goals (e.g. get a better job, leave public assistance, make more money) Personal goals (e.g. get a driver‟s licence, learn about cultural and recreational activities in the local area, find accommodation). These goals were dated and then reviewed periodically alongside the test assessments. A number of the ESL classes had made considerable efforts to base their lesson plans in civics on surveys of their students‟ expressed needs. At Napa Valley Adult School for example, a survey had shown „accessing and using free or low-cost medical and dental services‟ and „keeping your family safe and responding to emergencies‟ as the two top needs for their students, although the priorities also varied across the different ESL levels. These topics had therefore been the focus of the first sessions of these classes. This class‟ students were also encouraged to work in small groups on their civics programme, which the teachers felt helped to make the programme more learner-driven. A California official confirmed that teaching based on these sorts of learner surveys “has dramatically changed ESL teaching in a lot of centres” because of its contextualising of curriculum, rather than being predominantly workbook-driven. One California teacher, who had been part of the STAR pilot aimed at infusing research findings into ABE teaching, had told her students about the four main components of successful teaching (vocabulary, comprehension, alphabetics and fluency) and prefaced her teaching sessions by reference to how their session content related to these four components. TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 CASE STUDY – UNITED STATES – 27 She was delighted when recently talking about building vocabulary in a session, one of the students called out, “yeah, [be]cause that‟s one of the four things we need to read better.” She felt that it was important to share her teaching goals with her learners and was delighted that they had „taken them on board‟. Finally, one series of interchanges in an ABE class demonstrated how even simple tasks can generate considerable personal disclosure that helps build a learning environment of trust and support among the students. For example, this teacher prefaced her discussion with the statement, “it‟s OK if it‟s not right, [be]cause we can all fix it.” Asked for examples of how to use the word reverse in a sentence to demonstrate the word‟s meaning, one student said, “If there was one thing I wish that I could do in my life, it would be to reverse some of the decisions that my parents made [referring to a divorce] when I was young.” This sentence prompted a number of reactions from the other students of empathy and the teacher thanking her for her contribution as well as confirming the correct use of the word. The next student (a young woman who appeared to be a somewhat reluctant participant in the class up to this point) then offered her sentence (with some emotion), “there is nothing in my life that I would want to reverse, because it‟s all made me the strong person that I am.” A nearby student patted her on her shoulder, saying, “right on, that‟s cool.” The two contributions generated a warm buzz of affirmation between the students who clearly felt that it was „safe‟ to self-disclose, as well as completing the „technical‟ task set by the teacher. This class had a very high rate of attendance, which the centre‟s principal attributed in large part to the supportive environment that the teacher had created. Teacher questioning and learner discussions The use of questions is central to what all teachers do in their classroom activities. In terms of formative assessment, skilful questioning is an essential part of the teacher‟s repertoire. For example, how teachers pose their questions, how they time them, who they target and how they make use of the learners‟ responses were all integral to ensuring that learners became engaged, stayed engaged and actively worked on improving their skills. In the course of the observations for this study, a wide range of skill level was seen in relation to the teachers‟ questioning skills. Below are some specific examples of how teachers effectively used questioning to challenge their learners and ensure their engagement in learning:  Levels of questions: teachers pitched their questions at levels that were both attainable, but also challenging. TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 28 – CASE STUDY – UNITED STATES         Leaving reasonable wait-time for learners to respond to questions so that they clearly understood that a response was expected from them and would not be answered by the teacher. Ensuring that the students „did the work‟ by ensuring that they answered the questions and not the teacher – “so is that sentence OK? – anything that you would like to see changed with it?” (addressed to the whole class). Creating safe environments to ask questions: several instances demonstrated that teachers were genuinely interested in hearing students‟ questions – asking if anyone wanted anything explained further at the end of a session, one teacher waited a reasonable time for responses (rather than simply finishing the session without waiting for responses) and then when one student asked for clarification about a topic, then spent some time explaining it to the class. Students were given realistic feedback on their answers along with suggestions for reviewing why it was incorrect or incomplete – or additional questions to elicit another response. Promotion of critical thinking, for example by prompting students to make fine distinctions between similar words (fixable and feasible). „Tuning‟ of questions in response to learners‟ responses: teachers often opened with a general, open question (“so what do you all think about this?”), but then gradually focussed their questioning both in relation to more specific topics (“what do you think the writer is trying to do in this chapter?”) and also „fine-tuning‟ their questions in response to the learners‟ responses (“you think that it‟s all about control in this case or something else?”). There was quite a high level of student : student interaction in most of the classes due at least in part to the amount of questions that students asked each other, rather than being reliant on teacherdominated questioning. Allowing for diversity of answers so that students received a message that there are a number of ways to answer correctly. The use of students‟ responses as prompts for spontaneous teaching moments was noteworthy with several of the teachers. When a student identified Ottawa as the capital of Canada, the teacher immediately asked, “so who knows how to spell it?” which prompted a round of debate among the students about the answer. The teacher then asked them how they could TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 CASE STUDY – UNITED STATES – 29 check the spelling using a computer, which then led to a discussion about how to use Google as a learning resource. They had therefore not only debated an answer, but had learnt how to spell it and had a mini-lesson on learning how to learn – all within a period of five minutes. This interchange not only exemplified a teacher challenging their students‟ skills and knowledge, but also incorporated an element of „learning how to learn‟ – a skill that enables the students to then use in future learning episodes. There were a number of examples of teachers encouraging their students to reflect on their learning processes. These included the use of predicting texts in the teaching of reading, while another teacher prompted her students to “think out loud” about what they were doing as they went through some exercises calculating percentages – “show me your thinking as you go through the steps, so that we can see what you are doing here.” This teacher also encouraged her students to self-diagnose their learning needs (“so you think you need some more practice with maps, what makes you think that?”) The following exchange in an advanced maths class illustrates how a teacher encourages a student to reflect on her own learning: Learner: why couldn‟t you just knock off your zeroes, rather than doing all that? Teacher: I was hoping that you would work that out Learner: well, if you don‟t get the same as you started with [turning a fraction into a decimal and then back into a fraction], then it means that you‟ve messed up along the way somewhere right? Use of feedback The most prominent form of feedback undoubtedly comes from the array of standardised tests used in all of the programmes that were observed. The most frequently used test observed was the CASAS, but this was supplemented by a number of other tests including the Massachusetts Adult Performance Test (MAPT), BEST Plus (for ESL), REEP and a new assessment tool developed specifically for MAPT.16 These are standardized tests, but as they are skills-based, rather than tied to the curriculum, instructors cannot “teach to the test”, per se. A typical assessment schedule using these standardised tests from the Quinsigamond Community College shows three main components: 16 For details of these tests, see http://www.ncsall.net/?id= 574 TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 30 – CASE STUDY – UNITED STATES    An initial assessment where learners are tested in order to determine the class most appropriate for that learner Mid-term summative assessment where students are required to be tested halfway through the school year in order to assess the progress made and to determine if the student is ready to advance to the next level. Final summative assessment at the end of the year to determine progress since the mid-year assessment and to determine the class level for the following year. All students are required to be tested in order to return to the programme in the following year. The mid-term assessments are carried out once students have attended a certain amount of tuition („after instruction of sufficient duration and intensity‟ or usually after about 60-100 hours in California for example and 60 hours in Maryland). In practice, this can occur after as little as 12 hours (done because students were due to leave the district within the next week). It is important to note that eligibility for federal funding is dependent on the completion of standardised tests such as the CASAS and the TABE showing learner progression. Federal funding is 50% of total funding in many states. Even in those states where federal funding is a small share of overall funding,17 administrators value it for the extras that it enables them to offer that would not otherwise be available. Several centre administrators reported however that they are increasingly questioning the value of the federal funding vs. the amount of administrative work required to acquire it. In California, approximately half of the students are not eligible for the federal funding because they are not post-tested after the requisite period of instruction (usually after 60-100 hours in California). A number of people commented that while tests like the CASAS are accurate measures of literacy skills, they are not diagnostic, which limits their usefulness for teachers. The test results do, however, provide state administrators with useful information about variations in centres‟ performances that can provide useful monitoring data. For example in California, state officials were alerted to some large providers with poor performances relative to other providers through the monitoring of CASAS results. Follow-up with these providers showed that their poor performance was due primarily to poorly managed accountability systems, including not completing posttesting, even though some of these learners had 130+ hours of tuition. For those students who are preparing for the GED exam, there is frequent use of „practice tests‟ to provide feedback on their progress and 17 The balance comes predominantly from state funding, based on daily attendance) data. TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 CASE STUDY – UNITED STATES – 31 where they stand in relation to the GED standards. In addition to the feedback provided by the teacher, the GED workbooks also include explanations for the answers given in the appendices to enable learners to understand why answers are right or wrong. In most cases, the teachers set the timing of these tests (reviewing a practice test, the teacher concluded, “these all look pretty good, so I think you should think about taking the test over the next while”), but students can also request a test if they feel that they are ready for it. Several teachers said that they feel that it is important “not to over-test” and especially not to test before learners are ready for it. Some learners kept personal record sheets indicating what they had passed and what they still had to cover in their programme. Because learners in these GED classes worked mainly on an independent basis, they had very clear understandings of where they stood and their (mainly individual) tuition was closely geared to their individual learning needs. In terms of classroom interaction, the maths classes probably showed the best use of on-going assessment/feedback during the teaching sessions. There were many informal assessment activities built into these sessions and based on the results of the assessments, the students were encouraged to work on remedial or enrichment activities in keeping with the assessment results. A comparable situation could be seen in the ESL classes when students were working in pairs on set exercises and the teacher moved from pair to pair, giving critical feedback on their pronunciation, modelling correct pronunciation and then affirming the changes. The use of computers also provided a useful mechanism for providing accurate and immediate feedback for learners. In a family literacy programme for example, one of the parents (as her daughter looked on) worked studiously on her basic maths facts using a programme Fast Math, which is also used in the school where the programme is based. The programme incorporated a useful balance of consolidating what she already knew with practice in the operations that she had not yet mastered. The student was extremely engaged in the programme and enjoyed seeing how much she had progressed since the last session and the reward of a game on successful completion of a work unit. The teacher reported that all of the students enjoyed the programme and it had encouraged playful competition that had been generated between the parents and their children. At Napa, another teacher assigned some Internet-based language tasks that included periodic quizzes that the students also enjoyed, while in an advanced maths class in Baltimore, the teacher stopped the main class activity in order to do some basic numeracy facts practice (accessed off the Internet via a smart cart), “ [be]cause you‟re still making too many mistakes with them.” The effectiveness of the computer programmes for the learners in these cases appeared to be primarily about the immediacy of the feedback, the TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 32 – CASE STUDY – UNITED STATES ready adjusting of the content to the results being achieved and the independent, self-paced nature of the teaching process. For the teachers involved, the value of the computer in these cases appeared to be the convenience of readily accessing appropriate teaching material as well as the independent nature of the task, freeing them up to work individually with other students. Students as resources for learning There was considerable use of students as resources for learning in most of the classes observed. Typically, teachers would simply ask students to help their fellow students with tasks – “Adele, why don‟t you help her out there, you know how to do that don‟t you?” Adele would then move over and sit down with the student needing help to work on the problem. Several students would be helping in this way at any time, meaning that there was considerable movement around the classroom. Although a few students appeared to be called on to help more frequently than others, there was also a reasonable degree of reciprocity between the students. This form of peer teaching was particularly common in maths and ESL classes. For example in several of the ESL classes, students were working with other students with the same language, unprompted by the teacher and this informal teaching was seen as part and parcel of the usual teaching process for that classroom. Along similar lines, much of the time during the observations involved students working in pairs. A typical episode would involve the teacher instructing a point of grammar for example and then asking students to pair up and practise the point that had been taught. Another element of using students as resources, could be seen in several teachers who wrote learners‟ responses to written questions given for homework on the whiteboard – and then rather than simply correcting the responses herself, the teacher would then pose the question back to the learners, “so what do you think of this answer? [“it‟s not right”], “all right, what would a better answer be?” Students‟ incorrect or incomplete responses were then sometimes used as prompts for micro-teaching to clarify or correct the learners‟ responses. The students appeared to be more engaged with this process than in instances where the teacher simply corrected and/or supplied the correct answer. In addition to the use of peer teaching, one teacher gave an interesting reply to a student‟s request, “can I tell her?” [the answer]; she replied, “no, but you can help her understand it” – whereupon the two students then TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 CASE STUDY – UNITED STATES – 33 discussed the maths problem and the first student subsequently provided the correct answer. Although the teaching content for many of the sessions came from workbooks, the teachers constantly strove to contextualise the content in terms of the learners‟ backgrounds and living situations. Talking about voting, one teacher then quickly quizzed her students what the voting age was in their respective countries. In another class discussing accessing health services, the teacher asked students to recount recent experiences they had with these services. Conclusion This study of 14 literacy teachers in the states of California, Maryland and Massachusetts has shown that all of the teachers incorporate at least some elements of formative assessment in their teaching and many of the teachers used most of the formative elements reported. The teachers conveyed that they certainly understood the term formative assessment and its role in effective teaching, although people more commonly referred to „multiple assessments‟ or „continuous improvement‟. There is little doubt that the use of standardised tests such as the CASAS and the TABE is a distinctive feature of American provision in comparison with other countries. These tests are ubiquitous and increasingly, the cornerstone of most programmes. One state administrator concluded her review of assessment procedures in her state by saying that the use of standardised assessment “has probably been the prime focus of what we have been doing” and that the main concern has been “getting everyone up to speed doing it.” While these tests are strongly tied into the funding requirements, they are also an integral part of teachers‟ assessment regimes for the review of learners‟ progress. They are certainly used for summative purposes, but also as formative strategies, where the results are used to accurately place students in classes, move them to new ones in relation to their progress and to plan future teaching sessions. There were also a range of other formative assessment strategies observed in operation in these classrooms. Many of the teachers have designed their own assessment tools or use those developed by their centres. These tests tend to be used more for diagnostic purposes, showing learners‟ specific difficulties in their literacy, numeracy skills, which again are used to inform and shape future teaching content. As one state administrator said, “In our system, you‟re always assessing.” TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 34 – CASE STUDY – UNITED STATES In addition, there were also numerous examples of the sorts of formative assessment activities that have been identified in school-based research studies. These included mechanisms to help clarify and understand their learners‟ aims and to establish mutual criteria for monitoring success. In their teaching interactions, the teachers promoted and sustained effective classroom discussion, especially with the use of skilful questioning and providing feedback that helps their students to review their learning and move on to higher levels of literacy skills. Finally, the teachers were able to establish positive, supportive learning environments that reflected adult relationships and promoted students as owners of their own learning. In sum, there was considerable evidence of „pedagogies of engagement‟. References Black, P. et al. (2002), Working Inside the Black Box, NFER Nelson Publishing Co. Ltd., London. Black, P. et al. (2003), Assessment for Learning: Putting it into Practice, Open University Press, Maidenhead. Black, P. and D. Wiliam (1998), “Assessment and Classroom Learning”, Assessment in Education, CARFAX, Oxfordshire, Vol. 5, No. 1, pp. 7-74. Coben, D. et al. (2007), Effective Teaching and Learning Numeracy, National Research and Development Center for adult literacy and numeracy (NRDC), London. Comings, J.A. Sum and J. Uvin (2000), New Skills for a New Economy, MassINC, Boston, http://www.massinc.org/index.php?id= 216&pub_id = 1308 Nakosteen, R., M. Goodman and D. Ansel (2003), Mass.migration, MassINC, Boston, http://www.massinc.org/fileadmin/researchreports/ mass_migration/MassMigrationreport.pdf National Center for Education Statistics (1993), Adult Literacy in America: A First Look at the Findings of the National Adult Literacy Survey, NCES, Washington, DC, http://nces.ed.gov/pubs93/93275.pdf National Center for Educational Statistics (2006), “A first look at the literacy of America's Adults in the 21st century”, NCES, Washington, DC, pp. 2-3, http://nces.ed.gov/NAAL/PDF/2006470.PDF TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008 CASE STUDY – UNITED STATES – 35 National Institute for Literacy (1998), “Policy update: Workforce Investment Act offers opportunities for adult and family literacy”, NIFL, Washington, DC, http://www.nifl.gov/nifl/policy/updates/98-0923.html OECD (2005), Formative Assessment: Improving Learning in Secondary Classrooms, OECD, Paris. Robinson-Geller, P. (2007), “Individualized Group Instruction: A Reality of Adult Basic Education”,in J. Comings, B. Garner and C. Smith (eds.), Review of Adult Learning and Literacy,), Lawrence Erlbaum Associates, Mahwah, NJ, Vol. 7, pp. 137-174. Scogins, J. and S. Knell (2001), Observations in Adult Education Classrooms: What is Really Going On?, Illinois Literacy Resource Development Center, Champaign, Illinois. Sticht, T. (2002), “The Rise of the Adult Education and Literacy System in the United States:1600-2000”, in J. Comings, B. Garner and C. Smith (eds.), The Annual Review of Adult Learning and Literacy, Jossey-Bass, San Francisco, Vol. 3, pp. 10-43. Wiliam, D. (2007), Assessment for Learning: Why, What and How, www.dylanwiliam.net/, retrieved March 20, 2007. Workbase (2005), Supporting Workplace Literacy Providers Project, Report prepared for the Tertiary Education Commission, Workbase, Auckland. TEACHING, LEARNING AND ASSESSMENT FOR ADULTS: IMPROVING FOUNDATION SKILLS – ISBN-978-92-64-03990-2 © OECD 2008