研究ノート

Creating Formative Assessments for English A1 and B1 Classes

Ryan SPRING 1)*

1) Institute for Excellence in Higher Education, Tohoku University

1. Introduction and teachers realize where gaps in knowledge

Tohoku University authorized an English exist, and is meant to be a tool to enhance or
curriculum reform committee in 2019 to further learning going forward (Turner &
convene and make suggestions for a more Purpura, 2016). In the context of the English
well-defined English curriculum based on curriculum at Tohoku University, I hoped that
specific core skills. The English Teaching conducting a formal assessment of A1 and B1
Committee at Tohoku University approved a core skills at the beginning of students’ second
new curriculum to be implemented in 2020 that semester could help students to gauge their own
included 6 core skills for each of the first-year abilities, while also letting teachers know how
general English courses: English A1, A2, B1, well their students had mastered the core skills
and B2. The core skills selected for the first from A1 and B1 courses, so that they could
semester courses (English A1 and B1) included adjust their teaching accordingly. As such, I
a lot of information pertinent to the TOEFL worked with a team of teachers from the
ITP® (Test of English as a Foreign Language Institute for Excellence in Higher Education to
Institutional Testing Program) assessment, and create a short formative assessment of A1 and
it was hoped that improving upon these skills B1 skills to be given during the first day of A2
would increase TOEFL scores. However, it is and B2 classes. This report discusses how the
not always apparent to students and teachers formative assessments were created and the
whether or not students have learned the impact it had on student learning.
information required of them or acquired the
skills needed for their upper-division courses. 2. Creation of Formative Assessments
Here, formative assessment could play an As the purpose of the formative assessments
instrumental role. were to help students and teachers realize
Formative assessment has been noted as a student achievement levels of English A1 and
critical part of learning in general, and English B1 core skills, two separate tests were
specifically in the realm of second language created, one for each courses’ core skills. While
learning (Norris, Davis & Timpe-Laughlin, there are a number of different ways to conduct
2017). The major difference in formative formative assessment, I decided to use a simple,
assessment and other types of standard testing is multiple choice style test because such tests can
that the former is designed to help both students be checked automatically and google forms can
be used to automatically send corrective knowledge-based questions very simply based
feedback based on student scores. While a more on the information written in chapters A1.1
personalized formative assessment might garner through A1.6. The questions were designed to
more gains in student learning, teachers have a be relatively easy for students who had read and
large number of students and need to conduct remembered the basic information from these
and grade the tests themselves, so an chapters to solve. I first created questions and
assessment that could be automatized was reading passages, and then had the assessment
prioritized to lessen the burden to teachers. team assist me by editing, finding errors, and
pointing out questions in which distractors were
English A1 Formative Assessment too strong or not strong enough. Tests were then
I worked with the formative assessment edited and checked a second time before being
creation team to develop a number of sent to teachers for implementation.
multiple-choice questions that matched the
exact information in chapters A1.1 to A1.6 in English B1 Formative Assessment
Pathways to Academic English (2020) – one of English B1 formative assessments were created
the two textbooks required for students taking in a similar fashion. These tests were created
general education English classes in 2020. based on the information, vocabulary, phrases
Pathways to Academic English was designed to and skills represented in chapters B1.1 through
include all of the basic information regarding B1.6, but were all created as listening questions.
the core skills that student needed to know, as The test included eight questions related to
well as necessary vocabulary and phrases which ‘note-taking while listening’ and ‘orally
students were expected to memorize before summarizing from notes’ (chapters B1.1 and
completion of English A1. Therefore, creating B1.2) which involved listening to a lecture,
questions that matched the vocabulary, phrases, taking notes, answering questions about it and
information and skills of these chapters exactly then comparing their notes to an oral summary
seemed like an appropriate way to test how of the same lecture. 13 questions related to
much the students had mastered the core skills. ‘phrasal verbs’ and ‘idiomatic expressions’
The multiple-choice questions included 10 (chapters B1.3 and B1.4, respectively) were
vocabulary style questions based on synonyms created in the style of short conversations in
and word parts taken from chapters A1.1 and which understanding the meaning of the phrase
A1.2 (word parts and synonym vocabulary), 13 or expression was necessary for selecting the
reading questions designed to test acquisitions correct answer. Six questions were created to
of skills represented in chapters A1.3 through test the final two core skills represented in
A1.6 (skimming and scanning, summarizing chapters B1.5 and B1.6 (giving opinions and
and paraphrasing, denotative and connotative conjecturing, and interrogatives). These
meaning, and making inferences), and 10 questions were created by having students listen
to short conversations and selecting the missing chapter. In this way, students received not only
interrogative or statement of opinion that would information about which questions they
best fit in the conversation. Finally, 10 correctly answered (or didn’t), they could also
knowledge-based questions based on the get valuable information as to which core skills,
information in chapters B1.1 through B1.6 were in particular, they needed to review. Teachers
created, similar to those found at the end of the could also see this information, and adjust their
English A1 core skills formative assessment. lessons accordingly. For example, in my own
class, I noticed that most of my B2 students had
Providing Feedback not mastered phrasal verbs or idiomatic
While simply alerting students to the correct expressions at all, and I therefore instituted
answers is technically a form of feedback, it is additional homework wherein students had to
not as helpful as understanding what they slowly memorize the information from
misunderstood about the question and why. Pathways to Academic English chapters B1.3
Though overly specific information could not and B1.4.
be given individually, students could at least
understand which core skills they had mastered 3. Implementation of Formative
and which they had not by using the free Assessment
Google Sheets ad-on Flubaroo. Teachers were The formative assessments of A1 and B1 core
given a list of which core skill each question skills were made available to teachers of
was meant to test, and could then program the general education English classes to be used in
sheet to give students a response for each core the first class of English A2 or B2. I requested
skill based on their performance. For example, that teachers share data for their students to
in my own class, if students correctly answered validate the assessments and monitor their
80% or more of the questions pertaining to a impact on English learning. However, usage of
particular core skill, I programed Flubaroo to the formative assessments and sharing of data
include a message that the student showed was optional, and only a portion of teachers
“good mastery” of that particular skill. Similarly, (eight) chose to share data with me for analysis.
if students scored on average between 50 and I implemented the A1 core skills formative
79% on questions designed to test a particular assessment and in my A2 classes, and the B1
core skill, they received a message letting them assessment in my B2 class. I provided
know that they had some mastery, but should individualized feedback to students through the
probably review the Pathways to Academic method outlined in the previous section and
English Chapter related to the core skill. Finally, worked to help students with A1 and B1 core
students who failed to answer at least 50% of skills that they didn’t perform well on.
the questions correctly on any particular core Specifically, for classes that showed low
skill were asked to please review the related performance on particular core skills, I gave
extra homework and assigned self-study of valid and that students who performed well on
skills that students under-performed in. I also the formative assessments also tended to have
made lectures about these core skills that I had higher TOEFL ITP® test scores.
recorded in the previous semester available and The second evaluation of the formative
took student questions about the skills as assessments was conducted to check for the
necessary. impact of the formative assessment on learning
outcomes. Specifically, a comparison of TOEFL
4. Evaluating the Assessment ITP® test delta scores (the results of the test
The formative assessments were evaluated in taken in December, subtracted by the score
two ways. First, the validity of the tests was obtained by the same student in August) was
checked by performing a Pearson’s product made between students who took only the A1
momentum correlation analysis between test formative assessment, students who only took
scores from a TOEFL ITP® test that some the B1 formative assessment, and those who
students had taken in August and their overall took neither (no students happened to take both
scores on the formative assessments. The results formative assessments). This data is presented
of this analysis is shown in Table 1. in Table 2.

Table 1. Correlation between TOEFL ITP test Table 2. Average delta scores of students
scores and A1/B1 Formative Assessments divided by formative assessment participation
A1 Assessm. B1 Assessm. A1 Only B1 Only None
Overall r=.54, p<.001 r=.53, p<.001 Overall 19.99 22.35 17.64
Score Reading 1.24 .80 .34
Reading r=.42, p<.001 r=.40, p<.001 Written 1.87 2.44 1.44
Written r=.43, p<.001 r=.43, p<.001 Expression
Expression Listening 3.33 3.45 3.08
Listening r=.46, p<.001 r=.49, p<.001
A simple ANOVA analysis was performed to
The data from Table 1 shows that both of the check for the significance of differences in
formative assessment tests were highly these groups, but it did not show significantly
correlated to TOEFL ITP® test scores taken different results. However, the trend that having
around the same time. Furthermore, the A1 core taken at least one of the formative assessments
skills formative assessment was especially seemed to result in slightly overall higher
correlated with the reading section scores, and TOEFL ITP® test score improvement, and
the B1 core skills assessment was more specifically that A1 formative assessment
correlated with the listening section scores. This participation generally resulted in higher
indicates that the tests were at least somewhat reading scores while B1 formative assessment
participation generally resulted in higher written assessment (pp. 255-272). De Gruyter.
expression and listening scores is promising.
Though no significant differences were noticed
with this data, it was quite incomplete, as only a
small number of students took the TOEFL
ITP® test both in August and in December
(yielding a delta score) and also participated in
one of the formative assessment tests.
In the future, I hope that more teachers will
use the formative assessments so that we can
more closely monitor the impact that they have
on student learning. Furthermore, I hope that
more students and teachers will make use of
these assessments, as they are quite indicative
of better TOEFL ITP® test scores, and therefore
mastering the A1 and B1 core skills should lead
to more improvement in the future. Furthermore,
I hope to work with teachers to develop the
formative assessments to be more useful to both
students and teachers and to achieve an even
higher correlation with TOEFL ITP® test scores.
Finally, I would like to take student survey data
to see whether or not the formative assessments
actually encouraged them to review old material
or study harder.

References
Norris, J.M., Davis, J., & Timpe-Laughlin, V.(2017).
Second Language Educational Experiences for
Adult Learners. Routledge
Pathways to academic English (2020). Tohoku
University.
Turner, C.E., & Purpura, J.E. (2016).
Learning-oriented assessment in second and
foreign language classrooms. In D. Tsagari & J.
Banerjee (Eds.), Handbook of second language