The rating of proficiency tests that use the Inter-agency Roundtable (ILR) and American Council o... more The rating of proficiency tests that use the Inter-agency Roundtable (ILR) and American Council on the Teaching of Foreign Languages (ACTFL) guidelines claims that each major level is based on hierarchal linguistic functions that require mastery of multidimensional traits in such a way that each level subsumes the levels beneath it. These characteristics are part of what is commonly referred to as floor and ceiling scoring. In this binary approach to scoring that differentiates between sustained performance and linguistic breakdown, raters evaluate many features including vocabulary use, grammatical accuracy, pronunciation, and pragmatics, yet there has been very little empirical validation on the practice of floor/ceiling scoring. This study examined the relationship between temporal oral fluency, prompt type, and proficiency level based on a data set comprised of 147 Oral Proficiency Interview - computer (OPIc) exam responses whose ratings ranged from Intermediate Low to Advanced High [AH]. As speakers progressed in proficiency, they were more fluent. In terms of floor and ceiling scoring, the prompts that elicited speech a level above the sustained level generally resulted in speech that was slower and had more breakdown than the floor-level prompts, though the differences were slight and not significantly different. Thus, temporal fluency features alone are insufficient in floor/ceiling scoring but are likely a contributing feature.
Elicited imitation (EI) assessments have been shown to discriminate well between speakers across ... more Elicited imitation (EI) assessments have been shown to discriminate well between speakers across proficiency levels, but little has been reported on the effect L2 accent has on test-takers’ ability to understand and process the test items they hear. Furthermore, no study has investigated the effect of accent on test-taker perceptions of EI tests. This study examined the relationships among accent, accent familiarity, EI test item difficulty and test scores. To investigate, self-reports of students’ exposure to different varieties of English were obtained from a pre-assessment survey. An EI test (63 items) was then administered in which English language learners (n = 213) in the United States listened to test items in three varieties of English: American English, Australian English, and British English. A Rasch analysis found that the test had high reliability (person separation = .94), with intended item level and accent both having a significant effect on test item difficulty. Surv...
The rating of proficiency tests that use the Inter-agency Roundtable (ILR) and American Council o... more The rating of proficiency tests that use the Inter-agency Roundtable (ILR) and American Council on the Teaching of Foreign Languages (ACTFL) guidelines claims that each major level is based on hierarchal linguistic functions that require mastery of multidimensional traits in such a way that each level subsumes the levels beneath it. These characteristics are part of what is commonly referred to as floor and ceiling scoring. In this binary approach to scoring that differentiates between sustained performance and linguistic breakdown, raters evaluate many features including vocabulary use, grammatical accuracy, pronunciation, and pragmatics, yet there has been very little empirical validation on the practice of floor/ceiling scoring. This study examined the relationship between temporal oral fluency, prompt type, and proficiency level based on a data set comprised of 147 Oral Proficiency Interview - computer (OPIc) exam responses whose ratings ranged from Intermediate Low to Advanced ...
The ACTFL Oral Proficiency Interview (OPI) and accompanying proficiency guidelines have become a ... more The ACTFL Oral Proficiency Interview (OPI) and accompanying proficiency guidelines have become a mainstay in academic foreign language assessment. In 2006, an asynchronous electronic interface of the OPI was made available in which candidates are asked to record themselves responding to questions delivered aurally via computer, labeled the Oral Proficiency Interview‐computer (OPIc). Thompson, Cox, and Knapp () found that both instruments produced similar results but that many students in their sample who scored at the Advanced Low level on the traditional OPI scored one sublevel higher on the OPIc. The primary objective of the present study is to analyze lexical features, i.e., lexical diversity and lexical density, and discursive features, i.e., temporal and repair fluency, of students’ speech from a subset of OPI and OPIc exams to examine the relationship between the language elicited from each exam type, i.e., face‐to‐face and via computer. The research consists of two studies th...
This Dissertation is brought to you for free and open access by BYU ScholarsArchive. It has been ... more This Dissertation is brought to you for free and open access by BYU ScholarsArchive. It has been accepted for inclusion in All Theses and Dissertations by an authorized administrator of BYU ScholarsArchive. For more information, please contact scholarsarchive@byu.edu.
Part of the Educational Psychology Commons This Dissertation is brought to you for free and open ... more Part of the Educational Psychology Commons This Dissertation is brought to you for free and open access by BYU ScholarsArchive. It has been accepted for inclusion in All Theses and Dissertations by an authorized administrator of BYU ScholarsArchive. For more information, please contact scholarsarchive@byu.edu.
Pacific Rim Objective Measurement Symposium (PROMS) 2015 Conference Proceedings, 2016
Standards-based proficiency frameworks have become an integral part of the educational assessment... more Standards-based proficiency frameworks have become an integral part of the educational assessment landscape. These frameworks take complex, multidimensional competencies and attempt to represent them as a numerical value on a vertical scale that can be used by students, teachers, testing organizations, school admissions officers, employers, and others that want some certification of the proficiency of examinees.
The rating of proficiency tests that use the Inter-agency Roundtable (ILR) and American Council o... more The rating of proficiency tests that use the Inter-agency Roundtable (ILR) and American Council on the Teaching of Foreign Languages (ACTFL) guidelines claims that each major level is based on hierarchal linguistic functions that require mastery of multidimensional traits in such a way that each level subsumes the levels beneath it. These characteristics are part of what is commonly referred to as floor and ceiling scoring. In this binary approach to scoring that differentiates between sustained performance and linguistic breakdown, raters evaluate many features including vocabulary use, grammatical accuracy, pronunciation, and pragmatics, yet there has been very little empirical validation on the practice of floor/ceiling scoring. This study examined the relationship between temporal oral fluency, prompt type, and proficiency level based on a data set comprised of 147 Oral Proficiency Interview - computer (OPIc) exam responses whose ratings ranged from Intermediate Low to Advanced High [AH]. As speakers progressed in proficiency, they were more fluent. In terms of floor and ceiling scoring, the prompts that elicited speech a level above the sustained level generally resulted in speech that was slower and had more breakdown than the floor-level prompts, though the differences were slight and not significantly different. Thus, temporal fluency features alone are insufficient in floor/ceiling scoring but are likely a contributing feature.
Elicited imitation (EI) assessments have been shown to discriminate well between speakers across ... more Elicited imitation (EI) assessments have been shown to discriminate well between speakers across proficiency levels, but little has been reported on the effect L2 accent has on test-takers’ ability to understand and process the test items they hear. Furthermore, no study has investigated the effect of accent on test-taker perceptions of EI tests. This study examined the relationships among accent, accent familiarity, EI test item difficulty and test scores. To investigate, self-reports of students’ exposure to different varieties of English were obtained from a pre-assessment survey. An EI test (63 items) was then administered in which English language learners (n = 213) in the United States listened to test items in three varieties of English: American English, Australian English, and British English. A Rasch analysis found that the test had high reliability (person separation = .94), with intended item level and accent both having a significant effect on test item difficulty. Surv...
The rating of proficiency tests that use the Inter-agency Roundtable (ILR) and American Council o... more The rating of proficiency tests that use the Inter-agency Roundtable (ILR) and American Council on the Teaching of Foreign Languages (ACTFL) guidelines claims that each major level is based on hierarchal linguistic functions that require mastery of multidimensional traits in such a way that each level subsumes the levels beneath it. These characteristics are part of what is commonly referred to as floor and ceiling scoring. In this binary approach to scoring that differentiates between sustained performance and linguistic breakdown, raters evaluate many features including vocabulary use, grammatical accuracy, pronunciation, and pragmatics, yet there has been very little empirical validation on the practice of floor/ceiling scoring. This study examined the relationship between temporal oral fluency, prompt type, and proficiency level based on a data set comprised of 147 Oral Proficiency Interview - computer (OPIc) exam responses whose ratings ranged from Intermediate Low to Advanced ...
The ACTFL Oral Proficiency Interview (OPI) and accompanying proficiency guidelines have become a ... more The ACTFL Oral Proficiency Interview (OPI) and accompanying proficiency guidelines have become a mainstay in academic foreign language assessment. In 2006, an asynchronous electronic interface of the OPI was made available in which candidates are asked to record themselves responding to questions delivered aurally via computer, labeled the Oral Proficiency Interview‐computer (OPIc). Thompson, Cox, and Knapp () found that both instruments produced similar results but that many students in their sample who scored at the Advanced Low level on the traditional OPI scored one sublevel higher on the OPIc. The primary objective of the present study is to analyze lexical features, i.e., lexical diversity and lexical density, and discursive features, i.e., temporal and repair fluency, of students’ speech from a subset of OPI and OPIc exams to examine the relationship between the language elicited from each exam type, i.e., face‐to‐face and via computer. The research consists of two studies th...
This Dissertation is brought to you for free and open access by BYU ScholarsArchive. It has been ... more This Dissertation is brought to you for free and open access by BYU ScholarsArchive. It has been accepted for inclusion in All Theses and Dissertations by an authorized administrator of BYU ScholarsArchive. For more information, please contact scholarsarchive@byu.edu.
Part of the Educational Psychology Commons This Dissertation is brought to you for free and open ... more Part of the Educational Psychology Commons This Dissertation is brought to you for free and open access by BYU ScholarsArchive. It has been accepted for inclusion in All Theses and Dissertations by an authorized administrator of BYU ScholarsArchive. For more information, please contact scholarsarchive@byu.edu.
Pacific Rim Objective Measurement Symposium (PROMS) 2015 Conference Proceedings, 2016
Standards-based proficiency frameworks have become an integral part of the educational assessment... more Standards-based proficiency frameworks have become an integral part of the educational assessment landscape. These frameworks take complex, multidimensional competencies and attempt to represent them as a numerical value on a vertical scale that can be used by students, teachers, testing organizations, school admissions officers, employers, and others that want some certification of the proficiency of examinees.
Uploads
Papers by Troy Cox