Educational Innovations and Emerging Technologies, 2023
To enhance diverse students’ learning experiences, tertiary educational institutions are increasi... more To enhance diverse students’ learning experiences, tertiary educational institutions are increasingly adopting adaptive technologies. Although there are views that adaptive learning platforms contribute to learning within higher education, efficacy research of such systems yields mixed results. This is an indication that more could be studied concerning how adaptive learning can be efficacious within higher education contexts particularly when cognate studies have mostly discussed how cognitive measures differ between learners who engaged and those who did not engage with an adaptive learning system. Beyond cognitive measures such as achievement scores provided by adaptive learning systems, this brief paper proposes that learner satisfaction towards such systems - as an imperative factor due to its direct and significant relationship between users and the system - needs to be examined, in part to evaluate the efficacy of adaptive learning systems. Given the complexity of learner satisfaction as a construct, such investigations have the potential to illuminate important factors of system optimizations, developments, and interventions that provide crucial guidelines for higher education institutions to implement successful adaptive learning courses that meet students’ satisfaction.
Adaptive learning systems have gained popularity within higher education, given the affordances t... more Adaptive learning systems have gained popularity within higher education, given the affordances that claim to enhance student learning outcomes by providing personalised learning trajectories that allow students to interact with course content at their own pace. Nonetheless, studies investigating the impact of such systems on learning outcomes such as course scores have been mixed, in part due to the research approaches applied, as found by the review undertaken in this study. Yet, for purposes of accountability, it remains critical to investigate the efficacy of adaptive learning systems, at least for its relation to course scores when assessment stakes are involved. This study reports the efficacy of an in-house adaptive learning system used within an institution in terms of its impact on course scores, based upon propensity score analysis, a quasi-experimental approach considered as a feasible alternative to randomised controlled trials. Results of this study reported a difference in course scores, suggesting merit in using the in-house adaptive learning system, though the difference did not present statistically significant differences at the 95% confidence level. Directions for future research are also discussed.
It is well-established that English for academic purposes (EAP) programmes are critical for acade... more It is well-established that English for academic purposes (EAP) programmes are critical for academic success in English-medium universities. Nonetheless, there is significantly less research on how EAP programmes impact multilingual domestic university students, compared to that of international students who speak English as a foreign language. While an earlier study on a university in Singapore found that an EAP programme had a statistically significant and positive intervention effect on students’ grade point average of the first semester upon matriculation, this study sought to investigate the perceptions of students from the same university, as this would contribute to how EAP programmes could be refined to better support learning. Semi-structured interviews were conducted with 50 students invited based on a maximum variation strategy. Based on the thematic analyses undertaken, four themes (i.e., programme delivery, linguistic improvement, learning transfer and change in self-efficacy) were identified and discussed. These themes contributed to the formulation of SILVER, an innovative framework of components for consideration in EAP course design and delivery within higher education.
This article presents and focuses on the Institutional Support Questionnaire (ISQ) that was devel... more This article presents and focuses on the Institutional Support Questionnaire (ISQ) that was developed and validated to complement the Learning Needs Questionnaire (LNQ). While the LNQ, validated and published earlier, assessed students’ perceived learning needs, the ISQ assesses students’ psychological perspectives of their institution, particularly how they perceive their institution supports their learning. Both questionnaires work in tandem to support resource optimisation efforts in establishing targeted academic support structures within teaching-focused tertiary institutions. This study found that the 42-item ISQ had adequate psychometric properties and that institutional support could be represented by four factors (i.e., academic competency support, teaching practices, tutors’ characteristics, and use of technology in instruction) that reflected in large part the factors characterised by the LNQ (i.e., perceived academic competency, time management, preferred tutors’ characteristics, and use of technology). Practical applications of the use of both the ISQ and LNQ (i.e., how both could be applied in a tertiary education setting to identify perceived students’ learning needs and whether an institution is providing adequate support to meet these needs) and limitations on their use are discussed.
Tertiary educational institutions have continually implemented various educational technologies t... more Tertiary educational institutions have continually implemented various educational technologies to support student learning. An example is adaptive learning systems, within which learners take ownership of their learning experience and accelerate future learning. Despite the importance of considering learner satisfaction in the development of such systems given how it has been widely acknowledged as an indication of the success of e-learning systems, research in the area of adaptive learning technologies for education has concentrated more on tailoring instruction to implement personalised learning. A review of instruments measuring learner satisfaction of virtual learning environments found that the learner satisfaction questionnaire (LSQ) that was developed to measure learner satisfaction of e-learning systems, and preliminarily validated by means of exploratory factor analysis, was most suitable for adaptation. This study sought to adapt and validate the LSQ for the purpose of measuring learner satisfaction of an adaptive learning system developed in-house. A total of 121 tertiary students recruited via availability sampling took part in this study. Hierarchical confirmatory factor analysis was performed as part of the validation. Results presented the adapted LSQ as a 14-item instrument that can be readily deployed on a broad scale basis. The adapted LSQ also yielded valid and reliable satisfaction scores both at the subscale as well as the overall scale level. Practical implications are discussed, noting that such scores could inform the further development and refinement of AdLeS or similar systems, with the view of benefiting students.
Despite the criticality of considering student academic motivation as it influences learning, res... more Despite the criticality of considering student academic motivation as it influences learning, research within the field of adaptive learning technologies for education has so far focused more on customising instruction to implement personalised learning, than investigating how personalised learning is associated with learners’ motivation. Given this, a robust instrument is required to gather information about student academic motivation within the context of adaptive learning technologies. This study sought to validate the Academic Motivation Toward Mathematics Survey (AMTMS) currently used to measure motivation based on self-determination theory in mathematics education at pre-tertiary levels (grades 11 and 12) in Asia. A total of 196 participants recruited via availability sampling took part in this study, after interacting with an in-house mathematics adaptive learning system within a tertiary educational institution. The validation was performed based on modern test theory given that it overcomes some limitations of classical factor analytic approaches. Results supported the factorial structure of the AMTMS but 12 of the original 21 items had to be rescored to establish ordered thresholds. Further, the bifactor equivalent solution suggested the possibility of reporting a singular motivation index comprising the five factors within the AMTMS. Along with the results, this study offers researchers a robust and validated instrument to measure motivation toward mathematics that can be used within an adaptive learning environment.
Moral reasoning is a key element of the Singapore Ministry of Education's Character and Citizensh... more Moral reasoning is a key element of the Singapore Ministry of Education's Character and Citizenship Education curriculum, and educators in Singapore are currently encouraged to facilitate growth in students' moral reasoning based on the Stages of Moral Development established by Lawrence Kohlberg. To monitor students' progress in this area, however, a validated test of moral reasoning is needed. A review of existing moral reasoning assessments indicated that none provide a practical means by which large numbers of students in Singapore classrooms can be assessed in this area. In this study, the Moral Reasoning Questionnaire (MRQ) was developed and validated to assess moral reasoning development in Singapore secondary classrooms. The MRQ presents moral dilemmas that integrate prosocial, antisocial , and social pressure elements within situations that would be familiar to Singapore secondary (grades 7-12, aged between 12 and 18) students. Evidence supported the reliability and validity of the MRQ. The MRQ could be applicable as a practical means to assess moral reasoning on a broad-scale basis.
To support teachers in facilitating students’ moral reasoning development as specified within the... more To support teachers in facilitating students’ moral reasoning development as specified within the Singapore Ministry of Education Character and Citizenship Education curriculum, the Moral Reasoning Questionnaire (MRQ) was developed and underwent preliminary validation. Based upon expert reviews, cognitive interviews and a classical test theorybased factor analytic approach, the development and preliminary validation found evidence (i.e., content appropriateness, response processes and internal structure) to support the validity and reliability of the MRQ. This study aims to extend the validation by examining the purported MRQ items and scale at a deeper level on the Rasch Measurement Theory, given that it is the only model that presents appropriate properties of interval measurement on a log-linear scale. The Rasch analysis found anomalies including differential item functioning and disordered thresholds in the initial set of items. Upon remediation and a second Rasch analysis, the MRQ responses were consistent with that expressed by the Rasch model (i.e., an item with an endorsability higher than what a respondent would tend to endorse would have a lower probability of being endorsed than an item exhibiting an endorsability below what that respondent would tend to endorse) and hence, there was sufficient evidence to support measurement invariance, and that MRQ scores could be concluded to characterise persons invariantly across a continuum.
Journal of Higher Education Theory and Practice, 2021
Written with a tertiary educator in mind, this short paper aims to examine possibilities to maint... more Written with a tertiary educator in mind, this short paper aims to examine possibilities to maintain assessment score validity in light of the impact of COVID-19 on educational assessment. In doing so, this paper seeks to: (1) support some suggestions mooted by scholars or implemented by some educational systems, and (2) discuss how these may be applicable within tertiary education. The possibilities and a key threat to score validity (i.e., contract cheating) are discussed with respect to three of five facets of validity evidence (i.e., test content, internal structure and consequences of testing). The paper concludes by offering some views tertiary educators could consider when adjusting assessment to uphold score validity.
This article outlines the development and preliminary validation of the Learning Needs Questionna... more This article outlines the development and preliminary validation of the Learning Needs Questionnaire (LNQ). The LNQ is intended to provide information for optimising resources in establishing targeted academic support structures within institutions of higher learning with a focus on teaching. Owing in part to the lack of an instrument applicable to teaching universities with a diverse student population in Singapore, the LNQ sought to measure and provide information about moderators of student learning needs. Three stages were undertaken to establish evidence supporting the validity of the LNQ. Stage 1 saw the development of items based on feedback from university instructors, a review of relevant literature and expert judgement. Data was collected in Stage 2 as the LNQ was administered to students (N = 1178). In Stage 3, exploratory factor analysis (n = 589) and first-order confirmatory factor analysis (n = 589) suggested a four-factor model (perceived academic competency, time management, preferred tutor’s characteristics and use of technology) with a reduction of 41 to 33 items. A second-order confirmatory factor analysis suggested that the four identified factors could be associated with an overarching measure, though interpreting subscale scores presented a more plausible approach. Practical applications and limitations of the LNQ are discussed.
Endeavours to assess moral reasoning in education have largely been via established but less cont... more Endeavours to assess moral reasoning in education have largely been via established but less contemporary measures, with recent measures developed more than a decade ago. Together with the call to go beyond assessing cognitive constructs, and the intended learning outcomes of the Singapore Ministry of Education Character and Citizenship Education curriculum that requires teachers to help students progress in their moral reasoning stages, there is a need for a measure that affords consistency when evaluating students' attainment of learning outcomes stipulated in the curriculum. Guided by Messick's unitary concept of validity, this paper reviewed existing measures of moral reasoning for suitability, and found that established measures presented varying degrees of tenability in assessing moral reasoning. Findings related to content appropriateness and group administrability yielded a paucity of measures applicable for large-scale assessment of moral reasoning in Singapore secondary schools. To address some of these issues, this review suggests the development of a fit-for-purpose measure.
Proceedings of the Adult Education in Global Times Conference 2020, 2020
This paper reports a study of the exhibited learning behaviours of adult learners at a tertiary e... more This paper reports a study of the exhibited learning behaviours of adult learners at a tertiary education institution in the Republic of Singapore. The purpose is to provide nuances in the understanding of different learning behaviors and needs of adult learners so that adult educators can be more targeted in managing adult learners, and not simply treat adult learners as a single entity. Using a questionnaire, the responses collected showed that while the participants’ learning behaviours and habits generally do adhere to the consensus on how adult learners learn, there were also differences within, such as in the regulation of learning processes. Hence, it is important not to take a “one-size-fit-all” approach towards adult education.
The objective of this study is to contribute to the knowledge base of teacher professional develo... more The objective of this study is to contribute to the knowledge base of teacher professional development and formative assessment using Singapore as an example. The study was carried out through an exploratory case study in a Singapore school. Our study shows that while the teachers find formative assessment beneficial, they do not have sufficient knowledge of it and face the challenges of heavy workload, time pressure and lack of confidence. The results serve to inform policymakers and educators of the need to provide effective and sustained teacher professional development in formative assessment as well as continual teacher support and collaboration.
This article outlines the development and validation of the Computer-Delivered Test (CDT) Accepta... more This article outlines the development and validation of the Computer-Delivered Test (CDT) Acceptance Questionnaire (CTAQ). The CTAQ was designed to be a practical measure of CDT acceptance of Singapore secondary and high school students (Grades 7-12) toward taking tests within an e-assessment system. The stages of test (questionnaire item) content, response processes, and internal structure under Messick’s unified concept of validity suggested that the CTAQ had sound psychometric properties. Exploratory factor analysis (EFA; n = 485) and confirmatory factor analysis (CFA; n = 484) yielded a three-factor model (ease of use, involvement and experience) with a reduction of 21 to 13 items for CDT acceptance. Practical applications and limitations of the CTAQ are discussed.
The increasing focus to go beyond assessing cognitive constructs around the globe and in Singapor... more The increasing focus to go beyond assessing cognitive constructs around the globe and in Singapore (Ng in https://www.gov.sg/microsites/budget2017/press-room/news/content/speech-by-minister-ng-chee-meng-for-moe-schools-at-the-committee-of-supply-debate, 2017) such as cultural awareness, grit and moral reasoning and the call for schools to prepare students for citizenship the development of corresponding instruments applicable in a larger scale. Given the significance of moral reasoning and its assessment are key elements within the current Singapore Character and Citizenship Education (CCE) curriculum (Singapore Ministry of Education in https://www.moe.gov.sg/docs/default-source/document/education/syllabuses/character-citizenship-education/files/2014-character-and-citizenship-education-(secondary)-syllabus.pdf, 2012), there is room to explore how moral reasoning can be assessed practically in a large-scale setting. Undergirding this exploration requires a review of the epistemology of moral reasoning, corresponding theoretical frameworks, and how they might apply to moral education in Singapore schools. By reviewing perspectives of moral reasoning from relevant literature beyond educational research to business, medical and philosophical domains, this paper aims to propose an operational definition of moral reasoning suitable for use in Singapore schools, and to mitigate possible issues brought about by educators’ disparate or assumed understanding of moral reasoning. This can consequently support the enactment of the CCE curriculum and be applied to the development of assessment instruments suitable for use within Singapore schools and similar contexts.
Purpose – The purpose of this paper is to evaluate an English Proficiency (EP) programme and its ... more Purpose – The purpose of this paper is to evaluate an English Proficiency (EP) programme and its efficacy with respect to students’ academic performance in a university within a multi-lingual context, as the programme had been in effect for some years.
Design/methodology/approach – A quasi-experimental approach was used to study the efficacy of an EP programme in a university within a multilingual context. Data across two academic years were used, along with regression discontinuity design.
Findings – Results suggest that the EP programme had a significant and positive intervention effect on students’ initial semester grade point average. The programme effect size was found to be medium to large.
Research limitations/implications – It might be useful to extend the study for one more year for more concrete conclusions. As the study was anchored upon the structure of the 2016 EP programme, any major curricular/structural change to the programme warrants another study.
Practical implications – This study demonstrated that the implementation of EP programmes in higher education institutions is essential not only for international students who are foreign language speakers of English but also for domestic students in English-speaking countries, especially for bi/multilingual speakers.
Originality/value – Previous studies related to the efficacy of EP within higher education have focused on international students who speak English as an additional/foreign language. Further, most studies have focussed on students’ self-reported experiences and have yielded disparate findings. This study contributes to scholarship as it addresses the under-researched area related to domestic students who speak English as the first language in a bi/multi-lingual context.
Educational Innovations and Emerging Technologies, 2023
To enhance diverse students’ learning experiences, tertiary educational institutions are increasi... more To enhance diverse students’ learning experiences, tertiary educational institutions are increasingly adopting adaptive technologies. Although there are views that adaptive learning platforms contribute to learning within higher education, efficacy research of such systems yields mixed results. This is an indication that more could be studied concerning how adaptive learning can be efficacious within higher education contexts particularly when cognate studies have mostly discussed how cognitive measures differ between learners who engaged and those who did not engage with an adaptive learning system. Beyond cognitive measures such as achievement scores provided by adaptive learning systems, this brief paper proposes that learner satisfaction towards such systems - as an imperative factor due to its direct and significant relationship between users and the system - needs to be examined, in part to evaluate the efficacy of adaptive learning systems. Given the complexity of learner satisfaction as a construct, such investigations have the potential to illuminate important factors of system optimizations, developments, and interventions that provide crucial guidelines for higher education institutions to implement successful adaptive learning courses that meet students’ satisfaction.
Adaptive learning systems have gained popularity within higher education, given the affordances t... more Adaptive learning systems have gained popularity within higher education, given the affordances that claim to enhance student learning outcomes by providing personalised learning trajectories that allow students to interact with course content at their own pace. Nonetheless, studies investigating the impact of such systems on learning outcomes such as course scores have been mixed, in part due to the research approaches applied, as found by the review undertaken in this study. Yet, for purposes of accountability, it remains critical to investigate the efficacy of adaptive learning systems, at least for its relation to course scores when assessment stakes are involved. This study reports the efficacy of an in-house adaptive learning system used within an institution in terms of its impact on course scores, based upon propensity score analysis, a quasi-experimental approach considered as a feasible alternative to randomised controlled trials. Results of this study reported a difference in course scores, suggesting merit in using the in-house adaptive learning system, though the difference did not present statistically significant differences at the 95% confidence level. Directions for future research are also discussed.
It is well-established that English for academic purposes (EAP) programmes are critical for acade... more It is well-established that English for academic purposes (EAP) programmes are critical for academic success in English-medium universities. Nonetheless, there is significantly less research on how EAP programmes impact multilingual domestic university students, compared to that of international students who speak English as a foreign language. While an earlier study on a university in Singapore found that an EAP programme had a statistically significant and positive intervention effect on students’ grade point average of the first semester upon matriculation, this study sought to investigate the perceptions of students from the same university, as this would contribute to how EAP programmes could be refined to better support learning. Semi-structured interviews were conducted with 50 students invited based on a maximum variation strategy. Based on the thematic analyses undertaken, four themes (i.e., programme delivery, linguistic improvement, learning transfer and change in self-efficacy) were identified and discussed. These themes contributed to the formulation of SILVER, an innovative framework of components for consideration in EAP course design and delivery within higher education.
This article presents and focuses on the Institutional Support Questionnaire (ISQ) that was devel... more This article presents and focuses on the Institutional Support Questionnaire (ISQ) that was developed and validated to complement the Learning Needs Questionnaire (LNQ). While the LNQ, validated and published earlier, assessed students’ perceived learning needs, the ISQ assesses students’ psychological perspectives of their institution, particularly how they perceive their institution supports their learning. Both questionnaires work in tandem to support resource optimisation efforts in establishing targeted academic support structures within teaching-focused tertiary institutions. This study found that the 42-item ISQ had adequate psychometric properties and that institutional support could be represented by four factors (i.e., academic competency support, teaching practices, tutors’ characteristics, and use of technology in instruction) that reflected in large part the factors characterised by the LNQ (i.e., perceived academic competency, time management, preferred tutors’ characteristics, and use of technology). Practical applications of the use of both the ISQ and LNQ (i.e., how both could be applied in a tertiary education setting to identify perceived students’ learning needs and whether an institution is providing adequate support to meet these needs) and limitations on their use are discussed.
Tertiary educational institutions have continually implemented various educational technologies t... more Tertiary educational institutions have continually implemented various educational technologies to support student learning. An example is adaptive learning systems, within which learners take ownership of their learning experience and accelerate future learning. Despite the importance of considering learner satisfaction in the development of such systems given how it has been widely acknowledged as an indication of the success of e-learning systems, research in the area of adaptive learning technologies for education has concentrated more on tailoring instruction to implement personalised learning. A review of instruments measuring learner satisfaction of virtual learning environments found that the learner satisfaction questionnaire (LSQ) that was developed to measure learner satisfaction of e-learning systems, and preliminarily validated by means of exploratory factor analysis, was most suitable for adaptation. This study sought to adapt and validate the LSQ for the purpose of measuring learner satisfaction of an adaptive learning system developed in-house. A total of 121 tertiary students recruited via availability sampling took part in this study. Hierarchical confirmatory factor analysis was performed as part of the validation. Results presented the adapted LSQ as a 14-item instrument that can be readily deployed on a broad scale basis. The adapted LSQ also yielded valid and reliable satisfaction scores both at the subscale as well as the overall scale level. Practical implications are discussed, noting that such scores could inform the further development and refinement of AdLeS or similar systems, with the view of benefiting students.
Despite the criticality of considering student academic motivation as it influences learning, res... more Despite the criticality of considering student academic motivation as it influences learning, research within the field of adaptive learning technologies for education has so far focused more on customising instruction to implement personalised learning, than investigating how personalised learning is associated with learners’ motivation. Given this, a robust instrument is required to gather information about student academic motivation within the context of adaptive learning technologies. This study sought to validate the Academic Motivation Toward Mathematics Survey (AMTMS) currently used to measure motivation based on self-determination theory in mathematics education at pre-tertiary levels (grades 11 and 12) in Asia. A total of 196 participants recruited via availability sampling took part in this study, after interacting with an in-house mathematics adaptive learning system within a tertiary educational institution. The validation was performed based on modern test theory given that it overcomes some limitations of classical factor analytic approaches. Results supported the factorial structure of the AMTMS but 12 of the original 21 items had to be rescored to establish ordered thresholds. Further, the bifactor equivalent solution suggested the possibility of reporting a singular motivation index comprising the five factors within the AMTMS. Along with the results, this study offers researchers a robust and validated instrument to measure motivation toward mathematics that can be used within an adaptive learning environment.
Moral reasoning is a key element of the Singapore Ministry of Education's Character and Citizensh... more Moral reasoning is a key element of the Singapore Ministry of Education's Character and Citizenship Education curriculum, and educators in Singapore are currently encouraged to facilitate growth in students' moral reasoning based on the Stages of Moral Development established by Lawrence Kohlberg. To monitor students' progress in this area, however, a validated test of moral reasoning is needed. A review of existing moral reasoning assessments indicated that none provide a practical means by which large numbers of students in Singapore classrooms can be assessed in this area. In this study, the Moral Reasoning Questionnaire (MRQ) was developed and validated to assess moral reasoning development in Singapore secondary classrooms. The MRQ presents moral dilemmas that integrate prosocial, antisocial , and social pressure elements within situations that would be familiar to Singapore secondary (grades 7-12, aged between 12 and 18) students. Evidence supported the reliability and validity of the MRQ. The MRQ could be applicable as a practical means to assess moral reasoning on a broad-scale basis.
To support teachers in facilitating students’ moral reasoning development as specified within the... more To support teachers in facilitating students’ moral reasoning development as specified within the Singapore Ministry of Education Character and Citizenship Education curriculum, the Moral Reasoning Questionnaire (MRQ) was developed and underwent preliminary validation. Based upon expert reviews, cognitive interviews and a classical test theorybased factor analytic approach, the development and preliminary validation found evidence (i.e., content appropriateness, response processes and internal structure) to support the validity and reliability of the MRQ. This study aims to extend the validation by examining the purported MRQ items and scale at a deeper level on the Rasch Measurement Theory, given that it is the only model that presents appropriate properties of interval measurement on a log-linear scale. The Rasch analysis found anomalies including differential item functioning and disordered thresholds in the initial set of items. Upon remediation and a second Rasch analysis, the MRQ responses were consistent with that expressed by the Rasch model (i.e., an item with an endorsability higher than what a respondent would tend to endorse would have a lower probability of being endorsed than an item exhibiting an endorsability below what that respondent would tend to endorse) and hence, there was sufficient evidence to support measurement invariance, and that MRQ scores could be concluded to characterise persons invariantly across a continuum.
Journal of Higher Education Theory and Practice, 2021
Written with a tertiary educator in mind, this short paper aims to examine possibilities to maint... more Written with a tertiary educator in mind, this short paper aims to examine possibilities to maintain assessment score validity in light of the impact of COVID-19 on educational assessment. In doing so, this paper seeks to: (1) support some suggestions mooted by scholars or implemented by some educational systems, and (2) discuss how these may be applicable within tertiary education. The possibilities and a key threat to score validity (i.e., contract cheating) are discussed with respect to three of five facets of validity evidence (i.e., test content, internal structure and consequences of testing). The paper concludes by offering some views tertiary educators could consider when adjusting assessment to uphold score validity.
This article outlines the development and preliminary validation of the Learning Needs Questionna... more This article outlines the development and preliminary validation of the Learning Needs Questionnaire (LNQ). The LNQ is intended to provide information for optimising resources in establishing targeted academic support structures within institutions of higher learning with a focus on teaching. Owing in part to the lack of an instrument applicable to teaching universities with a diverse student population in Singapore, the LNQ sought to measure and provide information about moderators of student learning needs. Three stages were undertaken to establish evidence supporting the validity of the LNQ. Stage 1 saw the development of items based on feedback from university instructors, a review of relevant literature and expert judgement. Data was collected in Stage 2 as the LNQ was administered to students (N = 1178). In Stage 3, exploratory factor analysis (n = 589) and first-order confirmatory factor analysis (n = 589) suggested a four-factor model (perceived academic competency, time management, preferred tutor’s characteristics and use of technology) with a reduction of 41 to 33 items. A second-order confirmatory factor analysis suggested that the four identified factors could be associated with an overarching measure, though interpreting subscale scores presented a more plausible approach. Practical applications and limitations of the LNQ are discussed.
Endeavours to assess moral reasoning in education have largely been via established but less cont... more Endeavours to assess moral reasoning in education have largely been via established but less contemporary measures, with recent measures developed more than a decade ago. Together with the call to go beyond assessing cognitive constructs, and the intended learning outcomes of the Singapore Ministry of Education Character and Citizenship Education curriculum that requires teachers to help students progress in their moral reasoning stages, there is a need for a measure that affords consistency when evaluating students' attainment of learning outcomes stipulated in the curriculum. Guided by Messick's unitary concept of validity, this paper reviewed existing measures of moral reasoning for suitability, and found that established measures presented varying degrees of tenability in assessing moral reasoning. Findings related to content appropriateness and group administrability yielded a paucity of measures applicable for large-scale assessment of moral reasoning in Singapore secondary schools. To address some of these issues, this review suggests the development of a fit-for-purpose measure.
Proceedings of the Adult Education in Global Times Conference 2020, 2020
This paper reports a study of the exhibited learning behaviours of adult learners at a tertiary e... more This paper reports a study of the exhibited learning behaviours of adult learners at a tertiary education institution in the Republic of Singapore. The purpose is to provide nuances in the understanding of different learning behaviors and needs of adult learners so that adult educators can be more targeted in managing adult learners, and not simply treat adult learners as a single entity. Using a questionnaire, the responses collected showed that while the participants’ learning behaviours and habits generally do adhere to the consensus on how adult learners learn, there were also differences within, such as in the regulation of learning processes. Hence, it is important not to take a “one-size-fit-all” approach towards adult education.
The objective of this study is to contribute to the knowledge base of teacher professional develo... more The objective of this study is to contribute to the knowledge base of teacher professional development and formative assessment using Singapore as an example. The study was carried out through an exploratory case study in a Singapore school. Our study shows that while the teachers find formative assessment beneficial, they do not have sufficient knowledge of it and face the challenges of heavy workload, time pressure and lack of confidence. The results serve to inform policymakers and educators of the need to provide effective and sustained teacher professional development in formative assessment as well as continual teacher support and collaboration.
This article outlines the development and validation of the Computer-Delivered Test (CDT) Accepta... more This article outlines the development and validation of the Computer-Delivered Test (CDT) Acceptance Questionnaire (CTAQ). The CTAQ was designed to be a practical measure of CDT acceptance of Singapore secondary and high school students (Grades 7-12) toward taking tests within an e-assessment system. The stages of test (questionnaire item) content, response processes, and internal structure under Messick’s unified concept of validity suggested that the CTAQ had sound psychometric properties. Exploratory factor analysis (EFA; n = 485) and confirmatory factor analysis (CFA; n = 484) yielded a three-factor model (ease of use, involvement and experience) with a reduction of 21 to 13 items for CDT acceptance. Practical applications and limitations of the CTAQ are discussed.
The increasing focus to go beyond assessing cognitive constructs around the globe and in Singapor... more The increasing focus to go beyond assessing cognitive constructs around the globe and in Singapore (Ng in https://www.gov.sg/microsites/budget2017/press-room/news/content/speech-by-minister-ng-chee-meng-for-moe-schools-at-the-committee-of-supply-debate, 2017) such as cultural awareness, grit and moral reasoning and the call for schools to prepare students for citizenship the development of corresponding instruments applicable in a larger scale. Given the significance of moral reasoning and its assessment are key elements within the current Singapore Character and Citizenship Education (CCE) curriculum (Singapore Ministry of Education in https://www.moe.gov.sg/docs/default-source/document/education/syllabuses/character-citizenship-education/files/2014-character-and-citizenship-education-(secondary)-syllabus.pdf, 2012), there is room to explore how moral reasoning can be assessed practically in a large-scale setting. Undergirding this exploration requires a review of the epistemology of moral reasoning, corresponding theoretical frameworks, and how they might apply to moral education in Singapore schools. By reviewing perspectives of moral reasoning from relevant literature beyond educational research to business, medical and philosophical domains, this paper aims to propose an operational definition of moral reasoning suitable for use in Singapore schools, and to mitigate possible issues brought about by educators’ disparate or assumed understanding of moral reasoning. This can consequently support the enactment of the CCE curriculum and be applied to the development of assessment instruments suitable for use within Singapore schools and similar contexts.
Purpose – The purpose of this paper is to evaluate an English Proficiency (EP) programme and its ... more Purpose – The purpose of this paper is to evaluate an English Proficiency (EP) programme and its efficacy with respect to students’ academic performance in a university within a multi-lingual context, as the programme had been in effect for some years.
Design/methodology/approach – A quasi-experimental approach was used to study the efficacy of an EP programme in a university within a multilingual context. Data across two academic years were used, along with regression discontinuity design.
Findings – Results suggest that the EP programme had a significant and positive intervention effect on students’ initial semester grade point average. The programme effect size was found to be medium to large.
Research limitations/implications – It might be useful to extend the study for one more year for more concrete conclusions. As the study was anchored upon the structure of the 2016 EP programme, any major curricular/structural change to the programme warrants another study.
Practical implications – This study demonstrated that the implementation of EP programmes in higher education institutions is essential not only for international students who are foreign language speakers of English but also for domestic students in English-speaking countries, especially for bi/multilingual speakers.
Originality/value – Previous studies related to the efficacy of EP within higher education have focused on international students who speak English as an additional/foreign language. Further, most studies have focussed on students’ self-reported experiences and have yielded disparate findings. This study contributes to scholarship as it addresses the under-researched area related to domestic students who speak English as the first language in a bi/multi-lingual context.
Uploads
effect on students’ grade point average of the first semester upon matriculation, this study sought to investigate the perceptions of students from the same university, as this would contribute to how EAP
programmes could be refined to better support learning. Semi-structured interviews were conducted with 50 students invited based on a maximum variation strategy. Based on the thematic analyses undertaken, four themes (i.e., programme delivery, linguistic improvement, learning transfer and change in self-efficacy) were identified and discussed. These themes contributed to the formulation of SILVER, an innovative framework of components for consideration in EAP course design and delivery within higher education.
Education Character and Citizenship Education curriculum, the Moral Reasoning Questionnaire (MRQ) was developed
and underwent preliminary validation. Based upon expert reviews, cognitive interviews and a classical test theorybased factor analytic approach, the development and preliminary validation found evidence (i.e., content appropriateness,
response processes and internal structure) to support the validity and reliability of the MRQ. This study aims to extend
the validation by examining the purported MRQ items and scale at a deeper level on the Rasch Measurement Theory,
given that it is the only model that presents appropriate properties of interval measurement on a log-linear scale. The
Rasch analysis found anomalies including differential item functioning and disordered thresholds in the initial set of
items. Upon remediation and a second Rasch analysis, the MRQ responses were consistent with that expressed by
the Rasch model (i.e., an item with an endorsability higher than what a respondent would tend to endorse would
have a lower probability of being endorsed than an item exhibiting an endorsability below what that respondent
would tend to endorse) and hence, there was sufficient evidence to support measurement invariance, and that MRQ
scores could be concluded to characterise persons invariantly across a continuum.
Design/methodology/approach – A quasi-experimental approach was used to study the efficacy of an EP programme in a university within a multilingual context. Data across two academic years were used, along with regression discontinuity design.
Findings – Results suggest that the EP programme had a significant and positive intervention effect on students’ initial semester grade point average. The programme effect size was found to be medium to large.
Research limitations/implications – It might be useful to extend the study for one more year for more concrete conclusions. As the study was anchored upon the structure of the 2016 EP programme, any major curricular/structural change to the programme warrants another study.
Practical implications – This study demonstrated that the implementation of EP programmes in higher education institutions is essential not only for international students who are foreign language speakers of English but also for domestic students in English-speaking countries, especially for bi/multilingual speakers.
Originality/value – Previous studies related to the efficacy of EP within higher education have focused on international students who speak English as an additional/foreign language. Further, most studies have focussed on students’ self-reported experiences and have yielded disparate findings. This study contributes to scholarship as it addresses the under-researched area related to domestic students who speak English as the first language in a bi/multi-lingual context.
effect on students’ grade point average of the first semester upon matriculation, this study sought to investigate the perceptions of students from the same university, as this would contribute to how EAP
programmes could be refined to better support learning. Semi-structured interviews were conducted with 50 students invited based on a maximum variation strategy. Based on the thematic analyses undertaken, four themes (i.e., programme delivery, linguistic improvement, learning transfer and change in self-efficacy) were identified and discussed. These themes contributed to the formulation of SILVER, an innovative framework of components for consideration in EAP course design and delivery within higher education.
Education Character and Citizenship Education curriculum, the Moral Reasoning Questionnaire (MRQ) was developed
and underwent preliminary validation. Based upon expert reviews, cognitive interviews and a classical test theorybased factor analytic approach, the development and preliminary validation found evidence (i.e., content appropriateness,
response processes and internal structure) to support the validity and reliability of the MRQ. This study aims to extend
the validation by examining the purported MRQ items and scale at a deeper level on the Rasch Measurement Theory,
given that it is the only model that presents appropriate properties of interval measurement on a log-linear scale. The
Rasch analysis found anomalies including differential item functioning and disordered thresholds in the initial set of
items. Upon remediation and a second Rasch analysis, the MRQ responses were consistent with that expressed by
the Rasch model (i.e., an item with an endorsability higher than what a respondent would tend to endorse would
have a lower probability of being endorsed than an item exhibiting an endorsability below what that respondent
would tend to endorse) and hence, there was sufficient evidence to support measurement invariance, and that MRQ
scores could be concluded to characterise persons invariantly across a continuum.
Design/methodology/approach – A quasi-experimental approach was used to study the efficacy of an EP programme in a university within a multilingual context. Data across two academic years were used, along with regression discontinuity design.
Findings – Results suggest that the EP programme had a significant and positive intervention effect on students’ initial semester grade point average. The programme effect size was found to be medium to large.
Research limitations/implications – It might be useful to extend the study for one more year for more concrete conclusions. As the study was anchored upon the structure of the 2016 EP programme, any major curricular/structural change to the programme warrants another study.
Practical implications – This study demonstrated that the implementation of EP programmes in higher education institutions is essential not only for international students who are foreign language speakers of English but also for domestic students in English-speaking countries, especially for bi/multilingual speakers.
Originality/value – Previous studies related to the efficacy of EP within higher education have focused on international students who speak English as an additional/foreign language. Further, most studies have focussed on students’ self-reported experiences and have yielded disparate findings. This study contributes to scholarship as it addresses the under-researched area related to domestic students who speak English as the first language in a bi/multi-lingual context.