Papers by Rolf Vegar Olsen
Educational Research Review, 2016
This study systematically reviews literature on assessment instruments of primary and secondary s... more This study systematically reviews literature on assessment instruments of primary and secondary school students’ ICT literacy. It has three objectives: (1) Describe the development and characteristics of the assessments; (2) Present a synthesis of the facets of ICT literacy measured; and (3) Investigate to what extent information about the reliability and validity is provided. A total of 38 tests reported in 66 studies were included. The results indicate that most of the tests target lower secondary students, comprise multiple-choice item designs, and are evaluated by quantitative methodology. The majority of the tests measure facets such as searching, retrieving, and evaluating digital information, and technical skills. In particular, the access to tests measuring digital communication, collaboration, safety, and problem solving is limited. This review demonstrates that an adequate norm for documenting and reporting test quality is lacking. Our findings point to potential future directions in developing and reporting assessments of ICT literacy. http://www.sciencedirect.com/science/article/pii/S1747938X16300252
IEA Research for Education, 2016
ABSTRACT Den 31. august 2011 finner vi følgende overskrift på NRK sin nettside: ”Mer enn hver tre... more ABSTRACT Den 31. august 2011 finner vi følgende overskrift på NRK sin nettside: ”Mer enn hver tredje kommune strøk på NRKs måling av skolekvalitet. Bare 19 av landets 430 kommuner fikk toppkarakterer. Dette er skole-Norge 2011” (Krekling og Heggen 2011). I artikkelen vises det til en måling av kommunenes skolekvalitet som er gjennomført av NRK og Kommunal rapport. Kommunene har fått tildelt poeng ut fra informasjon om blant annet skolenes resultater på nasjonale prøver, avgangskarakterer i 10. klasse, elever per pc og hvor mange lærere uten godkjent utdanning som brukes i skolene. En liknende måling ble gjennomført av Dagbladet i juni, som ved å kombinere ulike typer registerdata, rangerte kommunene. I begge målingene omformes ulike typer informasjon om skolene i en kommune til poeng som gjør at kommunen til slutt får en samleskåre, og til sist brukes dette til å rangere kommunen i forhold til de 429 andre kommunene i landet. Dette ga godt stoff til skoledebatten i forbindelse med lokalvalget 2011. Spørsmålet er imidlertid om disse målingene gir en riktig framstilling av skolekvaliteten i kommunene, og om slike globale og sammensatte kvalitetsmål i det hele tatt gir mening? Skoler er komplekse organisasjoner med et svært sammensatt samfunnsoppdrag. Likevel er det lett å formulere sentrale kvaliteter som en god skole skal kjennetegnes ved. Skolen skal for eksempel være et sted hvor elevene kjenner seg trygge, hvor de blir inspirert, hvor de over tid utvikler kunnskaper, ferdigheter og holdninger, hvor det er god økonomisk styring, hvor lovfestede rettigheter etterleves – og slik kunne vi fortsette. Men så snart man skal forsøke å gi presise definisjoner av disse viktige kvalitetene, blir det straks vanskeligere. Vi kommer imidlertid ikke utenom denne øvelsen. Uten en klar forståelse av for eksempel hva det betyr at elevene skal kjenne seg trygge på skolen, kan man ikke observere og dokumentere på en systematisk måte hvorvidt, eller i hvilken grad, skolen kjennetegnes ved å ha en slik kvalitet. Dette kapitlet er et forsøk på å gi en kortfattet innføring i hva kvalitetsindikatorer er og hva de kan brukes til. Det er ikke rom for å presentere og drøfte alle de ulike indikatorene som finnes, men vi vil ta med noen eksempler som kan bidra til å synliggjøre ulike krav til ideelle kvalitetsindikatorer. Vi legger vekt på at dette er idealer som man så langt som mulig bør etterstrebe, men som det i praksis likevel kan være vanskelig å realisere, delvis fordi flere av disse kravene står i et motsetningsforhold til hverandre. I denne forbindelse trekker vi veksler på forskningslitteratur om indikatorer, testteori og psykometri I den videre framstillingen vi vi først gi en kort presentasjon av dagens situasjon i norsk skole med tanke på utvikling og bruk av kvalitetsindikatorer. Deretter drøfter vi viktige kjennetegn ved et godt kvalitetsindikatorsystem.
Bakgrunn: Det er i bruk mange forskjellige spørreskjemaer ment for ulike pasientgrupper lokalt i ... more Bakgrunn: Det er i bruk mange forskjellige spørreskjemaer ment for ulike pasientgrupper lokalt i helsetjenesten. Spørreskjemaene har varierende kvalitet og er lite egnet for sammenlikning av resultater på tvers av tjenester og pasientgrupper. Kunnskapssenteret ble derfor forespurt av Helse Vest RHF om å identifisere 8-10 kjernespørsmål til bruk i lokale pasienterfaringsundersøkelser i regionen, samt vurdere behovet for tilleggsspørsmål for ulike pasientgrupper. Metode: For å kunne besvare problemstillingene ble det gjennomført en pilotundersøkelse ved Helse Bergen HF med hovedfokus på pasientenes vurdering av viktigheten av en rekke pasienterfaringsområder. Undersøkelsen omfattet svarergrupper med erfaring fra ni ulike fagområder og omsorgsnivå: •Rehabilitering, poliklinikk •Somatisk avdeling, voksne, døgnopphold •Somatisk avdeling, voksne, poliklinikk/dagavdeling •Psykisk helsevern, voksne, døgnopphold •Psykisk helsevern, voksne, døgnopphold •Psykisk helsevern, voksne, poliklinikk •Somatisk avdeling, barn, døgnopphold• Psykisk helsevern, barn, poliklinikk •Avhengighetsbehandling, døgnopphold (fortsetter på baksiden) Pasienterfaringer i spesialisthelsetjenenesten. Et generisk, kort spørreskjema. Notat fra Kunnskapssenteret september 2009 Tittel: Pasienterfaringer i spesialisthelsetjenesten. Et generisk, kort spørreskjema. English title: Patient experiences in specialist health care. A generic, short questionnaire. Institusjon Nasjonalt kunnskapssenter for helsetjenesten Ansvarlig John-Arne Røttingen, direktør Forfattere Sjetne, Ingeborg Strømseng, forsker (prosjektleder) Bjertnaes, Øyvind Andresen, forskningsleder Iversen, Hilde Hestad, seniorforsker Olsen, Rolf Vegar, seniorforsker ISBN 978-82-8121-297-8 Notat Brukererfaringsundersøkelser Antall sider 74 med vedlegg Oppdragsgiver Helse Vest RHF Nøkkelord Generisk, pasienterfaring, spørreskjema Sitering Sjetne IS, Bjertnaes ØA, Iversen HH, Olsen RV. Pasienterfaringer i spesialisthelsetjenesten. Et generisk, kort spørreskjema. Notat, september 2009. Oslo: Nasjonalt kunnskapssenter for helsetjenesten, 2009. Nasjonalt kunnskapssenter for helsetjenesten fremskaffer og formidler kunnskap om effekt av metoder, virkemidler og tiltak og om kvalitet innen alle deler av helsetjenesten. Målet er å bidra til gode beslutninger slik at brukerne får best mulig helsetjenester. Senteret er formelt et forvaltningsorgan under Helsedirektoratet, uten myndighetsfunksjoner. Kunnskapssenteret kan ikke instrueres i faglige spørsmål. Kunnskapssenteret takker Anita Lyssand og Odd Rune Hegrenes for bidrag i dette prosjektet. Kunnskapssenteret tar det fulle ansvaret for synspunktene som er uttrykt i rapporten. Nasjonalt kunnskapssenter for helsetjenesten Oslo, september 2009 2 Oppsummering av felles kjernespørsmål (fet skrift og uthevet med farge) og tilleggsspørsmål for de enkelte svarergruppene (markert med X og uthevet med farge) Spørsmålnummer Spørsmål -felles kjernespørsmål uthevet med fet skrift og farge Rehab. Poli. Soma. v. døgn Soma. v. poli Psyk. v. døgn Psyk. v. poli Soma. b. døgn BUP Avheng. døgn Avheng. poli 4 Snakket behandlerne til deg slik at du forsto dem? 5 Har du tillit til behandlernes faglige dyktighet? 6 I hvilken grad opplevde du at behandlerne hadde omsorg for deg? 7 Opplevde du at behandlerne var interessert i din beskrivelse av egen situasjon? 8 Fikk du nok tid til samtale og kontakt med behandlerne? 9 Snakket det øvrige personalet til deg slik at du forsto dem? X X X X X 20 Synes du institusjonen har samarbeidet godt med X andre offentlige tjenester (for eksempel fastlege, NAV eller hjemmesykepleie)? X X X 21 Måtte du vente for å få tilbud ved institusjonen? X X 23 Fikk du inntrykk av at institusjonen for øvrig var i god stand? 24 Var hjelpen og behandlingen du fikk på institusjonen, alt i alt, tilfredsstillende? 25 Hvilket utbytte har du hatt, alt i alt, av behandlingen på institusjonen? 26 Mener du at du på noen måte ble feilbehandlet (etter det du selv kan bedømme)? 27 Hvis du ser behandlingen under ett, hvordan vil du vurdere institusjonen? 3
The previous chapter dealt with the development of the relative achievement profiles for countrie... more The previous chapter dealt with the development of the relative achievement profiles for countries across all the items within the domain of mathematical literacy in PISA 2003. Analyses of the overall pattern in these profiles indicated that the Nordic countries' profiles were relatively strongly correlated. Furthermore, the analysis also revealed that other groups of countries with similar relative achievement profiles existed. In this chapter these clusters are further analysed by studying characteristics of the items. The findings reveal that the profiles of the Nordic and the English-speaking countries are mainly accounted for by variables describing what could be termed 'realistic mathematics'. This finding is discussed in relation to curricular approaches to mathematics competency and learning in the Nordic countries.
Den 31. august 2011 finner vi følgende overskrift på NRK sin nettside: ”Mer enn hver tredje kommu... more Den 31. august 2011 finner vi følgende overskrift på NRK sin nettside: ”Mer enn hver tredje kommune strøk på NRKs måling av skolekvalitet. Bare 19 av landets 430 kommuner fikk toppkarakterer. Dette er skole-Norge 2011” (Krekling og Heggen 2011). I artikkelen vises det til en måling av kommunenes skolekvalitet som er gjennomført av NRK og Kommunal rapport. Kommunene har fått tildelt poeng ut fra informasjon om blant annet skolenes resultater på nasjonale prøver, avgangskarakterer i 10. klasse, elever per pc og hvor mange lærere uten godkjent utdanning som brukes i skolene. En liknende måling ble gjennomført av Dagbladet i juni, som ved å kombinere ulike typer registerdata, rangerte kommunene. I begge målingene omformes ulike typer informasjon om skolene i en kommune til poeng som gjør at kommunen til slutt får en samleskåre, og til sist brukes dette til å rangere kommunen i forhold til de 429 andre kommunene i landet. Dette ga godt stoff til skoledebatten i forbindelse med lokalvalge...
This chapter presents the analysis of the so-called item-by-country interactions for the cognitiv... more This chapter presents the analysis of the so-called item-by-country interactions for the cognitive items in the domain mathematical literacy in PISA 2003. By using cluster analysis the aim is to establish whether it is reasonable to speak of a distinct Nordic profile of achievement in mathematics as this is operationalised in PISA. The analyses presented give evidence for such a Nordic profile in mathematics. However, the Nordic profile is tightly linked to the profile of the English-speaking countries. There are some implicit messages for educational policy that may be drawn from the analyses presented in this chapter. It is, for instance, important to note that the success of Finnish students in mathematical literacy in PISA is not entirely due to a distinctly different profile. To some extent Finnish students have the same relative strengths and weaknesses as their Nordic peers. This implies that overall, the Finnish students are stronger than their Nordic peers in all aspects of...
Correspondence to r.v.olsen@ils.uio.no ABSTRACT The cognitive items covering the domain of scient... more Correspondence to r.v.olsen@ils.uio.no ABSTRACT The cognitive items covering the domain of scientific literacy in the Programme for International Student Assessment (PISA) are explored through a cluster analysis of the item p-value residuals. Such residuals are often referred to as item-by-country interactions. The analysis clearly indicates distinct clusters of countries with similar profiles. The most stable country clusters have been labelled 'English speaking countries', 'East-Asian countries', 'German speaking countries' and 'South American countries'. A more detailed inspection is done of the profiles for the Nordic countries, and they are shown to be members of a larger group of countries which is labelled North-West European countries. Some detailed features of the profiles are described using item characteristics such as the categories used in the operational definition of scientific literacy given in the framework. In projects like TIMSS and...
This chapter argues that PISA is more than a driver for policy decisions in many countries. The s... more This chapter argues that PISA is more than a driver for policy decisions in many countries. The study also provides unique data with the potential to engage educational researchers across the world in conducting a range of secondary analyses. The first section of the chapter describes how the primary purpose of such studies in general has gradually evolved. This description reflects how the studies have typically related to educational research. This section of the chapter is used as the general background for the second and major section, which presents a rationale for why educational researchers could or should be motivated to engage in analytical work relating to these studies. This is followed up by a provisional framework for how educational researchers may approach and make use of the data from these studies in secondary analyses. This framework is based on six generic analytical approaches derived from the study of a large number of examples of published secondary analyses.
International Journal of Science Education
The study reported in this paper is a survey (n=236) that examines how upper secondary students (... more The study reported in this paper is a survey (n=236) that examines how upper secondary students (18-19 years old) in Norway come to terms with the wave-particle duality as presented as part of a short introduction to quantum physics. The main conclusion is that this concept is poorly understood. Some students demonstrate clear and explicitly formulated misconceptions rooted in a classical physics world-view. Scholars in physics have stated that the concept of duality is unnecessary, but still included in school and university physics. It is argued that school physics should give a more explicit focus to the challenge that quantum physics presents to the classical worldview, thus introducing the importance of an affective dimension in the learning and teaching of quantum physics. This study replicates evidence from other studies that wave-particle duality does not introduce such a challenge.
Revue Francaise de Pedagogie
The present article presents a systematic overview of primary and secondary research perspectives... more The present article presents a systematic overview of primary and secondary research perspectives related to the large-scale international studies like PISA (Programme for International Student Assessment) and TIMSS (Trends in International Mathematics and Science Study). Firstly, the primary purpose for these studies is discussed, and it is described how the main purpose have gradually been changed from educational research to a means for providing educational indicators for control and effective national policy development. This change has largely left the area open for rich possibilities for a variety of secondary analyses of the data, which has been released for public use. The main part of the article goes on to describe various perspectives for relevant secondary analyses. For each of these perspectives a number of secondary studies are briefly discussed, and further research is suggested. Our main aim is to demonstrate the rich potential for further research based on the inte...
is a research fellow at the Department of Teacher Education and School Development at the Univers... more is a research fellow at the Department of Teacher Education and School Development at the University of Oslo. He is in the process of completing his Phd thesis. In his work he is particularly concerned about how the data in large scale international comparative assessments, such as PISA and TIMSS, may be used in secondary analysis to add-ress research questions in science education. He has a background in physics and prior to his Phd he wrote a Masters thesis about students understanding of quantum physics. Parallel to this he worked as physics teacher in upper secon-dary school. Beyond his work at the university he is the leader of the Norwegian association for physics teachers. Abstract The cognitive items covering the domain of scientific literacy in the Programme for International Student Assessment (PISA) are explored through an analysis of the item residuals (item-by-country interactions) with the aim of looking for a distinct Nordic pattern. The findings of a cluster analysis...
Framework, aims and population differs in TIMSS (8, grade study) and PISA. We see this as an oppo... more Framework, aims and population differs in TIMSS (8, grade study) and PISA. We see this as an opportunity to get more knowledge ,and insight in the educational system in different countries than one study alone could offer. Several countries participated in both studies in 2003. By comparing the countries’ ranks in the two studies it is evident that a
International Journal of Educational Research, 2009
Accompanying the rise in the number of working parents is a growing demand for afterschool care s... more Accompanying the rise in the number of working parents is a growing demand for afterschool care schemes for children. After-school care schemes, in addition to school, provide pupils with more learning opportunities than the experiences that school provides. The hypothesis is that after-school care schemes offer a better knowledge-basis for learning science than the school and home environments only. This article investigates how afterschool care schemes affect learning in grades 5 and 8. While taking into account the socioeconomic and ethnic backgrounds of pupils, results indicate that after-school care is of significance. The influence of after-school care is roughly explained by the increased number of learning opportunities. We discuss the implications for research and the limitations of this survey. ß article concerns the significant influence that ASC may exercise on pupils' science learning within the school context. Particularly, we examine the potential significance of ASC on pupils' performance in science tests in grades 5 and 8 (at ages 10 and 13). Firstly, we explain the particular institutional characteristics of ASC in Norway. We then, explore the theoretical mechanisms on which the hypothesis rests. Thereafter, we present the principal results from our study, investigating how ASC attendance may influence pupils' learning of science (as expressed through assessment test results). In this study, we attempt to safeguard against non-random selection. The complexity of factors associated with the influence of ASC on science achievement must be emphasized. Finally, we discuss these results and the limitations inherent in this survey, and we unpack the implications for research and educational practice.
Scandinavian Journal of Educational Research, 2004
In the Programme for International Student Assessment (PISA) the items are organised in small clu... more In the Programme for International Student Assessment (PISA) the items are organised in small clusters relating to the same stimulus material (called 'units'). Homogeneity analysis (HA) is used to develop a detailed description of the relationship between all the items in one unit, using the categorical information available in the PISA data. The main findings of the analyses presented are the following: (a) non-respondents are separated from those responding; (b) student patterns across the nominal variables cannot be used to develop a detailed insight into how students' thinking and knowledge is organised beyond the fact that good students perform well on most items; and (c) the profiles for countries are described by the same dimensions, indicating that the relative success of the countries can to a certain degree be explained differentially by these dimensions. From these findings implications for future assessments are suggested.
International Journal of Science Education, 2011
ABSTRACT The Programme for International Student Assessment in 2006 included several measures of ... more ABSTRACT The Programme for International Student Assessment in 2006 included several measures of students’ interest in science. These measures were constructed by combining information from several items where students are asked to respond to statements along Likert scale categories. Since there is evidence for Likert scales providing culturally biased country scores, we demonstrate in this article that the relative profiles of interest can be meaningfully analysed across countries. Hence, we have developed national relative profiles of interest in science constructed from the country‐ and item‐specific residuals at the item level. Subsequently, these relative profiles of interest have been used as input in a cluster analysis providing identification of distinct groups of countries with similar item‐by‐item patterns of interest in science. The most notable feature of the analysis is an overall division between two larger groups of countries, roughly corresponding to European/Western countries in one group and non‐European countries, with only a few exceptions, in the other group. A number of meaningful clusters of countries, partly defined by language and partly by localisation, are identified within each of the two main clusters. In order to develop a more detailed understanding of the characteristic features of the various clusters, descriptive information about the items is included in the analysis. The most notable finding is the strong relative preference for life and health issues among the non‐European countries, contrasted with the distinct favouring of items relating to physical/technological systems in the European/Western countries.
Uploads
Papers by Rolf Vegar Olsen