Suchergebnisse

International large-scale assessments (LSAs), such as the Programme for International Student Assessment (PISA), provide essential information about the distribution of student proficiencies across a wide range of countries. The repeated assessments of the distributions of these cognitive domains offer policymakers important information for evaluating educational reforms and received considerable attention from the media. Furthermore, the analytical strategies employed in LSAs often define methodological standards for applied researchers in the field. Hence, it is vital to critically reflect on the conceptual foundations of analytical choices in LSA studies. This article discusses the methodological challenges in selecting and specifying the scaling model used to obtain proficiency estimates from the individual student responses in LSA studies. We distinguish design-based inference from model-based inference. It is argued that for the official reporting of LSA results, design-based inference should be preferred because it allows for a clear definition of the target of inference (e.g., country mean achievement) and is less sensitive to specific modeling assumptions. More specifically, we discuss five analytical choices in the specification of the scaling model: (1) specification of the functional form of item response functions, (2) the treatment of local dependencies and multidimensionality, (3) the consideration of test-taking behavior for estimating student ability, and the role of country differential items functioning (DIF) for (4) cross-country comparisons and (5) trend estimation. This article's primary goal is to stimulate discussion about recently implemented changes and suggested refinements of the scaling models in LSA studies.

Zusammenfassung: Der big-fish-little-pond effect beschreibt das Phänomen, dass vergleichbar leistungsstarke Schülerinnen und Schüler in leistungsstärkeren Lerngruppen infolge sozialer Vergleiche niedrigere Fähigkeitsselbstkonzepte ausbilden als in Gruppen mit einem niedrigeren mittleren Leistungsniveau. In der vorliegenden Studie wird geprüft, ob dieser Effekt auch im Hinblick auf das sportbezogene Selbstkonzept zu finden ist. In einer längsschnittlich angelegten Studie wurde eine repräsentative Stichprobe von 1185 Schülerinnen und Schülern in der Grundschule (3. Klasse) und der Sekundarstufe I (6. Klasse) mit einem motorischen Test und per Fragebögen untersucht. In Mehrebenenanalysen konnte ein negativer Effekt der durchschnittlichen Sportleistung einer Klasse auf das sportbezogene Selbstkonzept der Schüler nachgewiesen werden, der auch noch nach dem Übergang in die Sekundarstufe I andauerte. Die Konfrontation mit sportlichen Mitschülern hat demnach persistente negative Effekte auf das sportbezogene Selbstkonzept.

Early detection of risk of failure on interactive tasks comes with great potential for better understanding how examinees differ in their initial behavior as well as for adaptively tailoring interactive tasks to examinees' competence levels. Drawing on procedures originating in shopper intent prediction on e-commerce platforms, we introduce and showcase a machine learning-based procedure that leverages early-window clickstream data for systematically investigating early predictability of behavioral outcomes on interactive tasks. We derive features related to the occurrence, frequency, sequentiality, and timing of performed actions from early-window clickstreams and use extreme gradient boosting for classification. Multiple measures are suggested to evaluate the quality and utility of early predictions. The procedure is outlined by investigating early predictability of failure on two PIAAC 2012 Problem Solving in Technology Rich Environments (PSTRE) tasks. We investigated early windows of varying size in terms of time and in terms of actions. We achieved good prediction performance at stages where examinees had, on average, at least two thirds of their solution process ahead of them, and the vast majority of examinees who failed could potentially be detected to be at risk before completing the task. In-depth analyses revealed different features to be indicative of success and failure at different stages of the solution process, thereby highlighting the potential of the applied procedure for gaining a finer-grained understanding of the trajectories of behavioral patterns on interactive tasks.

Internationale Schulleistungsvergleiche erlauben es, repräsentative Aussagen über die gesamte Schülerschaft, aber auch über Teilgruppen von Schülerinnen und Schülern zu treffen. In diesem Beitrag werden vielseitig hochkompetente fünfzehnjährige Jugendliche untersucht, die in den Untersuchungen des Programme for International Student Assessment (PISA) 2000 und 2003 in allen drei Kompetenzbereichen (Mathematik, Lesen bzw. Textverstehen und Naturwissenschaften) zum besten Zehntel der Stichproben gehören. Diese Gruppe umfasst in Deutschland etwas über drei Prozent aller Fünfzehnjährigen an allgemeinbildenden Schulen. In einem ersten Schritt wird gezeigt, wie das Kompetenzniveau dieser Gruppe im internationalen Vergleich einzuordnen ist. Anschließend werden die Jugendlichen nach ihrem Geschlecht, der Schulart und der Klassenstufe, die sie besuchen sowie nach dem Bundesland und dem sozioökonomischen Milieu, in dem sie leben, charakterisiert. Weiterhin wird gefragt, inwieweit die erhaltenen Befunde übereinstimmen, wenn man zwei unterschiedliche Kohorten (Fünfzehnjährige in PISA 2000 und Fünfzehnjährige in PISA 2003) betrachtet. (DIPF/Orig.).;;;Results from international student assessments are representative both for the student population in the participating countries, and for sub-groups of students in the population. The present paper examines highly competent 15-year old students in Germany, who have shown excellent performance in all competency tests of the Programme for International Student Assessment (PISA), implemented in 2000 and 2003. These students, about 3 percent of the population of 15-year-old students, were scored among the most successful ten percent of students tested in Germany in the domains of reading, mathematical and scientific literacy. After exploring how the performance of this group compares internationally, high-performers are characterized according to their gender, type of school, grade, as well as the German federal state and socio-economic milieu in which they live. Investigating two different cohorts (15-year old students in PISA 2000 and 15-year old students in PISA 2003) serves to assess the stability of the findings. (DIPF/Orig.).

This study aims to investigate how test scores from PIAAC (Programme for the International Assessment of Adult Competencies) can be interpreted, by comparing the PIAAC competencies literacy and numeracy to reasoning and perceptual speed. Dimensionality analyses supported, that the PIAAC competencies can be separated into a common factor overlapping with reasoning and perceptual speed, and domain-specific factors. For the common and specific factors, relations to other variables were analyzed. The nested factor for PIAAC literacy was as expected unrelated to age, positively related to learning opportunities during one's lifetime, and positively related to literacy skill use. The nested factor for PIAAC numeracy was also as expected unrelated to age, against expectation unrelated to learning opportunities during one's lifetime, and as expected positively related to numeracy skill use. Results support the validity of the intended test score interpretation for PIAAC literacy, while results for PIAAC numeracy were less clear.