Multiple Imputation of Missing Data at Level 2: A Comparison of Fully Conditional and Joint Modeling in Multilevel Designs

2017 ◽  
Vol 43 (3) ◽  
pp. 316-353 ◽  
Author(s):  
Simon Grund ◽  
Oliver Lüdtke ◽  
Alexander Robitzsch
Keyword(s):  
Missing Data ◽  
Multiple Imputation ◽  
International Student ◽  
Student Assessment ◽  
Joint Modeling ◽  
Computational Procedure ◽  
International Student Assessment ◽  
Fully Conditional Specification ◽  
Using Data ◽  
Level 2

Multiple imputation (MI) can be used to address missing data at Level 2 in multilevel research. In this article, we compare joint modeling (JM) and the fully conditional specification (FCS) of MI as well as different strategies for including auxiliary variables at Level 1 using either their manifest or their latent cluster means. We show with theoretical arguments and computer simulations that (a) an FCS approach that uses latent cluster means is comparable to JM and (b) using manifest cluster means provides similar results except in relatively extreme cases with unbalanced data. We outline a computational procedure for including latent cluster means in an FCS approach using plausible values and provide an example using data from the Programme for International Student Assessment 2012 study.

scholarly journals Determinants of Declining School Belonging 2000–2018: The Case of Sweden

2021 ◽  
Author(s):  
Björn Högberg ◽  
Solveig Petersen ◽  
Mattias Strandh ◽  
Klara Johansson
Keyword(s):  
Education Reform ◽  
School Environment ◽  
International Student ◽  
Student Assessment ◽  
School Belonging ◽  
Foreign Born ◽  
Bottom Part ◽  
Performance Orientation ◽  
International Student Assessment ◽  
Using Data

AbstractStudents’ sense of belonging at school has declined across the world in recent decades, and more so in Sweden than in almost any other high-income country. However, we do not know the characteristics or causes of these worldwide trends. Using data on Swedish students aged 15–16 years from the Programme for International Student Assessment (PISA) between 2000 and 2018, we show that the decline in school belonging in Sweden was driven by a disproportionately large decline at the bottom part of the distribution, and was greatest for foreign-born students, students from disadvantaged social backgrounds, and for low-achieving students. The decline cannot be accounted for by changes in student demographics or observable characteristics related to the school environment. The decline did, however, coincide with a major education reform, characterized by an increased use of summative evaluation, and an overall stronger performance-orientation.

Fabrication of interview data

2018 ◽  
Vol 26 (2) ◽  
pp. 213-226 ◽  
Author(s):  
Jörg Blasius
Keyword(s):  
Survey Research ◽  
International Student ◽  
Student Assessment ◽  
Content Type ◽  
International Student Assessment ◽  
Pisa 2012 ◽  
Research Organizations ◽  
Using Data ◽  
Copy And Paste ◽  
Practical Implications

Purpose Evidence from past surveys suggests that some interviewees simplify their responses even in very well-organized and highly respected surveys. This paper aims to demonstrate that some interviewers, too, simplify their task by at least partly fabricating their data, and that, in some survey research institutes, employees simplify their task by fabricating entire interviews via copy and paste. Design/methodology/approach Using data from the principal questionnaires in the Programme for International Student Assessment (PISA) 2012 and the Programme for the International Assessment of Adult Competencies (PIAAC) data, the author applies statistical methods to search for fraudulent methods used by interviewers and employees at survey research organizations. Findings The author provides empirical evidence for potential fraud performed by interviewers and employees of survey research organizations in several countries that participated in PISA 2012 and PIAAC. Practical implications The proposed methods can be used as early as the initial phase of fieldwork to flag potentially problematic interviewer behavior such as copying responses. Originality/value The proposed methodology may help to improve data quality in survey research by detecting fabricated data.

scholarly journals Data Analysis: Strengthening Inferences in Quantitative Education Studies Conducted by Novice Researchers

2020 ◽  
Vol 20 (1) ◽  
pp. 59-78
Author(s):  
Mohammed A. A. Abulela ◽  
Michael Harwell
Keyword(s):  
Data Analysis ◽  
Missing Data ◽  
International Student ◽  
Research Question ◽  
Student Assessment ◽  
Reliability And Validity ◽  
Multilevel Modelling ◽  
International Student Assessment ◽  
Data Analyses

Data analysis is a significant methodological component when conducting quantitative education studies. Guidelines for conducting data analyses in quantitative education studies are common but often underemphasize four important methodological components impacting the validity of inferences: quality of constructed measures, proper handling of missing data, proper level of measurement of a dependent variable, and model checking. This paper highlights these components for novice researchers to help ensure statistical inferences are valid. We used empirical examples involving contingency tables, group comparisons, regression analysis, and multilevel modelling to illustrate these components using the Program for International Student Assessment (PISA) data. For every example, we stated a research question and provided evidence related to the quality of constructed measures since measures with weak reliability and validity evidence can bias estimates and distort inferences. The adequate strategies for handling missing data were also illustrated. The level of measurement for the dependent variable was assessed and the proper statistical technique was utilized accordingly. Model residuals were checked for normality and homogeneity of variance. Recommendations for obtaining stronger inferences and reporting related evidence were also illustrated. This work provides an important methodological resource for novice researchers conducting data analyses by promoting improved practice and stronger inferences.

Moving beyond Country Rankings in International Assessments: The Case of PISA

2015 ◽  
Vol 117 (1) ◽  
pp. 1-10
Author(s):  
Nancy Perry ◽  
Kadriye Ercikan
Keyword(s):  
Teaching And Learning ◽  
International Student ◽  
Research Policy ◽  
Student Assessment ◽  
Approaches To Learning ◽  
International Student Assessment ◽  
International Assessments ◽  
Motivation For Learning ◽  
Using Data ◽  
Pisa Data

The Programme for International Student Assessment (PISA) was designed by the Organisation for Economic Cooperation and Development (OECD) to evaluate the quality, equity, and efficiency of school systems around the world. Specifically, the PISA has assessed 15-year-old students’ reading, mathematics, and science literacy on a 3-year cycle, since 2000. Also, the PISA collects information about how those outcomes are related to key demographic, social, economic, and educational variables. However, the preponderance of reports involving PISA data focus on achievement variables and cross-national comparisons of achievement variables. Challenges in evaluating achievement of students from different cultural and educational settings and data concerning students’ approaches to learning, motivation for learning, and opportunities for learning are rarely reported. A main goal of this themed issue of Teachers College Record (TCR) is to move the conversation about PISA data beyond achievement to also include factors that affect achievement (e.g., SES, home environment, strategy use). Also we asked authors to consider how international assessment data can be used for improving learning and education and what appropriate versus inappropriate inferences can be made from the data. In this introduction, we synthesize the six articles in this issue and themes that cut across them. Also we examine challenges associated with using data from international assessments, like the PISA, to inform education policy and practice within and across countries. We conclude with recommendations for collecting and using data from international assessments to inform research, policy, and teaching and learning.

scholarly journals Cognitive Diagnosis Modeling Incorporating Item-Level Missing Data Mechanism

2020 ◽  
Vol 11 ◽  
Author(s):  
Na Shan ◽  
Xiaofei Wang
Keyword(s):  
Missing Data ◽  
International Student ◽  
Student Assessment ◽  
Cognitive Diagnosis ◽  
Parameter Recovery ◽  
International Student Assessment ◽  
Item Level ◽  
Item Responses ◽  
Computer Based ◽  
Cognitive Diagnosis Modeling

The aim of cognitive diagnosis is to classify respondents' mastery status of latent attributes from their responses on multiple items. Since respondents may answer some but not all items, item-level missing data often occur. Even if the primary interest is to provide diagnostic classification of respondents, misspecification of missing data mechanism may lead to biased conclusions. This paper proposes a joint cognitive diagnosis modeling of item responses and item-level missing data mechanism. A Bayesian Markov chain Monte Carlo (MCMC) method is developed for model parameter estimation. Our simulation studies examine the parameter recovery under different missing data mechanisms. The parameters could be recovered well with correct use of missing data mechanism for model fit, and missing that is not at random is less sensitive to incorrect use. The Program for International Student Assessment (PISA) 2015 computer-based mathematics data are applied to demonstrate the practical value of the proposed method.

scholarly journals Trends in International PISA Scores over Time: Which Countries Are Actually Improving?

2019 ◽  
Vol 8 (8) ◽  
pp. 231 ◽  
Author(s):  
Kristie J. Rowley ◽  
Shelby M. McNeill ◽  
Mikaela J. Dufur ◽  
Chrisse Edmunds ◽  
Jonathan A. Jarvis
Keyword(s):  
International Student ◽  
Student Assessment ◽  
Accurate Assessment ◽  
Policy Decisions ◽  
International Student Assessment ◽  
Country Level ◽  
Form Of Government ◽  
Using Data ◽  
Over Time

Many countries attempt to increase their Program for International Student Assessment (PISA) rankings and scores over time. However, despite providing a more accurate assessment of the achievement-based improvements across countries, few studies have systematically examined growth in PISA scores over multiple assessments. Using data from the 2006, the 2009, and the 2012 PISA, we analyzed which countries experienced significant increases in their country-level average PISA scores between 2006 and 2012. To facilitate improved policy decisions, we also examined what country-level conditions were associated with such increases. Contrary to expectations, we found that few countries significantly increased their PISA scores over time. Countries that did experience meaningful improvements in PISA scores were more likely to have had lower PISA scores in 2006 and experienced country-level foundational advancements more recently, such as advancing to a more democratic form of government and/or a higher income classification.

scholarly journals The PISA performance gap between national and expatriate students in the United Arab Emirates

2020 ◽  
Author(s):  
Jose Marquez ◽  
Louise Lambert ◽  
Natasha Ridge ◽  
Stuart Walker
Keyword(s):  
Public Schools ◽  
United Arab Emirates ◽  
Private Schools ◽  
International Student ◽  
Linear Regression Analysis ◽  
Student Assessment ◽  
School Type ◽  
International Student Assessment ◽  
Immigrant Background ◽  
Using Data

In most education systems, students with an immigrant background perform worse academically compared to native students. However, in the United Arab Emirates (UAE), differences emerge in the opposite direction and the national-expatriate gap in academic competence is equivalent to almost three years of schooling. This gap is a concern in the UAE, where national students mainly attend public schools and expatriates, mostly private schools. To investigate the competence gap between national and expatriate students, we estimate group differences and conduct linear regression analysis using data from the 2018 Programme for International Student Assessment. Results show that the gap varies by emirate and country of origin and is greater among boys, better-off students and in private schools. Between 33% and 47% of this gap is explained by school type, whether public or private. We offer recommendations; however, in a country characterized by 85% expatriates and a maturing education policy, challenges remain, but may serve to pave the way for other high expatriate nations in development.

scholarly journals Educational test scores among adolescents in three-generational households in 20 countries

2017 ◽  
Vol 51 ◽  
pp. 3-22 ◽  
Author(s):  
Antti Olavi Tanskanen ◽  
Mirkka Danielsbacka ◽  
Jani Erola
Keyword(s):  
Parental Involvement ◽  
Test Scores ◽  
International Student ◽  
Resource Competition ◽  
Child Outcomes ◽  
Student Assessment ◽  
International Student Assessment ◽  
Educational Test ◽  
Parental Time ◽  
Using Data

Grandparental presence is often found to associate with improved grandchild wellbeing. However, studies have shown that the effect is not always positive. This could be explained by the fact that in some circumstances grandparents compete with grandchildren over parental time resources. We studied the assumption using data from the Program for International Student Assessment (PISA) from 20 Western countries (n=73,346 children at age 15). According to the results grandparental presence was associated with lower levels of parental involvement and decreased educational test scores among adolescents. Moreover, the results indicate that when the parental involvement is lower at the first place the grandparental presence tends to be associated with even weaker child outcomes. Finally, we found support that grandparental co-residence is a mediator of the association between parental involvement and child outcomes. These results are discussed with reference to the local resource competition model.

Pre-Print

2018 ◽  
Author(s):  
Daniel A Briley
Keyword(s):  
International Student ◽  
Student Assessment ◽  
Economic Systems ◽  
Student Interest ◽  
Similar Data ◽  
Strongly Correlated ◽  
Science Interest ◽  
International Student Assessment ◽  
The Family ◽  
Using Data

We replicated Tucker-Drob, Cheung, and Briley (2014) who found that the association between science interest and science knowledge depended on economic resources at the family, school, and national levels using data from the 2006 Program for International Student Assessment (PISA). In more economically prosperous families, schools, and nations, student interest was more strongly correlated with actual knowledge. Over roughly a decade, these results may no longer hold due to substantial changes to educational or economic systems. Using similar data from 2015 PISA (N = 537,170), we found largely consistent results. Students from more economically advantaged homes, schools, and nations exhibited a stronger link between interests and knowledge. However, these moderation effects were substantially reduced, and the main effect of science interest increased by nearly 25%, driven almost entirely by lower SES families and lower GDP nations. The interdependency of interests and resources is robust, but perhaps weakening with educational progress.

scholarly journals Joint Modeling of Compensatory Multidimensional Item Responses and Response Times

2019 ◽  
Vol 43 (8) ◽  
pp. 639-654 ◽  
Author(s):  
Kaiwen Man ◽  
Jeffrey R. Harring ◽  
Hong Jiao ◽  
Peida Zhan
Keyword(s):  
International Student ◽  
Response Times ◽  
Student Assessment ◽  
Joint Modeling ◽  
Model Parameters ◽  
Monte Carlo Simulation Study ◽  
Parameter Recovery ◽  
International Student Assessment ◽  
Modeling Approach ◽  
Item Responses

Computer-based testing (CBT) is becoming increasingly popular in assessing test-takers’ latent abilities and making inferences regarding their cognitive processes. In addition to collecting item responses, an important benefit of using CBT is that response times (RTs) can also be recorded and used in subsequent analyses. To better understand the structural relations between multidimensional cognitive attributes and the working speed of test-takers, this research proposes a joint-modeling approach that integrates compensatory multidimensional latent traits and response speediness using item responses and RTs. The joint model is cast as a multilevel model in which the structural relation between working speed and accuracy are connected through their variance-covariance structures. The feasibility of this modeling approach is investigated via a Monte Carlo simulation study using a Bayesian estimation scheme. The results indicate that integrating RTs increased model parameter recovery and precision. In addition, Program of International Student Assessment (PISA) 2015 mathematics standard unit items are analyzed to further evaluate the feasibility of the approach to recover model parameters.

Sign in / Sign up

Export Citation Format

Share Document