Evaluating the development of chemistry undergraduate researchers’ scientific thinking skills using performance-data: first findings from the performance assessment of undergraduate research (PURE) instrument

2017 ◽  
Vol 18 (3) ◽  
pp. 472-485 ◽  
Author(s):  
Joseph Harsh ◽  
John J. Esteb ◽  
Adam V. Maltese
Keyword(s):  
Undergraduate Research ◽  
Thinking Skills ◽  
Performance Data ◽  
Self Report ◽  
Scientific Thinking ◽  
Research Experiences ◽  
Chemistry Students ◽  
Primary Literature ◽  
Report Data ◽  
Self Report Data

National calls in science, technology, engineering, and technology education reform efforts have advanced the wide-scale engagement of students in undergraduate research for the preparation of a workforce and citizenry able to attend to the challenges of the 21st century. Awareness of the potential benefits and costs of these experiences has led to an emerging literature base outlining gains in participants’ cognitive, affective, and conative domains to support the impact of undergraduate research for students of all backgrounds; however, the majority of this work has relied on self-report data limiting inferences to the causal effects on student learning. As part of a larger project on apprentice-like undergraduate research experiences (UREs) in the physical sciences, the present exploratory study complemented indirect self-report data with direct performance data to assess the development of chemistry students’ scientific thinking skills over a research experience. Performance data were collected using the Performance assessment of Undergraduate Research Experiences (PURE) instrument, a validated tool designed to assess changes in chemistry students’ analytical and data driven decision-making skills through open-response tasks situated in real-world problems from primary literature. Twenty-four summer research students in chemistry (46% women; 50% 1st/2nd year students; 42% first time URE participant) from seven colleges and universities provided baseline and post-intervention performance data. Differences in pre/post-response task correctness provided a direct measure of individual changes in student competencies. Early study findings indicate the positive contributions of UREs to student's competencies in the areas of problem-solving, experimental design and the use of research techniques, data analysis and the interpretation of results, and the evaluation of primary literature. Survey data were also collected on students’ self-skill ratings to allow comparisons between perceived and demonstrated competencies, which were found to be weakly correlated. This work begins to offer direct evidence to the effect of UREs on student learning progressions as well as the potential use of performance test data in evaluating the success of research training interventions designed to improve scientific thinking skills.

Designing performance-based measures to assess the scientific thinking skills of chemistry undergraduate researchers

2016 ◽  
Vol 17 (4) ◽  
pp. 808-817 ◽  
Author(s):  
Joseph A. Harsh
Keyword(s):  
Research Training ◽  
Undergraduate Research ◽  
Thinking Skills ◽  
Self Report ◽  
Face Validity ◽  
Scientific Thinking ◽  
Research Experience ◽  
Good Reliability ◽  
Research Experiences ◽  
Chemistry Students

Undergraduate research (UR) is a vetted educational tool that is commonly perceived to prepare students for entering graduate school and careers in STEM fields; however, scholarly attention to date has largely relied on self-report data, which may limit inferences about the causal effects on student outcomes. In light of this, recent calls have been made for innovative and rigorous assessment strategies to better understand the efficacy and impact of UR on key disciplinary skills, both in classroom and internship UR models, that can help inform decisions about educational refinement. To more accurately measure the effect of UR on students, well-designed performance-based assessments can be used to provide direct evidence to the development of targeted skills during their research experience. Given the limited availability of tested, adaptable (and freely available) performance measures for assessing undergraduate chemistry students' scientific thinking skills, this article outlines a five-step process drawn from the literature about how reliable tasks and rubrics can be developed by faculty interested in assessing the effect of research training in the lab and classroom. For this purpose, as an applied example, the manuscript describes the development, testing, and validation of the Performance assessment of Undergraduate Research Experiences (PURE) instrument, which was designed to directly characterize the effects of research experiences on chemistry students' analytical and data-driven decision-making through open-response tasks situated in real-world scientific problems. Initial results reveal that the PURE instrument has high face validity and good reliability in measuring the scientific thinking skills of chemistry student researchers, and documents differences in UR students' answer quality over time supporting the effect of UR on research skill growth and the viability of performance data to assess these changes.

Correlating Personality and Actual Phone Usage

2014 ◽  
Vol 35 (3) ◽  
pp. 158-165 ◽  
Author(s):  
Christian Montag ◽  
Konrad Błaszkiewicz ◽  
Bernd Lachmann ◽  
Ionut Andone ◽  
Rayna Sariyska ◽  
...  
Keyword(s):  
Big Data ◽  
Social Network ◽  
Mobile Phone ◽  
Mobile Phones ◽  
Self Report ◽  
Psychological Variables ◽  
New Approach ◽  
Report Data ◽  
Self Report Data ◽  
Voice Calls

In the present study we link self-report-data on personality to behavior recorded on the mobile phone. This new approach from Psychoinformatics collects data from humans in everyday life. It demonstrates the fruitful collaboration between psychology and computer science, combining Big Data with psychological variables. Given the large number of variables, which can be tracked on a smartphone, the present study focuses on the traditional features of mobile phones – namely incoming and outgoing calls and SMS. We observed N = 49 participants with respect to the telephone/SMS usage via our custom developed mobile phone app for 5 weeks. Extraversion was positively associated with nearly all related telephone call variables. In particular, Extraverts directly reach out to their social network via voice calls.

Articles Published in EJPA 2009–2010

2011 ◽  
Vol 27 (4) ◽  
pp. 290-298 ◽  
Author(s):  
Tuulia M. Ortner ◽  
Isabella Vormittag
Keyword(s):  
Psychological Assessment ◽  
Rating Scales ◽  
Self Report ◽  
Related Field ◽  
Broad Scope ◽  
Future Challenges ◽  
Health Related ◽  
Report Data ◽  
Self Report Data

With reference to EJPA’s unique and broad scope, the current study analyzed the characteristics of the authors as well as the topics and research aims of the 69 empirical articles published in the years 2009–2010. Results revealed that more than one third of the articles were written by authors affiliated with more than one country. With reference to their research aims, an almost comparable number of articles (1) presented a new measure, (2) dealt with adaptations of measures, or (3) dealt with further research on existing measures. Analyses also revealed that most articles did not address any particular field of application. The second largest group was comprised of articles related to the clinical field, followed by the health-related field of application. The majority of all articles put their focus on investigating questionnaires or rating scales, and only a small number of articles investigated procedures classified as tests or properties of interviews. As to further characteristics of the method(s) used, a majority of EJPA contributions addressed self-report data. Results are discussed with reference to publication demands as well as the current and future challenges and demands of psychological assessment.

Does Dispositional Mindfulness Predict the Development of Grit?

2018 ◽  
Vol 39 (2) ◽  
pp. 76-87 ◽  
Author(s):  
Buaphrao Raphiphatthana ◽  
Paul Jose ◽  
Karen Salmon
Keyword(s):  
Undergraduate Students ◽  
Online Survey ◽  
Cross Sectional Study ◽  
Self Report ◽  
Dispositional Mindfulness ◽  
Cross Sectional ◽  
Report Data ◽  
Self Report Data ◽  
Acting With Awareness ◽  
Mindfulness Facets

Abstract. Grit, that is, perseverance and passion for long-term goals, is a novel construct that has gained attention in recent years ( Duckworth, Peterson, Matthews, & Kelly, 2007 ). To date, little research has been performed with the goal of identifying the antecedents of grit. Thus, in order to fill this gap in the literature, self-report data were collected to examine whether mindfulness, a mindset of being-in-the-present in a nonjudgmental way, plays a role in fostering grittiness. Three hundred and forty-three undergraduate students completed an online survey once in a cross-sectional study, and of these, 74 students completed the survey again 4.5 months later. Although the cross-sectional analyses identified a number of positive associations between mindfulness and grit, the longitudinal analysis revealed that the mindfulness facets of acting with awareness and non-judging were the most important positive predictors of grit 4.5 months later. This set of findings offers implications for future grit interventions.

The Effects of Questionnaire Length and Behavioral Consequences on Careless Responding

2020 ◽  
Vol 36 (2) ◽  
pp. 410-420 ◽  
Author(s):  
Anthony M. Gibson ◽  
Nathan A. Bowling
Keyword(s):  
Self Report ◽  
Randomized Experiments ◽  
High Quality ◽  
Buffering Effect ◽  
Careless Responding ◽  
Main Effect ◽  
Limited Support ◽  
Report Data ◽  
Self Report Data ◽  
Consistent Support

Abstract. The current paper reports the results of two randomized experiments designed to test the effects of questionnaire length on careless responding (CR). Both experiments also examined whether the presence of a behavioral consequence (i.e., a reward or a punishment) designed to encourage careful responding buffers the effects of questionnaire length on CR. Collectively, our two studies found (a) some support for the main effect of questionnaire length, (b) consistent support for the main effect of the consequence manipulations, and (c) very limited support for the buffering effect of the consequence manipulations. Because the advancement of many subfields of psychology rests on the availability of high-quality self-report data, further research should examine the causes and prevention of CR.

Why Ecological Momentary Assessment Surveys Go Incomplete?: When It Happens and How It Impacts Data

2020 ◽  
Author(s):  
Yu-Hsiang Wu ◽  
Jingjing Xu ◽  
Elizabeth Stangl ◽  
Shareka Pentony ◽  
Dhruv Vyas ◽  
...  
Keyword(s):  
Real Time ◽  
Self Report ◽  
Data Logging ◽  
Time Data ◽  
Real Time Data ◽  
Report Data ◽  
Self Report Data ◽  
Ecological Momentary ◽  
The Impact ◽  
Momentary Assessment

Abstract Background Ecological momentary assessment (EMA) often requires respondents to complete surveys in the moment to report real-time experiences. Because EMA may seem disruptive or intrusive, respondents may not complete surveys as directed in certain circumstances. Purpose This article aims to determine the effect of environmental characteristics on the likelihood of instances where respondents do not complete EMA surveys (referred to as survey incompletion), and to estimate the impact of survey incompletion on EMA self-report data. Research Design An observational study. Study Sample Ten adults hearing aid (HA) users. Data Collection and Analysis Experienced, bilateral HA users were recruited and fit with study HAs. The study HAs were equipped with real-time data loggers, an algorithm that logged the data generated by HAs (e.g., overall sound level, environment classification, and feature status including microphone mode and amount of gain reduction). The study HAs were also connected via Bluetooth to a smartphone app, which collected the real-time data logging data as well as presented the participants with EMA surveys about their listening environments and experiences. The participants were sent out to wear the HAs and complete surveys for 1 week. Real-time data logging was triggered when participants completed surveys and when participants ignored or snoozed surveys. Data logging data were used to estimate the effect of environmental characteristics on the likelihood of survey incompletion, and to predict participants' responses to survey questions in the instances of survey incompletion. Results Across the 10 participants, 715 surveys were completed and survey incompletion occurred 228 times. Mixed effects logistic regression models indicated that survey incompletion was more likely to happen in the environments that were less quiet and contained more speech, noise, and machine sounds, and in the environments wherein directional microphones and noise reduction algorithms were enabled. The results of survey response prediction further indicated that the participants could have reported more challenging environments and more listening difficulty in the instances of survey incompletion. However, the difference in the distribution of survey responses between the observed responses and the combined observed and predicted responses was small. Conclusion The present study indicates that EMA survey incompletion occurs systematically. Although survey incompletion could bias EMA self-report data, the impact is likely to be small.

scholarly journals Differential Self-control Effects: Moral Filtering and the Subsidiary Relevance of Self-control

2021 ◽  
Author(s):  
Helmut Hirtenlehner ◽  
Heinz Leitgöb
Keyword(s):  
Action Theory ◽  
Criminal Activity ◽  
Self Control ◽  
Self Report ◽  
Moral Norms ◽  
Situational Characteristics ◽  
Report Data ◽  
Self Report Data ◽  
The Individual ◽  
Low Self Control

AbstractCriminological research has identified low self-control as major cause of criminal activity. However, astonishingly little is known about the individual and situational characteristics that affect the functioning of self-control in relation to crime. Recent theorizing, especially in the context of Situational Action Theory, suggests that the interplay of personal and contextual morality creates a morally preselected choice set whose composition determines the relevance of self-control. Guided by the ideas of differential self-control effects and a moral filtering of action alternatives, the present inquiry investigates whether the role of self-control in crime causation depends on the power of moral factors to exclude crime from the set of the considered behavioral options. We argue that the significance of an individual’s capacity for self-control increases with a growing weakness of the moral filter, reaching its maximum when both personal and setting morality encourage criminal activity. Analyses of self-report data on adolescent vandalism delinquency provide support for differential self-control effects. The general picture is that self-control ability matters most when the strength of the moral filter hits a low, which is when both an individual’s own moral rules and the moral norms of the setting facilitate offending. Further evidence suggests that crime contemplation is highest when individual morality and setting morality jointly encourage vandalism. There is also indication that trait self-control has a greater effect on vandalism delinquency at higher levels of crime contemplation. All these results accord with the notion of a subsidiary relevance of control.

Sign in / Sign up

Export Citation Format

Share Document