Explore chapters and articles related to this topic
Validity II: Correlational-based
Published in Claudio Violato, Assessing Competence in Medicine and Other Health Professions, 2018
In psychological and medical educational measurement, examples of constructs include intelligence, scholastic aptitude, communications, clinical competence, honesty, clinical reasoning, and creativity. These constructs have been proposed to explain, summarize, and organize empirical relationships and response consistencies. Clinical competence, like gravity, cannot be directly measured; its existence must be inferred from behavioral measurements. Similarly, anxiety is indicated by behavioral and physiological markers, while creativity is indexed by products (e.g., paintings, novels, inventions) though it is thought to be a process. In any case—whether gravity, clinical competence, honesty, or anxiety—construct validity requires the gradual accumulation of information from a variety of sources. In effect, it is a special instance of the general procedure of validating a theory in any scientific endeavor. The ultimate purpose of validation is explanation, understanding, and prediction.
Validity, Invariant Measurement, and Rater-Mediated Assessments
Published in George Engelhard, Stefanie A. Wind, Invariant Measurement with Raters and Rating Scales, 2017
George Engelhard, Stefanie A. Wind
Recent discussions of validity have highlighted the notion that validity is defined within communities of practice (Behizadeh & Engelhard, 2015), and that content-area researchers may have unique definitions of validity and procedures for validation that reflect values within their communities. In order to improve the validity of assessment systems, communication is needed between content-area and measurement communities of practice. Along the same lines, Sireci (2013) pointed out that communication and agreement about purpose of assessment systems across communities of practice is essential in order to inform the validation process. Speaking to the educational measurement community, he observed: Tests are developed to fulfill one or more intended purposes. It is incumbent upon us as psychometricians to help those who commission these tests to articulate the intended purposes. Once these purposes are articulated, we know what we need to validate. We also know what it is we need to measure!(p. 100) The next section describes the current consensus definition of validity within the measurement community, as defined in the Standards for Educational and Psychological Testing. The consensus definition and related standards for validity are then considered as they apply specifically to rater-mediated assessments.
Introduction
Published in Steven P. Reise, Dennis A. Revicki, Handbook of Item Response Theory Modeling, 2014
Steven P. Reise, Dennis A. Revicki
In our judgment, applications of IRT in educational measurement have tended toward the more broadband constructs, such as verbal and quantitative aptitude, or comprehensive licensure testing contexts (which also involve competencies across a heterogeneous skill domain). In contrast, we argue that with few exceptions, applications of IRT in noneducational measurement have primarily been with constructs that are relatively conceptually narrow. As a consequence, IRT applications in noneducational measurement contexts present some unique challenges, and the results of such applications can be markedly different from a typical IRT application in education.
The unit log–log distribution: a new unit distribution with alternative quantile regression modeling and educational measurements applications
Published in Journal of Applied Statistics, 2023
Mustafa Ç. Korkmaz, Zehra Sedef Korkmaz
In educational measurement, it should be considered that the measurement process is done for an educational purpose. Educational measurement is the tool in order to obtain information about the characteristics of students and the educational politics of the countries as well as it provides information about the development levels of the countries. Among these educational measurements, the results obtained from international exams, educational attainment percentage of countries, education and school living conditions, data about education and training can be given as examples. These measurements of countries are easily accessible by the OECD. OECD has examined the metrics of countries related to education on two themes. The first of these is the education and learning theme, and the other is the Social Protection and Well-being theme. Using the data sets from OECD countries, there are many papers in the literature. It can be seen [1,2,4,7,13,14,18,21,28,30,52] for some of them. These data sets, which make up the educational measurements, vary according to the development status of the countries. Therefore, some countries' measurements may appear as outliers in the sets. For such situations, using statistically robust methods would be more appropriate for inferences. For example, quantile regression, introduced by [34], models can explain the linear relationship between response variable and independent variable with more robust results than ordinary regression models. One may see [2,13,14,23,37,54] for applications of the educational measurements based on the regression modeling.
How sure can we be that a student really failed? On the measurement precision of individual pass-fail decisions from the perspective of Item Response Theory
Published in Medical Teacher, 2020
Stefan K. Schauber, Martin Hecht
Importantly, procedures used to estimate the precision of a pass-fail decision are usually referred to as methods for estimating a given test’s classification accuracy. This issue has been reported on extensively in the broader literature on educational measurement and psychometrics (Huynh 1990; Kane 1996; Lathrop and Cheng 2014; Lee 2010; Lewis and Sheehan 1990; Subkoviak 1976; Rudner 2005; Webb et al. 2006; Wyse and Hao 2012). These works, and especially the approach described by Rudner (2005), form the psychometric background for the following illustrations. As we aim for a conceptual illustration, more technical elaborations are beyond the scope of this paper.
Interventions for undergraduate and postgraduate medical learners with academic difficulties: A BEME systematic review: BEME Guide No. 56
Published in Medical Teacher, 2019
Miriam Lacasse, Marie-Claude Audétat, Élisabeth Boileau, Nathalie Caire Fon, Marie-Hélène Dufour, Marie-Claude Laferrière, Alexandre Lafleur, Ève La Rue, Shirley Lee, Mathieu Nendaz, Emmanuelle Paquette Raynard, Caroline Simard, Yvonne Steinert, Johanne Théorêt
Although Cleland searched in multiple databases, some relevant databases for medical education were not explored (e.g. Education Source and PsycINFO). Their literature review focused primarily on educational measurement and program evaluation. Furthermore, both literature reviews extracted their data following the Kirkpatrick hierarchy only (Kirkpatrick 1994), and did not address other aspects of program evaluation, such as context, input or process (Stufflebeam 2003).