Experiments
Louis Cohen, Lawrence Manion, Keith Morrison in Research Methods in Education, 2017
The design experiment can be considered as a special case of a field experiment; it has its roots in experimental research, both in ‘true’ and quasi-experiments, and is intended to provide formative feedback on, for example, practical problems in, say, teaching and learning, and to bridge the potential gap between research and practice (Brown, 1992, p. 143; Reinking and Bradley, 2008; Bradley and Reinking, 2011; Engeström, 2011; Seel, 2011, p. 925; Anderson and Shattuck, 2012; Laurillard, 2012), in other words, to enhance the external validity of an experiment. The design experiment strives to avoid the artificial world of the laboratory and the lack of applicability to ‘real-world problems’ that follows from this artificial condition (Bradley and Reinking, 2011; Reinking and Bradley, 2008; Seel, 2011; Laurillard, 2012), and to have direct practical relevance to the complex world of teaching, learning and classrooms. Given their intended direct relevance to classrooms and the field nature – the diverse, complex, ‘real world’ of an actual classroom – design experiments may not be able to fulfil the requirements of a true experiment, for example, in randomization or in the application of controls. In these respects, design experiments are similar to action research (cf. Anderson and Shattuck, 2012).
External validity and public health
Sridhar Venkatapuram, Alex Broadbent in The Routledge Handbook of Philosophy of Public Health, 2023
The other reason they give for doubting the trade-off is because the notion of the “artificiality” of experimental settings, used to justify its existence, is vague and ambiguous (Jimenez-Buedo and Miller 2010: 307). The difficulty in blaming the artificiality of experimental environments for external validity failure is that none of the proponents of this view explain in exactly which respects the environments are meant to differ. In any case, it is clear that there is a lot more to the external validity problem than just a concern about the artificiality of experimental settings. This can be seen by considering the case of so-called field experiments or observational studies, where causes and effects are observed in real-world settings. In these studies, there is nothing artificial about the experimental context; yet, the external validity question is still a legitimate concern. This is because we can have cases where a field experiment exposes a clear causal relation evident in one context, but this causal relation does not hold in a new context because of different confounding factors, for example.
Randomization Tests or Permutation Tests? A Historical and Terminological Clarification
Vance W. Berger in Randomization, Masking, and Allocation Concealment, 2017
Interestingly, another statistical heavyweight, Jerzy Neyman, did something very similar with respect to giving credit to Fisher for the randomization design principle as Edward Pitman did with respect to giving credit to Fisher for developing the test. In his notorious paper, read before the Industrial and Agricultural Research Section of the Royal Statistical Society, Neyman (1935, p. 109) stated:Owing to the work of R. A. Fisher, “Student” and their followers, it is hardly possible to add anything essential to the present knowledge concerning local experiments…. One of the most important achievements of the English School is their method of planning field experiments known as the method of Randomized Blocks and Latin Squares.
Does an economic incentive affect provider behavior? Evidence from a field experiment on different payment mechanisms
Published in Journal of Medical Economics, 2019
Xiaoyu Xi, Ennan Wang, Qianni Lu, Piaopiao Chen, Tian Wo, Kammy Tang
We used a field experiment study design to examine the behaviors of physicians. If a laboratory experiment were performed instead of a field experiment, the conclusions might not be valid due to hyper-abstraction and simplification26. However, the participants in a field study were not restricted to college students, instead, they were adults in society. Moreover, the experimental environment was not confined to a laboratory. A field experiment, as defined by Harrison and List27, was an experiment conducted in multiple locations, including laboratories and actual environments. Its participants included both students and non-college adults. Therefore, under the real social conditional, the experiment subjects could make realistic choices. Above all, because of the differences between the experimental environment and subjects, the field experiment could represent actual conditions in a real environment, and subjects might act instinctively as they do in daily life, increasing the external validity of results28.
Employees’ Improvisational Behavior: Exploring the Role of Leader Grit and Humility
Published in Human Performance, 2022
Arménio Rego, Andreia Vitória, Miguel Pina e Cunha, Bradley P. Owens, Ana Ventura, Susana Leal, Camilo Valverde, Rui Lourenço-Gil
While providing overall support for the proposed causal direction of our model, our research is not without limitations. First, other causalities are possible. For example, employees may develop higher self-efficacy, hope, and optimism after making improvisations that are revealed to be successful. It is also possible that leaders adopt more perseverant efforts in pursuing challenging goals as a consequence of their higher employees’ PsyCap. Although the experiment was designed to enhance realism, which enhances confidence in hypothesized causality, it also suffers from modest external validity and other limitations (Lonati et al., 2018). Future studies should include covariates to rule out confounding and endogeneity effects, should adopt other experimental designs, and should be carried out in real organizational settings (Antonakis, Bendahan, Jacquart, & Lalive, 2014). A field experiment would represent a very important step forward in that endeavor, although abundant obstacles (methodological and practical) may make the endeavor unfeasible. Second, future studies may explore boundary conditions of the PsyCap-improvisation relationship. For example, is the relationship more positive when employees experience psychological safety?
Impact of safety training and interventions on training-transfer: targeting migrant construction workers
Published in International Journal of Occupational Safety and Ergonomics, 2020
Rahat Hussain, Akeem Pedro, Do Yeop Lee, Hai Chien Pham, Chan Sik Park
Notwithstanding, even though the culturally diverse nature of the work crews is a valuable aspect to evaluate migrant worker safety performance, it is hard to engage labourers for such practice. Similarly, the extent of interventions being implemented during and after training sessions is difficult to control. To address these challenges, the experiment design phase of this research has focused more on the reasons for training failures from both the literature and current migrant worker issues in industry. The approach presented in this study measures the combined impact of all interventions in a field experiment; however, in order to improve the reliability and validity in measurement, the individual effects of interventions should also be considered in further studies.
Related Knowledge Centers
- External Validity
- Randomization
- Statistical Inference
- Experiment
- Random Assignment
- Rubin Causal Model
- Standard Deviation
- Accuracy & Precision
- Sample Size Determination
- Stepped-Wedge Trial