Essays on a study of statistical power in economics. (2023)
Type of ContentTheses / Dissertations
Degree NameDoctor of Philosophy
Knowing the statistical power of an empirical analysis after it is completed can be very useful. Among other things, it can help one determine whether a finding of statistical insignificance is due to a small effect size or insufficient statistical power. This thesis consists of five studies linked together by my attempts to study how best to calculate ex post statistical power. Chapter One provides an introduction and the background for this thesis. In Chapter Two, I detail what is meant by statistical power (including ex ante and ex post power) and why it is important to researchers. I also identify the various factors that affect statistical power.
Chapter Three explains why ex post power has a “bad reputation.” A common practice for calculating ex post power employs an inappropriate method known as “observed power.” “Observed power” uses the estimated effect size as the assumed true effect size and then calculates the associated power. Though widely used, this method has been demonstrated to produce biased estimates of statistical power (Yuan & Maxwell, 2005). I present two approaches for calculating ex post power suggested by researchers to avoid the problems of “observed power”.
Chapter Four begins by replicating a recent paper by Brown, Lambert and Wojan (2019). BLW use a bootstrapping procedure to calculate ex post power and apply it to a benefit-cost analysis of a U.S. conservation program. I reproduce BLW’s results. I call their method for calculating ex post power BLW1. I then propose a variant of their method, which I call BLW2. I use Monte Carlo experiments to compare both methods in a simple data environment where there is no clustering.
In Chapter Five, I detail two more methods for calculating ex post power. The first procedure is taken from a blog post by David McKenzie and Owen Ozier (2019). I call this approach the SE-ES Method (for Standard Error – Effect Size). I then propose yet another variant of the BLW method, BLW3, which uses a wild-cluster bootstrap for handling clustered data. Chapter Five subjects all four methods to an extensive set of Monte Carlo experiments to assess their reliability in calculating ex post statistical power. I find that the SE-ES method is superior to BLW’s method (BLW1, BLW2 and BLW3) and has good overall performance.
Chapter Six applies the SE-ES method to a set of 23 development studies that were funded by the International Initiative for Impact Evaluation (3ie), a non-profit organization that supports research on ways to help the poor in low- and middle-income countries. I analyze the ex post power of these studies and explore factors that may be responsible for differences between ex ante and ex post statistical power. Chapter Seven concludes this thesis. It provides an overview of my chapters, as well as a summary of my main findings.
RightsAll Rights Reserved
Showing items related by title, author, creator and subject.
Research performance and age explain less than half of the gender pay gap in New Zealand universities Brower, Ann; James, Alex (Public Library of Science (PLoS), 2020)We use a globally unique dataset that scores every individual academic’s holistic research performance in New Zealand to test several common explanations for the gender pay gap in universities. We find a man’s odds of being ...
Neoliberal reform, contestation and relations of power in mining: Observations from Guinea and Mongolia Campbell, Bonnie; Hatcher, Pascale (Elsevier BV, 2019)© 2019 Elsevier Ltd Drawing on contributions from heterodox international political economy, this paper uses the notions of structural power and modes of governance to analyze the conditions that help explain emerging ...
Identity and Integration as Conflicting Forces Stimulating the Sunflower Movement and the Kuomintang's Loss in the 2014 Elections Clark C; Tan, Alex (2016)Over the past twenty years, there have been two important trends in Taiwan’s political economy whose contradictory implications provide an important explanation for the dramatic events of 201 4. The logic of each pulls ...