Both papers in this volume on which I was asked to comment (James E. Monogan III, “A Case for Registering Studies of Political Outcomes: An Application in the 2010 House Elections” and Macartan Humphreys, Raul Sanchez de la Sierra, and Peter van der Windt, “Fishing, Commitment, and Communication: A Proposal for Comprehensive Nonbinding Research Registration”) advocate registration regimes for our discipline. The recommendations in both are incremental [promoting, as Lindblom (1965) might have said it, the “intelligence of research” and cognizant of the costs in scientific learning from such a regime if rigidly enforced. Moreover, both papers cite studies by Gerber and various co-authors (e.g., Gerber, Green, and Nickerson 2001) demonstrating publication bias in political science, incentivizing researchers to manipulate their regression models until they can show a z-statistic ≥ 1.96, and thereby reaching standard levels of significance. I fully accept that Gerber et al.'s papers have detected a serious flaw in our scientific practices; there is a problem to be solved. The Monogan and Humphreys et al. proposals are therefore worthy of consideration.

How a treatment causes a particular outcome is a focus of inquiry in political science. When treatment data are either nonrandomly assigned or missing, the analyst will often invoke ignorability assumptions: that is, both the treatment and missingness are assumed to be as if randomly assigned, perhaps conditional on a set of observed covariates. But what if these assumptions are wrong? What if the analyst does not know why—or even if—a particular subject received a treatment? Building on Manski, Molinari offers an approach for calculating nonparametric identification bounds for the average treatment effect of a binary treatment under general missingness or nonrandom assignment. To make these bounds substantively more informative, Molinari's technique permits adding monotonicity assumptions (e.g., assuming that treatment effects are weakly positive). Given the potential importance of these assumptions, we develop a new Bayesian method for performing sensitivity analysis regarding them. This sensitivity analysis allows analysts to interpret the assumptions' consequences quantitatively and visually. We apply this method to two problems in political science, highlighting the method's utility for applied research.

In this article, we respond to Shultziner's critique that argues that identical twins are more alike not because of genetic similarity, but because they select into more similar environments and respond to stimuli in comparable ways, and that these effects bias twin model estimates to such an extent that they are invalid. The essay further argues that the theory and methods that undergird twin models, as well as the empirical studies which rely upon them, are unaware of these potential biases. We correct this and other misunderstandings in the essay and find that gene-environment (GE) interplay is a well-articulated concept in behavior genetics and political science, operationalized as gene-environment correlation and gene-environment interaction. Both are incorporated into interpretations of the classical twin design (CTD) and estimated in numerous empirical studies through extensions of the CTD. We then conduct simulations to quantify the influence of GE interplay on estimates from the CTD. Due to the criticism's mischaracterization of the CTD and GE interplay, combined with the absence of any empirical evidence to counter what is presented in the extant literature and this article, we conclude that the critique does not enhance our understanding of the processes that drive political traits, genetic or otherwise.

Survey questions about quantities offer a number of advantages over more common qualitative questions. However, concerns about survey respondents' abilities to accurately report numbers have limited the use of quantitative questions. This article shows quantitative questions are feasible and useful for the study of economic voting. First, survey respondents are capable of accurately assessing familiar economic quantities, such as the price of gas. Second, careful question design—in particular providing respondents with benchmark quantities—can reduce measurement error due to respondents not understanding the scale on which more complex quantities, such as the unemployment rate, are measured. Third, combining quantitative and qualitative questions sheds light on where partisan bias enters economic assessments: in perceiving, judging, or reporting economic quantities.

An increasing number of analyses in various subfields of political science employ Boolean algebra as proposed by Ragin's qualitative comparative analysis (QCA). This type of analysis is perfectly justifiable if the goal is to test deterministic hypotheses under the assumption of error-free measures of the employed variables. My contention is, however, that only in a very few research areas are our theories sufficiently advanced to yield deterministic hypotheses. Also, given the nature of our objects of study, error-free measures are largely an illusion. Hence, it is unsurprising that many studies employ QCA inductively and gloss over possible measurement errors. In this article, I address these issues and demonstrate the consequences of these problems with simple empirical examples. In an analysis similar to Monte Carlo simulation, I show that using Boolean algebra in an exploratory fashion without considering possible measurement errors may lead to dramatically misleading inferences. I then suggest remedies that help researchers to circumvent some of these pitfalls.

Political scientists frequently use instrumental variables (IV) estimation to estimate the causal effect of an endogenous treatment variable. However, when the treatment effect is heterogeneous, this estimation strategy only recovers the local average treatment effect (LATE). The LATE is an average treatment effect (ATE) for a subset of the population: units that receive treatment if and only if they are induced by an exogenous IV. However, researchers may instead be interested in the ATE for the entire population of interest. In this article, we develop a simple reweighting method for estimating the ATE, shedding light on the identification challenge posed in moving from the LATE to the ATE. We apply our method to two published experiments in political science in which we demonstrate that the LATE has the potential to substantively differ from the ATE.

Field survey experiments often measure amorphous concepts in discretely ordered categories, with postsurvey analytics that fail to account for the discrete attributes of the data. This article demonstrates the use of discrete distribution tests, specifically the chi-square test and the discrete Kolmogorov—Smirnov (KS) test, as simple devices for comparing and analyzing ordered responses typically found in surveys. In Monte Carlo simulations, we find the discrete KS test to have more power than the chi-square test when distributions are right or left skewed, regardless of the sample size or the number of alternatives. The discrete KS test has at least as much power as the chi-square, and sometimes more so, when distributions are bi-modal or approximately uniform and samples are small. After deriving rules of usage for the two tests, we implement them in two cases typical of survey analysis. Using our own data collected after Hurricanes Katrina and Rita, we employ our rules to both validate and assess treatment effects in a natural experimental setting.

In typical political experiments, researchers randomize a set of households, precincts, or individuals to treatments all at once, and characteristics of all units are known at the time of randomization. However, in many other experiments, subjects “trickle in” to be randomized to treatment conditions, usually via complete randomization. To take advantage of the rich background data that researchers often have (but underutilize) in these experiments, we develop methods that use continuous covariates to assign treatments sequentially. We build on biased coin and minimization procedures for discrete covariates and demonstrate that our methods outperform complete randomization, producing better covariate balance in simulated data. We then describe how we selected and deployed a sequential blocking method in a clinical trial and demonstrate the advantages of our having done so. Further, we show how that method would have performed in two larger sequential political trials. Finally, we compare causal effect estimates from differences in means, augmented inverse propensity weighted estimators, and randomization test inversion.

Two common problems in applications of two-stage least squares (2SLS) are nonrandom measurement error in the endogenous variable and weak instruments. In the presence of nonrandom measurement error, 2SLS yields inconsistent estimates. In the presence of weak instruments, confidence intervals and p-values can be severely misleading. This article introduces a rank-based estimator, grounded in randomization inference, which addresses both problems within a unified framework. Monte Carlo studies illustrate the deficiencies of 2SLS and the virtues of the rank-based estimator in terms of bias and efficiency. A replication of a study of the effect of economic shocks on democratic transitions demonstrates the practical implications of accounting for nonrandom measurement error and weak instruments.

Unemployment insurance policies are multidimensional objects, with variable waiting periods, eligibility duration, benefit levels, and asset tests, making intertemporal or international comparisons very difficult. Furthermore, labor market conditions, such as the likelihood and duration of unemployment, matter when assessing the generosity of different policies. In this article, we develop a new methodology to measure the generosity of unemployment insurance programs with a single metric. We build a first model with all characteristics of the complex unemployment insurance policy. Our model features heterogeneous agents that are liquidity constrained but can self-insure. We then build a second model, similar in all aspects but one: the unemployment insurance policy is one-dimensional (no waiting periods, eligibility limits, or asset tests, but constant benefits). We then determine which level of benefits in this second model makes society indifferent between both policies. We apply this measurement strategy to the unemployment insurance program of the United Kingdom.

If an experimental treatment is experienced by both treated and control group units, tests of hypotheses about causal effects may be difficult to conceptualize, let alone execute. In this article, we show how counterfactual causal models may be written and tested when theories suggest spillover or other network-based interference among experimental units. We show that the “no interference” assumption need not constrain scholars who have interesting questions about interference. We offer researchers the ability to model theories about how treatment given to some units may come to influence outcomes for other units. We further show how to test hypotheses about these causal effects, and we provide tools to enable researchers to assess the operating characteristics of their tests given their own models, designs, test statistics, and data. The conceptual and methodological framework we develop here is particularly applicable to social networks, but may be usefully deployed whenever a researcher wonders about interference between units. Interference between units need not be an untestable assumption; instead, interference is an opportunity to ask meaningful questions about theoretically interesting phenomena.

In this article, we show how to apply Bayesian methods to noisy ratio scale distances for both the classical similarities problem as well as the unfolding problem. Bayesian methods produce essentially the same point estimates as the classical methods, but are superior in that they provide more accurate measures of uncertainty in the data. Identification is nontrivial for this class of problems because a configuration of points that reproduces the distances is identified only up to a choice of origin, angles of rotation, and sign flips on the dimensions. We prove that fixing the origin and rotation is sufficient to identify a configuration in the sense that the corresponding maxima/minima are inflection points with full-rank Hessians. However, an unavoidable result is multiple posterior distributions that are mirror images of one another. This poses a problem for Markov chain Monte Carlo (MCMC) methods. The approach we take is to find the optimal solution using standard optimizers. The configuration of points from the optimizers is then used to isolate a single Bayesian posterior that can then be easily analyzed with standard MCMC methods.