The nonparametric Wilcoxon-Mann-Whitney test is commonly used by experimental economists for detecting differences in central tendency between two samples. This test is only theoretically appropriate under certain assumptions concerning the population distributions from which the samples are drawn, and is often used in cases where it is unclear whether these assumptions hold, and even when they clearly do not hold. Fligner and Pollicello's (1981, Journal of the American Statistical Association. 76, 162-168) robustrank-ordertestis a modification of the Wilcoxon-Mann-Whitney test, designed to be appropriate in more situations than Wilcoxon-Mann-Whitney. This paper uses simulations to compare the performance of the two tests under a variety of distributional assumptions. The results are mixed. The robust rank-order test tends to yield too many false positive results for mediumsized samples, but this liberalness is relatively invariant across distributional assumptions, and seems to be due to a deficiency of the normal approximation to its test statistic's distribution, rather than the test itself. The performance of the Wilcoxon-Mann-Whitney test varies hugely, depending on the distributional assumptions; in some cases, it is conservative, in others, extremely liberal. The tests have roughly similar power. Overall, the robust rank-order test performs better than Wilcoxon-Mann-Whitney, though when critical values for the robust rank-order test are not available, so that the normal approximation must be used, their relative performance depends on the underlying distributions, the sample sizes, and the level of significance used.

The classical trinity of tests is used to check for the presence of a tremble in economic experiments in which the response variable is binary. A tremble is said to occur when an agent makes a decision completely at random, without regard to the values taken by the explanatory variables. The properties of the tests are discussed, and an extension of the methodology is used to test for the presence of a tremble in binary panel data from a well-known economic experiment.

In a variety of measurement situations, the researcher may wish to compare the reliabilities of several instruments administered to the same sample of subjects. This paper presents eleven statistical procedures which test the equality of m coefficient alphas when the sample alpha coefficients are dependent. Several of the procedures are derived in detail, and numerical examples are given for two. Since all of the procedures depend on approximate asymptotic results, Monte Carlo methods are used to assess the accuracy of the procedures for sample sizes of 50, 100, and 200. Both control of Type I error and power are evaluated by computer simulation. Two of the procedures are unable to control Type I errors satisfactorily. The remaining nine procedures perform properly, but three are somewhat superior in power and Type I error control.

Diagnostic classification models (DCMs) have seen wide applications in educational and psychological measurement, especially in formative assessment. DCMs in the presence of testlets have been studied in recent literature. A key ingredient in the statistical modeling and analysis of testlet-based DCMs is the superposition of two latent structures, the attribute profile and the testlet effect. This paper extends the standard testlet DINA (T-DINA) model to accommodate the potential correlation between the two latent structures. Model identifiability is studied and a set of sufficient conditions are proposed. As a byproduct, the identifiability of the standard T-DINA is also established. The proposed model is applied to a dataset from the 2015 Programme for International Student Assessment. Comparisons are made with DINA and T-DINA, showing that there is substantial improvement in terms of the goodness of fit. Simulations are conducted to assess the performance of the new method under various settings.

In the social sciences we are often interested in comparing models specified by parametric equality or inequality constraints. For instance, when examining three group means $\{{\mathit{\mu }}_1,{\mathit{\mu }}_2,{\mathit{\mu }}_3\}$\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\{ \mu _1, \mu _2, \mu _3\}$$\end{document} through an analysis of variance (ANOVA), a model may specify that ${\mathit{\mu }}_1<{\mathit{\mu }}_2<{\mathit{\mu }}_3$\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\mu _1<\mu _2<\mu _3$$\end{document}, while another one may state that $\{{\mathit{\mu }}_1={\mathit{\mu }}_3\}<{\mathit{\mu }}_2$\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\{ \mu _1=\mu _3\} <\mu _2$$\end{document}, and finally a third model may instead suggest that all means are unrestricted. This is a challenging problem, because it involves a combination of nonnested models, as well as nested models having the same dimension. We adopt an objective Bayesian approach, requiring no prior specification from the user, and derive the posterior probability of each model under consideration. Our method is based on the intrinsic prior methodology, suitably modified to accommodate equality and inequality constraints. Focussing on normal ANOVA models, a comparative assessment is carried out through simulation studies. We also present an application to real data collected in a psychological experiment.

A paired composition is a response (upon a dependent variable) to the ordered pair <j, k> of stimuli, treatments, etc. The present paper develops an alternative analysis for the paired compositions layout previously treated by Bechtel's [1967] scaling model. The alternative model relaxes the previous one by including row and column scales that provide an expression of bias for each pair of objects. The parameter estimation and hypothesis testing procedures for this model are illustrated by means of a small group analysis, which represents a new approach to pairwise sociometrics and personality assessment.

A variety of distributional assumptions for dissimilarity judgments are considered, with the lognormal distribution being favored for most situations. An implicit equation is discussed for the maximum likelihood estimation of the configuration with or without individual weighting of dimensions. A technique for solving this equation is described and a number of examples offered to indicate its performance in practice. The estimation of a power transformation of dissimilarity is also considered. A number of likelihood ratio hypothesis tests are discussed and a small Monte Carlo experiment described to illustrate the behavior of the test of dimensionality in small samples.

The recent surge of interests in cognitive assessment has led to the development of cognitive diagnosis models. Central to many such models is a specification of the Q-matrix, which relates items to latent attributes that have natural interpretations. In practice, the Q-matrix is usually constructed subjectively by the test designers. This could lead to misspecification, which could result in lack of fit of the underlying statistical model. To test possible misspecification of the Q-matrix, traditional goodness of fit tests, such as the Chi-square test and the likelihood ratio test, may not be applied straightforwardly due to the large number of possible response patterns. To address this problem, this paper proposes a new statistical method to test the goodness fit of the Q-matrix, by constructing test statistics that measure the consistency between a provisional Q-matrix and the observed data for a general family of cognitive diagnosis models. Limiting distributions of the test statistics are derived under the null hypothesis that can be used for obtaining the test p-values. Simulation studies as well as a real data example are presented to demonstrate the usefulness of the proposed method.

A method for externally constraining certain distances in multidimensional scaling configurations is introduced and illustrated. The approach defines an objective function which is a linear composite of the loss function of the point configuration X relative to the proximity data P and the loss of X relative to a pseudo-data matrix R. The matrix R is set up such that the side constraints to be imposed on X’s distances are expressed by the relations among R’s numerical elements. One then uses a double-phase procedure with relative penalties on the loss components to generate a constrained solution X. Various possibilities for constructing actual MDS algorithms are conceivable: the major classes are defined by the specification of metric or nonmetric loss for data and/or constraints, and by the various possibilities for partitioning the matrices P and R. Further generalizations are introduced by substituting R by a set of R matrices, Ri, i = 1, …r, which opens the way for formulating overlapping constraints as, e.g., in patterns that are both row- and column-conditional at the same time.

The properties of nonmetric multidimensional scaling (NMDS) are explored by specifying statistical models, proving statistical consistency, and developing hypothesis testing procedures. Statistical models with errors in the dependent and independent variables are described for quantitative and qualitative data. For these models, statistical consistency often depends crucially upon how error enters the model and how data are collected and summarized (e.g., by means, medians, or rank statistics). A maximum likelihood estimator for NMDS is developed, and its relationship to the standard Shepard-Kruskal estimation method is described. This maximum likelihood framework is used to develop a method for testing the overall fit of the model.

Separation commonly occurs in political science, usually when a binary explanatory variable perfectly predicts a binary outcome. In these situations, methodologists often recommend penalized maximum likelihood or Bayesian estimation. But researchers might struggle to identify an appropriate penalty or prior distribution. Fortunately, I show that researchers can easily test hypotheses about the model coefficients with standard frequentist tools. While the popular Wald test produces misleading (even nonsensical) p-values under separation, I show that likelihood ratio tests and score tests behave in the usual manner. Therefore, researchers can produce meaningful p-values with standard frequentist tools under separation without the use of penalties or prior information.