Assuming a nonparametric family of item response theory models, a theory-based procedure for testing the hypothesis of unidimensionality of the latent space is proposed. The asymptotic distribution of the test statistic is derived assuming unidimensionality, thereby establishing an asymptotically valid statistical test of the unidimensionality of the latent trait. Based upon a new notion of dimensionality, the test is shown to have asymptotic power 1. A 6300 trial Monte Carlo study using published item parameter estimates of widely used standardized tests indicates conservative adherence to the nominal level of significance and statistical power averaging 81 out of 100 rejections for examinee sample sizes and psychological test lengths often incurred in practice.

While negative local item dependence (LID) has been discussed in numerous articles, its occurrence and effects often go unrecognized. This is due in part to confusion over what unidimensional latent trait is being utilized in evaluating the LID of multidimensional testing data. This article addresses this confusion by using an appropriately chosen latent variable to condition on. It then provides a proof that negative LID must occur when unidimensional ability estimates (such as number right score) are obtained from data which follow a very general class of multidimensional item response theory models. The importance of specifying what unidimensional latent trait is used, and its effect on the sign of the LIDs are shown to have implications in regard to a variety of foundational theoretical arguments, to the simulation of LID data sets, and to the use of testlet scoring for removing LID.

Knowledge space theory (KST) structures are introduced within item response theory (IRT) as a possible way to model local dependence between items. The aim of this paper is threefold: firstly, to generalize the usual characterization of local independence without introducing new parameters; secondly, to merge the information provided by the IRT and KST perspectives; and thirdly, to contribute to the literature that bridges continuous and discrete theories of assessment. In detail, connections are established between the KST simple learning model (SLM) and the IRT General Graded Response Model, and between the KST Basic Local Independence Model and IRT models in general. As a consequence, local independence is generalized to account for the existence of prerequisite relations between the items, IRT models become a subset of KST models, IRT likelihood functions can be generalized to broader families, and the issues of local dependence and dimensionality are partially disentangled. Models are discussed for both dichotomous and polytomous items and conclusions are drawn on their interpretation. Considerations on possible consequences in terms of model identifiability and estimation procedures are also provided.

This paper focuses on model interpretation issues and employs a geometric approach to compare the potential value of using the Grade of Membership (GoM) model in representing population heterogeneity. We consider population heterogeneity manifolds generated by letting subject specific parameters vary over their natural range, while keeping other population parameters fixed, in the marginal space (based on marginal probabilities) and in the full parameter space (based on cell probabilities). The case of a 2 × 2 contingency table is discussed in detail, and a generalization to 2J tables with J ≥ 3 is sketched. Our approach highlights the main distinction between the GoM model and the probabilistic mixture of classes by demonstrating geometrically the difference between the concepts of partial and probabilistic memberships. By using the geometric approach we show that, in special cases, the GoM model can be thought of as being similar to an item response theory (IRT) model in representing population heterogeneity. Finally, we show that the GoM item parameters can provide quantities analogous to more general logistic IRT item parameters. As a latent structure model, the GoM model might be considered a useful alternative for a data analysis when both classes of extreme responses, and additional heterogeneity that cannot be captured by those latent classes, are expected in the population.

This paper argues that test data are ordinal, that latent trait scores are only determined ordinally, and that test data are used largely for ordinal purposes. Therefore it is desirable to develop a test theory based only on ordinal assumptions. A set of ordinal assumptions is presented, including an ordinal version of local independence. From these assumptions it is first shown that the gamma-correlation between two tests is the product of their gamma-correlations with the true latent order. The theory is generalized to allow for heterogeneous tests by defining a weighted average local independence. The tau-correlations between total score and the latent order can be found in both homogeneous and heterogeneous cases, and a system of differential item weighting to maximize the tau-correlation between weighted items and the latent order is provided. Thus a purely ordinal test theory seems possible.

Two marginal one-parameter item response theory models are introduced, by integrating out the latent variable or random item parameter. It is shown that both marginal response models are multivariate (probit) models with a compound symmetry covariance structure. Several common hypotheses concerning the underlying covariance structure are evaluated using (fractional) Bayes factor tests. The support for a unidimensional factor (i.e., assumption of local independence) and differential item functioning are evaluated by testing the covariance components. The posterior distribution of common covariance components is obtained in closed form by transforming latent responses with an orthogonal (Helmert) matrix. This posterior distribution is defined as a shifted-inverse-gamma, thereby introducing a default prior and a balanced prior distribution. Based on that, an MCMC algorithm is described to estimate all model parameters and to compute (fractional) Bayes factor tests. Simulation studies are used to show that the (fractional) Bayes factor tests have good properties for testing the underlying covariance structure of binary response data. The method is illustrated with two real data studies.

The stochastic subject formulation of latent trait models contends that, within a given subject, the event of obtaining a certain response pattern may be probabilistic. Ordinary latent trait models do not imply that these within-subject probabilities are identical to the conditional probabilities specified by the model. The latter condition is called local homogeneity. It is shown that local homogeneity is equivalent to subpopulation invariance of the model. In case of the monotone IRT model, local homogeneity implies absence of item bias, absence of item specific traits, and the possibility to join overlapping subtests. The following characterization theorem is proved: the homogeneous monotone IRT model holds for a finite or countable item pool if and only if the pool is experimentally independent and pairwise nonnegative association holds in every positive subpopulation.

To capture the cognitive organization of a set of questions or problems pertaining to a body of information, Doignon and Falmagne have proposed, and analyzed in a number of papers, the concept of a knowledge space, that is, a distinguished collection of subsets of questions, representing the possible knowledge states. This collection of sets is assumed to satisfy a number of conditions. Since this concept is a deterministic one, the problem of empirical testing arises. A stochastic version of a knowledge space is developed in this paper, in which the knowledge states are considered as possible epochs in a subject's learning history. The knowledge space is decomposed as a union of a number of possible learning paths, called gradations. The model specifies how a subject is channelled through and progresses along a gradation. A probabilistic axiom of the “local indepencence” type relates the knowledge states to the observable responses. The predictions of this model are worked out in details in the case of parametric assumptions involving gamma distributions. An application of the model to artificial data is described, based on maximum likelihood methods. The statistical analysis is shown to be capable of revealing the combinatoric core of the model.

Using an infinite item test framework, it is argued that the usual assumption of local independence be replaced by a weaker assumption, essential independence. A fortiori, the usual assumption of unidimensionality is replaced by a weaker and arguably more appropriate statistically testable assumption of essential unidimennsionality. Essential unidimennsionnality implies the existence of a “unique” unidimensional latent ability. Essential unidimensionality is equivalent to the “consistent” estimation of this latnet ability in an ordinal scaling sense using any Balanced empirical scaling. A variation of this estimation approach allows consistent estimation of ability on the given latent ability scale.

Measurement invariance (lack of bias) of a manifest variable Y with respect to a latent variable W is defined as invariance of the conditional distribution of Y given W over selected subpopulations. Invariance is commonly assessed by studying subpopulation differences in the conditional distribution of Y given a manifest variable Z, chosen to substitute for W. A unified treatment of conditions that may allow the detection of measurement bias using statistical procedures involving only observed or manifest variables is presented. Theorems are provided that give conditions for measurement invariance, and for invariance of the conditional distribution of Y given Z. Additional theorems and examples explore the Bayes sufficiency of Z, stochastic ordering in W, local independence of Y and Z, exponential families, and the reliability of Z. It is shown that when Bayes sufficiency of Z fails, the two forms of invariance will often not be equivalent in practice. Bayes sufficiency holds under Rasch model assumptions, and in long tests under certain conditions. It is concluded that bias detection procedures that rely strictly on observed variables are not in general diagnostic of measurement bias, or the lack of bias.

Item response theory posits “local independence,” or conditional independence of item responses given item parameters and examinee proficiency parameters. The usual definition of local independence, however, addresses the context of fixed tests, and initially appears to yield incorrect response-pattern probabilities in the context of adaptive testing. The paradox is resolved by introducing additional notation to deal with the item selection mechanism.

The problem of deciding whether a set of mental test data is consistent with any one of a large class of item response models is considered. The “classical” assumption of locla independence is weakened to a new condition, local nonnegative dependence (LND). Necessary and sufficient conditions are derived for a LND item response model to fit a set of data. This leads to a condition that a set of data must satisfy if it is to be representable by any item response model that assumes both local independence and monotone item characteristic curves. An example is given to show that LND is strictly weaker than local independence. Thus rejection of LND models implies rejection of all item response models that assume local independence for a given set of data.

It is widely believed that a joint factor analysis of item responses and response time (RT) may yield more precise ability scores that are conventionally predicted from responses only. For this purpose, a simple-structure factor model is often preferred as it only requires specifying an additional measurement model for item-level RT while leaving the original item response theory (IRT) model for responses intact. The added speed factor indicated by item-level RT correlates with the ability factor in the IRT model, allowing RT data to carry additional information about respondents’ ability. However, parametric simple-structure factor models are often restrictive and fit poorly to empirical data, which prompts under-confidence in the suitablity of a simple factor structure. In the present paper, we analyze the 2015 Programme for International Student Assessment mathematics data using a semiparametric simple-structure model. We conclude that a simple factor structure attains a decent fit after further parametric assumptions in the measurement model are sufficiently relaxed. Furthermore, our semiparametric model implies that the association between latent ability and speed/slowness is strong in the population, but the form of association is nonlinear. It follows that scoring based on the fitted model can substantially improve the precision of ability scores.