To establish the existence of his abilities, a judge is given the task of classifying each of N= rs subjects into one of r known categories, each containing s of the subjects. An incomplete design is proposed whereby the judge is presented with b groups, each one containing n= rs/b<r subjects. The n different categories corresponding to members of the group are known. Using the total number of correct classifications, this method of grouping is compared to that in which the group size is equal to the number of categories. The incomplete grouping is shown to yield a more powerful test for discriminating between the null hypothesis that the judge is guessing the classifications and the alternative hypothesis that he has some definite abilities. The incomplete design is found to be most effective (powerful) when the number of subjects in a group is limited to two or three.

We propose a unifying framework for multilevel modeling of polytomous data and rankings, accommodating dependence induced by factor and/or random coefficient structures at different levels. The framework subsumes a wide range of models proposed in disparate methodological literatures. Partial and tied rankings, alternative specific explanatory variables and alternative sets varying across units are handled. The problem of identification is addressed. We develop an estimation and prediction methodology for the model framework which is implemented in the generally available gllamm software. The methodology is applied to party choice and rankings from the 1987–1992 panel of the British Election Study. Three levels are considered: elections, voters and constituencies.

Complete response vectors of all answer options in multiple-choice items can be used to estimate ability. The rising selection ratios criterion is necessary for scoring individuals because it implies that estimated ability always increases when the correct alternative is selected. This paper introduces the generalized DLT model, which assumes rising selection ratios and uses three parameters to describe each incorrect alternative. It is shown that the nominal categories and the Thissen and Steinberg model do not meet the criterion. A simulation study on goodness of recovery and an example with real data are also included.

We assume that a judge's task is to categorize each of N subjects into one of r known classes. The design of primary interest is employed if the judge is presented with s groups, each containing r subjects, such that each group of size r consists of exactly one subject of each of the r types. The probability distribution for the total number of correct choices is developed and used to test the null hypothesis that the judge is “guessing” in favor of the alternative that he or she is operating at a better than chance level. The power of the procedure is shown to be superior to two other procedures which appear in the literature.

Two designs for comparing a judge's ratings with a known standard are presented and compared. Design A pertains to the situation where the judge is asked to categorize each of N subjects into one of r (known) classes with no knowledge of the actual number in each class. Design B is employed when the judge is given the actual number in each class and is asked to categorize the individuals subject to these constraints. The probability distribution of the total number of correct choices is developed in each case. A power comparison of the two procedures is undertaken.

Previous studies of historical risk have used either nominal or real data to calculate risk measures for agricultural prices and income. However, the effects of using nominal and real data have not been evaluated. This study utilizes theoretical variance approximation relationships to examine variances from detrended real and nominal time series. The relationships between variances are derived for quarterly U.S. farm milk prices for 1960-72, 1973-80, and 1981-90. Contrary to common intuitive arguments, results indicate that variances of real time series can be larger than variances of nominal series. While definitive conclusions are not possible, several reasons for using nominal data in risk analysis are given.