We propose an alternative method to partial least squares for path analysis with components, called generalized structured component analysis. The proposed method replaces factors by exact linear combinations of observed variables. It employs a well-defined least squares criterion to estimate model parameters. As a result, the proposed method avoids the principal limitation of partial least squares (i.e., the lack of a global optimization procedure) while fully retaining all the advantages of partial least squares (e.g., less restricted distributional assumptions and no improper solutions). The method is also versatile enough to capture complex relationships among variables, including higher-order components and multi-group comparisons. A straightforward estimation algorithm is developed to minimize the criterion.

A modification of the TUCKALS3 algorithm is proposed that handles three-way arrays of order I × J × K for any I. When I is much larger than JK, the modified algorithm needs less work space to store the data during the iterative part of the algorithm than does the original algorithm. Because of this and the additional feature that execution speed is higher, the modified algorithm is highly suitable for use on personal computers.

An extension of Generalized Structured Component Analysis (GSCA), called Functional GSCA, is proposed to analyze functional data that are considered to arise from an underlying smooth curve varying over time or other continua. GSCA has been geared for the analysis of multivariate data. Accordingly, it cannot deal with functional data that often involve different measurement occasions across participants and a large number of measurement occasions that exceed the number of participants. Functional GSCA addresses these issues by integrating GSCA with spline basis function expansions that represent infinite-dimensional curves onto a finite-dimensional space. For parameter estimation, functional GSCA minimizes a penalized least squares criterion by using an alternating penalized least squares estimation algorithm. The usefulness of functional GSCA is illustrated with gait data.

This paper presents an overview of an approach to the quantitative analysis of qualitative data with theoretical and methodological explanations of the two cornerstones of the approach, Alternating Least Squares and Optimal Scaling. Using these two principles, my colleagues and I have extended a variety of analysis procedures originally proposed for quantitative (interval or ratio) data to qualitative (nominal or ordinal) data, including additivity analysis and analysis of variance; multiple and canonical regression; principal components; common factor and three mode factor analysis; and multidimensional scaling. The approach has two advantages: (a) If a least squares procedure is known for analyzing quantitative data, it can be extended to qualitative data; and (b) the resulting algorithm will be convergent. Three completely worked through examples of the additivity analysis procedure and the steps involved in the regression procedures are presented.

A general approach for fitting a model to a data matrix by weighted least squares (WLS) is studied. This approach consists of iteratively performing (steps of) existing algorithms for ordinary least squares (OLS) fitting of the same model. The approach is based on minimizing a function that majorizes the WLS loss function. The generality of the approach implies that, for every model for which an OLS fitting algorithm is available, the present approach yields a WLS fitting algorithm. In the special case where the WLS weight matrix is binary, the approach reduces to missing data imputation.

Millsap and Meredith (1988) have developed a generalization of principal components analysis for the simultaneous analysis of a number of variables observed in several populations or on several occasions. The algorithm they provide has some disadvantages. The present paper offers two alternating least squares algorithms for their method, suitable for small and large data sets, respectively. Lower and upper bounds are given for the loss function to be minimized in the Millsap and Meredith method. These can serve to indicate whether or not a global optimum for the simultaneous components analysis problem has been attained.

Homogeneity analysis, or multiple correspondence analysis, is usually applied to k separate variables. In this paper we apply it to sets of variables by using sums within sets. The resulting technique is called OVERALS. It uses the notion of optimal scaling, with transformations that can be multiple or single. The single transformations consist of three types: nominal, ordinal, and numerical. The corresponding OVERALS computer program minimizes a least squares loss function by using an alternating least squares algorithm. Many existing linear and nonlinear multivariate analysis techniques are shown to be special cases of OVERALS. An application to data from an epidemiological survey is presented.

A procedure is described for minimizing a class of matrix trace functions. The procedure is a refinement of an earlier procedure for minimizing the class of matrix trace functions using majorization. It contains a recently proposed algorithm by Koschat and Swayne for weighted Procrustes rotation as a special case. A number of trial analyses demonstrate that the refined majorization procedure is more efficient than the earlier majorization-based procedure.

A new method to estimate the parameters of Tucker’s three-mode principal component model is discussed, and the convergence properties of the alternating least squares algorithm to solve the estimation problem are considered. A special case of the general Tucker model, in which the principal component analysis is only performed over two of the three modes is briefly outlined as well. The Miller & Nicely data on the confusion of English consonants are used to illustrate the programs TUCKALS3 and TUCKALS2 which incorporate the algorithms for the two models described.

An important feature of distance-based principal components analysis, is that the variables can be optimally transformed. For monotone spline transformation, a nonnegative least-squares problem with a length constraint has to be solved in each iteration. As an alternative algorithm to Lawson and Hanson (1974), we propose the Alternating Length-Constrained Non-Negative Least-Squares (ALC-NNLS) algorithm, which minimizes the nonnegative least-squares loss function over the parameters under a length constraint, by alternatingly minimizing over one parameter while keeping the others fixed. Several properties of the new algorithm are discussed. A Monte Carlo study is presented which shows that for most cases in distance-based principal components analysis, ALC-NNLS performs as good as the method of Lawson and Hanson or sometimes even better in terms of the quality of the solution.

The problem of minimizing a general matrix, trace function, possibly subject to certain constraints, is approached by means of majorizing this function by one having a simple quadratic shape and whose minimum is easily found. It is shown that the parameter set that minimizes the majorizing function also decreases the matrix trace function, which in turn provides a monotonically convergent algorithm for minimizing the matrix trace function iteratively. Three algorithms based on majorization for solving certain least squares problems are shown to be special cases. In addition, by means of several examples, it is noted how algorithms may be provided for a wide class of statistical optimization tasks for which no satisfactory algorithms seem available.

Generalized structured component analysis (GSCA) is a component-based approach to structural equation modeling. In practice, researchers may often be interested in examining the interaction effects of latent variables. However, GSCA has been geared only for the specification and testing of the main effects of variables. Thus, an extension of GSCA is proposed to effectively deal with various types of interactions among latent variables. In the proposed method, a latent interaction is defined as a product of interacting latent variables. As a result, this method does not require the construction of additional indicators for latent interactions. Moreover, it can easily accommodate both exogenous and endogenous latent interactions. An alternating least-squares algorithm is developed to minimize a single optimization criterion for parameter estimation. A Monte Carlo simulation study is conducted to investigate the parameter recovery capability of the proposed method. An application is also presented to demonstrate the empirical usefulness of the proposed method.

The DEDICOM method for the analysis of asymmetric data tables gives representations that are identified only up to a nonsingular transformation. To identify solutions it is proposed to impose subspace constraints on the stimulus coefficients. Most attention is paid to the case where different subspace constraints are imposed on different dimensions. The procedures are discussed both for the case where the complete table is fitted, and for cases where only offdiagonal elements are fitted. The case where the data table is skew-symmetric is treated separately as well.

We present a new algorithm, MAPCLUS (MAthematical Programming CLUStering), for fitting the Shepard-Arabie ADCLUS (for ADditive CLUStering) model. MAPCLUS utilizes an alternating least squares method combined with a mathematical programming optimization procedure based on a penalty function approach, to impose discrete (0,1) constraints on parameters defining cluster membership. This procedure is supplemented by several other numerical techniques (notably a heuristically based combinatorial optimization procedure) to provide an efficient general-purpose computer implemented algorithm for obtaining ADCLUS representations. MAPCLUS is illustrated with an application to one of the examples given by Shepard and Arabie using the older ADCLUS procedure. The MAPCLUS solution uses half as many clusters to achieve nearly the same level of goodness-of-fit. Finally, we consider an extension of the present approach to fitting a three-way generalization of the ADCLUS model, called INDCLUS (INdividual Differences CLUStering).

An extension of multiple correspondence analysis is proposed that takes into account cluster-level heterogeneity in respondents’ preferences/choices. The method involves combining multiple correspondence analysis and k-means in a unified framework. The former is used for uncovering a low-dimensional space of multivariate categorical variables while the latter is used for identifying relatively homogeneous clusters of respondents. The proposed method offers an integrated graphical display that provides information on cluster-based structures inherent in multivariate categorical data as well as the interdependencies among the data. An empirical application is presented which demonstrates the usefulness of the proposed method and how it compares to several extant approaches.

We consider the problem of least-squares fitting of squared distances in unfolding. An alternating procedure is proposed which fixes the row or column configuration in turn and finds the global optimum of the objective criterion with respect to the free parameters, iterating in this fashion until convergence is reached. A considerable simplification in the algorithm results, namely that this conditional global optimum is identified by performing a single unidimensional search for each point, irrespective of the dimensionality of the unfolding solution.

Generalized Structured Component Analysis (GSCA) was recently introduced by Hwang and Takane (2004) as a component-based approach to path analysis with latent variables. The parameters of GSCA are estimated by pooling data across respondents under the implicit assumption that they all come from a single, homogenous group. However, as has been empirically demonstrated by various researchers across a number of areas of inquiry, such aggregate analyses can often mask the true structure in data when respondent heterogeneity is present. In this paper, GSCA is generalized to a fuzzy clustering framework so as to account for potential group-level respondent heterogeneity. An alternating least-squares procedure is developed and technically described for parameter estimation. A small-scale Monte Carlo study involving synthetic data is carried out to compare the performance between the proposed method and an extant approach. In addition, an empirical application concerning alcohol use among adolescents from US northwestern urban areas is presented to illustrate the usefulness of the proposed method. Finally, a number of directions for future research are provided.

Takane, Young, and de Leeuw proposed a procedure called FACTALS for the analysis of variables of mixed measurement levels (numerical, ordinal, or nominal). Mooijaart pointed out that their algorithm does not necessarily converge, and Nevels proposed a new algorithm for the case of nominal variables. In the present paper it is shown that Nevels' procedure is incorrect, and a new procedure for handling nominal variables is proposed. In addition, a procedure for handling ordinal variables is proposed. Using these results, a monotonically convergent algorithm is constructed for FACTALS of any mixture of variables.

Partial least squares path modeling has been widely used for component-based structural equation modeling, where constructs are represented by weighted composites or components of observed variables. This approach remains a limited-information method that carries out two separate stages sequentially to estimate parameters (component weights, loadings, and path coefficients), indicating that it has no single optimization criterion for estimating the parameters at once. In general, limited-information methods are known to provide less efficient parameter estimates than full-information ones. To address this enduring issue, we propose a full-information method for partial least squares path modeling, termed global least squares path modeling, where a single least squares criterion is consistently minimized via a simple iterative algorithm to estimate all the parameters simultaneously. We evaluate the relative performance of the proposed method through the analyses of simulated and real data. We also show that from algorithmic perspectives, the proposed method can be seen as a block-wise special case of another full-information method for component-based structural equation modeling—generalized structured component analysis.