Principal components analysis of employment in Eastern Europe

AbstractGood empirical applications of geometric morphometrics (GMM) typically involve several times more variables than specimens, a situation the statistician refers to as “highp/n,” wherepis the count of variables andnthe count of specimens. This note calls your attention to two predictable catastrophic failures of one particular multivariate statistical technique, between-groups principal components analysis (bgPCA), in this high-p/nsetting. The more obvious pathology is this: when applied to the patternless (null) model ofpidentically distributed Gaussians over groups of the same size, both bgPCA and its algebraic equivalent, partial least squares (PLS) analysis against group, necessarily generate the appearance of huge equilateral group separations that are actually fictitious (absent from the statistical model). When specimen counts by group vary greatly or when any group includes fewer than about ten specimens, an even worse failure of the technique obtains: the smaller the group, the more likely a bgPCA is to fictitiously identify that group as the end-member of one of its derived axes. For these two reasons, when used in GMM and other high-p/nsettings the bgPCA method very often leads to invalid or insecure bioscientific inferences. This paper demonstrates and quantifies these and other pathological outcomes both for patternless models and for models with one or two valid factors, then offers suggestions for how GMM practitioners should protect themselves against the consequences for inference of these lamentably predictable misrepresentations. The bgPCA method should never be used unskeptically — it is never authoritative — and whenever it appears in partial support of any biological inference it must be accompanied by a wide range of diagnostic plots and other challenges, many of which are presented here for the first time.

Download Full-text

How Likely is Simpson's Paradox in Path Models?

Remote Work and Collaboration ◽

10.4018/978-1-5225-1918-8.ch034 ◽

2017 ◽

pp. 660-666

Author(s):

Ned Kock

Keyword(s):

Latent Variables ◽

Empirical Studies ◽

Statistical Analyses ◽

Multivariate Methods ◽

Multivariate Statistical Analyses ◽

Simpson’S Paradox ◽

Multivariate Statistical ◽

General Inequality ◽

Path Models ◽

Simpson's Paradox

Simpson's paradox is a phenomenon arising from multivariate statistical analyses that often leads to paradoxical conclusions in the field of e-collaboration as well as many other fields where multivariate methods are employed. This work derives a general inequality for the occurrence of Simpson's paradox in path models with or without latent variables. The inequality is then used to estimate the probability that Simpson's paradox would occur at random in path models with two predictors and one criterion variable. This probability is found to be approximately 12.8 percent, slightly higher than 1 occurrence per 8 path models. This estimate suggests that Simpson's paradox is likely to occur in empirical studies, in the field of e-collaboration and other fields, frequently enough to be a source of concern.

Download Full-text

How Likely is Simpson's Paradox in Path Models?

International Journal of e-Collaboration ◽

10.4018/ijec.2015010101 ◽

2015 ◽

Vol 11 (1) ◽

pp. 1-7 ◽

Cited By ~ 7

Author(s):

Ned Kock

Keyword(s):

Latent Variables ◽

Empirical Studies ◽

Statistical Analyses ◽

Multivariate Methods ◽

Multivariate Statistical Analyses ◽

Criterion Variable ◽

Simpson’S Paradox ◽

Multivariate Statistical ◽

Path Models ◽

Simpson's Paradox

Simpson's paradox is a phenomenon arising from multivariate statistical analyses that often leads to paradoxical conclusions in the field of e-collaboration as well as many other fields where multivariate methods are employed. This work derives a general inequality for the occurrence of Simpson's paradox in path models with or without latent variables. The inequality is then used to estimate the probability that Simpson's paradox would occur at random in path models with two predictors and one criterion variable. This probability is found to be approximately 12.8 percent, slightly higher than 1 occurrence per 8 path models. This estimate suggests that Simpson's paradox is likely to occur in empirical studies, in the field of e-collaboration and other fields, frequently enough to be a source of concern.

Download Full-text

Adaptive multiscale principal components analysis for online monitoring of wastewater treatment

Water Science & Technology ◽

10.2166/wst.2002.0593 ◽

2002 ◽

Vol 45 (4-5) ◽

pp. 227-235 ◽

Cited By ~ 29

Author(s):

J. Lennox ◽

C. Rosen

Keyword(s):

Wastewater Treatment ◽

Principal Components Analysis ◽

Principal Components ◽

Fault Detection And Isolation ◽

Scale Model ◽

Multivariate Statistical ◽

Data Set ◽

Significant Difference ◽

Components Analysis ◽

Adaptive Pca

Fault detection and isolation (FDI) are important steps in the monitoring and supervision of industrial processes. Biological wastewater treatment (WWT) plants are difficult to model, and hence to monitor, because of the complexity of the biological reactions and because plant influent and disturbances are highly variable and/or unmeasured. Multivariate statistical models have been developed for a wide variety of situations over the past few decades, proving successful in many applications. In this paper we develop a new monitoring algorithm based on Principal Components Analysis (PCA). It can be seen equivalently as making Multiscale PCA (MSPCA) adaptive, or as a multiscale decomposition of adaptive PCA. Adaptive Multiscale PCA (AdMSPCA) exploits the changing multivariate relationships between variables at different time-scales. Adaptation of scale PCA models over time permits them to follow the evolution of the process, inputs or disturbances. Performance of AdMSPCA and adaptive PCA on a real WWT data set is compared and contrasted. The most significant difference observed was the ability of AdMSPCA to adapt to a much wider range of changes. This was mainly due to the flexibility afforded by allowing each scale model to adapt whenever it did not signal an abnormal event at that scale. Relative detection speeds were examined only summarily, but seemed to depend on the characteristics of the faults/disturbances. The results of the algorithms were similar for sudden changes, but AdMSPCA appeared more sensitive to slower changes.

Download Full-text

Basic Multivariate Statistical Methods for Environmental Monitoring Data Mining: Introductory Course for Master Students

Chemistry-Didactics-Ecology-Metrology ◽

10.2478/cdem-2020-0002 ◽

2020 ◽

Vol 25 (1-2) ◽

pp. 35-56

Author(s):

Vasil Simeonov

Keyword(s):

Environmental Monitoring ◽

Principal Components Analysis ◽

Statistical Methods ◽

Principal Components ◽

Monitoring Data ◽

Multivariate Statistical Methods ◽

Multivariate Statistical ◽

Introductory Course ◽

Components Analysis ◽

Environmental Monitoring Data

Abstract The present introductory course of lectures summarizes the principles and algorithms of several widely used multivariate statistical methods: cluster analysis, principal components analysis, principal components regression, N-way principal components analysis, partial least squares regression and self-organizing maps with respect to their possible application in intelligent analysis, classification, modelling and interpretation to environmental monitoring data. The target group of possible users is master program students (environmental chemistry, analytical chemistry, environmental modelling and risk assessment etc.).

Download Full-text

Pathologies of Between-Groups Principal Components Analysis in Geometric Morphometrics

Evolutionary Biology ◽

10.1007/s11692-019-09484-8 ◽

2019 ◽

Vol 46 (4) ◽

pp. 271-302 ◽

Cited By ~ 9

Author(s):

Fred L. Bookstein

Keyword(s):

Geometric Morphometrics ◽

Principal Components Analysis ◽

Principal Components ◽

Null Model ◽

Statistical Technique ◽

Multivariate Statistical ◽

Wide Range ◽

Pls Analysis ◽

Components Analysis ◽

First Time

Abstract Good empirical applications of geometric morphometrics (GMM) typically involve several times more variables than specimens, a situation the statistician refers to as “high p/n,” where p is the count of variables and n the count of specimens. This note calls your attention to two predictable catastrophic failures of one particular multivariate statistical technique, between-groups principal components analysis (bgPCA), in this high-p/n setting. The more obvious pathology is this: when applied to the patternless (null) model of p identically distributed Gaussians over groups of the same size, both bgPCA and its algebraic equivalent, partial least squares (PLS) analysis against group, necessarily generate the appearance of huge equilateral group separations that are fictitious (absent from the statistical model). When specimen counts by group vary greatly or when any group includes fewer than about ten specimens, an even worse failure of the technique obtains: the smaller the group, the more likely a bgPCA is to fictitiously identify that group as the end-member of one of its derived axes. For these two reasons, when used in GMM and other high-p/n settings the bgPCA method very often leads to invalid or insecure biological inferences. This paper demonstrates and quantifies these and other pathological outcomes both for patternless models and for models with one or two valid factors, then offers suggestions for how GMM practitioners should protect themselves against the consequences for inference of these lamentably predictable misrepresentations. The bgPCA method should never be used unskeptically—it is always untrustworthy, never authoritative—and whenever it appears in partial support of any biological inference it must be accompanied by a wide range of diagnostic plots and other challenges, many of which are presented here for the first time.

Download Full-text

Zooplancton limnétique de 46 lacs et 17 rivières du territoire de la baie de James

Canadian Journal of Zoology ◽

10.1139/z79-220 ◽

1979 ◽

Vol 57 (8) ◽

pp. 1693-1709 ◽

Cited By ~ 8

Author(s):

Bernadette Pinel-Alloul ◽

Pierre Legendre ◽

Etienne Magnin

Keyword(s):

Principal Components Analysis ◽

Principal Components ◽

Statistical Analyses ◽

Environmental Data ◽

James Bay ◽

Single Linkage ◽

Widespread Species ◽

Bay Area ◽

Components Analysis ◽

Western Portion

From June through October 1973, 335 samples of limnetic plankton were collected from 46 lakes and 17 rivers of the James Bay area. Sixty zooplanktonic species were identified (20 Copepoda, 27 Cladocera, and 13 Rotifera). The most common and widespread species are cold stenotherms (Leptodiaptomus minutus, Diacyclops bicuspidatus thomasi, Epischura lacustris, Holopedium gibberum, Bosmina longirostris, Daphnia longiremis, and Kellicottia longispina). In order to study the typology of the lake samples, the data were subjected to three types of statistical analyses: principal components analysis, single linkage, and complete linkage clustering. Five groups of lakes emerged from these analyses: types IV and V are located in the northeastern portion of the studied area, whereas types I and II were identified in the western portion, corresponding with the area occupied by the Tyrrell glacial sea. Type III fills an intermediate position. Types II, III, and V are smalt lakes. The characteristic zooplanktonic communities of each group are described, whereas the principal components and the components of the diversity are correlated with the environmental data.

Download Full-text