Multi-criteria Ranking Based on Joint Distributions

AbstractA quality model is a conceptual decomposition of an abstract notion of quality into relevant, possibly conflicting characteristics and further into measurable metrics. For quality assessment and decision making, metrics values are aggregated to characteristics and ultimately to quality scores. Aggregation has often been problematic as quality models do not provide the semantics of aggregation. This makes it hard to formally reason about metrics, characteristics, and quality. We argue that aggregation needs to be interpretable and mathematically well defined in order to assess, to compare, and to improve quality. To address this challenge, we propose a probabilistic approach to aggregation and define quality scores based on joint distributions of absolute metrics values. To evaluate the proposed approach and its implementation under realistic conditions, we conduct empirical studies on bug prediction of ca. 5000 software classes, maintainability of ca. 15000 open-source software systems, and on the information quality of ca. 100000 real-world technical documents. We found that our approach is feasible, accurate, and scalable in performance.

Download Full-text

Error and Reliability in Stochastic Processes and Psychological Measurement

Psychological Reports ◽

10.2466/pr0.1972.31.1.131 ◽

1972 ◽

Vol 31 (1) ◽

pp. 131-140 ◽

Cited By ~ 2

Author(s):

Donald W. Zimmerman

Keyword(s):

Stochastic Processes ◽

Random Error ◽

Random Sampling ◽

Probability Model ◽

Random Variables ◽

Formal Structure ◽

Intermediate Structure ◽

Psychological Measurement ◽

Joint Distributions ◽

True Values

The concepts of random error and reliability of measurements that are familiar in traditional theories based on the notions of “true values” and “errors” can be represented by a probability model having a simpler formal structure and fewer special assumptions about random sampling and independence of measurements. In this model formulas that relate observable events are derived from probability axioms and from primitive terms that refer to observable events, without an intermediate structure containing variances and correlations of “true” and “error” components of scores. While more economical in language and formalism, the model at the same time is more general than classical theories and applies to stochastic processes in which joint distributions of many dependent random variables are of interest. In addition, it clarifies some long-standing problems concerning “experimental independence” of measurements and the relation of sampling of individuals to sampling of measurements.

Download Full-text

The distributional structure of finite moving-average processes

Journal of Applied Probability ◽

10.1017/s002190020004095x ◽

1988 ◽

Vol 25 (02) ◽

pp. 313-321 ◽

Cited By ~ 4

Author(s):

ED McKenzie

Keyword(s):

Moving Average ◽

Markov Property ◽

Covariance Structure ◽

Time Reversibility ◽

Linear Processes ◽

Joint Distributions ◽

Moving Average Processes ◽

Standard Models ◽

Non Gaussian ◽

Analysis Of Time Series

Analysis of time-series models has, in the past, concentrated mainly on second-order properties, i.e. the covariance structure. Recent interest in non-Gaussian and non-linear processes has necessitated exploration of more general properties, even for standard models. We demonstrate that the powerful Markov property which greatly simplifies the distributional structure of finite autoregressions has an analogue in the (non-Markovian) finite moving-average processes. In fact, all the joint distributions of samples of a qth-order moving average may be constructed from only the (q + 1)th-order distribution. The usefulness of this result is illustrated by references to three areas of application: time-reversibility; asymptotic behaviour; and sums and associated point and count processes. Generalizations of the result are also considered.

Download Full-text