# Standard deviation

## Comments

## Transcription

Standard deviation

Statistics PDF generated using the open source mwlib toolkit. See http://code.pediapress.com/ for more information. PDF generated at: Wed, 22 Jan 2014 22:53:40 UTC Contents Articles Standard deviation 1 Samuelson's inequality 15 Distance correlation 16 Geometric standard deviation 21 Bregman divergence 23 Bhattacharyya distance 26 Hellinger distance 28 Mahalanobis distance 31 Similarity learning 33 Yamartino method 35 References Article Sources and Contributors 37 Image Sources, Licenses and Contributors 38 Article Licenses License 39 Standard deviation 1 Standard deviation In statistics and probability theory, the standard deviation (represented by the Greek letter sigma, σ) shows how much variation or dispersion from the average exists. A low standard deviation indicates that the data points tend to be very close to the mean (also called expected value); a high standard deviation indicates that the data points are spread out over a large range of values. The standard deviation of a random variable, statistical population, data set, or probability distribution is the square root of its variance. It is algebraically simpler though in practice less robust than the average absolute deviation. A useful property of the standard deviation is that, unlike the variance, it is expressed in the same units as the data. Note, however, that for measurements with percentage as the unit, the standard deviation will have percentage points as the unit. A plot of a normal distribution (or bell-shaped curve) where each band has a width of 1 standard deviation – See also: 68–95–99.7 rule Cumulative probability of a normal distribution with expected value 0 and standard deviation 1. In addition to expressing the variability of a population, the standard deviation is commonly used to measure confidence in statistical conclusions. For example, the margin of error in polling data is determined by calculating the expected standard deviation in the results if the same poll were to be conducted multiple times. The reported margin of error is typically about twice the standard deviation – the half-width of a 95 percent confidence interval. In science, researchers commonly report the standard deviation of experimental data, and only effects that fall much farther than one standard deviation away from what would have been expected are considered statistically significant – normal random error or variation in the measurements is in this way distinguished from causal variation. The standard deviation is also important in finance, where the standard deviation on the rate of return on an investment is a measure of the volatility of the investment. When only a sample of data from a population is available, the term standard deviation of the sample or sample standard deviation can refer to either the above-mentioned quantity as applied to those data or to a modified quantity that is a better estimate of the population standard deviation (the standard deviation of the entire population). Standard deviation Basic examples For a finite set of numbers, the standard deviation is found by taking the square root of the average of the squared differences of the values from their average value. For example, consider a population consisting of the following eight values: These eight data points have the mean (average) of 5: First, calculate the difference of each data point from the mean, and square the result of each: Next, calculate the mean of these values, and take the square root: This quantity is the population standard deviation, and is equal to the square root of the variance. This formula is valid only if the eight values we began with form the complete population. If the values instead were a random sample drawn from some larger parent population, then we would have divided by 7 (which is n−1) instead of 8 (which is n) in the denominator of the last formula, and then the quantity thus obtained would be called the sample standard deviation. Dividing by n−1 gives a better estimate of the population standard deviation than dividing by n. As a slightly more complicated real-life example, the average height for adult men in the United States is about 70 inches, with a standard deviation of around 3 inches. This means that most men (about 68 percent, assuming a normal distribution) have a height within 3 inches of the mean (67–73 inches) – one standard deviation – and almost all men (about 95%) have a height within 6 inches of the mean (64–76 inches) – two standard deviations. If the standard deviation were zero, then all men would be exactly 70 inches tall. If the standard deviation were 20 inches, then men would have much more variable heights, with a typical range of about 50–90 inches. Three standard deviations account for 99.7 percent of the sample population being studied, assuming the distribution is normal (bell-shaped). Definition of population values Let X be a random variable with mean value μ: Here the operator E denotes the average or expected value of X. Then the standard deviation of X is the quantity (derived using the properties of expected value). In other words the standard deviation σ (sigma) is the square root of the variance of X; i.e., it is the square root of the average value of (X − μ)2. 2 Standard deviation The standard deviation of a (univariate) probability distribution is the same as that of a random variable having that distribution. Not all random variables have a standard deviation, since these expected values need not exist. For example, the standard deviation of a random variable that follows a Cauchy distribution is undefined because its expected value μ is undefined. Discrete random variable In the case where X takes random values from a finite data set x1, x2, ..., xN, with each value having the same probability, the standard deviation is or, using summation notation, If, instead of having equal probabilities, the values have different probabilities, let x1 have probability p1, x2 have probability p2, ..., xN have probability pN. In this case, the standard deviation will be Continuous random variable The standard deviation of a continuous real-valued random variable X with probability density function p(x) is and where the integrals are definite integrals taken for x ranging over the set of possible values of the random variable X. In the case of a parametric family of distributions, the standard deviation can be expressed in terms of the parameters. For example, in the case of the log-normal distribution with parameters μ and σ2, the standard deviation is [(exp(σ2) − 1)exp(2μ + σ2)]1/2. Estimation One can find the standard deviation of an entire population in cases (such as standardized testing) where every member of a population is sampled. In cases where that cannot be done, the standard deviation σ is estimated by examining a random sample taken from the population and computing a statistic of the sample, which is used as an estimate of the population standard deviation. Such a statistic is called an estimator, and the estimator (or the value of the estimator, namely the estimate) is called a sample standard deviation, and is denoted by s (possibly with modifiers). However, unlike in the case of estimating the population mean, for which the sample mean is a simple estimator with many desirable properties (unbiased, efficient, maximum likelihood), there is no single estimator for the standard deviation with all these properties, and unbiased estimation of standard deviation is a very technical involved problem. Most often, the standard deviation is estimated using the corrected sample standard deviation (using N − 1), defined below, and this is often referred to as the "sample standard deviation", without qualifiers. However, other estimators are better in other respects: the uncorrected estimator (using N) yields lower mean squared error, while using N − 1.5 (for the normal distribution) almost completely eliminates bias. 3 Standard deviation 4 Uncorrected sample standard deviation Firstly, the formula for the population standard deviation (of a finite population) can be applied to the sample, using the size of the sample as the size of the population (though the actual population size from which the sample is drawn may be much larger). This estimator, denoted by sN, is known as the uncorrected sample standard deviation, or sometimes the standard deviation of the sample (considered as the entire population), and is defined as follows:[citation needed] where are the observed values of the sample items and is the mean value of these observations, while the denominator N stands for the size of the sample. This is a consistent estimator (it converges in probability to the population value as the number of samples goes to infinity), and is the maximum-likelihood estimate when the population is normally distributed.[citation needed] However, this is a biased estimator, as the estimates are generally too low. The bias decreases as sample size grows, dropping off as 1/n, and thus is most significant for small or moderate sample sizes; for the bias is below 1%. Thus for very large sample sizes, the uncorrected sample standard deviation is generally acceptable. This estimator also has a uniformly smaller mean squared error than the corrected sample standard deviation. Corrected sample standard deviation When discussing the bias, to be more precise, the corresponding estimator for the variance, the biased sample variance: equivalently the second central moment of the sample (as the mean is the first moment), is a biased estimator of the variance (it underestimates the population variance). Taking the square root to pass to the standard deviation introduces further downward bias, by Jensen's inequality, due to the square root being a concave function. The bias in the variance is easily corrected, but the bias from the square root is more difficult to correct, and depends on the distribution in question. An unbiased estimator for the variance is given by applying Bessel's correction, using N − 1 instead of N to yield the unbiased sample variance, denoted s2: This estimator is unbiased if the variance exists and the sample values are drawn independently with replacement. N − 1 corresponds to the number of degrees of freedom in the vector of residuals, Taking square roots reintroduces bias, and yields the corrected sample standard deviation, denoted by s: While s2 is an unbiased estimator for the population variance, s is a biased estimator for the population standard deviation, though markedly less biased than the uncorrected sample standard deviation. The bias is still significant for small samples (n less than 10), and also drops off as 1/n as sample size increases. This estimator is commonly used, and generally known simply as the "sample standard deviation". Standard deviation Unbiased sample standard deviation For unbiased estimation of standard deviation, there is no formula that works across all distributions, unlike for mean and variance. Instead, s is used as a basis, and is scaled by a correction factor to produce an unbiased estimate. For the normal distribution, an unbiased estimator is given by s/c4, where the correction factor (which depends on N) is given in terms of the Gamma function, and equals: This arises because the sampling distribution of the sample standard deviation follows a (scaled) chi distribution, and the correction factor is the mean of the chi distribution. An approximation is given by replacing N − 1 with N − 1.5, yielding: The error in this approximation decays quadratically (as 1/N2), and it is suited for all but the smallest samples or highest precision: for n = 3 the bias is equal to 1.3%, and for n = 9 the bias is already less than 0.1%. For other distributions, the correct formula depends on the distribution, but a rule of thumb is to use the further refinement of the approximation: where γ2 denotes the population excess kurtosis. The excess kurtosis may be either known beforehand for certain distributions, or estimated from the data. Confidence interval of a sampled standard deviation The standard deviation we obtain by sampling a distribution is itself not absolutely accurate, both for mathematical reasons (explained here by the confidence interval) and for practical reasons of measurement (measurement error). The mathematical effect can be described by the confidence interval or CI. To show how a larger sample will increase the confidence interval, consider the following examples: For a small population of N=2, the 95% CI of the SD is from 0.45*SD to 31.9*SD. In other words, the standard deviation of the distribution in 95% of the cases can be larger by a factor of 31 or smaller by a factor of 2. For a larger population of N=10, the CI is 0.69*SD to 1.83*SD. So even with a sample population of 10, the actual SD can still be almost a factor 2 higher than the sampled SD. For a sample population N=100, this is down to 0.88*SD to 1.16*SD. To be more certain that the sampled SD is close to the actual SD we need to sample a large number of points. Identities and mathematical properties The standard deviation is invariant under changes in location, and scales directly with the scale of the random variable. Thus, for a constant c and random variables X and Y: The standard deviation of the sum of two random variables can be related to their individual standard deviations and the covariance between them: 5 Standard deviation where 6 and stand for variance and covariance, respectively. The calculation of the sum of squared deviations can be related to moments calculated directly from the data. The standard deviation of the sample can be computed as: The sample standard deviation can be computed as: For a finite population with equal probabilities at all points, we have This means that the standard deviation is equal to the square root of (the average of the squares less the square of the average). See computational formula for the variance for proof, and for an analogous result for the sample standard deviation. Interpretation and application A large standard deviation indicates that the data points are far from the mean and a small standard deviation indicates that they are clustered closely around the mean. For example, each of the three populations {0, 0, 14, 14}, {0, 6, 8, 14} and {6, 6, 8, 8} has a mean of 7. Their standard deviations are 7, 5, and 1, respectively. The third population has a much smaller standard deviation than the other two because its values are all close to 7. It will have the same units as the data points themselves. If, for instance, the data set {0, 6, 8, 14} Example of two sample populations with the same mean and different standard represents the ages of a population of deviations. Red population has mean 100 and SD 10; blue population has mean 100 and four siblings in years, the standard SD 50.Wikipedia:Disputed statement deviation is 5 years. As another example, the population {1000, 1006, 1008, 1014} may represent the distances traveled by four athletes, measured in meters. It has a mean of 1007 meters, and a standard deviation of 5 meters. Standard deviation may serve as a measure of uncertainty. In physical science, for example, the reported standard deviation of a group of repeated measurements gives the precision of those measurements. When deciding whether measurements agree with a theoretical prediction, the standard deviation of those measurements is of crucial importance: if the mean of the measurements is too far away from the prediction (with the distance measured in standard deviations), then the theory being tested probably needs to be revised. This makes sense since they fall outside the range of values that could reasonably be expected to occur, if the prediction were correct and the standard deviation appropriately quantified. See prediction interval. Standard deviation While the standard deviation does measure how far typical values tend to be from the mean, other measures are available. An example is the mean absolute deviation, which might be considered a more direct measure of average distance, compared to the root mean square distance inherent in the standard deviation. Application examples The practical value of understanding the standard deviation of a set of values is in appreciating how much variation there is from the average (mean). Climate As a simple example, consider the average daily maximum temperatures for two cities, one inland and one on the coast. It is helpful to understand that the range of daily maximum temperatures for cities near the coast is smaller than for cities inland. Thus, while these two cities may each have the same average maximum temperature, the standard deviation of the daily maximum temperature for the coastal city will be less than that of the inland city as, on any particular day, the actual maximum temperature is more likely to be farther from the average maximum temperature for the inland city than for the coastal one. Particle physics Particle physics uses a standard of "5 sigma" for the declaration of a discovery. At five-sigma there is only one chance in nearly two million that a random fluctuation would yield the result. This level of certainty prompted the announcement that a particle consistent with the Higgs boson has been discovered in two independent experiments at CERN. Finance In finance, standard deviation is often used as a measure of the risk associated with price-fluctuations of a given asset (stocks, bonds, property, etc.), or the risk of a portfolio of assets (actively managed mutual funds, index mutual funds, or ETFs). Risk is an important factor in determining how to efficiently manage a portfolio of investments because it determines the variation in returns on the asset and/or portfolio and gives investors a mathematical basis for investment decisions (known as mean-variance optimization). The fundamental concept of risk is that as it increases, the expected return on an investment should increase as well, an increase known as the risk premium. In other words, investors should expect a higher return on an investment when that investment carries a higher level of risk or uncertainty. When evaluating investments, investors should estimate both the expected return and the uncertainty of future returns. Standard deviation provides a quantified estimate of the uncertainty of future returns. For example, let's assume an investor had to choose between two stocks. Stock A over the past 20 years had an average return of 10 percent, with a standard deviation of 20 percentage points (pp) and Stock B, over the same period, had average returns of 12 percent but a higher standard deviation of 30 pp. On the basis of risk and return, an investor may decide that Stock A is the safer choice, because Stock B's additional two percentage points of return is not worth the additional 10 pp standard deviation (greater risk or uncertainty of the expected return). Stock B is likely to fall short of the initial investment (but also to exceed the initial investment) more often than Stock A under the same circumstances, and is estimated to return only two percent more on average. In this example, Stock A is expected to earn about 10 percent, plus or minus 20 pp (a range of 30 percent to −10 percent), about two-thirds of the future year returns. When considering more extreme possible returns or outcomes in future, an investor should expect results of as much as 10 percent plus or minus 60 pp, or a range from 70 percent to −50 percent, which includes outcomes for three standard deviations from the average return (about 99.7 percent of probable returns). Calculating the average (or arithmetic mean) of the return of a security over a given period will generate the expected return of the asset. For each period, subtracting the expected return from the actual return results in the difference from the mean. Squaring the difference in each period and taking the average gives the overall variance of the return of the asset. The larger the variance, the greater risk the security carries. Finding the square root of this 7 Standard deviation 8 variance will give the standard deviation of the investment tool in question. Population standard deviation is used to set the width of Bollinger Bands, a widely adopted technical analysis tool. For example, the upper Bollinger Band is given as x + nσx. The most commonly used value for n is 2; there is about a five percent chance of going outside, assuming a normal distribution of returns. Unfortunately, financial time series are known to be non-stationary series, whereas the statistical calculations above, such as standard deviation, apply only to stationary series. Whatever apparent "predictive powers" or "forecasting ability" that may appear when applied as above is illusory. To apply the above statistical tools to non-stationary series, the series first must be transformed to a stationary series, enabling use of statistical tools that now have a valid basis from which to work. Geometric interpretation It is requested that a diagram or diagrams be included in this article to improve its quality. Specific illustrations, plots or diagrams can be requested at the Graphic Lab. For more information, refer to discussion on this page and/or the listing at Wikipedia:Requested images. To gain some geometric insights and clarification, we will start with a population of three values, x1, x2, x3. This defines a point P = (x1, x2, x3) in R3. Consider the line L = {(r, r, r) : r ∈ R}. This is the "main diagonal" going through the origin. If our three given values were all equal, then the standard deviation would be zero and P would lie on L. So it is not unreasonable to assume that the standard deviation is related to the distance of P to L. And that is indeed the case. To move orthogonally from L to the point P, one begins at the point: whose coordinates are the mean of the values we started out with. A little algebra shows that the distance between P and M (which is the same as the orthogonal distance between P and the line L) is equal to the standard deviation of the vector x1, x2, x3, multiplied by the square root of the number of dimensions of the vector (3 in this case.) Chebyshev's inequality An observation is rarely more than a few standard deviations away from the mean. Chebyshev's inequality ensures that, for all distributions for which the standard deviation is defined, the amount of data within a number of standard deviations of the mean is at least as much as given in the following table. Minimum population Distance from mean 50% √2 75% 2 89% 3 94% 4 96% 5 97% 6 [1] Standard deviation 9 Rules for normally distributed data The central limit theorem says that the distribution of an average of many independent, identically distributed random variables tends toward the famous bell-shaped normal distribution with a probability density function of: where μ is the expected value of the random variables, σ equals their distribution's standard deviation 1/2 divided by n , and n is the number of random variables. The standard deviation therefore is simply a scaling variable that adjusts how broad the curve will be, though it also appears in the normalizing constant. Dark blue is one standard deviation on either side of the mean. For the normal distribution, this accounts for 68.27 percent of the set; while two standard deviations from the mean (medium and dark blue) account for 95.45 percent; three standard deviations (light, medium, and dark blue) account for 99.73 percent; and four standard deviations account for 99.994 percent. The two points of the curve that are one standard deviation from the mean are also the inflection points. If a data distribution is approximately normal, then the proportion of data values within z standard deviations of the mean is defined by: Proportion = where is the error function. If a data distribution is approximately normal then about 68 percent of the data values are within one standard deviation of the mean (mathematically, μ ± σ, where μ is the arithmetic mean), about 95 percent are within two standard deviations (μ ± 2σ), and about 99.7 percent lie within three standard deviations (μ ± 3σ). This is known as the 68-95-99.7 rule, or the empirical rule. For various values of z, the percentage of values expected to lie in and outside the symmetric interval, CI = (−zσ, zσ), are as follows: zσ Percentage within CI Percentage outside CI Fraction outside CI 0.674490σ 50% 50% 1 / 2 0.994458σ 68% 32% 1 / 3.125 1σ 31.7310508% 1 / 3.1514872 1.281552σ 80% 20% 1 / 5 1.644854σ 90% 10% 1 / 10 1.959964σ 95% 5% 1 / 20 2σ 4.5500264% 1 / 21.977895 2.575829σ 99% 1% 1 / 100 3σ 0.2699796% 1 / 370.398 3.290527σ 99.9% 0.1% 1 / 1000 3.890592σ 99.99% 0.01% 1 / 10000 4σ 0.006334% 1 / 15787 0.001% 1 / 100000 68.2689492% 95.4499736% 99.7300204% 99.993666% 4.417173σ 99.999% Standard deviation 10 4.5σ 99.9993204653751% 0.0006795346249% 3.4 / 1000000 (on each side of mean) 4.891638σ 99.9999% 0.0001% 1 / 1000000 5σ 0.0000573303% 1 / 1744278 5.326724σ 99.99999% 0.00001% 1 / 10000000 5.730729σ 99.999999% 0.000001% 1 / 100000000 6σ 0.0000001973% 1 / 506797346 6.109410σ 99.9999999% 0.0000001% 1 / 1000000000 6.466951σ 99.99999999% 0.00000001% 1 / 10000000000 6.806502σ 99.999999999% 0.000000001% 1 / 100000000000 7σ 0.000000000256% 1 / 390682215445 99.9999426697% 99.9999998027% 99.9999999997440% Relationship between standard deviation and mean The mean and the standard deviation of a set of data are descriptive statistics usually reported together. In a certain sense, the standard deviation is a "natural" measure of statistical dispersion if the center of the data is measured about the mean. This is because the standard deviation from the mean is smaller than from any other point. The precise statement is the following: suppose x1, ..., xn are real numbers and define the function: Using calculus or by completing the square, it is possible to show that σ(r) has a unique minimum at the mean: Variability can also be measured by the coefficient of variation, which is the ratio of the standard deviation to the mean. It is a dimensionless number. Standard deviation of the mean Often, we want some information about the precision of the mean we obtained. We can obtain this by determining the standard deviation of the sampled mean. Assuming statistical independence of the values in the sample, the standard deviation of the mean is related to the standard deviation of the distribution by: where N is the number of observations in the sample used to estimate the mean. This can easily be proven with (see basic properties of the variance): hence Resulting in: Standard deviation 11 It should be emphasized that in order to estimate standard deviation of the mean it is necessary to know standard deviation of the entire population beforehand. However, in most applications this parameter is unknown. For example, if series of 10 measurements of previously unknown quantity is performed in laboratory, it is possible to calculate resulting sample mean and sample standard deviation, but it is impossible to calculate standard deviation of the mean. Rapid calculation methods The following two formulas can represent a running (repeatedly updated) standard deviation. A set of two power sums s1 and s2 are computed over a set of N values of x, denoted as x1, ..., xN: Given the results of these running summations, the values N, s1, s2 can be used at any time to compute the current value of the running standard deviation: Where : Similarly for sample standard deviation, In a computer implementation, as the three sj sums become large, we need to consider round-off error, arithmetic overflow, and arithmetic underflow. The method below calculates the running sums method with reduced rounding errors. This is a "one pass" algorithm for calculating variance of n samples without the need to store prior data during the calculation. Applying this method to a time series will result in successive values of standard deviation corresponding to n data points as n grows larger with each new sample, rather than a constant-width sliding window calculation. For k = 1, ..., n: where A is the mean value. Note: since Sample variance: Population variance: or Standard deviation Weighted calculation When the values xi are weighted with unequal weights wi, the power sums s0, s1, s2 are each computed as: And the standard deviation equations remain unchanged. Note that s0 is now the sum of the weights and not the number of samples N. The incremental method with reduced rounding errors can also be applied, with some additional complexity. A running sum of weights must be computed for each k from 1 to n: and places where 1/n is used above must be replaced by wi/Wn: In the final division, and where n is the total number of elements, and n' is the number of elements with non-zero weights. The above formulas become equal to the simpler formulas given above if weights are taken as equal to one. Combining standard deviations Population-based statistics The populations of sets, which may overlap, can be calculated simply as follows: Standard deviations of non-overlapping (X ∩ Y = ∅) sub-populations can be aggregated as follows if the size (actual or relative to one another) and means of each are known: For example, suppose it is known that the average American man has a mean height of 70 inches with a standard deviation of three inches and that the average American woman has a mean height of 65 inches with a standard deviation of two inches. Also assume that the number of men, N, is equal to the number of women. Then the mean and standard deviation of heights of American adults could be calculated as: 12 Standard deviation 13 For the more general case of M non-overlapping populations, X1 through XM, and the aggregate population : where If the size (actual or relative to one another), mean, and standard deviation of two overlapping populations are known for the populations as well as their intersection, then the standard deviation of the overall population can still be calculated as follows: If two or more sets of data are being added together datapoint by datapoint, the standard deviation of the result can be calculated if the standard deviation of each data set and the covariance between each pair of data sets is known: For the special case where no correlation exists between any pair of data sets, then the relation reduces to the root-mean-square: Sample-based statistics Standard deviations of non-overlapping (X ∩ Y = ∅) sub-samples can be aggregated as follows if the actual size and means of each are known: For the more general case of M non-overlapping data sets, X1 through XM, and the aggregate data set : Standard deviation where: If the size, mean, and standard deviation of two overlapping samples are known for the samples as well as their intersection, then the standard deviation of the aggregated sample can still be calculated. In general: History The term standard deviation was first used in writing by Karl Pearson in 1894, following his use of it in lectures. This was as a replacement for earlier alternative names for the same idea: for example, Gauss used mean error. It may be worth noting in passing that the mean error is mathematically distinct from the standard deviation. References [1] Ghahramani, Saeed (2000). Fundamentals of Probability (2nd Edition). Prentice Hall: New Jersey. p. 438. External links • Hazewinkel, Michiel, ed. (2001), "Quadratic deviation" (http://www.encyclopediaofmath.org/index. php?title=p/q076030), Encyclopedia of Mathematics, Springer, ISBN 978-1-55608-010-4 • A simple way to understand Standard Deviation (http://standard-deviation.appspot.com/) • Standard Deviation – an explanation without maths (http://www.techbookreport.com/tutorials/stddev-30-secs. html) • Standard Deviation, an elementary introduction (http://davidmlane.com/hyperstat/A16252.html) • Standard Deviation while Financial Modeling in Excel (http://www.edupristine.com/blog/ what-is-standard-deviation) • Standard Deviation, a simpler explanation for writers and journalists (http://www.robertniles.com/stats/stdev. shtml) • The concept of Standard Deviation is shown in this 8-foot-tall (2.4 m) Probability Machine (named Sir Francis) comparing stock market returns to the randomness of the beans dropping through the quincunx pattern. (http:// www.youtube.com/watch?v=AUSKTk9ENzg) from Index Funds Advisors IFA.com (http://www.ifa.com) 14 Samuelson's inequality 15 Samuelson's inequality In statistics, Samuelson's inequality, named after the economist Paul Samuelson,[1] also called the Laguerre–Samuelson inequality,[2] after the mathematician Edmond Laguerre, proved that every one of any collection x1, ..., xn, is within √(n − 1) sample standard deviations of their sample mean. In other words, if we let be the sample mean and be the standard deviation of the sample, then [3] Equality holds on the left if and only if the n − 1 smallest of the n numbers are equal to each other, and on the right iff the n − 1 largest ones are equal. Samuelson's inequality may be considered a reason why studentization of residuals should be done externally. Relationship to polynomials Samuelson was not the first to describe this relationship. The first to discover this relationship was probably Laguerre in 1880 while investigating the roots (zeros) of polynomials.[4][5] Consider a polynomial Without loss of generality let and let and Then and In terms of the coefficients Laguerre showed that the roots of this polynomial were bounded by where Inspection shows that is the mean of the roots and that b is the standard deviation of the roots. Laguerre failed to notice this relationship with the means and standard deviations of the roots being more interested in the bounds themselves. This relationship permits a rapid estimate of the bounds of the roots and may be of use in their location. Samuelson's inequality 16 Note When the coefficients and are both zero no information can be obtained about the location of the roots. References [1] Paul Samuelson, "How Deviant Can You Be?", Journal of the American Statistical Association, volume 63, number 324 (December, 1968), pp. 1522–1525 [2] Jensen, Shane Tyler (1999) The Laguerre–Samuelson Inequality with Extensions and Applications in Statistics and Matrix Theory (http:/ / www. collectionscanada. gc. ca/ obj/ s4/ f2/ dsk1/ tape10/ PQDD_0027/ MQ50799. pdf) MSc Thesis. Department of Mathematics and Statistics, McGill University. [3] Advances in Inequalities from Probability Theory and Statistics, by Neil S. Barnett and Sever Silvestru Dragomir, Nova Publishers, 2008, page 164 [4] Jensen, Shane Tyler (1999) The Laguerre–Samuelson Inequality with Extensions and Applications in Statistics and Matrix Theory (http:/ / www. collectionscanada. gc. ca/ obj/ s4/ f2/ dsk1/ tape10/ PQDD_0027/ MQ50799. pdf) MSc Thesis. Department of Mathematics and Statistics, McGill University [5] Laguerre E. (1880) Mémoire pour obtenir par approximation les racines d'une équation algébrique qui a toutes les racines réelles. Nouv Ann Math 2e série, 19, 161-172, 193-202 Distance correlation In statistics and in probability theory, distance correlation is a measure of statistical dependence between two random variables or two random vectors of arbitrary, not necessarily equal dimension. An important property is that this measure of dependence is zero if and only if the random variables are statistically independent. This measure is derived from a number of other quantities that are used in its specification, specifically: distance variance, distance standard deviation and distance covariance. These take the same roles as the ordinary moments with corresponding names in the specification of the Pearson product-moment correlation coefficient. These distance-based measures can be put into an indirect relationship to the ordinary moments by an alternative formulation (described below) using ideas related to Brownian motion, and this has led to the use of names such as Brownian covariance and Brownian distance covariance. Background The classical measure of dependence, the Pearson correlation coefficient,[1] is mainly sensitive to a linear relationship between two variables. Distance correlation was introduced in 2005 by Gabor J Szekely in several lectures to address this deficiency of Pearson’s correlation, namely that it can easily be zero for dependent variables. Correlation = 0 (uncorrelatedness) does not imply independence while distance correlation = 0 does imply Several sets of (x, y) points, with the Distance correlation coefficient of x and y for each set. Compare to the graph on correlation independence. The first results on distance correlation were published in 2007 and 2009.[2][3] It was proved that distance covariance is the same as the Brownian covariance. These measures are examples of energy distances. Distance correlation 17 Definitions Distance covariance Let us start with the definition of the sample distance covariance. Let (Xk, Yk), k= 1, 2, ..., n be a statistical sample from a pair of real valued or vector valued random variables (X, Y). First, compute all pairwise distances where || ⋅ || denotes Euclidean norm. That is, compute the n by n distance matrices (aj, k) and (bj, k). Then take all doubly centered distances where is the j-th row mean, is thek-th column mean, and is the grand mean of the distance matrix of the X sample. The notation is similar for the b values. (In the matrices of centered distances (Aj, k) and (Bj,k) all rows and all columns sum to zero.) The squared sample distance covariance is simply the arithmetic average of the products Aj, kBj, k: The statistic Tn = n dCov2n(X, Y) determines a consistent multivariate test of independence of random vectors in arbitrary dimensions. For an implementation see dcov.test function in the energy package for R.[4] The population value of distance covariance can be defined along the same lines. Let X be a random variable that takes values in a p-dimensional Euclidean space with probability distribution μ and let Y be a random variable that takes values in a q-dimensional Euclidean space with probability distribution ν, and suppose that X and Y have finite expectations. Write Finally, define the population value of squared distance covariance of X and Y as One can show that this is equivalent to the following definition: where E denotes expected value, and and are independent and identically distributed. Distance covariance can be expressed in terms of Pearson’s covariance, cov, as follows: This identity shows that the distance covariance is not the same as the covariance of distances, cov(||X-X' ||, ||Y-Y' ||). This can be zero even if X and Y are not independent. Alternately, the squared distance covariance can be defined as the weighted L2 norm of the distance between the joint characteristic function of the random variables and the product of their marginal characteristic functions:[5] where ϕX, Y(s, t), ϕX(s), and ϕY(t) are the characteristic functions of (X, Y), X, and Y, respectively, p, q denote the Euclidean dimension of X and Y, and thus of s and t, and cp, cq are constants. The weight function is chosen to produce a scale equivariant and rotation invariant measure that doesn't go to Distance correlation 18 zero for dependent variables. One interpretation of the characteristic function definition is that the variables eisX and eitY are cyclic representations of X and Y with different periods given by s and t, and the expression ϕX, Y(s, t) - ϕX(s) ϕY(t) in the numerator of the characteristic function definition of distance covariance is simply the classical covariance of eisX and eitY. The characteristic function definition clearly shows that dCov2(X, Y) = 0 if and only if X and Y are independent. Distance variance The distance variance is a special case of distance covariance when the two variables are identical. The population value of distance variance is the square root of where denotes the expected value, independent of and is an independent and identically distributed copy of and has the same distribution as and and is . The sample distance variance is the square root of which is a relative of Corrado Gini’s mean difference introduced in 1912 (but Gini did not work with centered distances). Distance standard deviation The distance standard deviation is the square root of the distance variance. Distance correlation The distance correlation of two random variables is obtained by dividing their distance covariance by the product of their distance standard deviations. The distance correlation is and the sample distance correlation is defined by substituting the sample distance covariance and distance variances for the population coefficients above. For easy computation of sample distance correlation see the dcor function in the energy package for R. Properties Distance correlation (i) and (ii) if and only if (iii) . and are independent. implies that dimensions of the linear spaces spanned by and are almost surely equal and if we assume that these subspaces are equal, then in this subspace some vector , scalar , and orthonormal matrix . samples respectively for Distance correlation 19 Distance covariance (i) and . (ii) for all constant vectors , and orthonormal matrices (iii) If the random vectors Equality holds if and only if , scalars . and and are independent then are both constants, or and are both constants, or are mutually independent. (iv) if and only if and are independent. This last property is the most important effect of working with centered distances. The statistic . Under independence of X and Y [6] is a biased estimator of Distance variance (i) if and only if (ii) almost surely. if and only if every sample observation is identical. (iii) (iv) If for all constant vectors and , scalars , and orthonormal matrices are independent then . . Equality holds in (iv) if and only if one of the random variables or is a constant. Generalization Distance covariance can be generalized to include powers of Euclidean distance. Define Then for every , and are independent if and only if note that this characterization does not hold for exponent . It is important to ; in this case for bivariate [2] is a deterministic function of the Pearson correlation. the corresponding distances, , then If and are , powers of sample distance covariance can be defined as the nonnegative number for which One can extend with metric to metric-space-valued random variables , then define , has finite first moment), and : If has law , and (provided . Then if in a metric space is finite, i.e., has law (in a possibly different metric space with finite first moment), define This is non-negative for all such has negative type if negative type, then iff both metric spaces have negative type.[7] Here, a metric space is isometric to a subset of a Hilbert space.[8] If both metric spaces have strong iff are independent. Distance correlation Alternative formulation: Brownian covariance Brownian covariance is motivated by generalization of the notion of covariance to stochastic processes. The square of the covariance of random variables X and Y can be written in the following form: where E denotes the expected value and the prime denotes independent and identically distributed copies. We need the following generalization of this formula. If U(s), V(t) are arbitrary random processes defined for all real s and t then define the U-centered version of X by whenever the subtracted conditional expected value exists and denote by YV the V-centered version of Y.[9][10] The (U,V) covariance of (X,Y) is defined as the nonnegative number whose square is whenever the right-hand side is nonnegative and finite. The most important example is when U and V are two-sided independent Brownian motions /Wiener processes with expectation zero and covariance |s| + |t| - |s-t| = 2 min(s,t). (This is twice the covariance of the standard Wiener process; here the factor 2 simplifies the computations.) In this case the (U,V) covariance is called Brownian covariance and is denoted by There is a surprising coincidence: The Brownian covariance is the same as the distance covariance: and thus Brownian correlation is the same as distance correlation. On the other hand, if we replace the Brownian motion with the deterministic identity function id then Covid(X,Y) is simply the absolute value of the classical Pearson covariance, Notes [1] Pearson (1895) [2] Székely, Rizzo and Bakirov (2007) [3] Székely & Rizzo (2009) [4] energy package for R (http:/ / cran. us. r-project. org/ web/ packages/ energy/ index. html) [5] Székely & Rizzo (2009) Theorem 7, (3.7), p. 1249. [6] Székely and Rizzo (2009), Rejoinder [7] Lyons, R. (2011) "Distance covariance in metric spaces". [8] Klebanov, L. B. (2005) N-distances and their Applications, Karolinum Press, Charles University, Prague. [9] Bickel & Xu (2009) [10] Kosorok (2009) 20 Distance correlation 21 References • Bickel, P.J. and Xu, Y. (2009) "Discussion of: Brownian distance covariance", Annals of Applied Statistics, 3 (4), 1266–1269. doi: 10.1214/09-AOAS312A (http://dx.doi.org/10.1214/09-AOAS312A) Free access to article (http://arxiv4.library.cornell.edu/PS_cache/arxiv/pdf/0912/0912.3295v2.pdf) • Gini, C. (1912). Variabilità e Mutabilità. Bologna: Tipografia di Paolo Cuppini. • Pearson, K. (1895). "Note on regression and inheritance in the case of two parents", Proceedings of the Royal Society, 58, 240–242 • Pearson, K. (1920). "Notes on the history of correlation", Biometrika, 13, 25–45. • Székely, G. J. Rizzo, M. L. and Bakirov, N. K. (2007). "Measuring and testing independence by correlation of distances", Annals of Statistics, 35/6, 2769–2794. doi: 10.1214/009053607000000505 (http://dx.doi.org/10. 1214/009053607000000505) Reprint (http://personal.bgsu.edu/~mrizzo/energy/AOS0283-reprint.pdf) • Székely, G. J. and Rizzo, M. L. (2009). "Brownian distance covariance", Annals of Applied Statistics, 3/4, 1233–1303. doi: 10.1214/09-AOAS312 (http://dx.doi.org/10.1214/09-AOAS312) Reprint (http://personal. bgsu.edu/~mrizzo/energy/AOAS312.pdf) • Kosorok, M. R. (2009) "Discussion of: Brownian Distance Covariance", Annals of Applied Statistics, 3/4, 1270–1278. doi: 10.1214/09-AOAS312B (http://dx.doi.org/10.1214/09-AOAS312B) Free access to article (http://arxiv.org/PS_cache/arxiv/pdf/1010/1010.0822v1.pdf) External links • E-statistics (energy statistics) (http://personal.bgsu.edu/~mrizzo/energy.htm) Geometric standard deviation In probability theory and statistics, the geometric standard deviation describes how spread out are a set of numbers whose preferred average is the geometric mean. For such data, it may be preferred to the more usual standard deviation. Note that unlike the usual arithmetic standard deviation, the geometric standard deviation is a multiplicative factor, and thus is dimensionless, rather than having the same dimension as the input values. Definition If the geometric mean of a set of numbers {A1, A2, ..., An} is denoted as μg, then the geometric standard deviation is Derivation If the geometric mean is then taking the natural logarithm of both sides results in The logarithm of a product is a sum of logarithms (assuming is positive for all ), so Geometric standard deviation It can now be seen that 22 is the arithmetic mean of the set , therefore the arithmetic standard deviation of this same set should be This simplifies to Geometric standard score The geometric version of the standard score is If the geometric mean, standard deviation, and z-score of a datum are known, then the raw score can be reconstructed by Relationship to log-normal distribution The geometric standard deviation is related to the log-normal distribution. The log-normal distribution is a distribution which is normal for the logarithm transformed values. By a simple set of logarithm transformations we see that the geometric standard deviation is the exponentiated value of the standard deviation of the log transformed values (e.g. exp(stdev(ln(A)))); As such, the geometric mean and the geometric standard deviation of a sample of data from a log-normally distributed population may be used to find the bounds of confidence intervals analogously to the way the arithmetic mean and standard deviation are used to bound confidence intervals for a normal distribution. See discussion in log-normal distribution for details. Bregman divergence 23 Bregman divergence In mathematics, a Bregman divergence or Bregman distance is similar to a metric, but does not satisfy the triangle inequality nor symmetry. There are two ways in which Bregman divergences are important. Firstly, they generalize squared Euclidean distance to a class of distances that all share similar properties. Secondly, they bear a strong connection to exponential families of distributions; as has been shown by (Banerjee et al. 2005), there is a bijection between regular exponential families and regular Bregman divergences. Bregman divergences are named after L. M. Bregman, who introduced the concept in 1967. More recently researchers in geometric algorithms have shown that many important algorithms can be generalized from Euclidean metrics to distances defined by Bregman divergence (Banerjee et al. 2005; Nielsen and Nock 2006; Boissonnat et al. 2010). Definition Let be a continuously-differentiable real-valued and strictly convex function defined on a closed convex set . The Bregman distance associated with F for points is the difference between the value of F at point p and the value of the first-order Taylor expansion of F around point q evaluated at point p: Properties • Non-negativity: for all p, q. This is a consequence of the convexity of F. • Convexity: is convex in its first argument, but not necessarily in the second argument (see [1]) • Linearity: If we think of the Bregman distance as an operator on the function F, then it is linear with respect to non-negative coefficients. In other words, for strictly convex and differentiable, and , • Duality: The function F has a convex conjugate . The Bregman distance defined with respect to has an interesting relationship to Here, is the dual point corresponding to p • A key result about Bregman divergences is that, given a random vector, the mean vector minimizes the expected Bregman divergence from the random vector. This result generalizes the textbook result that the mean of a set minimizes total squared error to elements in the set. This result was proved for the vector case by (Banerjee et al. 2005), and extended to the case of functions/distributions by (Frigyik et al. 2008). This result is important because it further justifies using a mean as a representative of a random set, particularly in Bayesian estimation. Bregman divergence 24 Examples • Squared Euclidean distance is the canonical example of a Bregman distance, generated by the convex function • The squared Mahalanobis distance, which is generated by the convex function . This can be thought of as a generalization of the above squared Euclidean distance. • The generalized Kullback–Leibler divergence is generated by the convex function • The Itakura–Saito distance, is generated by the convex function Generalizing projective duality A key tool in computational geometry is the idea of projective duality, which maps points to hyperplanes and vice versa, while preserving incidence and above-below relationships. There are numerous analytical forms of the projective dual: one common form maps the point to the hyperplane . This mapping can be interpreted (identifying the hyperplane with its normal) as the convex conjugate mapping that takes the point p to its dual point , where F defines the d-dimensional paraboloid . If we now replace the paraboloid by an arbitrary convex function, we obtain a different dual mapping that retains the incidence and above-below properties of the standard projective dual. This implies that natural dual concepts in computational geometry like Voronoi diagrams and Delaunay triangulations retain their meaning in distance spaces defined by an arbitrary Bregman divergence. Thus, algorithms from "normal" geometry extend directly to these spaces (Boissonnat, Nielsen and Nock, 2010) Matrix Bregman divergences, functional Bregman divergences and the submodular Bregman divergences Bregman divergences can also be defined between matrices, between functions, and between measures (distributions). Bregman divergences between matrices include the Stein's loss and von Neumann entropy. Bregman divergences between functions include total squared error, relative entropy, and squared bias; see the references by Frigyik et al. below for definitions and properties. Similarly Bregman divergences have also been defined over sets, through a submodular set function which is known as the discrete analog of a convex function. The submodular Bregman divergences subsume a number of discrete distance measures, like the Hamming distance, precision and recall, mutual information and some other set based distance measures (see Iyer & Bilmes, 2012) for more details and properties of the submodular Bregman.) For a list of common matrix Bregman divergences, see Table 15.1 in.[2] Bregman divergence References [1] "Joint and separate convexity of the Bregman Distance", by H. Bauschke and J. Borwein, in D. Butnariu, Y. Censor, and S. Reich, editors, Inherently Parallel Algorithms in Feasibility and Optimization and their Applications, Elsevier 2001 [2] "Matrix Information Geometry", R. Nock, B. Magdalou, E. Briys and F. Nielsen, pdf (http:/ / www1. univ-ag. fr/ ~rnock/ Articles/ Drafts/ book12-nmbn. pdf), from this book (http:/ / www. springerlink. com/ index/ 10. 1007/ 978-3-642-30232-9) • Banerjee, Arindam; Merugu, Srujana; Dhillon, Inderjit S.; Ghosh, Joydeep (2005). "Clustering with Bregman divergences" (http://jmlr.csail.mit.edu/papers/v6/banerjee05b.html). Journal of Machine Learning Research 6: 1705–1749. • Bregman, L. M. (1967). "The relaxation method of finding the common points of convex sets and its application to the solution of problems in convex programming". USSR Computational Mathematics and Mathematical Physics 7 (3): 200–217. doi: 10.1016/0041-5553(67)90040-7 (http://dx.doi.org/10.1016/ 0041-5553(67)90040-7). • Frigyik, Bela A.; Srivastava, Santosh; Gupta, Maya R. (2008). "Functional Bregman Divergences and Bayesian Estimation of Distributions" (http://www.ee.washington.edu/research/guptalab/publications/ FrigyikSrivastavaGupta.pdf). IEEE Transactions on Information Theory 54 (11): 5130–5139. doi: 10.1109/TIT.2008.929943 (http://dx.doi.org/10.1109/TIT.2008.929943). • Iyer, Rishabh.; Bilmes, Jeff (2012). "The Submodular Bregman divergences and Lovasz Bregman divergences with Applications". Conference on Neural Information Processing Systems. • Frigyik, Bela A.; Srivastava, Santosh; Gupta, Maya R. (2008). An Introduction to Functional Derivatives (http:// www.ee.washington.edu/research/guptalab/publications/functionalDerivativesIntroduction.pdf). UWEE Tech Report 2008-0001. University of Washington, Dept. of Electrical Engineering. • Nielsen, Frank; Nock, Richard (2009). "The dual Voronoi diagrams with respect to representational Bregman divergences" (http://www.lix.polytechnique.fr/~nielsen/ISVD09-GenBregmanVD.pdf). Proc. 6th International Symposium on Voronoi Diagrams. IEEE. doi: 10.1109/ISVD.2009.15 (http://dx.doi.org/10. 1109/ISVD.2009.15). • Nielsen, Frank; Nock, Richard (2007). "On the Centroids of Symmetrized Bregman Divergences". arXiv: 0711.3242 (http://arxiv.org/abs/0711.3242) [ cs.CG (http://arxiv.org/archive/cs.CG)]. • Nielsen, Frank; Boissonnat, Jean-Daniel; Nock, Richard (2007). "On Visualizing Bregman Voronoi diagrams" (http://www.csl.sony.co.jp/person/nielsen/PT/SoCG07/). Proc. 23rd ACM Symposium on Computational Geometry (video track). • Boissonnat, Jean-Daniel; Nielsen, Frank; Nock, Richard (2010). "Bregman Voronoi Diagrams" (http://hal. archives-ouvertes.fr/hal-00488441/en/). Discrete and Computational Geometry. 44 (2). • Nielsen, Frank; Nock, Richard (2006). "On approximating the smallest enclosing Bregman Balls". Proc. 22nd ACM Symposium on Computational Geometry. pp. 485–486. doi: 10.1145/1137856.1137931 (http://dx.doi.org/ 10.1145/1137856.1137931). External links • Bregman divergence interactive applet (http://www.csl.sony.co.jp/person/nielsen/BregmanDivergence/ index.html) • Bregman Voronoi diagram applet (http://www.csl.sony.co.jp/person/nielsen/BVDapplet/index.html) • Exact Smallest Enclosing Bregman Ball applet (http://www.sonycsl.co.jp/person/nielsen/BregmanBall/ MINIBALL/) • Approximating smallest enclosing Bregman ball applet (http://www.sonycsl.co.jp/person/nielsen/ BregmanBall/BBC/index.html) • Sided and Symmetrized Bregman centroids (http://www.sonycsl.co.jp/person/nielsen/BregmanCentroids/) 25 Bhattacharyya distance 26 Bhattacharyya distance In statistics, the Bhattacharyya distance measures the similarity of two discrete or continuous probability distributions. It is closely related to the Bhattacharyya coefficient which is a measure of the amount of overlap between two statistical samples or populations. Both measures are named after A. Bhattacharya, a statistician who worked in the 1930s at the Indian Statistical Institute. The coefficient can be used to determine the relative closeness of the two samples being considered. It is used to measure the separability of classes in classification and it is considered to be more reliable than the Mahalanobis distance, as the Mahalanobis distance is a particular case of the Bhattacharyya distance when the standard deviations of the two classes are the same. Therefore, when two classes have the similar means but different standard deviations, the Mahalanobis distance would tend to zero, however, the Bhattacharyya distance would grow depending on the difference between the standard deviations. Definition For discrete probability distributions p and q over the same domain X, it is defined as: where: is the Bhattacharyya coefficient. For continuous probability distributions, the Bhattacharyya coefficient is defined as: In either case, distance and . does not obey the triangle inequality, but the Hellinger does obey the triangle inequality. In its simplest formulation, the Bhattacharyya distance between two classes under the normal distribution can be calculated [1] by extracting the mean and variances of two separate distributions or classes: where: is the Bhattacharyya distance between p and q distributions or classes, is the variance of the p-th distribution, is the mean of the p-th distribution, and are two different distributions. The Mahalanobis distance used in Fisher's Linear discriminant analysis is a particular case of the Bhattacharyya Distance. When the variances of the two distributions are the same the first term of the distance is zero as this term depends solely on the variances of the distributions (left case of the figure). The first term will grow as the variances differ (right case of the figure). The second term, on the other hand, will be zero if the means are equal and is inversely proportional to the variances. For multivariate normal distributions , , where and are the means and covariances of the distributions, and Bhattacharyya distance 27 . Note that, in this case, the first term in the Bhattacharyya distance is related to the Mahalanobis distance. Bhattacharyya coefficient The Bhattacharyya coefficient is an approximate measurement of the amount of overlap between two statistical samples. The coefficient can be used to determine the relative closeness of the two samples being considered. Calculating the Bhattacharyya coefficient involves a rudimentary form of integration of the overlap of the two samples. The interval of the values of the two samples is split into a chosen number of partitions, and the number of members of each sample in each partition is used in the following formula, [2] where considering the samples a and b, n is the number of partitions, and , are the number of members of samples a and b in the i'th partition. This formula hence is larger with each partition that has members from both sample, and larger with each partition that has a large overlap of the two sample's members within it. The choice of number of partitions depends on the number of members in each sample; too few partitions will lose accuracy by overestimating the overlap region, and too many partitions will lose accuracy by creating individual partitions with no members despite being in a surroundingly populated sample space. The Bhattacharyya coefficient will be 0 if there is no overlap at all due to the multiplication by zero in every partition. This means the distance between fully separated samples will not be exposed by this coefficient alone. Applications The Bhattacharyya distance is widely used in research of feature extraction and selection,[3] image processing,[4] speaker recognition,[5] phone clustering.[6] A "Bhattacharyya Space" has been proposed as a feature selection technique that can be applied to texture segmentation.[7] A "Bhattacharyya coefficient" has also been proposed as a feature selection technique that can be used to estimate the given distance between the Indraneel Bhattacharyya number and any given Nesli coordinate.[7] References [1] Guy B. Coleman, Harry C. Andrews, Image Segmentation by Clustering, Proc IEEE, Vol. 67, No. 5, pp. 773-785,1979 [2] D. Comaniciu, V. Ramesh, P. Meer: Real-Time Tracking of Non-Rigid Objects using Mean Shift (http:/ / coewww. rutgers. edu/ riul/ research/ papers/ pdf/ trackmo. pdf), BEST PAPER AWARD, IEEE Conf. Computer Vision and Pattern Recognition (CVPR'00), Hilton Head Island, South Carolina, Vol. 2, 142-149, 2000 [3] Euisun Choi, Chulhee Lee, Feature extraction based on the Bhattacharyya distance, Pattern Recognition, Volume 36, Issue 8, August 2003, Pages 1703–1709 [4] François Goudail, Philippe Réfrégier, Guillaume Delyon, Bhattacharyya distance as a contrast parameter for statistical processing of noisy optical images, JOSA A, Vol. 21, Issue 7, pp. 1231−1240 (2004) [5] Chang Huai You, An SVM Kernel With GMM-Supervector Based on the Bhattacharyya Distance for Speaker Recognition, Signal Processing Letters, IEEE, Vol 16, Is 1, pp. 49 - 52 [6] Mak, B., Phone clustering using the Bhattacharyya distance, Spoken Language, 1996. ICSLP 96. Proceedings., Fourth International Conference on, Vol 4, pp. 2005 - 2008 vol.4, 3−6 Oct 1996 [7] Reyes-Aldasoro, C.C., and A. Bhalerao, The Bhattacharyya space for feature selection and its application to texture segmentation, Pattern Recognition, (2006) Vol. 39, Issue 5, May 2006, pp. 812-826 • Nielsen, F.; Boltz, S. (2010). "The Burbea-Rao and Bhattacharyya centroids". IEEE Transactions on Information Theory 57 (8): 5455–5466. arXiv: 1004.5049 (http://arxiv.org/abs/1004.5049). doi: Bhattacharyya distance 10.1109/TIT.2011.2159046 (http://dx.doi.org/10.1109/TIT.2011.2159046). • Bhattacharyya, A. (1943). "On a measure of divergence between two statistical populations defined by their probability distributions". Bulletin of the Calcutta Mathematical Society 35: 99–109. MR 0010358 (http://www. ams.org/mathscinet-getitem?mr=0010358). • Kailath, T. (1967). "The Divergence and Bhattacharyya Distance Measures in Signal Selection". IEEE Transactions on Communication Technology 15 (1): 52–60. doi: 10.1109/TCOM.1967.1089532 (http://dx.doi. org/10.1109/TCOM.1967.1089532). • Djouadi, A.; Snorrason, O.; Garber, F. (1990). "The quality of Training-Sample estimates of the Bhattacharyya coefficient". IEEE Transactions on Pattern Analysis and Machine Intelligence 12 (1): 92–97. doi: 10.1109/34.41388 (http://dx.doi.org/10.1109/34.41388). • For a short list of properties, see: http://www.mtm.ufsc.br/~taneja/book/node20.html External links • Hazewinkel, Michiel, ed. (2001), "Bhattacharyya distance" (http://www.encyclopediaofmath.org/index. php?title=p/b110490), Encyclopedia of Mathematics, Springer, ISBN 978-1-55608-010-4 Hellinger distance In probability and statistics, the Hellinger distance is used to quantify the similarity between two probability distributions. It is a type of f-divergence. The Hellinger distance is defined in terms of the Hellinger integral, which was introduced by Ernst Hellinger in 1909. Definition Measure theory To define the Hellinger distance in terms of measure theory, let P and Q denote two probability measures that are absolutely continuous with respect to a third probability measure λ. The square of the Hellinger distance between P and Q is defined as the quantity Here, dP / dλ and dQ / dλ are the Radon–Nikodym derivatives of P and Q respectively. This definition does not depend on λ, so the Hellinger distance between P and Q does not change if λ is replaced with a different probability measure with respect to which both P and Q are absolutely continuous. For compactness, the above formula is often written as 28 Hellinger distance 29 Probability theory using Lebesgue measure To define the Hellinger distance in terms of elementary probability theory, we take λ to be Lebesgue measure, so that dP / dλ and dQ / dλ are simply probability density functions. If we denote the densities as f and g, respectively, the squared Hellinger distance can be expressed as a standard calculus integral where the second form can be obtained by expanding the square and using the fact that the integral of a probability density over its domain must be one. The Hellinger distance H(P, Q) satisfies the property (derivable from the Cauchy-Schwarz inequality) Discrete distributions For two discrete probability distributions and , their Hellinger distance is defined as which is directly related to the Euclidean norm of the difference of the square root vectors, i.e. Connection with the statistical distance The Hellinger distance and the total variation distance (or statistical distance) are related as [1] follows: These inequalities follow immediately from the inequalities between the 1-norm and the 2-norm. Properties The maximum distance 1 is achieved when P assigns probability zero to every set to which Q assigns a positive probability, and vice versa. Sometimes the factor 1/2 in front of the integral is omitted, in which case the Hellinger distance ranges from zero to the square root of two. The Hellinger distance is related to the Bhattacharyya coefficient as it can be defined as Hellinger distances are used in the theory of sequential and asymptotic statistics.[2] Hellinger distance 30 Examples The squared Hellinger distance between two normal distributions The squared Hellinger distance between two exponential distributions The squared Hellinger distance between two Weibull distributions common shape parameter and and is: and and (where is a are the scale parameters respectively): The squared Hellinger distance between two Poisson distributions with rate parameters and is: and , so that , is: Notes [1] Harsha's lecture notes on communication complexity (http:/ / www. tcs. tifr. res. in/ ~prahladh/ teaching/ 2011-12/ comm/ lectures/ l12. pdf) [2] Erik Torgerson (1991) Comparison of Statistical Experiments, volume 36 of Encyclopedia of Mathematics. Cambridge University Press. References • Yang, Grace Lo; Le Cam, Lucien M. (2000). Asymptotics in Statistics: Some Basic Concepts. Berlin: Springer. ISBN 0-387-95036-2. • Vaart, A. W. van der. Asymptotic Statistics (Cambridge Series in Statistical and Probabilistic Mathematics). Cambridge, UK: Cambridge University Press. ISBN 0-521-78450-6. • Pollard, David E. (2002). A user's guide to measure theoretic probability. Cambridge, UK: Cambridge University Press. ISBN 0-521-00289-3. Mahalanobis distance 31 Mahalanobis distance The Mahalanobis distance is a descriptive statistic that provides a relative measure of a data point's distance (residual) from a common point. It is a unitless measure introduced by P. C. Mahalanobis in 1936. The Mahalanobis distance is used to identify and gauge similarity of an unknown sample set to a known one. It differs from Euclidean distance in that it takes into account the correlations of the data set and is scale-invariant. In other words, it has a multivariate effect size. Definition The Mahalanobis distance of a multivariate vector from a group of values with mean and covariance matrix S is defined as: [1] Mahalanobis distance (or "generalized squared interpoint distance" for its squared value[2]) can also be defined as a dissimilarity measure between two random vectors and of the same distribution with the covariance matrix S: If the covariance matrix is the identity matrix, the Mahalanobis distance reduces to the Euclidean distance. If the covariance matrix is diagonal, then the resulting distance measure is called a normalized Euclidean distance: where si is the standard deviation of the xi and yi over the sample set. Intuitive explanation Consider the problem of estimating the probability that a test point in N-dimensional Euclidean space belongs to a set, where we are given sample points that definitely belong to that set. Our first step would be to find the average or center of mass of the sample points. Intuitively, the closer the point in question is to this center of mass, the more likely it is to belong to the set. However, we also need to know if the set is spread out over a large range or a small range, so that we can decide whether a given distance from the center is noteworthy or not. The simplistic approach is to estimate the standard deviation of the distances of the sample points from the center of mass. If the distance between the test point and the center of mass is less than one standard deviation, then we might conclude that it is highly probable that the test point belongs to the set. The further away it is, the more likely that the test point should not be classified as belonging to the set. This intuitive approach can be made quantitative by defining the normalized distance between the test point and the set to be . By plugging this into the normal distribution we can derive the probability of the test point belonging to the set. The drawback of the above approach was that we assumed that the sample points are distributed about the center of mass in a spherical manner. Were the distribution to be decidedly non-spherical, for instance ellipsoidal, then we would expect the probability of the test point belonging to the set to depend not only on the distance from the center of mass, but also on the direction. In those directions where the ellipsoid has a short axis the test point must be closer, while in those where the axis is long the test point can be further away from the center. Putting this on a mathematical basis, the ellipsoid that best represents the set's probability distribution can be estimated by building the covariance matrix of the samples. The Mahalanobis distance is simply the distance of the Mahalanobis distance 32 test point from the center of mass divided by the width of the ellipsoid in the direction of the test point. Discussion In general, given a normal (Gaussian) random variable normal random variable can be defined in terms of with variance and mean by the equation , any other Conversely, to recover a normalized random variable from any normal random variable, one can typically solve for . If we square both sides, and take the square-root, we will get an equation for a metric that looks a lot like the Mahalanobis distance: The resulting magnitude is always non-negative and varies with the distance of the data from the mean, attributes that are convenient when trying to define a model for the data. Relationship to leverage Mahalanobis distance is closely related to the leverage statistic, h, but has a different scale:[3] Squared Mahalanobis distance = (N − 1)(h − 1/N). Applications Mahalanobis's discovery was prompted by the problem of identifying the similarities of skulls based on measurements in 1927.[4] Mahalanobis distance is widely used in cluster analysis and classification techniques. It is closely related to Hotelling's T-square distribution used for multivariate statistical testing and Fisher's Linear Discriminant Analysis that is used for supervised classification.[5] In order to use the Mahalanobis distance to classify a test point as belonging to one of N classes, one first estimates the covariance matrix of each class, usually based on samples known to belong to each class. Then, given a test sample, one computes the Mahalanobis distance to each class, and classifies the test point as belonging to that class for which the Mahalanobis distance is minimal. Mahalanobis distance and leverage are often used to detect outliers, especially in the development of linear regression models. A point that has a greater Mahalanobis distance from the rest of the sample population of points is said to have higher leverage since it has a greater influence on the slope or coefficients of the regression equation. Mahalanobis distance is also used to determine multivariate outliers. Regression techniques can be used to determine if a specific case within a sample population is an outlier via the combination of two or more variable scores. A point can be a multivariate outlier even if it is not a univariate outlier on any variable (consider a probability density similar to a hollow cube in three dimensions, for example). Mahalanobis distance was also widely used in biology, such as predicting protein structural class, predicting membrane protein type,[6] predicting protein subcellular localization,[7]as well as predicting many other attributes of proteins through their pseudo amino acid composition or Chou's PseAAC, based on Chou's invariance theorem, as done in the papers. Mahalanobis distance 33 References [1] De Maesschalck, Roy; Jouan-Rimbaud, Delphine; and Massart, Désiré L. (2000); The Mahalanobis distance, Chemometrics and Intelligent Laboratory Systems 50:1–18 [2] Gnanadesikan, Ramanathan; and Kettenring, John R. (1972); Robust estimates, residuals, and outlier detection with multiresponse data, Biometrics 28:81–124 [3] Schinka, John A.; Velicer, Wayne F.; and Weiner, Irving B. (2003); Handbook of psychology: Research methods in psychology, John Wiley and Sons [4] Mahalanobis, Prasanta Chandra (1927); Analysis of race mixture in Bengal, Journal and Proceedings of the Asiatic Society of Bengal, 23:301–333 [5] McLachlan, Geoffrey J. (1992); Discriminant Analysis and Statistical Pattern Recognition, Wiley Interscience, p. 12. ISBN 0-471-69115-1 [6] Chou, Kuo-Chen; and Elrod, David W. (1999); Prediction of membrane protein types and subcellular locations, Proteins: Structure, Function, and Genetics, 34, 137–153 [7] Chou, Kuo-Chen; and Elrod, David W. (1999); Protein subcellular location prediction, Protein Engineering, 12, 107–118 External links • Hazewinkel, Michiel, ed. (2001), "Mahalanobis distance" (http://www.encyclopediaofmath.org/index. php?title=p/m062130), Encyclopedia of Mathematics, Springer, ISBN 978-1-55608-010-4 • Mahalanobis distance tutorial (http://people.revoledu.com/kardi/tutorial/Similarity/MahalanobisDistance. html) – interactive online program and spreadsheet computation • Mahalanobis distance (Nov-17-2006) (http://matlabdatamining.blogspot.com/2006/11/mahalanobis-distance. html) – overview of Mahalanobis distance, including MATLAB code • What is Mahalanobis distance? (http://blogs.sas.com/content/iml/2012/02/15/what-is-mahalanobis-distance/ ) – intuitive, illustrated explanation, from Rick Wicklin on blogs.sas.com Similarity learning Similarity learning is one type of a supervised machine learning task in artificial intelligence. It is closely related to regression and classification, but the goal is to learn from examples a function that measure how similar or related two objects are. It has applications in ranking and in recommendation systems. Learning setup There are three common setups for similarity and metric distance learning. • Regression similarity learning. In this setup, pairs of objects are given similarity . The goal is to learn a function that approximates triplet example together with a measure of their for every new labeled . This is typically achieved by minimizing a regularized loss . • Classification similarity learning. Given are pairs of similar objects An equivalent formulation is that every pair and non similar objects . is given together with a binary label that determines if the two objects are similar or not. The goal is again to learn a classifier that can decide if a new pair of objects is similar or not. • Ranking similarity learning. Given are triplets of objects predefined order: is known to be more similar to for any new triplet of objects , it obeys than to whose relative similarity obey a . The goal is to learn a function such that . This setup assumes a weaker form of supervision than in regression, because instead of providing an exact measure of similarity, one only has to provide the relative order of similarity. For this reason, ranking-based similarity learning is easier to apply in real large scale applications. Similarity learning 34 A common approach for learning similarity, is to model the similarity function as a bilinear form. For example, in the case of ranking similarity learning, one aims to learn a matrix W that parametrizes the similarity function . Metric learning Similarity learning is closely related to distance metric learning. Metric learning is the task of learning a distance function over objects. A metric or distance function has to obey three axioms: non-negativity, symmetry and subadditivity / triangle inequality. When the objects are vectors in the space of x through the bilinear form , then any positive definite matrix defines a distance function of . Some well known approaches for metric learning include Large margin nearest neighbor , Information theoretic metric learning (ITML). In statistics, the covariance matrix of the data is sometimes used to define a distance metric called Mahalanobis distance. Applications Similarity learning is used in information retrieval for learning to rank, and in recommendation systems. Also, many machine learning approaches rely on some metric. This include unsupervised learning such as clustering, which groups together close or similar objects. It also includes supervised approaches like K-nearest neighbor algorithm which rely on labels of nearby objects to decide on the label of a new object. Metric learning has been proposed as a preprocessing step for many of these approaches Further reading For further information on this topic, see the survey on metric and similarity learning by Bellet et al. References Yamartino method 35 Yamartino method The Yamartino method (introduced by Robert J. Yamartino in 1984) is an algorithm for calculating an approximation to the standard deviation σθ of wind direction θ during a single pass through the incoming data. The standard deviation of wind direction is a measure of lateral turbulence, and is used in a method for estimating the Pasquill stability category. The typical method for calculating standard deviation requires two passes through the list of values. The first pass determines the average of those values; the second pass determines the sum of the squares of the differences between the values and the average. This double-pass method requires access to all values. A single-pass method can be used for normal data but is unsuitable for angular data such as wind direction where the 0°/360° (or +180°/-180°) discontinuity forces special consideration. For example, the directions 1°, 0°, and 359° (or -1°) should not average to the direction 120°! The Yamartino method solves both problems. The United States Environmental Protection Agency (EPA) has chosen it as the preferred way to compute the standard deviation of wind direction.[1] A further discussion of the Yamartino method, along with other methods of estimating the standard deviation of wind direction can be found in Farrugia & Micallef [2]. Algorithm Over the time interval to be averaged across, n measurements of wind direction (θ) will be made and two totals are accumulated without storage of the n individual values. At the end of the interval the calculations are as follows: with the average values of sinθ and cosθ defined as Then the average wind direction is given via the four-quadrant arctan(x,y) function as From twenty different functions for σθ using variables obtained in a single-pass of the wind direction data, Yamartino found the best function to be where The key here is to remember that sin2θ + cos2θ = 1 so that for example, with a constant wind direction at any value of θ, the value of will be zero, leading to a zero value for the standard deviation. The use of alone produces a result close to that produced with a double-pass when the dispersion of angles is small (not crossing the discontinuity), but by construction it is always between 0 and 1. Taking the arcsine then produces the double-pass answer when there are just two equally common angles: in the extreme case of an oscillating wind blowing backwards and forwards, it produces a result of radians, i.e. a right angle. The final factor adjusts this figure upwards so that it produces the double-pass result of radians for an almost uniform distribution of angles across all directions, while making minimal change to results for small dispersions. The theoretical maximum error against the correct double-pass σθ is therefore about 15% with an oscillating wind. Comparisons against Monte Carlo generated cases indicate that Yamartino's algorithm is within 2% for more realistic distributions. Yamartino method A variant might be to weight each wind direction observation by the wind speed at that time. References [1] Meteorological Monitoring Guidance for Regulatory Modeling Applications (section 6.2.1) (http:/ / www. epa. gov/ scram001/ guidance/ met/ mmgrma. pdf) [2] http:/ / journals. cambridge. org/ production/ action/ cjoGetFulltext?fulltextid=408748 36 Article Sources and Contributors Article Sources and Contributors Standard deviation Source: https://en.wikipedia.org/w/index.php?oldid=591840563 Contributors: 1exec1, 83d40m, A.amitkumar, AJR, AManWithNoPlan, Abalter, Aberglaube, Abscissa, AbsolutDan, Abtin, Ace of Spades, Adamjslund, Addshore, Adi4094, Admissions, Aednichols, Aeriform, Afa86, Alansohn, Ale jrb, Alex.g, Alexandrov, AllCluesKey, Allessia67, Alvinwc, Amahoney, Amatulic, Amead, Amitch, Amitchell125, Amorim Parga, Anameofmyveryown, Andraaide, Andre Engels, Andres, AndrewWTaylor, Andy Marchbanks, Andycjp, Andysor, AngelOfSadness, Anomie, Anon lynx, Anonymous Dissident, Anonymous editor, Anonymousuk2015, Anrnusna, Ansa211, Anwar saadat, Arbitrarily0, Arnaugir, Aroundthewayboy, Arthur Rubin, Artichoker, Artorius, Asaba, Ashawley, Ashiabor, Astatine211, Atethnekos, AugPi, AxelBoldt, Bart133, Bdesham, Beefyt, Beetstra, Behco, Beland, BenFrantzDale, Bgwhite, Bha100710, BiT, Billgordon1099, Blah314, Blehfu, Bo Jacoby, Bobo192, Bodnotbod, Brianga, Brutha, BryanG, Bsodmike, Btyner, Buchanan-Hermit, Bulgaroctonus, Butcheries, CJLL Wright, CRGreathouse, CSWarren, CWii, CYD, Calabe1992, Calculator1000, Calvin 1998, CambridgeBayWeather, Captain-n00dle, Carmichael, Cathardic, Ccdbasmile735593, Cenkuyan, Ceyockey, Charles Matthews, Chatfecter, Chillwithabong, Chris the speller, ChrisFontenot13, ChrisGualtieri, Chrism, Christopher Parham, Chrysrobyn, Cjarrodsnow, Ck lostsword, Clemwang, Cmichael, Coffee, Coffee2theorems, Conversion script, Coppertwig, Corrigann, Crazy Boris with a red beard, Crisófilax, Cryptic C62, Cutler, DARTH SIDIOUS 2, DRHagen, DRTllbrg, DVD R W, DVdm, Daev, Danger, DanielCD, Danielb613, Danski14, Darko.veberic, Dave6, DavidLeighEllis, DavidMcKenzie, DavidSJ, DavidWBrooks, Davidkazuhiro, Davidwbulger, Dcoetzee, Ddiazhn, Ddofborg, Ddr, Decayintodust, DeeDeeKerby, Dekuntz, Delldot, Den fjättrade ankan, Denisarona, Dennyhelper, DerHexer, Derekleungtszhei, Dgw, Dhanya139, Dick Beldin, DieterVanUytvanck, Diomidis Spinellis, Dirkbb, Discospinster, Dmcq, Dmr2, Doctorambient, DomCleal, Dominus, Donner60, Dr.alaagad, DrMicro, Drappel, Dreslough, Duoduoduo, Dycedarg, Dylan Lake, Dymitr, EJM86, EPublicRelationsMT, Earth, Eb Oesch, Economist 2007, Egriffin, Elanb, Elaragirl, Elliskev, Emerah, Endobrendo, Enigmaman, Epbr123, Eric Olson, Erutuon, Esrever, Eve Teschlemacher, Everyking, Excirial, Falcon8765, Falcon9x5, Fatla00, Felixrising, Fgnievinski, FilipeS, Flamurai, Forlornturtle, Forty two, Frehley, Frencheigh, Fsiler, Furrykef, Fæ, G.engelstein, G716, Gabbe, Gail, Gary King, Gatoclass, Gauge, Gauravm1312, Gemini1980, Geneffects, George Drummond, George Ponderevo, Gerriet42, Gherghel, Giftlite, Gilliam, Gingemonkey, Giraffedata, Gjshisha, GlassCobra, Glen, Gogowitsch, Gouveia2, Graham87, Greentryst, Greg L, GregWooledge, Gurch, Gvanrossum, Gyro Copter, Gzkn, Gökhan, HaakonHjortland, Hadleywickham, Haham hanuka, Haizum, HalfShadow, Harmil, HatlessAtlas, Hawaiian717, Hede2000, Heezy, Helix84, Hellknowz, Henrygb, Hgberman, Hgrenbor, Hilgerdenaar, Hu12, Hut 8.5, Iccaldwell, Imagine Reason, Imeriki al-Shimoni, Inomyabcs, Intangir, Iridescent, Isaac Dupree, Isis, Isomorphic, IustinPop, Izno, JA(000)Davidson, JForget, JNW, JRBrown, Jacob grace, JadeInOz, Jake04961, Jamamala, Jamesx12345, Jamned, Janderk, JanetteDoe, Jcw69, Jean15paul, Jeremy68, Jeremykemp, Jfitzg, Jim.belk, Jim.henderson, JinJian, Jmoorhouse, Jni, Joerite, John, John Newbury, John11235813, JohnBlackburne, JohnCD, JorisvS, Jprg1966, Jratt, Jrennie, Jts10101, JulioSergio, Justanyone, Justinep, KJS77, KKoolstra, Kainaw, Kastchei, Kbolino, Kelvie, Khalanil, Khazar2, Khunglongcon, Kidakaka, Kingpin13, Kingturtle, Kirachinmoku, Kiril Simeonovski, Kjtobo, Knkw, Knutux, Krinkle, Krishna.91, Kungfuadam, Kuratowski's Ghost, Kuru, Kvng, Kwamikagami, Kyle824, LGW3, LWG, Lambiam, Larry_Sanger, Ldm, Learning4ever, LeaveSleaves, Legare, Legoman 86, Lethe, Lgauthie, Lilac Soul, LizardJr8, Loadmaster, Loodog, Lord Jubjub, Lucasgw8, Lugia2453, Luna Santin, LysolPionex, M1arvin, M2Ys4U, M360 Real, MBisanz, MCepek, MER-C, MONGO, MZMcBride, Macronencer, Madir, Madoka, MagneticFlux, Magnus Bakken, Malo, Mapley, Marcos, Marek69, MarkSweep, Markhebner, Markkawika, Markpravda, Marokwitz, Materialscientist, Matthew Yeager, Maurice Carbonaro, Mavenp, Mbloore, Mbroshi, Mbweissman, McKay, Mcorson, Mdann52, Mehtaba, Melchoir, Melcombe, Mercuryeagle, Mesoderm, Methecooldude, Mets501, Mgiganteus1, Mhinckley, Miaow Miaow, Michael Hardy, Miguel.mateo, Mike Rosoft, Mikhail Dvorkin, Mikhail Ryazanov, Milliemchi, Mimithebrain, MisterSheik, Mjg3456789, Mjrice, Mjroyster, Moeron, Mollerup, Monteboi1, Mooli, MrOllie, Ms2ger, Msm, Mud4t, Munckin, Murb:, Mwtoews, NHRHS2010, Nakon, Nathandean, NawlinWiki, Nbarth, Nectarflowed, NeilRickards, Neo Poz, Neudachnik, Neuralwarp, Neutrality, Ngoddard, Niallharkin, Nigholith, Noe, Nonagonal Spider, Norazoey, Normy rox, NorwegianBlue, Not-just-yeti, NotAnonymous0, Novalis, Nwbeeson, O.Koslowski, O18, Octahedron80, Ocvailes, Oddbodz, Ohl4xoaS, Oleg Alexandrov, Oliphaunt, Omegatron, Omicronpersei8, Owisped, Oxymoron83, P toolan, P.Silveira, PMHauge, Pak21, Pakaran, Paul August, PeteLaud, Peter ryan 1976, Pharaoh of the Wizards, PhilKnight, Philip Trueman, Pickledweller, Pinethicket, Piotrus, Pkomarov, Policron, Polyester, Poochy, Ppardal, Psb777, Pstanton, Psychlohexane, Publius3, Purplewowies, Qwfp, RDBury, RJaguar3, RadioKirk, Raffamaiden, Ranjithsutari, RayAYang, Razorflame, Rdbrid, Rdjere, Rednblu, Reedy, Regitmail, Rememberway, Rettetast, Revipm, Rich Farmbrough, Richard001, Rickyfrizzlebum, Robo Cop, Rocketrod1960, Rompe, Ronz, Rose Garden, Rseay267, Rspence1234, Rushbugled13, Ryk, Salix alba, Sam Korn, Sameer r, SamuelTheGhost, Sander123, Satellizer, Savidan, Savie Kumara, Scooterpooting, Seaphoto, SebastianHelm, Seresin, Shanes, Shaun ward, ShelfSkewed, Sietse Snel, Sigma 7, SimonEast, Singhalawap, Skbkekas, SkerHawx, Skittle, Slakr, Slick0805, Slon02, Slowking Man, Smokingduck, Some jerk on the Internet, Someguy1221, Sonicblade128, Spacepotato, SpeedyGonsales, Speedyboy, Spinality, Spliffy, Sstoneb, Statisfactions, Stemonitis, Stepheng3, Stevvers, StewartMH, Storkk, Stpasha, Suffusion of Yellow, Surachit, Susurrus, Svick, Swat671, Swcurran, SwisterTwister, THEN WHO WAS PHONE?, Takanoha, Taxman, Tayste, TedE, Tempodivalse, Thadius856, The Thing That Should Not Be, The sock that should not be, Thingg, Thingstofollow, Thomag, ThomasNichols, ThomasStrohmann, Thr4wn, Tide rolls, Time9, Titoxd, Tlroche, Tom harrison, Tomi, Tompa, Torsionalmetric, Tosayit, Tpbradbury, TradingBands, Triwbe, Urdutext, Useight, Ute in DC, V DESAI UK, Vaughan Pratt, Verbum Veritas, Versus22, Vice regent, VictorAnyakin, Vinod.kumar2, VladimirReshetnikov, W4chris, Waggers, Warniats, Wavelength, Wayne Slam, Wbm1058, Widr, Wikipelli, Wildingd, William Avery, Winchelsea, Winsteps, Wmahan, Wolfkeeper, Wonglkd, Woood, Wykypydya, X-Fi6, Yaara dildaara, Yachtsman1, Yamaguchi先 生, Ylloh, Yochai Twitto, Zafiroblue05, Zenkat, Zenomax, Zhieaanm, Zigger, Zvika, ﺯﺭﺷﮏ, 1711 anonymous edits Samuelson's inequality Source: https://en.wikipedia.org/w/index.php?oldid=561618657 Contributors: Cybercobra, DrMicro, Drpickem, FrankSamuelson, Headbomb, JmCor, JulienC, Michael Hardy, Qwfp Distance correlation Source: https://en.wikipedia.org/w/index.php?oldid=567253966 Contributors: AdventurousSquirrel, Alcides, Andrewman327, Angelorf, Duoduoduo, Fgnievinski, Gjshisha, Joel7687, Mathstat, Melcombe, Michael Hardy, Naught101, Queenmomcat, Qweyui, Rjwilmsi, 16 anonymous edits Geometric standard deviation Source: https://en.wikipedia.org/w/index.php?oldid=575081167 Contributors: AugPi, Betzkorn, CWenger, Cholling, Ciemo, Clhtnk, Crystallina, Deadbadger, Leon Granowitz, Leyo, Melcombe, Michael Hardy, Mikael Häggström, MrOllie, Pfeilspitze, Pjrm, Sdurietz, 19 anonymous edits Bregman divergence Source: https://en.wikipedia.org/w/index.php?oldid=580391573 Contributors: BradBeattie, BrotherE, Chire, Daniel Mietchen, David Eppstein, Dfrankow, Dicklyon, Epistemenical, Folajimi, Headbomb, Jheald, Ladybirdintheuk, Lavaka, Linas, MDReid, Melcombe, Michael Hardy, Oleg Alexandrov, Omnipaedista, Politepunk, Prashantva, RJHall, Reyk, RoboDave, Schizoid, Tekhnofiend, Yasmar, 45 anonymous edits Bhattacharyya distance Source: https://en.wikipedia.org/w/index.php?oldid=586121601 Contributors: 3mta3, Amcbride, AtroX Worf, Barticus88, Bluemoose, Btyner, Charles Matthews, Chris the speller, ChrisGualtieri, Chriscing, CommonsDelinker, Creyes, Ctwardy, Giftlite, Headbomb, HeinzRos, Hoonose, IByte, Jason Quinn, Jheald, Jimfaed, John Palkovic, JohnBlackburne, Jppellet, Jrvz, KYN, Mairi, Mathfreq, MattWatt, Mcld, Melcombe, Memming, Mukerjee, MwGamera, Myasuda, Oleg Alexandrov, Papadim.G, Policron, SFK2, Seabhcan, Sgsourav, Szzoli, Tgillet1, Thetawave, Tosha, Vanzandtj, YurritAvonds, 28 anonymous edits Hellinger distance Source: https://en.wikipedia.org/w/index.php?oldid=579262540 Contributors: 3mta3, Aetheling, Djozwebo, Epistemenical, Giftlite, GregorB, Kiefer.Wolfowitz, Linas, Lucaswilkins, Magog the Ogre, Melcombe, Michael Hardy, Mkayala, Nbarth, Oleg Alexandrov, Qwertyus, R.e.b., Repied, SFK2, Schlurcher, Simbalis, Skbkekas, Stpasha, Szzoli, Vanzandtj, Ylloh, 15 anonymous edits Mahalanobis distance Source: https://en.wikipedia.org/w/index.php?oldid=590498814 Contributors: Aetheling, Ahennell, Antegallya, Anubhab91, BenFrantzDale, Bluemoose, Bobke, CaAl, Calimo, Charles Matthews, Cherkash, Crasshopper, CrazyChemGuy, Dawn Bard, Den fjättrade ankan, Dfrankow, Donner60, Donnesimon, EffeX2, EoGuy, Flammifer, Frecklefoot, Ged.R, Giftlite, Ilmari Karonen, Innotata, Jbzdak, JohnBlackburne, Jppellet, Kku, Kovlan, Lambiam, Linas, Low-frequency internal, Mahalanobis51, Mahalanobis52, Marsias, Materialscientist, Mcld, Mdd4696, Melcombe, Mentifisto, Michael Hardy, Moti Levy, Muscicapa, Nesbit, Oleg Alexandrov, Pagh, Papadim.G, Policron, Pot, Qwfp, Raguks, Rlopez, Seabhcan, Seegoon, Semifinalist, Sgb235, Shabbychef, Shyamal, Silverfish, Sourangshu, Tiha, Timflutre, Timothy Gu, Toniher, Tosha, Urhixidur, Waldir, Warbola, WereSpielChequers, Wimmerm, Y(J)S, 133 anonymous edits Similarity learning Source: https://en.wikipedia.org/w/index.php?oldid=562650972 Contributors: Excirial, Gal chechik, OrenBochman, R'n'B, 6 anonymous edits Yamartino method Source: https://en.wikipedia.org/w/index.php?oldid=576002382 Contributors: Ajakef, Andreas Kaufmann, Bbik, CharlotteWebb, Fgnievinski, Gouveia2, Headbomb, Hwestbrook, Mbeychok, Melcombe, Michael Hardy, NickyMcLean, Omnipaedista, Panarchy, Pascal.Tesson, Pcirrus, Rumping, Stemonitis, Wilderness, 5 anonymous edits 37 Image Sources, Licenses and Contributors Image Sources, Licenses and Contributors File:standard deviation diagram.svg Source: https://en.wikipedia.org/w/index.php?title=File:Standard_deviation_diagram.svg License: Creative Commons Attribution 2.5 Contributors: Mwtoews File:cumulativeSD.svg Source: https://en.wikipedia.org/w/index.php?title=File:CumulativeSD.svg License: Public Domain Contributors: Normal_Distribution_CDF.svg: Inductiveload derivative work: Wolfkeeper (talk) File:Comparison standard deviations.svg Source: https://en.wikipedia.org/w/index.php?title=File:Comparison_standard_deviations.svg License: Public Domain Contributors: JRBrown Image:Nuvola apps kchart.svg Source: https://en.wikipedia.org/w/index.php?title=File:Nuvola_apps_kchart.svg License: GNU Lesser General Public License Contributors: en:David Vignoni, User:Stannered Image:standard deviation diagram.svg Source: https://en.wikipedia.org/w/index.php?title=File:Standard_deviation_diagram.svg License: Creative Commons Attribution 2.5 Contributors: Mwtoews Image:Distance Correlation Examples.svg Source: https://en.wikipedia.org/w/index.php?title=File:Distance_Correlation_Examples.svg License: Creative Commons Attribution-Sharealike 3.0 Contributors: Naught101 38 License License Creative Commons Attribution-Share Alike 3.0 //creativecommons.org/licenses/by-sa/3.0/ 39