Squared distance measures; Chi-squared and Mahalanobis Distance

 Chi-squared and Mahalanobis Distance
(Squared standardized distance to mean)

1.      Using standardized distance to mean

Ø  For univariate normal data, you determine whether an observation is likely to belong to a predefined population, according to how many standard deviations (eg. Z-score), it is away from the mean.


Ø  An observation can be higher or lower than mean. So, standardized distance of an observation, from mean, can be positive or negative value.


Ø  Likewise for a multinormal distribution, the distance from the mean or center of the measurements can be used to determine likelihood of membership of predefined population.


Ø  For multivariate data, there is no specific positive or negative direction from center of the multivariate data. Need different metric.


2.      Using Squared standardized distance to mean

Ø  For univariate data, you can calculate squared standardized distance to the mean, termed as Chi-square value for an observation.


Ø  Chi-square value/score for the univariate normal observations is expected to follow Chi-square distribution with 1 degree of freedom, which is positive Skewed distribution.


Ø  When there is one variable measured, the chi squared distribution can be simply related to the normal distribution. The proportion of observations less than 4 chi-squared unit is the same as the proportion of observations within two normalized unit on either side of the mean as ND is symmetric.


Ø  For multivariate data, covariance in the variable determines relativeness of the observation to the center of multivariate data, in addition to variance of respective variables.


Ø  A new measurement of distance which considers covariance amongst the variable is required. Another squared distance, Mahalanobis distance, which consider variance and covariance of variable is to be considered.


Ø  Mahalanobis distance for the multivariate normal observations is expected to follow chi-square distribution with k degree of freedom, where k is number of variables.

Ø  Once k is greater than or equal to 3, it no longer has a maximum at 0. This means that the center of the population is no longer the place where it is most likely to find the data. This is because although the density still is the greatest at the center, the hypersurface of the hypersphere that represents equal Mahalanobis distance from the center expands faster initially, so it is a trade-off between density and area.

Ø  When k is large, the chi squared distribution resembles a normal distribution.


Ø The distribution is positively skewed, but skewness decreases with more degrees of freedom.


Ø  Mahalanobis distance for multivariate data obeys chi-square distribution only if the underlying distribution is multivariate normal.


Ø  The mean of the chi-squared distribution equals k, variance equals 2k.

a.      If K ≥ 2, the maximum value occurs at a value of chi-square of k-2.


Reference:

1. Chemometric column by Richard B. Grereton, J. Chemometrics 2015, 29: 9–12

2. Chi-square distribution - Minitab
3. 

Comments

Post a Comment

Popular posts from this blog

Clear Understanding on Sin, Cos and Tan (Trigonometric Functions)

Image
  Clear Understanding on Sin, Cos and Tan (Trigonometric Functions) Trigonometry  (from  Ancient Greek   τρίγωνον  ( trígōnon )  'triangle', and  μέτρον  ( métron )  'measure')  is a branch of  mathematics  concerned with relationships between  angles  and side lengths of triangles.  All trigonometric formulas are divided into two major systems: Trigonometric Functions Trigonometric Identities In particular, the  trigonometric functions    relate the angles of a  right triangle  with  ratios  of its side lengths. Also, called as trigonometric rations. Trigonometric functions  are also known as  Circular or cyclic Functions. Trig functions  can be simply defined as the functions of an angle of a triangle. It means that the relationship between the angles and sides of a triangle are given by these trig functions. There are basically 6 ratios used for finding the elements in Trigonometry, those are  sine(sin), cosine(cos),  tangent(tan),  cotangent(cot), secant(sec) and co
Read more

Clear Understanding on Mahalanobis Distance

Image
  Clear Understanding on Mahalanobis Distance In multivariate/ multicharacteristics data, a measure of divergence or distance between groups in terms of  multiple characteristics is required. Lets consider, you are interested in measuring the difference (distance) between groups G1 and G2 (each of p-dimensional). A common assumption is to take the p-dimensional random vector X  , from each group, as having same variation about its mean within either group. The difference between the groups can be considered in terms of difference between mean vectors of X, in each group relative to the common within-group variation  (using common (pooled) covariance matrix). The most often used measure for multiple characteristics data is, Mahalanobis distance (Mahalanobis  Δ , where  Δ  is Uppercase Delta). The square of Mahalanobis distance is given by,  Δ 2 =  (µ 1 -µ 2 ) T Σ -1 (µ 1 -µ 2 ) or          Δ 2 =  (µ 1 -µ 2 ) ′ Σ -1 (µ 1 -µ 2 ) where the superfix T or  ′  denotes matrix transpose,   
Read more

Vignettes for Matrix concepts, related operations

  Geometric Transformations, compute the matrix of a rotation transformation and visualize it. Wolfram|Alpha Examples: Geometric Transformations (wolframalpha.com) Reference Change of Basis – GeoGebra Eigenvalues of a Matrix Calculator - Online Eigen Values λ Finder (dcode.fr) Eigenvectors of a Matrix Calculator (with Eigenvalues) - Online (dcode.fr) Matrix Visualization | Sarah Greer (sygreer.com)   Matrix visualizer (utexas.edu)  gives rotation by angle. Eigenvalue Calculator: Wolfram|Alpha (wolframalpha.com) Determinant Calculator: Wolfram|Alpha (wolframalpha.com)
Read more