# Variance Covariance Matrix

Description
Variance Covariance Matrix and properties
Categories
Published

View again

All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
Share
Transcript
Variance-Covariance Matrix Assembled by Ben DanforthJune 1, 2009 1 Covariance Covariance measures the degree to which two variables change or vary together (i.e. co-vary).On the one hand, the covariance of two variables is positive if they vary together in the samedirection relative to their expected values (i.e. if one variable moves above its expectedvalue, then the other variable also moves above its expected value). On the other hand, if one variable tends to be above its expected value when the other is below its expected value,then the covariance between the two variables is negative. If there is no linear dependencybetween the two variables, then the covariance is 0.The covariance of two random variables,  X  i  and  X   j , can be mathematically representedas cov ( X  i ,X   j ) =  E  [( X  i − µ i )( X   j − µ  j )] (1)where  µ i  =  E  ( X  i ) and  µ  j  =  E  ( X   j ). 1 This relationship can be further generalized to amultivariate situation and for estimated coeﬃcients (i.e. ˆ β  s) using matrix notation cov (ˆ β  ) =  (ˆ β  − µ )(ˆ β  − µ ) T    =  E  (ˆ β  ˆ β  T  ) − µµ T  (2)where ˆ β   is a vector of estimated coeﬃcients and  µ  =  E  (ˆ β  ) =  β  . This relationship can bemore explicitly represented as cov (ˆ β  ) =  cov (ˆ β  1 ,  ˆ β  1 )  cov (ˆ β  1 ,  ˆ β  2 )  ... cov (ˆ β  1 ,  ˆ β  k ) cov (ˆ β  2 ,  ˆ β  1 )  cov (ˆ β  2 ,  ˆ β  2 )  ... cov (ˆ β  2 ,  ˆ β  k ) ... ... ... ...cov (ˆ β  k ,  ˆ β  1 )  cov (ˆ β  k ,  ˆ β  2 )  ... cov (ˆ β  k ,  ˆ β  k )  ,  (3)where the diagonal of this matrix represents the  variance   of the vector ˆ β   with  k  elements.Therefore, the matrix can be rewritten as cov (ˆ β  ) =  var (ˆ β  1 )  cov (ˆ β  1 ,  ˆ β  2 )  ... cov (ˆ β  1 ,  ˆ β  k ) cov (ˆ β  2 ,  ˆ β  1 )  var (ˆ β  2 )  ... cov (ˆ β  2 ,  ˆ β  k ) ... ... ... ...cov (ˆ β  k ,  ˆ β  1 )  cov (ˆ β  k ,  ˆ β  2 )  ... var (ˆ β  k )  .  (4)This essentially represents the  covariance   or  variance-covariance matrix  . 1 Sometimes  cov ( X  i ,X  j ) is denoted as Σ i,j . 1  2 Deriving the Variance-Covariance Matrix First, begin with the OLS estimator in matrix notationˆ β   = ( X  T  X  ) − 1 X  T  y  (5)Next, substitute  y  =  Xβ   +  u  into the preceding equation to arrive atˆ β   = ( X  T  X  ) − 1 X  T  ( Xβ   +  u )Then distribute the ﬁrst set of termsˆ β   = ( X  T  X  ) − 1 X  T  Xβ   + ( X  T  X  ) − 1 X  T  u And simplify, recalling that ( X  T  X  ) − 1 ( X  T  X  ) = 1ˆ β   =  β   + ( X  T  X  ) − 1 X  T  u Finally, subtract  β   from both sides to getˆ β  − β   = ( X  T  X  ) − 1 X  T  u  (6)Recalling Eq. 2, with  β   substituted for  µcov (ˆ β  ) =  E   (ˆ β  − β  )(ˆ β  − β  ) T    (7)Insert Eq. 6 into Eq. 7 and then reorder the terms (remember ( AB ) T  =  B T  A T  ) cov (ˆ β  ) =  E   (( X  T  X  ) − 1 X  T  u )(( X  T  X  ) − 1 X  T  u ) T   cov (ˆ β  ) =  E   ( X  T  X  ) − 1 X  T  uu T  X  ( X  T  X  ) − 1  Take the expectation (the  X  s are non-stochastic, and it is assumed that  E  ( uu T  ) =  σ 2 I  ) cov (ˆ β  ) = ( X  T  X  ) − 1 X  T  E  ( uu T  ) X  ( X  T  X  ) − 1 cov (ˆ β  ) = ( X  T  X  ) − 1 X  T  σ 2 IX  ( X  T  X  ) − 1 And simplify, producing the ﬁnal equation for the variance-covariance matrix cov (ˆ β  ) =  σ 2 ( X  T  X  ) − 1 (8)In this last equation,  σ 2 is the homoskedastic variance of   u i  and ( X  T  X  ) − 1 is the inversematrix appearing in the OLS estimator. cov (ˆ β  ) =  σ 2 ( X  T  X  ) − 1 2  3 Estimating the Variance-Covariance Matrix To estimate the variance-covariance matrix, the variance is needed. Since the true varianceis unknown, it must be estimated. An unbiased estimator of   σ 2 for the  k -th variable case isˆ σ 2 =   ˆ u 2 i n − k  (9)which is equivalent toˆ σ 2 = ˆ u T  ˆ un − k  (10)where ˆ u  is the estimated residuals,  n  is the number of observations, and  k  the number of parameters being estimated.Although ˆ σ 2 can be computed directly from the estimated residuals, there is a secondapproach to calculating it. First, recall that TSS   =  ESS   +  RSS   (11)where  TSS   is the total sum of squares, ESS is the explained sum of squares, and RSS is theresidual sum of squares. This equation can be rewritten as RSS   =  TSS  − ESS   (12)Next, remember that RSS   = ˆ u T  ˆ u  (13) TSS   =  y T  y − n ¯ Y  2 (14) ESS   = ˆ β  T  X  T  y T  − n ˆ Y  2 (15)where the term  n ¯ Y  2 is known as the correction for mean. Therefore, substituting Eqs. 13-15into Eq. 12 producesˆ u T  ˆ u  =  y T  y −  ˆ β  T  X  T  y T  (16) 4 Covariance vs. Correlation Covariance and correlation are related but not equivalent statistical measures. In particular,the correlation of two variables,  X  i  and  X   j , is their normalized covariance, which is deﬁnedas ρ i,j  =  E   [( X  i − µ i )( X   j − µ  j )] σ i σ  j ,  (17)where  ρ i,j  is the correlation coeﬃcient,  µ i  =  E  ( X  i ),  µ  j  =  E  ( X   j ),  σ i  is the standard deviationof   X  i , and  σ  j  is the standard deviation of   X   j . Because the correlation is normalized, it isdimensionless (i.e. it is a pure number without a unit of measure). By contrast, covariancedoes have a unit of measure—the product of the units of two variables.3  5 Excercise You are given the following pieces of information: n  = 15¯ Y   = 68 . 7333ˆ β   =  5 . 5027 4 . 1535 3 . 3787 8 . 2903  y T  y  = 259651 X  T  y  =  1031113703598110176  1. Compute the variance-covariance matrix. You can also compute  R 2 .2. Use the variance-covariance matrix to make inferences about the estimated coeﬃcients.3. Is there any evidence of violations of the OLS assumptions given the above informationand your calculations?4. The covariance matrix of the three variables of interest ( X  1 ,  X  2 ,  X  3 ) is  X  1  X  2  X  3 X  1  53 . 5523 167 . 2238 4 . 1667 X  2  167 . 2238 522 . 9238 12 . 5238 X  3  4 . 16667 12 . 5238 12 . 3810  Given that the standard deviations between  X  1  and  X  2  are 7.3179 and 22.8675, re-spectively, what is the correlation between these two variables?4

Jul 23, 2017

#### Training And Development

Jul 23, 2017
Search
Similar documents

View more...
Tags

Related Search