Definition
Covariance is a measure of the degree to which two Random Variables, X and Y, vary together. If X and Y vary in the same direction, they have a positive covariance. If they vary in opposite directions, their covariance is negative. If X and Y are independent, the covariance is zero.
Note: The converse of the last statement is not true - if covariance(X,Y) is zero, this does not necessarily imply that X and Y are independent. This is because covariance measures only linear association between the two variables. Thus, X and Y may have a non-linear relationship which would still result in a covariance of zero.
Examples
Application
Given a sample of size n of measurements on X and Y: (xi, yi), i = 1, 2, ....n
The covariance between X and Y is given by:
The covariance of X and Y takes on the units of measurement of both variables, so it cannot be used to compare relationships of variables across different datasets.