Linear Algebra: Linear Independence of Vectors

The set of vectors is said to be linearly independent when none of the vectors in the set can be expressed as a linear combination of the other vectors in the set. In other words, a set of vectors {x_1, x_2, x_3, x_4, …., x_n} is said to be linearly independent, if only if the below equation.

has a trivial solution, that is,

However, if the solution is not trivial, then the set of vectors is dependent. Moreover, it is important to note that, independence and dependence of the vector is discussed in context of a collection of a vectors. We cant say that vector a is independent or dependent since it doesn’t make any sense.

Now that we have discussed the definition, let us see how we can find out if the set of vectors is independent or not.

Approach 1: Determinant

Given the set of vectors, create a matrix with columns being the vectors. If the determinant of the matrix is zero, then one or more of the columns is dependent on the other. However, if the determinant is non-zero, then the set of vectors is linearly independent.

Approach 2: Rank of the matrix

The rank of the matrix is the number of linearly independent columns of the matrix. If the rank of the matrix is equal to the number of vectors in the set, then vectors are linearly independent else they are dependent. To find the rank of the matrix, use the row echelon form of the matrix. To know more about the row echelon form of the matrix, read the blog, https://medium.com/@praggrt/linear-algebra-finding-basis-for-row-space-of-a-matrix-and-its-connection-to-row-echelon-form-a6567a38ce13. The rank of the matrix is the number of non — zero rows in the matrix.

Linear dependence doesn't imply causation. For example, if vectors v1 and v2 are dependent, then v1 can be expressed in terms of v2 and vice-versa. However, it can't be inferred that v1 causes v2 or vice-versa.

Food for thought: Granger Causality.

Linear independence doesn't imply independence. Given that vectors are linearly independent, it does not rule out the possibility of a non-linear relationship between vectors. Hence, vectors might be linearly independent however they might have a non-linear relationship. If the vectors are linearly independent, they are said to be uncorrelated vectors. Independence is a stronger notion than linear independence.