Subspaces, Spans, and Linear Independence

09 Subspaces, Spans, and Linear Independence

Chapter Two, Sections 1.II and 2.I look at several different kinds of subset of a vector space.

A subspace of a vector space $(V, +, \cdot)$ is a subset of $V$ that is itself a vector space, using the vector addition and scalar multiplication that are inherited from $V$ . (This means that for $\vec{v}$ and $\vec{u}$ in the subspace and a real number $r$ , $\vec{v} + \vec{u}$ and $r \cdot \vec{v}$ have the same values whether $\vec{v}$ and $\vec{u}$ are thought of as elements of the subspace as they do when $\vec{v}$ and $\vec{u}$ are thought of as elements of $V$ )

Suppose $V$ is a vector space and $W \subseteq V$ , and you want to show that $W$ is a subspace of $V$ . This means verifying that $(W, +, \cdot)$ satisfies the ten properties listed in the definition of vector space. However, $W$ satisfies many of those properties automatically, because they are already known to hold in $V$ . For example, if $\vec{v}, \vec{u} \in W$ , we know without checking it that $\vec{v} + \vec{u} = \vec{u} + \vec{v}$ because that equation is true for all elements of $V$ . To show that $W$ is a subspace of $V$ , it suffices to check just three things:

$W$ is closed under vector addition. That is for all $\vec{v}, \vec{u} \in W$ , $\vec{v} + \vec{u} \in W$ .
$W$ is closed under scalar multiplication. That is, for all $\vec{v} \in W$ and $r \in R$ , $r \cdot \vec{v} \in W$ .
$W$ is not empty. (Usually, this is shown by showing $\vec{0} \in W$ .)

It's usually fairly easy to check that a subset is a subspace. For example the set $D$ of differentiable functions from $R$ to $R$ is a subspace of the vector space, $F$ , of all functions from $R$ to $R$ . This is because (1) the sum of two differentiable functions is differentiable; (2) a constant times a differentiable function is differentiable; and (3) the constant function $f (x) = 0$ (which is the additive identity in $F$ ) is differentiable.

If you take some arbitrary subset of a vectors space $V$ , it is probably not a subspace. However, you can "generate" a subspace from any subset of $V$ by taking the "span" of that subset.

Definition: Suppose that $(V, +, \cdot)$ is a vector space, and $S$ is any non-empty subset of $V$ . The span of $S$ is defined to be the subset, $[S]$ , of $V$ given by $[S] = {a_{1} {\vec{v}}_{1} + a_{2} {\vec{v}}_{2} + \dots a_{k} {\vec{v}}_{k} | k \in N and a_{1}, a_{2}, \dots a_{k} \in R and {\vec{v}}_{1}, {\vec{v}}_{2}, \dots {\vec{v}}_{k} \in S}$ That is, $[S]$ is the set of all linear combinations of any number of vectors in $S$ . The notation $s p a n (S)$ is also used instead of $[S]$ . In addition, wel say that $s p a n (\emptyset) = {\vec{0}}$ ; that is, the span of the empty set is the trivial subspace that contains only the single vector $\vec{0} .$

Theorem: If $(V, +, \cdot)$ is a vector space and $S \subseteq V$ , then the span, $[S]$ , of $S$ is a subspace of $V$ .

Proof: If $S = \emptyset$ , then $[S] = {\vec{0}}$ , which is a subspace of $V$ . Suppose that $S$ is non-empty. Then

$[S]$ is not empty. For, since $S \neq \emptyset$ , there is some $\vec{v} \in S$ . Now, $1 \cdot \vec{v}$ is a linear combination of the single vector $\vec{v}$ , so $1 \cdot \vec{v} \in [S]$ by defintion of $[S]$ . So $[S] \neq \emptyset$ . [Since $1 \cdot \vec{v} = \vec{v}$ , we see that $\vec{v} \in S$ . This shows that $S \subseteq [S]$ .]
$[S]$ is closed under vector addition. For suppose that $\vec{α}, \vec{β} \in [S]$ . We must show that $\vec{α} + \vec{β} \in [S]$ . But, by definition of $S$ , we can write $\vec{α} = a_{1} {\vec{v}}_{1} + \dots + a_{k} {\vec{v}}_{k}$ and $\vec{β} = b_{1} {\vec{u}}_{1} + \dots + b_{ℓ} {\vec{u}}_{ℓ}$ , were $a_{1}, \dots, a_{k}, b_{1}, \dots, b_{ℓ} \in R$ and ${\vec{v}}_{1}, \dots, {\vec{v}}_{k}, {\vec{u}}_{1}, \dots, {\vec{u}}_{ℓ} \in S$ . But then, $\vec{α} + \vec{β} = a_{1} {\vec{v}}_{1} + \dots + a_{k} {\vec{v}}_{k} + b_{1} {\vec{u}}_{1} + \dots + b_{ℓ} {\vec{u}}_{ℓ}$ , which is in $[S]$ since it is a linear combination of elements of $S$ . [Note that we do not require the vectors in a linear combination to be distinct, so it's OK if some of the $\vec{v}'s$ are equal to some of the $\vec{u}'s$ .]
$[S]$ is closed under scalar multiplication. For let $\vec{α} \in [S]$ and $r \in R$ . Write $\vec{α} = a_{1} {\vec{v}}_{1} + \dots + a_{k} {\vec{v}}_{k}$ where $a_{1}, \dots, a_{k} \in R$ and ${\vec{v}}_{1}, \dots, {\vec{v}}_{k} \in S$ . Then $r \cdot \vec{α} = (r a_{1}) \cdot {\vec{v}}_{1} + \dots (r a_{k}) \cdot {\vec{v}}_{k}$ , which is in $[S]$ since it is a linear combination of elements of $S$ .

Since $[S]$ has these three properties, it is a subspace.

If $[S] = W$ , we say that $S$ spans $W$ or generates $W$ , and that $S$ is a spanning set for $W .$

We have actually been working with spans for a while. If $S$ consists of a single non-zero vector $\vec{v}$ , then $[S]$ is the set of all scalar multiples of $\vec{v}$ . That is, $[S] = {a \vec{v} | a \in R}$ . So the span of $S$ is the line through the origin in direction $\vec{v}$ . If we add another vector, $\vec{u}$ , to $S$ , the span of $S$ would be ${a \vec{v} + b \vec{u} | a, b \in R}$ . If $\vec{u}$ is a multiple of $\vec{v}$ , then it adds nothing new to the span; you still just get the line through the origin in direction $\vec{v}$ . But if $\vec{u}$ does not lie on that line, then $\vec{v}$ and $\vec{u}$ generate a plane that contains the origin. If we then add a third vector, $\vec{w}$ , to $S$ , the possibilities are that either $\vec{w}$ lies on the plane spanned by $\vec{v}$ and $\vec{u}$ (in which case, it adds nothing new to the span), or $\vec{w}$ is not on the plane (in which case $\vec{v}$ , $\vec{u}$ , and $\vec{w}$ span a three-dimensional linear space). I will also note that the solution set of a homogeneous linear system written as a set ${a_{1} {\vec{v}}_{1} + a_{2} {\vec{v}}_{2} + \dots + a_{k} {\vec{v}}_{k} | a_{1}, a_{2}, \dots, a_{k} \in R}$ is the span of the set ${{\vec{v}}_{1}, {\vec{v}}_{2}, \dots, {\vec{v}}_{k}}$ . In particular, it is a subspace. It is also easy to prove that fact directly.

Note by the way that when we take the span of a finite set of vectors, say $S = {{\vec{β}}_{1}, {\vec{β}}_{2}, \dots, {\vec{β}}_{m}}$ , then the span can be written $[S] = {a_{1} {\vec{β}}_{1} + a_{2} {\vec{β}}_{2} + \dots + a_{m} {\vec{β}}_{m} | a_{1}, a_{2}, \dots, a_{m} \in R}$ . That is, we can include all $m$ vectors from $S$ in every linear combination in the span, even though the definition allows linear combinations with any number of vectors from $S$ . This is because some or all of the coefficients, $a_{i}$ , can be zero in a linear combination, and because if the same vector is used several times in a linear combination, all the terms that use it can be added together to get a single term (for example, $3 {\vec{β}}_{i} + 2 {\vec{β}}_{i} - 1 {\vec{β}}_{i}$ can be combined to make $5 {\vec{β}}_{i}$ ).

The idea that adding another vector to a spanning set will add something new to the span, unless that new vector is a linear combination of the vectors already in the set, is an essential one. It leads to the idea of linear dependence and linear independence. Spanning and linear independence are closely related concepts that are absolutely central to linear algebra.

Definition: Let $(V, +, \cdot)$ be a vector space, and let $S$ be a subset of $V$ . We say that $S$ is linearly independent (or that the vectors in $S$ are linearly independent) if no element of $S$ can be written as a linear combination of other elements of $S$ . A set that is not linearly independent is called linearly dependent.

I actually prefer a different definition of linear dependence and independence, and I believe that this alternative definition is more common in math books.

Definition: Let $(V, +, \cdot)$ be a vector space, and let $S$ be a subset of $V$ . We say that $S$ is linearly independent if in any linear combination that adds up to zero, $a_{1} {\vec{v}}_{1} + a_{2} {\vec{v}}_{2} + \dots + a_{k} {\vec{v}}_{k} = \vec{0}$ where ${\vec{v}}_{1}, {\vec{v}}_{2}, \dots, {\vec{v}}_{k} \in S$ , all of the coefficients $a_{i}$ must be zero. And $S$ is linearly dependent if there is a linear combination of elements of $S$ that adds up to $\vec{0}$ , and in which at least one of the coefficients is non-zero.

I will accept either definition. It is easy to prove that the two definitions are equivalent.

But there is another complication. Sometimes, instead of dealing with sets of vectors, we work with lists of vectors. The difference is that a list is ordered and can contain duplicates while a set cannot contain duplicates and the order of the elements in a set does not matter. So, for example, $3, 1, 2$ and $2, 3, 1$ are different lists, but ${3, 1, 2}$ and ${2, 3, 1}$ are the same set. And $2, 3, 2$ and $2, 3$ are different lists while ${2, 3, 2}$ is just another way of writing the set ${2, 3}$ . Order is not really important for linear independence, but there is a subtle difference between linear independence of lists and linear independence of sets that arises because of the possibility of duplicates in a list. Consider this definition, and note this in this case, all of the elements from the list are used in the linear combination:

Definition: Let $(V, +, \cdot)$ be a vector space, and let ${\vec{v}}_{1}, {\vec{v}}_{2}, \dots, {\vec{v}}_{k}$ be a list of elements of $V$ . We say that this list of vectors is linearly independent if in any linear combination that adds up to zero, $a_{1} {\vec{v}}_{1} + a_{2} {\vec{v}}_{2} + \dots + a_{k} {\vec{v}}_{k} = \vec{0}$ all of the coefficients $a_{i}$ must be zero. And the list is linearly dependent if there is a linear combination of this form that adds up to $\vec{0}$ , and in which at least one of the coefficients is non-zero.

The subtle difference turns up when you consider a list that contains duplicates. A list of vectors that contains duplicates cannot be linearly independent. For example, the list $\vec{v}, \vec{w}, \vec{v}$ is linearly dependent because $1 \cdot \vec{v} + 0 \cdot \vec{w} + (- 1) \cdot \vec{v} = 0$ . However, the set ${\vec{v}, \vec{w}, \vec{v}}$ is really just the set ${\vec{v}, \vec{w}}$ , which is linearly independent unless one of the two vectors is a multiple of the other. Usually, it is clear whether we are working with lists or with sets of vectors.

Many things can be proved about linearly independent sets and their spans. Mostly they amount to understanding that a set $S$ is linearly independent if it is a minimal spanning set for the subspace that it spans. That is, removing any element from a linearly independent set will remove some things from the span. On the other hand, if a set is linearly dependent, then there is at least one element that you can remove without affecting the span. That would be an element that is a linear combination of other elements from the set. That element adds nothing new to the span of the other elements.

(back to contents)