You can also call this exclusive union, especially if you want to use it analogously to exclusive disjunction; then the exclusive union of sets consists of those elements that belong to exactly one of the sets.
The term symmetric difference, however, is usually used as the addition in a Boolean ring and so interpreted as an associative operation; then the symmetric difference of sets consists of those elements that belong to an odd number of the sets.
In constructive mathematics, the symmetic difference cannot be proved associative and so becomes less useful (unless one restricts attention to sets with decidable equality, where it can be proved associative). This is a problem in measure theory; see Cheng measurable space for one way to fix this.
Note that the union, symmetric difference, and (internal, or external up to natural isomorphism) disjoint union of a family of (pairwise) disjoint sets are all the same. For a family that is not disjoint, however, the union and symmetric difference are different, the internal disjoint union does not make sense, and the external disjoint union is not isomorphic to either the union or the symmetric difference (at least not naturally, and in some cases not at all).