A full triangulated subcategory is thick (or épaisse) if it is closed under extensions.
Sometimes the same definition is used in abelian categories as well. However, for many authors, including Pierre Gabriel, in abelian categories, this term denotes the stronger notion of a topologizing subcategory closed under extensions; in other words, a nonempty full subcategory $T$ of an abelian category $A$ is thick (in the strong sense) iff for every exact sequence
in $A$, the object $M''$ is in $T$ iff $M$ and $M'$ are in $T$.
For some authors the thick subcategory (strong version) is called a Serre subcategory (in a weak sense), the term which we reserve for (generally) a stronger notion.
For any subcategory of an abelian category $A$ one denotes by $\bar{T}$ the full subcategory of $A$ generated by all objects $N$ for which any (nonzero) subquotient of $N$ in $T$ has a (nonzero) subobject from $T$. This becomes an idempotent operation on the class of subcategories of $A$ where $T\subset \bar{T}$ iff $T$ is topologizing. Moreover $\bar{T}$ is always thick in the stronger sense. Serre subcategories in the strong sense are those (nonempty) subcategories which are stable under the operation $T\mapsto\bar{T}$.
Following the extensions of an early work of Serre by Grothendieck and Gabriel, for a thick subcategory $T$ in an abelian category $A$, one defines the (Serre) quotient category $A/T$ as the one having the same objects as $T$ and
where the colimit runs through all subobjects $X'\subset X$, $Y'\subset Y$ such that $X/X' \in Ob T$, $Y'\in Ob T$. The quotient functor $Q: A\to A/T$ is obvious.
Notice that the set of morphisms is small, so that the Serre quotient category exists. On the other hand, one can construct an equivalent localization by the Gabriel-Zisman localizing at the class $\Sigma$ of all morphisms whose kernel and cokernel are in $T$. Although $\Sigma$ admits the calculus of fractions, this method does not guarantee the existence in general.
A thick subcategory (here always in the strong sense) is said to be localizing if $T$ is thick and the canonical functor $Q$ admits a right adjoint $A/T\to A$, often called the section functor. In other words $A/T$ is a reflective subcategory of $A$. Every coreflective thick subcategory $T$ admits a section functor, and the converse holds if $A$ has injective envelopes. A thick subcategory $T\subset A$ is a coreflective iff $(T,F)$ is a torsion theory where
Recall that a class $\Sigma$ of morphisms with category of fractions $\mathcal{C}_\Sigma$ is called saturated if $\Sigma$ coincides with the class of morphisms inverted by the canonical functor $\mathcal{C}\to \mathcal{C}_\Sigma$.
Let $A$ be an abelian category, $T$ be a thick subcategory in the strong sense and let $\Sigma_T$ be the class of morphisms in $A$ such that their kernel and cokernel are in $T$. Then $\Sigma_T$ has a right and a left calculus of fractions and is saturated for the canonical functor $p:A\to A/T$. Furthermore, $p$ maps to zero objects precisely the objects in $T$.
Conversely, let $\Sigma$ be a saturated class of morphisms of $A$ with a calculus of fractions on the right and on the left. Then the full subcategory on the objects that are mapped to zero by the canonical $p:A\to A_\Sigma$ is thick. In other words, there is a bijection between thick subcategories in the strong sense and saturated classes of morphisms with a calculus of fractions on the right and on the left.
(For this material see Schubert 1970, pp.105-107).
Pierre Gabriel, Des catégories abéliennes, Bulletin de la Société Mathématique de France, 90 (1962), p. 323-448 (numdam)
Jacob Lurie, Chromatic Homotopy Theory, lecture 26, Thick subcategories (pdf)
A. L. Rosenberg, Noncommutative algebraic geometry and representations of quantized algebras, MIA 330, Kluwer Academic Publishers Group, Dordrecht, 1995. xii+315 pp. ISBN: 0-7923-3575-9
H. Schubert, Kategorien II , Springer Heidelberg 1970.