A full triangulated subcategory is thick (or épaisse) if it is closed under extensions.
Sometimes the same definition is used in abelian categories as well. However, for many authors, including Pierre Gabriel, in abelian categories, this term denotes the stronger notion of a topologizing subcategory closed under extensions; in other words, a nonempty full subcategory $T$ of an abelian category $A$ is thick (in the strong sense) iff for every exact sequence
in $A$, the object $M''$ is in $T$ iff $M$ and $M'$ are in $T$.
For some authors the thick subcategory (strong version) is called a Serre subcategory (in a weak sense), the term which we reserve for (generally) a stronger notion.
For any subcategory of an abelian category $A$ one denotes by $\bar{T}$ the full subcategory of $A$ generated by all objects $N$ for which any (nonzero) subquotient of $N$ in $T$ has a (nonzero) subobject from $T$. This becomes an idempotent operation on the class of subcategories of $A$ where $T\subset \bar{T}$ iff $T$ is topologizing. Moreover $\bar{T}$ is always thick in the stronger sense. Serre subcategories in the strong sense are those (nonempty) subcategories which are stable under the operation $T\mapsto\bar{T}$.
Following the extensions of an early work of Serre by Grothendieck and Gabriel, for a thick subcategory $T$ in an abelian category $A$, one defines the (Serre) quotient category $A/T$ has the same objects as $T$ and
where the colimit runs through all subobjects $X'\subset X$, $Y'\subset Y$ such that $X/X' \in Ob T$, $Y'\in Ob T$. The quotient functor $Q: A\to A/T$ is obvious.
Notice that the set of morphisms is small, so that the Serre quotient category exists. On the other hand, one can construct an equivalent localization by the Gabriel-Zisman localizing at the class $\Sigma$ of all morphisms whose kernel and cokernel are in $T$. Although $\Sigma$ admits the calculus of fractions, this method does not guarantee the existence in general.
A thick subcategory (here always in strong sense) is said to be localizing if and $Q$ admits a right adjoint $A/T\to A$, often called the section functor. In other words $A/T$ is a reflective subcategory of $A$. Every coreflective thick subcategory $T$ admits a section functor, and the converse holds if $A$ has injective envelopes. A thick subcategory $T\subset A$ is a coreflective iff $(T,F)$ is a torsion theory where
Pierre Gabriel, Des catégories abéliennes, Bulletin de la Société Mathématique de France, 90 (1962), p. 323-448 (numdam)
A. L. Rosenberg, Noncommutative algebraic geometry and representations of quantized algebras, MIA 330, Kluwer Academic Publishers Group, Dordrecht, 1995. xii+315 pp. ISBN: 0-7923-3575-9
Jacob Lurie, Chromatic Homotopy Theory, lecture 26, Thick subcategories (pdf)
Springer eom: localization of categories