Quite generally, the term Mackey functor refers to an additive functor from a (subcategory of) a category of correspondences (in a disjunctive category $\mathcal{C}$) to possibly any other additive category which however usually is the “base” abelian category. More generally the term now refers to the fairly obvious homotopy theoretic or higher categorical refinements of this concept.
Therefore the concept of Mackey functors is similar to that of sheaves with transfer and as such appears (implicitly) in the discussion of motives (explicitly e.g. in Kahn-Yamazaki 11, section 2, where $\mathcal{C}$ is a category of suitable schemes).
The concept was however introduced and named as such in the context of representation theory (Dress 71) and equivariant homotopy theory/equivariant cohomology (May 96). In that context it first received its full (∞,1)-category-theoretic forumation and discussion (Barwick 14).
We follow the modern account in (Barwick 14).
Let $\mathcal{C}$ be a disjunctive (∞,1)-category and write $Corr_1(\mathcal{C})^\otimes$ for the (∞,1)-category of correspondences in $\mathcal{C}$, regarded as a symmetric monoidal (∞,1)-category with respect to its coproduct (which is a biproduct by disjunctiveness of $\mathcal{C}$).
Write $\mathcal{C} =$Spectra${}^\otimes$ for the (∞,1)-category of spectra regarded as a symmetric monoidal (∞,1)-category with respect to the smash product of spectra. More generally $\mathcal{A}$ could be any symmetric monoidal stable (∞,1)-category
Then a (spectral) Mackey functor on $\mathcal{C}$ is a monoidal (∞,1)-functor of the form
Notice that this means that $S$ is in particular
a covariant (∞,1)-functor $(-)_\ast \colon\mathcal{C} \to \mathcal{A}$;
a contravariant (∞,1)-functor, hence $(-)^\ast \colon\mathcal{C}^{op} \to \mathcal{A}$;
satisfying the Beck-Chevalley condition.
(More generally one may specify suitably chosen sub-$(\infty,1)$-categories $\mathcal{C}^\dagger, \mathcal{C}_\dagger \subset \mathcal{C}$ and restrict $Corr_1$ to correspondences whose left leg is in $\mathcal{C}_\dagger$ and whose right leg is in $\mathcal{C}^\dagger$ (Barwick 14, section 5).)
For $\mathcal{A}$ taken to be (the derived category) of an abelian category (or better: postcomposed with a homological functor ) this definition reduces (Barwick 14) to that of Mackey functors as originally defined in (Dress 71).
For $G$ a finite group let $\mathcal{C}= G Set$ be its category of permutation representations. Then $Corr_1(\mathcal{C})$ is essentially what is called the Burnside category of $G$ ()possibly after abelianizing/stabilizing the hom-spaces suitably, but as (Barwick 14) highlights, this is unnecessary when one is mapping out of this into something abelian/stable, as is the case here.
Then Mackey functors on $\mathcal{C}$ are equivalently genuine G-spectra (Guillou-May 11, theorem 0.1, Barwick 14, below example B.6).
(…)
For $E$ a genuine G-spectrum then the corresponding Mackey functor us given by the assignment
where on the right we have the $G$-equivariant mapping spectrum from the (equivariant) suspension spectrum of the orbit $G/H$ to $E$.
(e.g. Guillou-May 11, remark 2.5)
(…)
The original article is
Reviews and surveys include
Peter May, section IX.4 of Equivariant homotopy and cohomology theory CBMS Regional Conference Series in Mathematics, vol. 91, Published for the Conference Board of the Mathematical Sciences, Washington, DC, 1996. With contributions by M. Cole, G. Comeza˜na, S. Costenoble, A. D. Elmenddorf, J. P. C. Greenlees, L. G. Lewis, Jr., R. J. Piacenza, G. Triantafillou, and S. Waner. (pdf)
Peter Webb, A Guide to Mackey Functors
Michael Hill, Michael Hopkins, Douglas Ravenel, section 4 of The Arf-Kervaire problem in algebraic topology: Sketch of the proof (pdf)
(with an eye towards application to the Arf-Kervaire invariant problem)
See also
Tammo tom Dieck, Transformation groups, Studies in Mathematics, vol. 8, Walter de Gruyter, Berlin, New York, 1987, x + 311 pp.,
chapter 1 of Serge Bouc, Green Functors and G-sets, LNM 1671 (1997; paperback 2008) doi:10.1007/BFb0095821
Tammo tom Dieck, Equivariant homology and Mackey functors, Mathematische Annalen 206, no.1, pp. 67–78, 1973 doi:10.1007/BF01431529
D. Tambara, The Drinfeld center of the category of Mackey functors, J. Algebra 319, 10, pp. 4018-4101 (2008) doi:10.1016/j.jalgebra.2008.02.011
Elango Panchadcharam, Categories of Mackey Functors, PhD thesis, Macquarie Univ. 2006
A comprehensive construction of equivariant stable homotopy theory in terms of Mackey functors is in the series
Bert Guillou, Peter May, Models of $G$-spectra as presheaves of spectra, (arXiv:1110.3571)
Permutative $G$-categories in equivariant infinite loop space theory (arXiv:1207.3459)
Clark Barwick, Spectral Mackey functors and equivariant algebraic K-theory (I) (arXiv:1404.0108)
Application of Mackey functors to the theory of motives includes