An integral transform on functions is a linear map between functions on spaces $X$, $Y$ encoded by a function $K$ on the product space $X \times Y$ and given by a formula of the type
where on the right we have some kind of integration over $X$.
Here $K$ is called the integral kernel of the transformation.
Typically the definition of an integral transform on functions involves some delicate technical issues concerning the precise nature of the function space, the measure with respect to which the integral is defined, etc.
On the other hand, one may understand the general form of an integral transform as the decategorification of a very natural general abstract construction in higher category theory: that of integral transforms on sheaves given by spans of base change geometric morphisms.
Special cases of such categorified integral transforms are discussed at
The simplest example is matrix multiplication, which corresponds to the case where $X$ and $Y$ are discrete spaces.
Standard examples involving genuine functional analysis are for instance the Fourier transform and related Laplace transform?, Mellin transform, Hankel transform?, Hilbert transform?.
Also the path integral in quantum mechanics and quantum field theory is supposed to be a class of examples of a (secondary) integral kernel.
In noncommutative algebraic geometry, one of the most important results is Dmitri Orlov’s representability theorem (1997) which states that every fully faithful triangulated functor between the derived categories of coherent sheaves on two smooth projective varieties is representable by some “integral kernel”, i.e., a coherent complex on the product. One would like to remove the “fully faithful” assumption, but this has proved extremely difficult so far. In the context of dg-categories the analogous result, discovered by Bertrand Toen in 2004, does have the clean form one would like it to. Along with other problems with triangulated categories, this has been one of the motivations for people like Maxim Kontsevich and Goncalo Tabuada to start doing noncommutative geometry with pretriangulated dg-categories instead of triangulated categories.
On the other hand pretriangulated dg-categories are known to provide a model for linear stable (infinity,1)-categories. Using a different model, like quasi-categories, would be more convenient for extending Toen’s theorem from (smooth proper) schemes to (smooth proper) derived stacks. This was done in Ben-Zvi & Francis & Nadler 08. In the followup (Ben-Zvi & Nadler & Preygel 13) the authors have also extended these results to the non-smooth case.
Discussion of integral transforms in derived algebraic geometry (see also at geometric infinity-function theory) is in
David Ben-Zvi, John Francis, David Nadler, Integral Transforms and Drinfeld Centers in Derived Algebraic Geometry (arXiv:0805.0157)
David Ben-Zvi, David Nadler, Anatoly Preygel, Integral transforms for coherent sheaves (arXiv:1312.7164)
Comments on the formalization of integral transforms and quantization in dependent linear type theory are at