Types of quantum field thories
While the Einstein-Hilbert action (or any of its variants) that defines the dynamics of gravity makes perfect sense at low energies as the definition of an effective quantum field theory (see there for details), it is not expected to be a non-effective theory at arbitrary high energies, because it is not renormalizable: all non-renormalizable QFTs that have appeared in practice turned out to be effective QFT approximations (see there) to renormalizable UV-completions.
The idea has been explored that gravity is simply to be regarded as a classical field theory, but trying to couple such a classical field theory to a stress-energy tensor of a matter quantum field theory leads to conceptual problems that break the very foundation of what is known about theoretical physics. The technical problem is this: the equations of motion of classical Einstein gravity (Einstein's equations) assert that the Einstein tensor of the field (physics) of gravity equals the energy-momentum tensor of all other force and matter fields. Here into the energy-momentum tensor enters the expression for energy and momentum of classical matter fields. However, according to the standard model of particle physics, these matter fields are not described by classical field theory. This then is the technical problem: there is no consistent way in sight to modify Einstein's equations so as to have a quantum EM tensor in it. The simple idea of taking the expectation value of the quantum matter EM-tensor leads to all kinds of theoretical problems.
Thus, this idea seems to be very unlikely. At the same time, it is clear that present experiments cannot directly probe the effects of that supposed UV-completion of Einstein-gravity.
Approaches to a full quantization of gravity therefore roughly fall into two different strategies
One assumes that the Einstein-Hilbert action is indeed the effective QFT that approximates a “UV-completion”, a more fundamental theory valid at all energies. This is the approach taken for instance in string theory.
One assumes that by some other fact that has been overlooked, one can make sense of a non-perturbative quantization of the EH action at arbitrary energies after all. This is for instance the case in speculations that EH-gravity has a UV fixed point?.
Accordingly, there are various proposals for what quantum gravity might be, but a comprehensive theory is missing to date. This is in stark contrast to the situation for the other fundamental forces of the standard model of particle physics: these are Yang-Mills fields, whose nature as QFTs is, while subtle, well understood. Indeed, the theoretical and experimental sucess of Yang-Mills theory is the main reason for the belief that QFT is the right framework for fundamental physics, and that also the field of gravity must have a description in this form.
Indeed, a model of fundamental physics where Yang-Mills fields are quantized but gravity is not is incomplete and arguably even inconsistent. Central motivating examples for this come from black hole-physics:
The study of quantum fields on the classical gravitational background of a black hole spacetime suggest that in a full theory of quantum gravity black holes could be shown to emit radiation and to eventually even decay in some way or other. This expected decay process cannot be described without a theory of quantum gravity.
Related to black hole radiation is a whole series of observations in classical general relativity that seem to indicate that black holes need to be thought of as systems possessing physical microstates with an entropy that cannot be accounted for by classical general relativity. This has led to the expectation that these microstates are states in quantum theory of gravity and that their understanding is necessary for understanding the physics of black holes.
More generally, appearances of unphysical singularities in general relativity, such as in black holes or cosmological singularities such as the big bang, have been argued to be artefacts of a non-quantum description of gravity. Various toy models of quantum gravity have been proposed where notably the big-bang singularity is just a classical approximation to a non-singular and well-defined quantum dynamics.
An evident definition of quantum gravity would seem to simply be the quantization of the Einstein-Hilbert action by the rules perturbation theory and renormalization. This straightforward approach however fails, as it can be demonstrated that this action functional is not renormalizable.
More recently it has been argued that there is evidence that the EH-action has a nontrivial UV-fixed point? after all. If true this would mean that a quantization of gravity in standard QFT is possible.
Another possibility is that this is only an apparent problem, and that standard gravity is nothing but an effective quantum field theory. (See there for references on gravity).
But most approaches to the problem have taken the non-renormalizability of classical gravity to be an indication that the standard concepts of QFT might need refinement in order to include gravity.
A common idea is that the assumption that the space of configurations in gravity is really that of smooth spacetimes, i.e. of smooth manifolds, is wrong. A plethora of variants of the notion of smooth manifold have been proposed as ingredients for a theory of quantum gravity. Notably concepts in noncommutative geometry. For instance Alain Connes’s spectral action generalizes the Einstein-Hilbert action functional from Riemannian manifolds to spectral triples. However, none of the modifications of the spaces of configuratations of gravity proposed so far have been shown to support action functionals that can actually be quantized, in one sense or another.
Accordingly there are suggestions to modify instead the principles of perturbative quantum field theory. In string theory the fundamental assumption is that the Feynman perturbation series over 1-dimensional graphs of traditional perturbative QFT is to be refined by a similar series over 2-dimensional surfaces. It has been shown that this string perturbation series needs no renormalization and that it does describe Yang-Mills fields and a gravitational field. This led to huge hopes and large activity in the study of string theory as a theory of quantum gravity. But various central questions remain open and the state of the theory remains somewhat inconclusive.
While perturbative string theory did demonstrate that there is a variant of the perturbation series for gravity which is renormalized, it can, by definition of perturbation theory, be just an approximation to a more fundamental non-perturbatively defined theory. Various candidates for the non-perturbative theory to which the string perturbation series might be an approximation have been proposed. One that enjoyed a large popularity in the 1990s was called matrix mechanics but the high hopes connected with it were supported mostly by toy example computations. More recently the AdS/CFT correspondence is widely thought of as providing a non-perturbative definition of string theory and of quantum gravity. This correspondence asserts that observables of a string theory on an asymptotically anti-de Sitter spacetime may be matched equivalently to certain observables in an auxiliary super Yang-Mills theory. If true, this would allow to take the well understood non-perturbative definition of Yang-Mills theory and deduce from it a theory of quantum gravity.
Still another suggestion has been that ordinary quantum field theory might be seen to apply to the ordinary configuration space and action functional of general relativity, if only a suitably adapted parameterization of the configuration space is chosen. Notably one observes that Riemannian manifolds may be encoded in terms of connections on the tangent bundle – the Levi-Civita connection – this being manifestly analogous to the nature of the fields in Yang-Mills theory. (See first-order formulation of gravity).
It has been suggested that therefore quantum gravity may have a description in terms of the holonomy-observables of these connections, their parallel transport around loops. Research in this direction has therefore become known as loop quantum gravity . However, little progress has been made in understanding the configuration space of smooth connections in terms of loop observables. In most of the LQG-literature instead it is assumed from the outset that it makes sense to pass to generalized connections : while a smooth connection is the same as a smooth parallel transport-functor on the path groupoid , the “generalized connections” in this context are taken to be non-smooth and non-continuous such functors on sub-categories of paths. It is is not clear how this configuration space relates to that of ordinary gravity.
A plethora of further suggestions has developed out of this, which in the literature often still go under the name “LQG”, even though they may abandon the description of configuration spaces of metrics in terms of loop observables. There is for instance the approach of spin foams or dynamical triangulations , and that of causets , to name just a few.
The problem with discriminating between all these proposals is the combination of two problems.
There exist insufficient theoretical tools for analyzing in detail the consequences of the various proposals.
There are few to no experiments available that would be able to observe the phenomena that a theory of quantum gravity is supposed to describe.
As a result, the field of fundamental physics today finds itself in a somewhat awkward position where the need for theoretical progress concerning quantum gravity for conceptual reasons is widely appreciated, but where disagreements about the viability of various proposals finds no conclusive resolution and divides the community.
Some of the standard lore about necessity of and the problems of quantum gravity are discussed in reviews such as
R. P. Woodard, How Far Are We from the Quantum Theory of Gravity? (arXiv:0907.4238)
Exposition from the point of view of worldline formalism is in
M. H. Goroff, A. Sagnotti, Quantum Gravity At Two Loops, Phys. Lett. B 160 (1985) 81.
A. E. M. van de Ven, Two loop quantum gravity, Nucl. Phys. B 378 (1992) 309.
A brief survey of the relevant issued with more pointers to the literature is in