geometry, complex numbers, complex line
$dim = 1$: Riemann surface, super Riemann surface
A modular form is a holomorphic function on the upper half-plane that satisfies certain transformation property under the action of the modular group. Abstractly this transformation property makes the function a section of a certain line bundle on the quotient of the upper half plane that makes it the moduli stack of elliptic curves (over the complex numbers) or more generally the Riemann moduli space for Riemann surfaces of given genus.
Modular forms are also often called classical automorphic forms, see below
Modular forms appear as the coefficient ring of the Witten genus on manifolds with rational string structure. For manifolds with actual string structure this refines to topological modular forms, which are the homotopy groups of the spectrum tmf.
An (integral) modular form of weight $w$ is a holomorphic function on the upper half-plane
(complex numbers with strictly positive imaginary part)
such that
if $A = \left( \array{a & b \\ c& d}\right) \in SL_2(\mathbb{Z})$ acting by $A : \tau \mapsto = \frac{a \tau + b }{c \tau + d}$ we have
(notice that for $A = \left( \array{1 & 1 \\ 0& 1}\right)$ then $f(\tau + 1) = f(\tau)$)
$f$ has at worst a pole at $\{0\}$ (for weak modular forms this condition is relaxed)
it follows that $f = f(q)$ with $q = e^{2 \pi i \tau}$ is a meromorphic funtion on the open disk.
integrality $\tilde f(q) = \sum_{k = -N}^\infty a_k \cdot q^k$ then $a_k \in \mathbb{Z}$
More generally there is such a definition for $SL(2,\mathbb{Z})$ replaced by any other arithmetic subgroup $\Gamma \subset SL(2,\mathbb{R})$ (e.g. Litt, def.1), giving modular forms on a Riemann moduli spaces.
More abstractly, for $\mathcal{M}_{ell}$ the moduli stack of elliptic curves (or rather its Deligne-Mumford compactification) and $A \to \mathcal{M}_{ell}$ the corresponding universal bundle, write $\Omega^1_{A/S}$ for the line bundle of fiberwise Kähler differential forms. Write $e$ for the 0-section of this line bundle. Then
is a line bundle over the moduli stack of elliptic curves. A modular form of weight $k$ is a section of $\omega^{\otimes k}$
Similarly one considers modular forms for congruence subgroups of the full modular group, hence on the space of elliptic curves with level structure.
Instead of regarding, as above, modular forms as sections of a line bundle on a quotient of the upper half plane, one may regard them alternatively as plain functions, but on the (projective) special linear group $SL(2,\mathbb{R})$. (e.g. Martin 13, section 2, Litt, section 2).
As such these functions are then invariant under the action of the modular subgroup $SL(2,\mathbb{Z})\hookrightarrow SL(2,\mathbb{R})$ and hence are really functions on the coset space $SL(2,\mathbb{R})/SL(2,\mathbb{Z})$ (for forms on moduli of elliptic curves) or more generally $\Gamma \backslash SL(2,\mathbb{Z})$ (for forms on moduli of more general Riemann surfaces).
This generalizes to the case of other congruence subgroups (as above). Generally such functions on coset spaces like this are called automorphic forms. See there for more.
For the history of the terminology “modular form”/“automorphic form” see also this MO comment.
Write $\Gamma_0(2) \hookrightarrow SL_2(\mathbb{Z})$ for the subgroup of the modular group on those elements $\left(\array{a & b \\ c & d}\right)$ for which $c = 0\, mod\, 2$.
A modular function for $\Gamma_0(2)$ is a meromorphic function on the upper half plane which transforms as a modular form under the action of $\Gamma_0(2) \hookrightarrow SL_2(\mathbb{Z})$. Write $MF_\bullet(\Gamma_0(2))$ for the ring of these.
There is a natural isomorphism
(see at elliptic genus) for the notation.
(Landweber-Ravenel-Stong 93, theorem 1.5 and sections 5.3, 5.8)
For $E$ the elliptic cohomology theory associated to the elliptic curve $C$, then
(where $\omega$ is the line bundle from above)
and
generalization to functions on moduli of higher dimensional abelian varieties: Hilbert modular form, Siegel modular forms
A basic and handy reference is
Textbook accounts include
Lecture notes and reviews include
Richard Hain, section 4 of Lectures on Moduli Spaces of Elliptic Curves (arXiv:0812.1803)
Charles Rezk, section 10 of pdf
Daniel Litt, Automorphic forms notes, part I (pdf)
Jan Hendrik Bruinier, Gerard van der Geer, Günter Harder, Don Zagier, The 1-2-3 of modular forms, Lectures at a Summer School 2004 in Nordfjordeid, Norway; Universitext, Springer 2008.
Kimball Martin, A brief overview of modular and automorphic forms, 2013 pdf
Wikipedia, Modular form
Original discussion in the context of elliptic genera and elliptic cohomology includes
Peter Landweber, Douglas Ravenel, Robert Stong, Periodic cohomology theories defined by elliptic curves, in Haynes Miller et. al. (eds.), The Cech centennial: A conference on homotopy theory, June 1993, AMS (1995) (pdf)
Peter Landweber, Elliptic Cohomology and Modular Forms, in Elliptic Curves and Modular Forms in Algebraic Topology, Lecture Notes in Mathematics Volume 1326, 1988, pp 55-68 (LandweberEllipticModular.pdf?)
Reviews of this include