How to use this course
Norbert Wiener entered college at eleven and took a Harvard doctorate at eighteen. He then spent forty-five years at the Massachusetts Institute of Technology, where he built a body of work that runs from pure analysis to engineering. He gave the first rigorous construction of Brownian motion, found the exact condition for solving the Dirichlet problem, created generalized harmonic analysis, proved the general Tauberian theorem, and with Raymond Paley and Eberhard Hopf developed the complex-variable methods that bear their names. During the Second World War he turned these tools on the problem of predicting an aircraft’s path, and from that work came the Wiener filter and the science he named cybernetics.
- Real analysis and Lebesgue integration; the Fourier transform on \(L^1\) and \(L^2\).
- Probability: Gaussian random variables, independence, the strong law of large numbers.
- Complex analysis: analytic functions, Liouville’s theorem, contour integration (Lessons 07 and 08).
- Some potential theory (harmonic functions, Green’s functions) helps for Lesson 05.
- The Shannon course on this site is a companion to Lessons 09 and 10.
Each technical lesson opens with a paragraph for the general reader, then gives precise statements, proof sketches and exercises with folded solutions. We write the Fourier transform as \(\hat f(\xi)=\int_{\mathbb R} f(x)e^{-i\xi x}\,dx\) unless stated otherwise. Lessons 04 and 05 follow his work of 1920–1924; Lessons 06 to 08 the harmonic analysis of 1925–1934; Lessons 09 and 10 the wartime and postwar work.
Timeline
| 1894 | Born 26 November in Columbia, Missouri, first child of Leo Wiener, a language teacher from the Russian Empire, and Bertha Kahn. |
| 1906 | Graduates from Ayer High School, Massachusetts, aged 11. Enters Tufts College in September. |
| 1909 | B.A. in mathematics from Tufts, cum laude, aged 14. Begins graduate study in zoology at Harvard. |
| 1910–11 | Studies philosophy at Cornell on a scholarship. |
| 1912 | M.A. from Harvard. |
| 1913 | PhD from Harvard, aged 18, on the algebra of relations in Schröder and in Whitehead and Russell. Travelling fellowship to Cambridge, England. |
| 1913–14 | Studies with Bertrand Russell and G. H. Hardy at Cambridge; spring 1914 at Göttingen with David Hilbert and Edmund Landau. |
| 1914 | Shows that ordered pairs can be defined from sets. |
| 1915–18 | Teaches philosophy at Harvard; works at General Electric and for the Encyclopedia Americana. |
| 1918 | Ballistics computation at the Aberdeen Proving Ground, Maryland; then enlists as a private. |
| 1919 | Instructor in mathematics at MIT. |
| 1920 | International Congress at Strasbourg; works with Maurice Fréchet on normed vector spaces. |
| 1923 | “Differential-space”: the construction of Brownian motion. Begins work on the Dirichlet problem; Wiener’s test follows in 1924. |
| 1925–26 | Collaborates with Max Born on an operator form of quantum mechanics during Born’s visit to MIT. Assistant professor (1925). |
| 1926 | Marries Margaret Engemann. Guggenheim fellowship, 1926–27, in Göttingen and Copenhagen. |
| 1929 | Associate professor at MIT. |
| 1930 | “Generalized harmonic analysis”, Acta Mathematica. |
| 1931 | Wiener–Hopf paper with Eberhard Hopf. 1931–32 in Cambridge with Hardy. |
| 1932 | “Tauberian theorems”, Annals of Mathematics. Full professor at MIT. |
| 1933 | Bôcher Memorial Prize, shared with Marston Morse. Elected to the National Academy of Sciences. Paley dies in April. |
| 1934 | Fourier Transforms in the Complex Domain, with Paley. |
| 1935–36 | Visiting professor at Tsing Hua University, Peking. |
| 1936 | Plenary address at the International Congress of Mathematicians, Oslo. |
| 1938 | “The homogeneous chaos”. |
| 1941 | Resigns from the National Academy of Sciences. Begins anti-aircraft prediction work for the National Defense Research Committee. |
| 1942 | Classified report on extrapolation, interpolation and smoothing of stationary time series. |
| 1943 | “Behavior, purpose and teleology” with Arturo Rosenblueth and Julian Bigelow. |
| 1947 | Public letter refusing to share his work with military research, in the Atlantic Monthly. |
| 1948 | Cybernetics: or Control and Communication in the Animal and the Machine. |
| 1950 | The Human Use of Human Beings. Plenary address at the International Congress in Cambridge, Massachusetts. |
| 1953 | Ex-Prodigy, first volume of autobiography; I Am a Mathematician follows in 1956. |
| 1959 | Institute Professor at MIT; emeritus from 1960. |
| 1964 | January: receives the National Medal of Science (awarded for 1963). Dies 18 March in Stockholm, aged 69. |
The making of a prodigy, 1894–1909
Norbert Wiener was born on 26 November 1894 in Columbia, Missouri, the first child of Leo Wiener and Bertha Kahn. Leo Wiener was a Jew from the Russian Empire who had studied medicine at Warsaw and engineering in Berlin before emigrating to America. He was a gifted linguist. After teaching languages at the University of Missouri he moved to Harvard, where he became professor of Slavic languages. Bertha Kahn came from a German Jewish family. Of Norbert’s two younger sisters, Constance later married the MIT mathematician Philip Franklin.
Leo Wiener had firm views on education and applied them to his son. His father taught him at home, drilling him in mathematics and languages with a severity that the son later described with mixed feelings: he credited his father with his knowledge and blamed him for a childhood of humiliation at every mistake. He was also severely short-sighted, and for a time doctors forbade him to read; his father continued the lessons orally, and the boy did algebra in his head.
Ayer and Tufts
In 1903, aged nine, he entered Ayer High School in Massachusetts and graduated in 1906, aged eleven. That September he entered Tufts College. Socially he was a child among young men, and he felt it keenly. He graduated in 1909, aged fourteen, with a degree in mathematics, cum laude. The newspapers had followed him throughout, and he grew up with the label “prodigy”, which he later examined at length in Ex-Prodigy (1953).
Harvard, Cambridge, Göttingen, 1909–1915
Wiener began graduate work in zoology at Harvard in 1909. He was clumsy in the laboratory, and the experiment failed. In 1910 he moved to Cornell on a scholarship to study philosophy, which went no better. His father then brought him back to Harvard, where he studied philosophy and mathematical logic. He took an M.A. in 1912 and a PhD in 1913, aged eighteen. His thesis compared the treatment of the algebra of relations by Ernst Schröder with that of Alfred North Whitehead and Bertrand Russell in Principia Mathematica. His supervisor was Karl Schmidt; Josiah Royce and Edward Huntington also guided him.
Russell and Hardy
A Harvard travelling fellowship took him to Cambridge, England, in 1913. Russell told him that a logician needed more mathematics and sent him to G. H. Hardy’s lectures. Hardy’s analysis was a revelation, and Wiener came to regard him as his principal teacher in mathematics. In the spring of 1914 he went to Göttingen, where he heard David Hilbert on differential equations and Edmund Landau on group theory, and attended the philosopher Edmund Husserl’s courses.
His most lasting logical result came from this period. In “A simplification of the logic of relations” (1914) he showed that the ordered pair, which Principia had taken as primitive, could be defined from sets, so that the theory of relations reduces to the theory of classes. Kazimierz Kuratowski gave the simpler definition used today in 1921 (Exercise 2.1). The war broke into his European studies. After a further stay in Cambridge he returned to America in 1915.
Exercise 2.1Kuratowski defined \((a,b)=\{\{a\},\{a,b\}\}\). Prove that \((a,b)=(c,d)\) implies \(a=c\) and \(b=d\). Wiener’s 1914 definition was \(\{\{\{a\},\emptyset\},\{\{b\}\}\}\); why might he have needed the extra layers?
Show solution
Wandering years and MIT, 1915–1925
The next four years were unsettled. Wiener taught philosophy at Harvard in 1915–16, then worked for General Electric and as a staff writer for the Encyclopedia Americana in Albany, and briefly as a journalist. His poor eyesight kept him out of the officer corps. In the summer of 1918, at the invitation of Oswald Veblen, he joined the team computing artillery range tables at the Aberdeen Proving Ground in Maryland. Before the war ended he enlisted as a private; he was discharged in February 1919.
That year William Fogg Osgood of Harvard told him of a vacancy in the mathematics department of MIT, then an engineering school with little research tradition in mathematics. Wiener became an instructor in 1919 and stayed for the rest of his life. He believed that antisemitism had cost him a permanent post at Harvard.
The first discoveries
MIT gave him engineers, physicists and real problems. He later recalled watching the waves of the Charles River from the Institute and asking how one could describe the motion of its waves mathematically. He took up Brownian motion, which Albert Einstein had explained in 1905 and Jean Perrin had measured, and by 1923 had constructed a probability measure on the space of continuous paths (Lesson 04). At the 1920 International Congress in Strasbourg he worked with Maurice Fréchet and arrived, at the same time as Stefan Banach, at the notion of a complete normed vector space; he soon dropped the subject, and the spaces carry Banach’s name. From 1923 to 1925 he solved the general Dirichlet problem (Lesson 05).
In the winter of 1925–26 Max Born visited MIT. Heisenberg’s matrix mechanics had just appeared, and it applied only to periodic systems. Born and Wiener reformulated it with linear operators, so that it could describe aperiodic motion, in a joint paper of 1926. It was one of the first uses of operators in quantum mechanics, and Wiener took pride in it, though he did not continue in physics. He was promoted to assistant professor in 1925.
Brownian motion and Wiener measure
For the general reader. A pollen grain in water jitters because molecules strike it from all sides. Einstein showed in 1905 that its average squared displacement grows in proportion to time. Wiener asked a mathematical question: is there a consistent way to assign probabilities to whole paths of such a particle? In 1923 he answered yes. The paths turned out to be continuous everywhere and smooth nowhere. His construction is the foundation of stochastic analysis and of the mathematics of finance.
- State the existence theorem for Wiener measure.
- Construct Brownian motion as a random Fourier series.
- Prove the quadratic variation formula and understand why paths are rough.
There is a unique probability measure \(W\) on the space \(C[0,\infty)\) of continuous real functions, with its Borel σ-algebra for the topology of uniform convergence on compact sets, such that under \(W\) the coordinate process \(B(t)(\omega)=\omega(t)\) satisfies: \(B(0)=0\); for \(0\le t_0< t_1<\cdots< t_n\) the increments \(B(t_k)-B(t_{k-1})\) are independent; and \(B(t)-B(s)\) is Gaussian with mean 0 and variance \(t-s\) for \(s< t\).
The process \(B\) is Brownian motion, or the Wiener process, and \(W\) is Wiener measure. Equivalently, \(B\) is a centred Gaussian process with covariance \(\mathbb E[B(s)B(t)]=\min(s,t)\) and continuous paths.
A construction by random Fourier series
Wiener’s 1923 paper built the measure from the distributions at finitely many times and proved that it lives on continuous functions. In the early 1930s, with Paley, he gave a cleaner construction. Let \(\xi_0,\xi_1,\xi_2,\dots\) be independent standard Gaussians and set, for \(0\le t\le\pi\),
\[ B(t)=\frac{\xi_0\,t}{\sqrt\pi}+\sqrt{\frac2\pi}\sum_{n=1}^{\infty}\xi_n\,\frac{\sin nt}{n}. \]The series converges in \(L^2\) for each \(t\), and a suitable grouping of terms (blocks \(2^m\le n<2^{m+1}\)) converges uniformly in \(t\) almost surely, so \(B\) is continuous. Being a sum of independent Gaussians, \(B(t)\) is Gaussian, and the covariance is \(\min(s,t)\) (Exercise 4.1). Paul Lévy later gave a construction by successive midpoint interpolation, and the same ideas extend to every interval.
Roughness
(i) For partitions \(0=t_0< t_1<\dots< t_n=T\) with mesh \(\delta\to0\), \(\sum_k\big(B(t_k)-B(t_{k-1})\big)^2\to T\) in \(L^2\). (ii) Almost every Brownian path is nowhere differentiable. (iii) Almost every path is Hölder continuous of every order \(\gamma<\tfrac12\) and of no order \(\gamma>\tfrac12\).
Part (ii) is the theorem of Paley, Wiener and Antoni Zygmund; parts (i) and (iii) are due mainly to Paul Lévy. Part (i) is a short computation (Exercise 4.2). It already shows that paths have infinite length: a function of bounded variation \(V\) that is continuous has \(\sum(\Delta B)^2\le V\cdot\max|\Delta B|\to0\). Part (i) is the seed of Itô’s calculus, where \((dB)^2=dt\) replaces the rules of ordinary calculus. Wiener’s 1938 paper “The homogeneous chaos” decomposed functionals of the Brownian path into orthogonal pieces of increasing degree; Robert Cameron and William Martin proved in 1947 that these pieces span all square-integrable functionals. This Wiener chaos expansion is a basic tool of stochastic analysis.
Exercise 4.1Using \(\sum_{n\ge1}\frac{\sin^2 nt}{n^2}=\frac{t(\pi-t)}{2}\) for \(0\le t\le\pi\), show that the series above has \(\operatorname{Var}B(t)=t\).
Show solution
Exercise 4.2Prove part (i) of the theorem: with \(Q=\sum_k(\Delta_kB)^2\) and \(\Delta_kt=t_k-t_{k-1}\), show \(\mathbb E\,Q=T\) and \(\operatorname{Var}Q=2\sum_k(\Delta_kt)^2\le 2T\delta\).
Show solution
Exercise 4.3Brownian scaling. For \(c>0\) let \(\tilde B(t)=c^{-1/2}B(ct)\). Show that \(\tilde B\) is again a Brownian motion. What does this say about the picture in the hero image if the time axis is stretched by a factor 4?
Show solution
The Dirichlet problem and capacity
For the general reader. Fix the temperature on the boundary of a region and wait: the temperature inside settles into a harmonic function. Does it always take the prescribed boundary values? Not quite. At some sharp inward spikes of the boundary the solution can miss its target. In 1924 Wiener found the exact test that decides, point by point, whether this happens, using a measure of size called capacity.
- Understand regular and irregular boundary points.
- Define Newtonian capacity and compute it for a ball.
- State Wiener’s test and apply it to cones, punctures and Lebesgue’s spine.
The problem
Let \(D\subset\mathbb R^n\) be a bounded open set and \(g\) a continuous function on \(\partial D\). The Dirichlet problem asks for \(u\) harmonic in \(D\), continuous on \(\overline D\), with \(u=g\) on \(\partial D\). Stanisław Zaremba showed in 1911 that it can fail: for a punctured disc, the value at the centre cannot be prescribed. Henri Lebesgue found in 1913 a subtler example in \(\mathbb R^3\), a domain whose boundary has a sharp inward cusp, the Lebesgue spine. Wiener, and in parallel Oskar Perron, showed that every continuous \(g\) has a generalized solution \(u_g\), harmonic in \(D\) and depending linearly and monotonically on \(g\). A boundary point \(x_0\) is regular if \(u_g(x)\to g(x_0)\) as \(x\to x_0\) for every continuous \(g\).
Capacity
For a compact \(K\subset\mathbb R^n\), \(n\ge3\), define the Newtonian capacity
\[ \operatorname{cap}(K)=\inf\Big\{\int_{\mathbb R^n}|\nabla\varphi|^2\,dx\ :\ \varphi\in C_c^\infty(\mathbb R^n),\ \varphi\ge1\text{ on }K\Big\}. \]Physically, it is the charge a conductor of shape \(K\) holds at unit potential, up to a constant. It is monotone, subadditive, and scales as \(\operatorname{cap}(\lambda K)=\lambda^{n-2}\operatorname{cap}(K)\). Sets of capacity zero, such as points and, in \(\mathbb R^3\), line segments, are invisible to harmonic functions.
Let \(D\subset\mathbb R^n\), \(n\ge3\), be bounded and open, \(x_0\in\partial D\), and for \(k\ge1\) let \(E_k=\{x\notin D:\ 2^{-k-1}\le|x-x_0|\le2^{-k}\}\). Then \(x_0\) is regular if and only if
\[ \sum_{k=1}^{\infty}2^{k(n-2)}\operatorname{cap}(E_k)=\infty. \]The weights \(2^{k(n-2)}\) undo the scaling of capacity, so each term measures how “thick” the complement of \(D\) is at scale \(2^{-k}\) relative to that scale. The test says that \(x_0\) is regular when the complement is thick at infinitely many scales in a summed sense. In the plane the same theorem holds with logarithmic capacity and weights \(k\).
Proof idea. Let \(v\) be the equilibrium potential of the set \(\complement D\cap\overline{B(x_0,r)}\), equal to 1 on that set (up to capacity zero) and harmonic off it. Regularity of \(x_0\) is equivalent to \(v(x)\to1\) as \(x\to x_0\) for all small \(r\). Splitting the set into the shells \(E_k\) and estimating each piece’s potential at \(x_0\) by \(c\,2^{k(n-2)}\operatorname{cap}(E_k)\) gives both directions, the hard part being to show that a convergent series leaves a gap.
The probabilistic reading
Shizuo Kakutani showed in 1944 that \(u_g(x)=\mathbb E_x\,g(B_\tau)\), where \(B\) is Brownian motion started at \(x\) and \(\tau\) its first exit time from \(D\). A point \(x_0\) is regular exactly when Brownian motion started at \(x_0\) leaves \(D\) immediately with probability one. Wiener’s two great works of the early 1920s, on Brownian motion and on the Dirichlet problem, turned out to be one subject.
Exercise 5.1Show that the ball \(\overline{B(0,r)}\subset\mathbb R^3\) has capacity \(4\pi r\), using the trial function \(\varphi=\min(1,r/|x|)\) (which is admissible after smoothing) and the fact that it is the minimizer.
Show solution
Exercise 5.2(a) Suppose the complement of \(D\) contains a cone with vertex \(x_0\). Show that \(x_0\) is regular. (b) Let \(D\) be a punctured ball \(B(x_0,1)\setminus\{x_0\}\) in \(\mathbb R^3\). Show \(x_0\) is irregular.
Show solution
Exercise 5.3Lebesgue’s spine. In \(\mathbb R^3\) remove from a ball the thin solid of revolution \(\{(x,y,z):z\ge0,\ \sqrt{x^2+y^2}\le f(z)\}\) with \(f\) decreasing to 0 very fast as \(z\downarrow0\). Granting that a cylinder of length \(\ell\) and radius \(a\ll\ell\) has capacity about \(4\pi\ell/(2\log(\ell/a))\), explain why \(0\) is irregular when \(f(z)=e^{-1/z}\).
Show solution
Generalized harmonic analysis
For the general reader. A musical chord can be split into pure tones by Fourier analysis. But noise, the hiss of a radio or the light of a lamp, never repeats and never dies away, so it has neither a Fourier series nor a Fourier integral. Wiener showed how to give such signals a spectrum anyway, by first measuring how a signal correlates with a shifted copy of itself. The power spectrum of every signal processor and every optical spectrometer rests on this idea.
- Define the autocorrelation and spectrum of a signal that is neither periodic nor integrable.
- State the Wiener–Khinchin theorem for stationary processes.
- Compute spectra of standard examples.
Signals of finite power
Let \(f:\mathbb R\to\mathbb C\) be measurable with finite mean power, and suppose that for every \(\tau\) the limit
\[ \varphi(\tau)=\lim_{T\to\infty}\frac1{2T}\int_{-T}^{T}f(t+\tau)\,\overline{f(t)}\,dt \]exists. Wiener called \(\varphi\) the autocorrelation of \(f\). It is positive definite: \(\sum_{j,k}c_j\bar c_k\varphi(\tau_j-\tau_k)\ge0\). If \(\varphi\) is continuous, the theorem of Gustav Herglotz and Salomon Bochner gives \(\varphi(\tau)=\int e^{i\omega\tau}\,dS(\omega)\) with \(S\) bounded and nondecreasing. Wiener’s 1930 memoir in Acta Mathematica took \(S\), the integrated spectrum, as the spectrum of \(f\). He also showed how to obtain \(S\) from \(f\) itself, through a generalized Fourier transform of \(f\) whose increments over short frequency intervals carry the power. A spectrum therefore exists for every signal whose autocorrelation exists, and it can have jumps (pure tones), a density (noise), or a singular part.
Stationary processes
Let \(X(t)\) be a wide-sense stationary process: \(\mathbb E X(t)=0\), \(\mathbb E|X(t)|^2<\infty\), and \(R(\tau)=\mathbb E\,X(t+\tau)\overline{X(t)}\) does not depend on \(t\). If \(R\) is continuous at 0, there is a unique bounded nondecreasing right-continuous \(F\) with \(F(-\infty)=0\) such that
\[ R(\tau)=\int_{-\infty}^{\infty}e^{i\omega\tau}\,dF(\omega). \]If \(R\in L^1\), then \(F\) has the continuous density \(S(\omega)=\frac1{2\pi}\int R(\tau)e^{-i\omega\tau}\,d\tau\ge0\), the power spectral density.
Aleksandr Khinchin stated the theorem for stationary processes in 1934, and it is usually called the Wiener–Khinchin theorem. When the process is ergodic, the time averages of Wiener’s definition equal the ensemble averages of Khinchin’s, which is why the two approaches agree in practice.
Exercise 6.1Let \(f(t)=\sum_{k=1}^{m}a_ke^{i\lambda_kt}\) with distinct real \(\lambda_k\). Compute \(\varphi\) and describe \(S\).
Show solution
Exercise 6.2A stationary process has \(R(\tau)=\sigma^2e^{-a|\tau|}\), \(a>0\) (the Ornstein–Uhlenbeck process). Find its spectral density and its behaviour for large \(\omega\).
Show solution
Tauberian theorems
For the general reader. An averaging procedure can make sense of a divergent series, and a smoother average gives more series a value. A Tauberian theorem runs the other way: it says that if a smooth average converges and the terms are not too wild, then a rougher average converges too. Wiener found that all such theorems follow from one statement about Fourier transforms. As a by-product he obtained a new proof of the prime number theorem.
- State Wiener’s approximation theorem and his general Tauberian theorem.
- Prove Wiener’s lemma on absolutely convergent Fourier series by Gelfand’s method.
- Understand the Wiener–Ikehara route to the prime number theorem.
From Abel and Tauber to Wiener
Abel’s theorem says that if \(\sum a_n=s\) then \(\sum a_nx^n\to s\) as \(x\uparrow1\). The converse fails, but Alfred Tauber proved in 1897 that it holds if \(na_n\to0\), and John Littlewood weakened this in 1911 to \(na_n\) bounded, opening a long collaboration with Hardy on such theorems. Each such theorem needed its own proof. In “Tauberian theorems” (1932), a hundred-page paper in the Annals of Mathematics, Wiener transferred the problem to the real line, where averaging becomes convolution, and found the common mechanism.
Let \(K\in L^1(\mathbb R)\). The linear span of the translates \(K(\cdot-a)\), \(a\in\mathbb R\), is dense in \(L^1(\mathbb R)\) if and only if \(\hat K(\xi)\ne0\) for every real \(\xi\).
Let \(K\in L^1(\mathbb R)\) with \(\hat K\) nowhere zero, and let \(\phi\in L^\infty(\mathbb R)\). If \(\displaystyle\lim_{x\to\infty}(K*\phi)(x)=A\int K\), then \(\displaystyle\lim_{x\to\infty}(G*\phi)(x)=A\int G\) for every \(G\in L^1(\mathbb R)\).
The second follows from the first: the set of \(G\in L^1\) for which the conclusion holds is a closed subspace, invariant under translation, containing \(K\); by the approximation theorem it is all of \(L^1\). The approximation theorem in turn rests on a discrete lemma.
If \(f(\theta)=\sum_{n\in\mathbb Z}c_ne^{in\theta}\) with \(\sum|c_n|<\infty\) and \(f(\theta)\ne0\) for all \(\theta\), then \(1/f\) also has an absolutely convergent Fourier series.
Proof (Gelfand, 1941). The functions with absolutely convergent Fourier series form a commutative Banach algebra \(A\) under pointwise multiplication, with norm \(\sum|c_n|\). Every maximal ideal of a commutative Banach algebra with unit is the kernel of a nonzero homomorphism \(h:A\to\mathbb C\). Such an \(h\) is determined by \(h(e^{i\theta})=z\), and \(|z|\le1\), \(|1/z|=|h(e^{-i\theta})|\le1\), so \(z=e^{i\theta_0}\) and \(h(f)=f(\theta_0)\). If \(f\) never vanishes it lies in no maximal ideal, hence is invertible in \(A\). ∎ Wiener’s own proof was a direct local argument. Israel Gelfand’s proof made the lemma the founding example of the theory of Banach algebras.
The prime number theorem
Let \(A\) be nondecreasing on \([0,\infty)\), and suppose \(F(s)=\int_0^\infty e^{-sx}\,dA(x)\) converges for \(\operatorname{Re}s>1\) and that \(F(s)-\frac{c}{s-1}\) extends continuously to \(\operatorname{Re}s\ge1\). Then \(e^{-x}A(x)\to c\) as \(x\to\infty\).
Take \(A(x)=\psi(e^x)\), where \(\psi(y)=\sum_{n\le y}\Lambda(n)\) is Chebyshev’s function. Then \(F(s)=-\zeta'(s)/\zeta(s)\), which has a simple pole with residue 1 at \(s=1\) and is otherwise continuous on \(\operatorname{Re}s\ge1\) because \(\zeta(1+it)\ne0\) for real \(t\ne0\). The theorem gives \(\psi(y)\sim y\), which is the prime number theorem. Shikao Ikehara, Wiener’s first doctoral student, published the theorem in 1931. Wiener emphasized that the non-vanishing of \(\zeta\) on the line \(\operatorname{Re}s=1\) is exactly the Tauberian condition that \(\hat K\) never vanishes.
Exercise 7.1Show that \(\sum(-1)^n\) is Abel summable to \(\tfrac12\), and explain why no Tauberian condition of Hardy–Littlewood type can hold for it.
Show solution
Exercise 7.2(a) Let \(f(\theta)=1-ce^{i\theta}\) with \(|c|<1\). Find the Fourier series of \(1/f\) and its \(A\)-norm. (b) Explain what goes wrong as \(|c|\to1\).
Show solution
Exercise 7.3Let \(K(x)=e^{-|x|}\). Compute \(\hat K\) and conclude that the translates of \(K\) span a dense subspace of \(L^1(\mathbb R)\). Give an \(L^1\) function whose translates are not dense.
Show solution
Paley–Wiener and Wiener–Hopf
For the general reader. A signal that lasts only a finite time has a spectrum that is very smooth: it extends to an analytic function of a complex frequency. A filter that cannot see into the future has a response with a similar analytic property. In the early 1930s Wiener, with the young English mathematician Raymond Paley, turned these facts into exact theorems. With Eberhard Hopf he used the same ideas to solve a class of integral equations on a half-line that arise in radiation, diffraction and, later, prediction.
- State the Paley–Wiener theorem for band-limited and time-limited functions.
- State the Paley–Wiener criterion for causal filters.
- Understand the factorization method of Wiener and Hopf.
The Paley–Wiener theorem
Let \(A>0\). An entire function \(F\) has the form \(F(z)=\int_{-A}^{A}f(t)e^{izt}\,dt\) with \(f\in L^2(-A,A)\) if and only if \(F\) is of exponential type at most \(A\), that is, \(|F(z)|\le C_\varepsilon e^{(A+\varepsilon)|z|}\) for every \(\varepsilon>0\), and \(\int_{\mathbb R}|F(x)|^2dx<\infty\).
Read in reverse, a function in \(L^2(\mathbb R)\) whose spectrum is confined to \([-A,A]\), a band-limited signal, extends to an entire function of exponential type. Such a function cannot vanish on an interval unless it vanishes identically, so no nonzero signal is limited both in time and in frequency. This is the mathematical background of the sampling theorem in the Shannon course.
Let \(\Phi\ge0\) be in \(L^2(\mathbb R)\). There is a function \(h\in L^2(\mathbb R)\) vanishing for \(t<0\) with \(|\hat h(\omega)|=\Phi(\omega)\) almost everywhere if and only if
\[ \int_{-\infty}^{\infty}\frac{|\log\Phi(\omega)|}{1+\omega^2}\,d\omega<\infty. \]A filter whose impulse response vanishes for \(t<0\) is causal: its output never depends on future input. The criterion says which amplitude responses a causal filter can have. In particular \(\Phi\) cannot vanish on a set of positive measure, so an ideal filter that cuts off a band exactly is physically unrealizable. Paley died on 7 April 1933, aged 26, in an avalanche while skiing near Banff in Canada. Wiener completed their book, Fourier Transforms in the Complex Domain, published in 1934.
The Wiener–Hopf method
Eberhard Hopf, an Austrian-born mathematician then at MIT, had been studying Edward Arthur Milne’s equation for radiative transfer in a stellar atmosphere. Together, in 1931, Wiener and Hopf solved the class of equations
\[ f(x)-\int_0^\infty k(x-y)\,f(y)\,dy=g(x),\qquad x>0, \]with \(k\in L^1(\mathbb R)\). The difficulty is that the kernel is translation invariant, which suggests the Fourier transform, while the equation holds only on a half-line. The method has four steps.
- Extend the equation to all \(x\) by defining \(f=g=0\) for \(x<0\) and letting \(h\) be the unknown left side for \(x<0\). Transforms of functions supported on \(x>0\) (written \(F_+\), with \(F_+(\xi)=\int_0^\infty f(x)e^{i\xi x}dx\)) are analytic in the upper half-plane; those supported on \(x<0\) (written \(H_-\)) are analytic in the lower half-plane.
- Transform: \(\big(1-\hat k(\xi)\big)F_+(\xi)=G_+(\xi)+H_-(\xi)\) on the real line.
- Factorize \(1-\hat k=M_+M_-\), with \(M_\pm\) analytic and nonvanishing in the upper and lower half-planes respectively, and split \(G_+/M_-=[G_+/M_-]_++[G_+/M_-]_-\).
- Then \(M_+F_+-[G_+/M_-]_+=[G_+/M_-]_-+H_-/M_-\). The left side is analytic above, the right below; together they define an entire function, which is zero by growth conditions and Liouville’s theorem. Hence \(F_+=[G_+/M_-]_+/M_+\).
The factorization exists under mild conditions when \(1-\hat k\) does not vanish and has winding number zero; it is given by \(\log M_\pm\) = the upper and lower parts of \(\log(1-\hat k)\). The method became a standard tool for diffraction by a half-plane, for crack problems in elasticity, and, through Lesson 09, for optimal prediction.
Exercise 8.1Let \(f=\mathbf 1_{[-A,A]}\). Compute \(F(z)=\int_{-A}^Af(t)e^{izt}dt\) and check the hypotheses of the Paley–Wiener theorem.
Show solution
Exercise 8.2Can a causal filter have the Gaussian amplitude response \(\Phi(\omega)=e^{-\omega^2}\)? What about \(\Phi(\omega)=1/(1+\omega^2)\)?
Show solution
Exercise 8.3With the convention that \(+\) functions are analytic in \(\operatorname{Im}\xi>0\), factorize \(\frac{\xi^2+b^2}{\xi^2+a^2}\) (\(a,b>0\)) as \(M_+M_-\).
Show solution
Prediction and filtering
For the general reader. An anti-aircraft gun must fire at where the aircraft will be, not where it is, and the radar track is noisy. In 1940–42 Wiener treated this as a problem of statistics: given the past of a noisy signal, find the best linear estimate of its future. The answer, the Wiener filter, separates signal from noise using only their spectra. The same theory was found independently in Moscow by Andrey Kolmogorov, and its descendants run in every phone, radar and navigation system.
- Derive the non-causal Wiener filter from the orthogonality principle.
- State the Wiener–Hopf equation for the causal filter.
- State the Kolmogorov–Szegő formula for the one-step prediction error.
The smoothing problem
Let \(x(t)=s(t)+n(t)\), where signal \(s\) and noise \(n\) are jointly wide-sense stationary with spectral densities \(S_s\), \(S_n\) and cross-spectrum \(S_{sn}\). We seek a filter \(h\) with output \(\hat s(t)=\int h(\tau)x(t-\tau)\,d\tau\) minimizing \(\mathbb E|s(t)-\hat s(t)|^2\). The error is minimal exactly when it is orthogonal to every observation: \(\mathbb E\big[(s(t)-\hat s(t))\overline{x(t-u)}\big]=0\) for every admissible \(u\).
(i) If all \(u\in\mathbb R\) are admissible (smoothing with the whole record), the optimal filter has frequency response \(H(\omega)=S_{sx}(\omega)/S_x(\omega)\); if \(s\) and \(n\) are uncorrelated,
\[ H(\omega)=\frac{S_s(\omega)}{S_s(\omega)+S_n(\omega)},\qquad \text{minimal error}=\int_{-\infty}^{\infty}\frac{S_s(\omega)S_n(\omega)}{S_s(\omega)+S_n(\omega)}\,d\omega. \](ii) If only the past is admissible (\(h(\tau)=0\) for \(\tau<0\)), \(h\) solves the Wiener–Hopf equation
\[ \int_0^\infty h(\tau)R_x(u-\tau)\,d\tau=R_{sx}(u),\qquad u\ge0, \]and is found by factorizing \(S_x=|M_+|^2\) as in Lesson 08.
Here spectral densities are normalized so that \(R(\tau)=\int e^{i\omega\tau}S(\omega)\,d\omega\). The filter passes frequencies where the signal dominates and suppresses those where noise dominates. Prediction \(\alpha\) seconds ahead is the case \(R_{sx}(u)=R_s(u+\alpha)\) with no noise.
Wiener’s report, bound in yellow covers and nicknamed the “yellow peril” by engineers who struggled with its mathematics, was classified in 1942 and published in 1949 as Extrapolation, Interpolation, and Smoothing of Stationary Time Series. Kolmogorov had solved the discrete-time problem in papers of 1939 and 1941. Rudolf Kalman’s 1960 filter recast the problem in state-space form and allowed nonstationary signals.
How predictable is a process?
Let \(X_n\) be a stationary sequence with spectral density \(S\) on \([-\pi,\pi]\), normalized by \(R(k)=\frac1{2\pi}\int_{-\pi}^{\pi}e^{ik\lambda}S(\lambda)\,d\lambda\). The minimal mean square error in predicting \(X_0\) linearly from \(X_{-1},X_{-2},\dots\) is
\[ \sigma^2=\exp\Big(\frac1{2\pi}\int_{-\pi}^{\pi}\log S(\lambda)\,d\lambda\Big). \]In particular the future is perfectly predictable from the past (\(\sigma^2=0\)) exactly when \(\int\log S=-\infty\), for instance when \(S\) vanishes on an interval. This is the discrete form of the Paley–Wiener criterion.
Exercise 9.1Signal and noise are uncorrelated with \(S_s(\omega)=\frac{1}{1+\omega^2}\) and white noise \(S_n(\omega)=N\). Find the non-causal Wiener filter and describe it for small \(N\).
Show solution
Exercise 9.2An AR(1) sequence \(X_n=aX_{n-1}+\varepsilon_n\), \(|a|<1\), \(\varepsilon_n\) white with variance \(v\), has \(S(\lambda)=v/|1-ae^{-i\lambda}|^2\). Use the Kolmogorov–Szegő formula to find \(\sigma^2\) and check it directly.
Show solution
Cybernetics: feedback and purpose
For the general reader. When you reach for a glass, your eyes report the error between hand and glass, and your muscles correct it continuously. A thermostat and a ship’s steering engine do the same. Wiener saw in the 1940s that machines and animals share a common logic of control through feedback and communication, and he named the science of that logic cybernetics, from the Greek for steersman. The idea shaped control engineering, neuroscience, early artificial intelligence and the language of the information age.
- Model a feedback loop with delay and find its stability boundary.
- Relate overcorrection to oscillation, Wiener’s explanation of intention tremor.
- Understand the cybernetic programme and its measure of information.
Purpose as feedback
The fire-control project showed Wiener and his engineer colleague Julian Bigelow that the human gunner was part of the control loop. With the physiologist Arturo Rosenblueth they published “Behavior, purpose and teleology” in Philosophy of Science in 1943. It argued that purposeful behaviour can be defined by negative feedback: a system acts to reduce the difference between its state and a goal. Rosenblueth had described patients with cerebellar damage whose hands oscillate more and more widely as they reach for an object. Wiener and Bigelow recognized this intention tremor as the oscillation of a feedback loop with too much gain.
Let \(k>0\) and \(\tau\ge0\). Every solution of \(\dot x(t)=-k\,x(t-\tau)\) tends to 0 if and only if \(k\tau<\pi/2\). At \(k\tau=\pi/2\) there are persistent oscillations of angular frequency \(k\), period \(4\tau\). Non-oscillating solutions exist exactly when \(k\tau\le1/e\); for \(k\tau>1/e\) every nonzero solution oscillates.
Proof sketch. Solutions are combinations of \(e^{st}\) with \(s+ke^{-s\tau}=0\). A root crosses the imaginary axis at \(s=i\omega\) when \(i\omega=-ke^{-i\omega\tau}\), that is \(\omega=k\) and \(\omega\tau=\pi/2\). For small \(k\tau\) all roots have negative real part, and they stay there until the first crossing. A real root exists when \(\min_{s}(s+ke^{-s\tau})\le0\), which happens exactly when \(k\tau\le1/e\). ∎
The cybernetic programme
Cybernetics: or Control and Communication in the Animal and the Machine (1948) gathered prediction, feedback, the nervous system, computing machines and information into one book. Wiener defined the amount of information in a continuous distribution with density \(p\) as \(\int p\log_2p\,dx\), the negative of what Claude Shannon published the same year as entropy; Wiener regarded information as negative entropy, a measure of organization. The book sold widely despite pages of difficult mathematics. The interdisciplinary Macy conferences on feedback mechanisms (1946–1953), chaired by Warren McCulloch, brought together Wiener, John von Neumann, Walter Pitts, Margaret Mead, Gregory Bateson and others. A second edition of Cybernetics in 1961 added chapters on learning, self-reproducing machines and brain waves.
Exercise 10.1A discrete controller corrects a fraction \(k\) of the error at each step: \(e_{n+1}=e_n-k\,e_n\). For which \(k\) does the error tend to zero? When does it overshoot and oscillate?
Show solution
Exercise 10.2A reaching movement is corrected through a visual feedback loop with delay \(\tau=0.1\) s. Using the theorem, estimate the largest stable gain and the frequency of the tremor that appears when it is exceeded.
Show solution
Exercise 10.3Verify the condition for a real root: show that \(s+ke^{-s\tau}=0\) has a real solution iff \(k\tau\le1/e\).
Show solution
Fame, war and conscience, 1926–1950
In 1926 Wiener married Margaret Engemann, who taught modern languages at Juniata College in Pennsylvania; their daughters Barbara and Margaret were born in 1928 and 1929. A Guggenheim fellowship took him to Göttingen and Copenhagen in 1926–27. He was promoted to associate professor in 1929 and full professor in 1932, after a year in Cambridge with Hardy in 1931–32 that produced the collaboration with Paley. In 1933 he shared the Bôcher Memorial Prize of the American Mathematical Society with Marston Morse, for the Tauberian work, and was elected to the National Academy of Sciences. He gave the Society’s Colloquium Lectures in 1934 and a plenary address at the International Congress in Oslo in 1936.
He supervised students from mathematics and electrical engineering alike. Among them were Shikao Ikehara (1930), Yuk-Wing Lee (1930), with whom he designed and patented electrical networks, and Norman Levinson (1935), who became one of the leading analysts of his generation. Brockway McMillan, later a pioneer of information theory and an official of the US Air Force, was another. In 1935–36 Wiener spent a year as visiting professor at Tsing Hua University in Peking, at Lee’s invitation; he later dated from that year his maturity as an independent scientist.
The war and after
Wiener disliked honorific bodies, and in 1941 he resigned from the National Academy, which accepted the resignation with regret. That year, invited by Vannevar Bush, he began the anti-aircraft prediction project for the National Defense Research Committee described in Lesson 09. The theory was a success; the predictor built with Bigelow was not used, because its advantage over simpler devices was small. He left the project in 1943.
The atomic bombing of Hiroshima and Nagasaki changed his view of military science. In January 1947 an engineer at an aircraft company asked him for a copy of a wartime paper. Wiener refused in a letter published in the Atlantic Monthly under the title “A scientist rebels”: he would no longer publish work that could harm in the hands of irresponsible militarists. He withdrew from a symposium funded by the military and declined defence contracts thereafter. In The Human Use of Human Beings (1950) he warned that automation would displace workers as the industrial revolution had, and wrote to labour leaders to warn them. He gave a second plenary address at the International Congress in Cambridge, Massachusetts, in 1950.
DiscussionWiener developed fire-control theory during the war and refused military funding after it. Was his position consistent? How should a mathematician weigh the possible uses of results that are themselves neutral, such as the theory of prediction?
Last years, death and legacy, 1950–1964
In his last fifteen years Wiener was a public figure. He lectured on cybernetics in Paris as a Fulbright professor at the Collège de France (1951), in India at the Indian Statistical Institute in Calcutta in the mid-1950s, and in Naples (1960–61). He wrote two volumes of autobiography, Ex-Prodigy (1953) and I Am a Mathematician (1956), a novel, The Tempter (1959), and Nonlinear Problems in Random Theory (1958), which applied the homogeneous chaos to the analysis of nonlinear systems. MIT made him an Institute Professor in 1959. Stories of his absent-mindedness circulated widely at MIT; his family said many were exaggerated.
In January 1964 President Lyndon Johnson presented him with the National Medal of Science, awarded for 1963, for his contributions to pure and applied mathematics and his penetration into engineering and biology. Two months later, on 18 March 1964, he died of a heart attack in Stockholm, aged 69, while on a lecture visit to Europe. He is buried in Vittum Hill Cemetery in Sandwich, New Hampshire, in the New Hampshire hills where he spent his summers. His last book, God and Golem, Inc., on the moral questions posed by learning and self-reproducing machines, appeared that year and won the National Book Award in 1965.
Legacy
- Probability: Wiener measure is the starting point of stochastic analysis: Itô calculus, stochastic differential equations, the Feynman–Kac formula, and the Black–Scholes model of finance.
- Analysis: the general Tauberian theorem and Wiener’s lemma founded the theory of Banach algebras and the spectral synthesis problem; the Paley–Wiener theorems are basic in harmonic analysis and PDE.
- Potential theory: Wiener’s test and his notion of capacity are the model for regularity criteria for elliptic and parabolic equations.
- Engineering: the Wiener filter and the Wiener–Hopf method are standard in signal processing, control and diffraction theory; the Kalman filter is their successor.
- Cybernetics: the word and the programme influenced control theory, neuroscience, cognitive science, systems biology and artificial intelligence, and gave the prefix “cyber” to the language.
- Honours: the Norbert Wiener Prize in Applied Mathematics of the American Mathematical Society and the Society for Industrial and Applied Mathematics; the Norbert Wiener Award for social responsibility in computing; a lunar crater.
Final projectChoose one of Lessons 04, 07 or 09. Write a two-page account for a strong undergraduate that states the main theorem, works one example in full, and explains how the result connects to another lesson of this course (for example, Brownian motion to the Dirichlet problem, or Tauberian theorems to prediction).
Connections
People linked to Norbert Wiener across this collection. A name without a link has no page here yet. See Norbert Wiener in the network of all 337 lives →
Teachers and advisors
- Bertrand Russell 1913-1914
- Karl Schmidt 1913
Students
- Brockway McMillan
- Norman Levinson 1935
- Shikao Ikehara 1930
- Yuk-Wing Lee 1930
Influenced by
- Albert Einstein
- Bertrand Russell 1913-1914
- David Hilbert 1914
- G. H. Hardy 1913-1932
- Henri Lebesgue
- Edmund Landau 1914
- Josiah Royce
Influenced
Colleagues and collaborators
- Claude Shannon
- John von Neumann
- Maurice Fréchet 1920
- Max Born 1925-1926
- Paul Lévy
- Aleksandr Khinchin
- Antoni Zygmund 1933
- Arturo Rosenblueth 1943-1964
- Eberhard Hopf 1930-1931
- Julian Bigelow 1941-1943
- Raymond Paley 1931-1933
- Vannevar Bush
- Walter Pitts
- Warren McCulloch
Rivals
Family
Also mentioned in this life
- Niels Henrik Abel
- Pafnuty Chebyshev
- Peter Gustav Lejeune Dirichlet
- Leonhard Euler
- Joseph Fourier
- Werner Heisenberg
- Kazimierz Kuratowski
- Joseph-Louis Lagrange
- Joseph Liouville
- Jean Perrin
Fields
- harmonic analysis 3 lives
- probability theory 23 lives
- potential theory 9 lives
Sources and further reading
Biography
- J. J. O’Connor and E. F. Robertson, Norbert Wiener, MacTutor History of Mathematics.
- Norbert Wiener, Wikipedia.
- Norbert Wiener 1894–1964, National Academy of Sciences, Biographical Memoirs 61 (1992).
- N. Levinson, Wiener’s life, Bull. Amer. Math. Soc. 72 (1966), part II, 1–32 (special issue on Wiener’s work).
- N. Wiener, Ex-Prodigy: My Childhood and Youth, Simon and Schuster, 1953; I Am a Mathematician, Doubleday, 1956.
- P. R. Masani, Norbert Wiener 1894–1964, Birkhäuser, 1990.
- F. Conway and J. Siegelman, Dark Hero of the Information Age, Basic Books, 2005.
- G. Strand, The “rebellious scientist” who inspired Kurt Vonnegut, Science Friday, 2015.
Mathematics
- N. Wiener, A simplification of the logic of relations, Proc. Cambridge Philos. Soc. 17 (1914) 387–390.
- N. Wiener, Differential-space, J. Math. and Phys. 2 (1923) 131–174.
- N. Wiener, The Dirichlet problem, J. Math. and Phys. 3 (1924) 127–146.
- M. Born and N. Wiener, Eine neue Formulierung der Quantengesetze für periodische und nicht periodische Vorgänge, Z. Physik 36 (1926) 174–187.
- N. Wiener, Generalized harmonic analysis, Acta Math. 55 (1930) 117–258.
- N. Wiener and E. Hopf, Über eine Klasse singulärer Integralgleichungen, Sitzungsber. Preuss. Akad. Wiss. (1931) 696–706.
- N. Wiener, Tauberian theorems, Ann. of Math. 33 (1932) 1–100.
- R. E. A. C. Paley and N. Wiener, Fourier Transforms in the Complex Domain, AMS Colloquium Publications 19, 1934.
- N. Wiener, The homogeneous chaos, Amer. J. Math. 60 (1938) 897–936.
- A. Rosenblueth, N. Wiener and J. Bigelow, Behavior, purpose and teleology, Philosophy of Science 10 (1943) 18–24.
- N. Wiener, Extrapolation, Interpolation, and Smoothing of Stationary Time Series, MIT Press, 1949.
- N. Wiener, Cybernetics, MIT Press, 1948; 2nd ed. 1961.
- I. M. Gelfand, Über absolut konvergente trigonometrische Reihen und Integrale, Mat. Sbornik 9 (1941) 51–66.
- J. Korevaar, Tauberian Theory: A Century of Developments, Springer, 2004.
- P. Mörters and Y. Peres, Brownian Motion, Cambridge University Press, 2010.