The idea behind Fourier series is to try and express some function on a domain $[-L,L]$ into a sum of complex exponentials of the form $\frac{1}{\sqrt{2L}}e^{2\pi i \ nx/L}$. One of the reasons this is interesting is that the complex exponentials are orthonormal system under the dot product $\int f(x)\overline{g(x)}\ dx$.
One can start by considering the vector space $V$ of all square-integrable functions on $[-L,L]$ -- this gives us a vector space with an inner product. Specifically, we're interested in the subspace $V_n$ that is the span of complex exponentials upto $n$ and $-n$.Then given a vector $f$ in $V$, we can ask for its projection $f_n$ onto $V_n$.
As the complex exponentials are already orthonormal, it is easy to calculate this projection in their basis:
\[\begin{gathered}
{a_k} = \left\langle {f,\frac{1}{\sqrt{2L}}{e^{2\pi i\;nx/L}}} \right\rangle = \int\limits_{ - L}^L {f(x)\frac{e^{ - 2\pi i\;kx/L}}{\sqrt{2L}}dx} \hfill \\
{f_n}(x) = \sum\limits_{|k|\le n} {{a_k}\frac{e^{2\pi i\;kx/L}}{\sqrt{2L}}} \hfill \\
\end{gathered} \]
Notably this implies by Cauchy-Schwarz that:
\[{\left| f \right|^2} \geqslant \sum\limits_{|k| \leqslant n} {{{\left| {{a_k}} \right|}^2}} \]
This really just is Cauchy-Schwarz, and is known as Bessel's inequality. If we can show that the Fourier series approaches $f$, i.e. that $\left\|f-f_n\right\|\to 0$, then it would be obvious that
\[{\left| f \right|^2} = \sum\limits_{|k| \in \mathbb{Z}} {{{\left| {{a_k}} \right|}^2}} \]
Which is just the Pythagoras theorem, and is known as Parseval's theorem. Obviously, these theorems exist in the general theory of Hilbert spaces.
Insights on interesting things.
Showing posts with label fourier. Show all posts
Showing posts with label fourier. Show all posts
Contour Integration II: everything about singularities; what gives life to Pi
Stuff we'll cover in this article:
The story so far: if a function has no screw-up points within a closed contour, its integral on that contour is zero. But if it does, it may not be.
And by a screw-up point, we just mean a point at which the function isn't (or rather cannot be -- this is what we call a non-removable singularity) holomorphic.
But why? Let's look at the antiderivative of $1/z$ -- $\mathrm{log}(z)$. It's fundamentally a multivalued function -- and here's what's interesting: a loop around the origin isn't actually a loop on this graph -- it brings you to a higher level than you were previously (specifically $2\pi i$ higher than you were previously).
And you can kinda see why this comes about -- the derivative of this function not being holomorphic at 0 is encapsulated by the fact that the function is all torn up at 0 -- its slope must be different in different directions, because you have multiple planes stuck to that point. This is the idea behind a branch point -- a point such that the function is discontinuous when going about an arbitrarily small circuit about the point.
One can now start to see how integrals that do loop around the origin behave -- for starters, the winding number of the contour corresponds to the number of levels you climb during the integral. Also, the distance between levels should depend purely on the local nature of the function around the branch point (because the antiderivative has a defined derivative (the function $f$, in this case $1/z$), so the spacing must remain constant). By similar reasoning, encircling multiple poles means climbing all their levels (so the total distance climbed adds up).
This, above, is the residue theorem. For a function $f$, a simply connected open set $U$ such that $f$ is holomorphic on $U-\{a_1,\dots a_k\}$, and a closed contour $\gamma$ contained in this punctured set:
$$\oint_\gamma f(z) dz = \sum_{k=1}^n \mathrm{W}(\gamma, a_k)\mathrm{Res}(f,a_k)$$
Where $\mathrm{Res}(f,a)$ is the residue of $f$ at $a$, which is the "local quantity" that equals the "distance between layers" of the multivalued antiderivative.
This is, obviously, awesome. But thinking about generally handling and computing these residues $\mathrm{Res}(f,a)=\oint_{\circ}f(z)dz$ leads us to wonder about the general nature of branch cuts.
Here's an idea: we know exactly what the residue for $f(z)=z^{-1}$ at 0 is -- it's $2\pi i$. One also easily sees that the residue of $f(z)=z^n$ for any other $n$ is zero (the antiderivative $z^{n+1}/(n+1)$ does not have branch cuts).
I wonder if -- like how Taylor series allow us to represent a function near a general point $a$ as a sum of $(z-a)^n$s for nonnegative $n$ -- we could represent a function near a singularity $a$ as a sum of $(z-a)^n$s for all integer $n$s.
I wonder if all "serious" singularities are just ultimately $1/z^k$-style singularities.
This is, in fact, what is known as the Laurent series -- a representation $f(z)=\sum_{n=-\infty}^\infty c_n (z-a)^n $. Then the distance climbed by $f(z)$ is equal to the distance climbed by the $(z-a)^{-1}$ term, which is just $2\pi i c_{-1}$.
So the point of this "Laurent series interpretation" of the residue is this:
But to actually "prove" this interpretation means to:
The first one is easy. Obviously we need ${c_{ - 1}} = \frac{1}{{2\pi i}}\oint_\circ {f(z)\,dz}$. Analogously by considering this expression for the function $\frac{{f(z)}}{{{{(z - a)}^{n + 1}}}}$ to extract the other coefficients, we see that:
$${c_n} = \frac{1}{{2\pi i}}\oint_\circ {\frac{{f(z)}}{{{{(z - a)}^{n + 1}}}}\,dz} $$
The second and third are actually challenging -- but there's a remarkable fortunate observation one can make. The fact that the Laurent series is defined on an annulus (each of the power series has a radius of convergence, which puts a maximum bound on both $z$ and $|z|$) is incredibly suggestive of describing a periodic function. Indeed, a variable substitution $z=\rho e^{i\theta}$ turns the Laurent series into a Fourier series. The properties 2 and 3 then follow from properties of the Fourier series.
This "Fourier series" interpretation of the Laurent series also makes it easy to see Cauchy's integral formula and holomorphicity implies analyticity.
By the way, I would argue that the notion of a residue and the Laurent series is the fundamental source of the importance of $\pi$. Perhaps the standard motivation for $\pi$ is that $2\pi i$ is the period of the exponential function. This is equivalent to saying it's the residue of the logarithm function. The existence of Laurent series expansions -- i.e. all branch cuts being logarithmic in nature -- is what gives importance to these residues.
Exercise: prove the Cauchy differentiation formula -- for a holomorphic function $f$,
$$f^{(n)}(a)=\frac{n!}{2\pi i}\oint_\gamma \frac{f(z)}{(z-a)^{n+1}}dz $$
- Residues as "climbing between the values of a multivalued antiderivative"
- Winding numbers and the residue theorem
- All residues are logarithmic residues: the Laurent series
- "Proving Laurent series": Laurent series as Fourier series
- How the residue theorem gives life to $\pi$
The story so far: if a function has no screw-up points within a closed contour, its integral on that contour is zero. But if it does, it may not be.
And by a screw-up point, we just mean a point at which the function isn't (or rather cannot be -- this is what we call a non-removable singularity) holomorphic.
But why? Let's look at the antiderivative of $1/z$ -- $\mathrm{log}(z)$. It's fundamentally a multivalued function -- and here's what's interesting: a loop around the origin isn't actually a loop on this graph -- it brings you to a higher level than you were previously (specifically $2\pi i$ higher than you were previously).
And you can kinda see why this comes about -- the derivative of this function not being holomorphic at 0 is encapsulated by the fact that the function is all torn up at 0 -- its slope must be different in different directions, because you have multiple planes stuck to that point. This is the idea behind a branch point -- a point such that the function is discontinuous when going about an arbitrarily small circuit about the point.
One can now start to see how integrals that do loop around the origin behave -- for starters, the winding number of the contour corresponds to the number of levels you climb during the integral. Also, the distance between levels should depend purely on the local nature of the function around the branch point (because the antiderivative has a defined derivative (the function $f$, in this case $1/z$), so the spacing must remain constant). By similar reasoning, encircling multiple poles means climbing all their levels (so the total distance climbed adds up).
This, above, is the residue theorem. For a function $f$, a simply connected open set $U$ such that $f$ is holomorphic on $U-\{a_1,\dots a_k\}$, and a closed contour $\gamma$ contained in this punctured set:
$$\oint_\gamma f(z) dz = \sum_{k=1}^n \mathrm{W}(\gamma, a_k)\mathrm{Res}(f,a_k)$$
Where $\mathrm{Res}(f,a)$ is the residue of $f$ at $a$, which is the "local quantity" that equals the "distance between layers" of the multivalued antiderivative.
This is, obviously, awesome. But thinking about generally handling and computing these residues $\mathrm{Res}(f,a)=\oint_{\circ}f(z)dz$ leads us to wonder about the general nature of branch cuts.
Here's an idea: we know exactly what the residue for $f(z)=z^{-1}$ at 0 is -- it's $2\pi i$. One also easily sees that the residue of $f(z)=z^n$ for any other $n$ is zero (the antiderivative $z^{n+1}/(n+1)$ does not have branch cuts).
I wonder if -- like how Taylor series allow us to represent a function near a general point $a$ as a sum of $(z-a)^n$s for nonnegative $n$ -- we could represent a function near a singularity $a$ as a sum of $(z-a)^n$s for all integer $n$s.
I wonder if all "serious" singularities are just ultimately $1/z^k$-style singularities.
This is, in fact, what is known as the Laurent series -- a representation $f(z)=\sum_{n=-\infty}^\infty c_n (z-a)^n $. Then the distance climbed by $f(z)$ is equal to the distance climbed by the $(z-a)^{-1}$ term, which is just $2\pi i c_{-1}$.
So the point of this "Laurent series interpretation" of the residue is this:
- The key conceptual takeaway from the Laurent series is that all branch cuts of the antiderivatives of holomorphic functions are "logarithmic". (This does not, e.g. apply to the branch cut of $\sqrt{z}$, as its derivative isn't holomorphic.)
- When actually calculating residues, we can find the Laurent series by other means (such as by patching together different Taylor series) and use its $c_{-1}$ coefficient as the residue.
But to actually "prove" this interpretation means to:
- Write down what its coefficients should be -- this is our candidate series which justifies using it to calculate residues.
- Show that this candidate is a valid Laurent series, i.e. that it actually converges to $f(z)$ on some region.
- Show that it is the valid Laurent series, i.e. that the Laurent series is unique. So we can calculate the Laurent series however we want and use its coefficients to calculate residues.
The Laurent series is not the Taylor series, and the nonnegative coefficients of the Laurent series are not generally the coefficients of the Taylor series. So its coefficients cannot be interpreted as higher derivatives of the function or anything (that doesn't even make sense at that point).
The first one is easy. Obviously we need ${c_{ - 1}} = \frac{1}{{2\pi i}}\oint_\circ {f(z)\,dz}$. Analogously by considering this expression for the function $\frac{{f(z)}}{{{{(z - a)}^{n + 1}}}}$ to extract the other coefficients, we see that:
$${c_n} = \frac{1}{{2\pi i}}\oint_\circ {\frac{{f(z)}}{{{{(z - a)}^{n + 1}}}}\,dz} $$
The second and third are actually challenging -- but there's a remarkable fortunate observation one can make. The fact that the Laurent series is defined on an annulus (each of the power series has a radius of convergence, which puts a maximum bound on both $z$ and $|z|$) is incredibly suggestive of describing a periodic function. Indeed, a variable substitution $z=\rho e^{i\theta}$ turns the Laurent series into a Fourier series. The properties 2 and 3 then follow from properties of the Fourier series.
This "Fourier series" interpretation of the Laurent series also makes it easy to see Cauchy's integral formula and holomorphicity implies analyticity.
By the way, I would argue that the notion of a residue and the Laurent series is the fundamental source of the importance of $\pi$. Perhaps the standard motivation for $\pi$ is that $2\pi i$ is the period of the exponential function. This is equivalent to saying it's the residue of the logarithm function. The existence of Laurent series expansions -- i.e. all branch cuts being logarithmic in nature -- is what gives importance to these residues.
Exercise: prove the Cauchy differentiation formula -- for a holomorphic function $f$,
$$f^{(n)}(a)=\frac{n!}{2\pi i}\oint_\gamma \frac{f(z)}{(z-a)^{n+1}}dz $$
Discovering the Fourier transform
Key ideas in this post
Consider a function with period 1 -- computing its Fourier series, you write it as:
\[f(x) = \sum\limits_{n = - \infty }^\infty {{a_n}{e^{i2\pi \,\,nx}}} \]
Where
\[{a_n} = \int_{-1}^1 {f(x){e^{ - 2\pi inx}}dx} \]
That's all standard and trivial. But suppose you wanted to study a function with a higher period (we will tend this period to infinity) -- what would that look like? Well, consider $g(x)=f(x/L)$, which is this function we're looking for -- then we can rewrite the above identities as:
\[g(xL) = \sum\limits_{n = - \infty }^\infty {{a_n}{e^{i2\pi {\kern 1pt} {\kern 1pt} nx}}} \Rightarrow g(x) = \sum\limits_{n = - \infty }^\infty {{a_n}{e^{i2\pi {\kern 1pt} {\kern 1pt} nx/L}}} \]
\[{a_n} = \int_{-1}^1 {g(xL){e^{ - 2\pi inx}}dx} \Rightarrow {a_n} = \int_{-L}^L {g(x){e^{ - 2\pi inx/L}}dx/L} \]
Where we transformed $x\to x/L$.
This seems all too trivial and useless, and maybe you're looking for a little trick to turn this into something interesting. But tricks must typically also arise from some sort of insight. Let's assume for a moment that we didn't know anything about variable substitutions or transformations like the kind we did above (and indeed, the idea behind variable substitutions also comes from a geometric understanding of the corresponding transformation) and think about how we may re-think the Fourier transform in its context.
Well, if the function's period is $P$, in other words it is stretched out by $P$, the same logic must be used to derive the Fourier series for the new function as for the function with period 1 -- specifically, sines and cosines with longer periods than $P$ don't matter (their coefficient must be zero, because otherwise you've introduced an element into the function that doesn't repeat with that period), but those with shorter, divisible periods matter, because they influence the value of the function within the period, perturbing it by little bits to get to the right function.
So when dealing with our new period $L$, one would expect periods that are fractions of $L$, i.e. $L/n$, as opposed to just $1/n$. So $n/L$ is "more important" than $n$, and indeed it seems very easy to transform the summation into one in terms of this new variable, which we will still call $n$ (i.e. transform $n/L\to n$):
\[g(x) = \sum\limits_n^{} {{a_n}{e^{i2\pi nx}}} \]
\[{a_n} = \frac{1}{L}\int_{-L}^L {g(x){e^{ - 2\pi inx}}dx} \]
Where we labeled $a_{nL}$ as just $a_n$, because that's just a subscript, the labeling doesn't matter. Just remember that $n$ is no longer just an integer/multiple of 1, but a multiple of any fraction $1/L$.
Now note how a non-periodic function is just a function with infinite period, i.e. $L\to\infty$. So $n$ stops being a discrete integer and starts approaching a continuous variable, which we'll call $s$, writing $a_n$ as $a(s)ds$ (why the $ds$? because the increment in $n$ is just $1/L$, which appears in the expression for $a_n$).
\[g(x) = \int_{ - \infty }^\infty {ds\,\,a(s){e^{i2\pi sx}}} \]
\[a(s) = \int_{ - \infty }^\infty {dx\,\,g(x){e^{ - i2\pi sx}}} \]
Which is just a pretty satisfying result.
Recall again the expressions we got for the Fourier transform and its inverse:
\[f(t) = \int_{ - \infty }^\infty {ds\,\,\hat f(s){e^{i2\pi ts}}} \]
\[\hat f(s) = \int_{ - \infty }^\infty {dt{\kern 1pt} {\kern 1pt} f(t){e^{ - i2\pi st}}} \]
(We typically say the Fourier transform maps time-domain functions to frequency-domain ones, so we consider the latter to be the Fourier transform and the first equation to be its inverse.) Note how you can easily turn the first one into an actual Fourier transform, by transforming $s\to -s$:
\[f(t) = \int_{ - \infty }^\infty {ds\,\,\hat f( - s){e^{ - i2\pi ts}}} \]
In other words:
\[{\mathcal{F}^{ - 1}}\left\{ {f(s)} \right\} = \mathcal{F}\left\{ {f( - s)} \right\}\]
And of course that means ${\mathcal{F}^4} = I$, the identity operator (kind of like the derivative on complex exponentials/sine and cosine, is it not?).
- The Fourier series is a decomposition of a periodic function into sums of sinusoids with periods less than it. Representing a function with longer period requires sinusoids with longer periods, which is the same as requiring a denser range of frequencies.
- A non-periodic function can be understood as a function with an infinitely long period, which requires a frequency range of all real numbers.
- So the Fourier transform can be understood as a generalization of the Fourier series that represents all functions as sums of sinusoids. Interestingly, the inverse Fourier series (i.e. the expression for the coefficients) was already an integral, albeit with finite domain, while the Fourier series already had an infinite domain (all integers), albeit not an integral. In the limit where you get the Fourier transform, though, they both magically become the same.
- Anyway, it's then straightforward to see the Fourier relationship between the sinusoid and the Dirac delta function, etc.
- In general, the Fourier transform can be seen as a "change of basis" for functions. Looking at the sinusoids as a basis in this sense, one can immediately infer e.g. Parseval's theorem and its generalisation the Rayleigh energy theorem as the "Pythagoras theorem" for this basis.
- Other changes of basis or linear transformations lead to other integral transforms.
Consider a function with period 1 -- computing its Fourier series, you write it as:
\[f(x) = \sum\limits_{n = - \infty }^\infty {{a_n}{e^{i2\pi \,\,nx}}} \]
Where
\[{a_n} = \int_{-1}^1 {f(x){e^{ - 2\pi inx}}dx} \]
That's all standard and trivial. But suppose you wanted to study a function with a higher period (we will tend this period to infinity) -- what would that look like? Well, consider $g(x)=f(x/L)$, which is this function we're looking for -- then we can rewrite the above identities as:
\[g(xL) = \sum\limits_{n = - \infty }^\infty {{a_n}{e^{i2\pi {\kern 1pt} {\kern 1pt} nx}}} \Rightarrow g(x) = \sum\limits_{n = - \infty }^\infty {{a_n}{e^{i2\pi {\kern 1pt} {\kern 1pt} nx/L}}} \]
\[{a_n} = \int_{-1}^1 {g(xL){e^{ - 2\pi inx}}dx} \Rightarrow {a_n} = \int_{-L}^L {g(x){e^{ - 2\pi inx/L}}dx/L} \]
Where we transformed $x\to x/L$.
This seems all too trivial and useless, and maybe you're looking for a little trick to turn this into something interesting. But tricks must typically also arise from some sort of insight. Let's assume for a moment that we didn't know anything about variable substitutions or transformations like the kind we did above (and indeed, the idea behind variable substitutions also comes from a geometric understanding of the corresponding transformation) and think about how we may re-think the Fourier transform in its context.
Well, if the function's period is $P$, in other words it is stretched out by $P$, the same logic must be used to derive the Fourier series for the new function as for the function with period 1 -- specifically, sines and cosines with longer periods than $P$ don't matter (their coefficient must be zero, because otherwise you've introduced an element into the function that doesn't repeat with that period), but those with shorter, divisible periods matter, because they influence the value of the function within the period, perturbing it by little bits to get to the right function.
So when dealing with our new period $L$, one would expect periods that are fractions of $L$, i.e. $L/n$, as opposed to just $1/n$. So $n/L$ is "more important" than $n$, and indeed it seems very easy to transform the summation into one in terms of this new variable, which we will still call $n$ (i.e. transform $n/L\to n$):
\[g(x) = \sum\limits_n^{} {{a_n}{e^{i2\pi nx}}} \]
\[{a_n} = \frac{1}{L}\int_{-L}^L {g(x){e^{ - 2\pi inx}}dx} \]
Where we labeled $a_{nL}$ as just $a_n$, because that's just a subscript, the labeling doesn't matter. Just remember that $n$ is no longer just an integer/multiple of 1, but a multiple of any fraction $1/L$.
Now note how a non-periodic function is just a function with infinite period, i.e. $L\to\infty$. So $n$ stops being a discrete integer and starts approaching a continuous variable, which we'll call $s$, writing $a_n$ as $a(s)ds$ (why the $ds$? because the increment in $n$ is just $1/L$, which appears in the expression for $a_n$).
\[g(x) = \int_{ - \infty }^\infty {ds\,\,a(s){e^{i2\pi sx}}} \]
\[a(s) = \int_{ - \infty }^\infty {dx\,\,g(x){e^{ - i2\pi sx}}} \]
Which is just a pretty satisfying result.
Recall again the expressions we got for the Fourier transform and its inverse:
\[f(t) = \int_{ - \infty }^\infty {ds\,\,\hat f(s){e^{i2\pi ts}}} \]
\[\hat f(s) = \int_{ - \infty }^\infty {dt{\kern 1pt} {\kern 1pt} f(t){e^{ - i2\pi st}}} \]
(We typically say the Fourier transform maps time-domain functions to frequency-domain ones, so we consider the latter to be the Fourier transform and the first equation to be its inverse.) Note how you can easily turn the first one into an actual Fourier transform, by transforming $s\to -s$:
\[f(t) = \int_{ - \infty }^\infty {ds\,\,\hat f( - s){e^{ - i2\pi ts}}} \]
In other words:
\[{\mathcal{F}^{ - 1}}\left\{ {f(s)} \right\} = \mathcal{F}\left\{ {f( - s)} \right\}\]
And of course that means ${\mathcal{F}^4} = I$, the identity operator (kind of like the derivative on complex exponentials/sine and cosine, is it not?).
Subscribe to:
Posts (Atom)