Mathematical Foundations Reference

Cambridge University Engineering Department

Mathematical Foundations Reference

This document consolidates transforms, probability, and statistics content from the Mathematics and Information databooks.

1. Fourier Series

Periodic function with period TT:

f(t)=d+∑n=1∞(ancos⁡2πntT+bnsin⁡2πntT)f(t) = d + \sum_{n=1}^\infty \left( a_n \cos\frac{2\pi nt}{T} + b_n \sin\frac{2\pi nt}{T} \right)

Complex Form

f(t)=∑n=−∞∞cneinω0tf(t) = \sum_{n=-\infty}^\infty c_n e^{in\omega_0 t}

where ω0=2π/T\omega_0 = 2\pi/T.

2. Fourier Transforms

Let ω=2πf\omega = 2\pi f.

Definition

G(ω)=∫−∞∞g(t)e−jωt dtG(\omega) = \int_{-\infty}^{\infty} g(t) e^{-j\omega t} \, dt

g(t)=12π∫−∞∞G(ω)ejωt dωg(t) = \frac{1}{2\pi} \int_{-\infty}^{\infty} G(\omega) e^{j\omega t} \, d\omega

Common Transform Pairs

Time domain g(t)g(t)Frequency domain G(ω)G(\omega)
DC level: 112πδ(ω)2\pi \delta(\omega)
Unit step: u(t)u(t)πδ(ω)+1jω\pi\delta(\omega) + \frac{1}{j\omega}
Complex exponential: ejω0te^{j\omega_0 t}2πδ(ω−ω0)2\pi\delta(\omega - \omega_0)
cos⁡(ω0t)\cos(\omega_0 t)π[δ(ω−ω0)+δ(ω+ω0)]\pi[\delta(\omega-\omega_0)+\delta(\omega+\omega_0)]
sin⁡(ω0t)\sin(\omega_0 t)πj[δ(ω−ω0)−δ(ω+ω0)]\frac{\pi}{j}[\delta(\omega-\omega_0)-\delta(\omega+\omega_0)]
Impulse train: ∑nδ(t−nT)\sum_n \delta(t-nT)2πT∑mδ(ω−2πmT)\frac{2\pi}{T}\sum_m \delta(\omega-\frac{2\pi m}{T})
Rectangular pulse∝sinc(ωb/2)\propto \mathrm{sinc}(\omega b/2)
Triangular pulse∝sinc2(ωb/2)\propto \mathrm{sinc}^2(\omega b/2)

where sinc(x)=sin⁡xx\mathrm{sinc}(x) = \frac{\sin x}{x}.

Properties

PropertyTime domainFrequency domain
Time shiftg(t−t0)g(t-t_0)e−jωt0G(ω)e^{-j\omega t_0} G(\omega)
Frequency shiftejω0tg(t)e^{j\omega_0 t} g(t)G(ω−ω0)G(\omega-\omega_0)
Differentiationdngdtn\frac{d^n g}{dt^n}(jω)nG(ω)(j\omega)^n G(\omega)
Convolutiong1(t)∗g2(t)g_1(t) * g_2(t)G1(ω)G2(ω)G_1(\omega) G_2(\omega)
Multiplicationg1(t)g2(t)g_1(t) g_2(t)12π(G1∗G2)(ω)\frac{1}{2\pi}(G_1 * G_2)(\omega)

Duality: If g(t)↔p(ω)g(t) \leftrightarrow p(\omega), then p(t)↔2πg(−ω)p(t) \leftrightarrow 2\pi g(-\omega).

Parseval’s Theorem: ∫−∞∞∣g(t)∣2 dt=12π∫−∞∞∣G(ω)∣2 dω\int_{-\infty}^{\infty} |g(t)|^2 \, dt = \frac{1}{2\pi} \int_{-\infty}^{\infty} |G(\omega)|^2 \, d\omega

3. Discrete Fourier Transform (DFT)

For xnx_n, n=0,…,N−1n = 0, \ldots, N-1:

Xk=∑n=0N−1xne−i2πkn/N,0≤k≤N−1X_k = \sum_{n=0}^{N-1} x_n e^{-i2\pi kn/N}, \quad 0 \le k \le N-1

Inverse: xn=1N∑k=0N−1Xkei2πkn/Nx_n = \frac{1}{N} \sum_{k=0}^{N-1} X_k e^{i2\pi kn/N}

Sampling Interpretation

  • Sampling frequency: f0f_0
  • Total time: T=N/f0T = N/f_0
  • Frequency resolution: 1/T1/T
  • Nyquist frequency: f0/2f_0/2

4. Z-Transforms

For sequence gkg_k:

G(z)=∑k=0∞gkz−kG(z) = \sum_{k=0}^{\infty} g_k z^{-k}

Key Properties

Time shift: gk+m⟷zmG(z)−zmg0−⋯−zgm−1g_{k+m} \longleftrightarrow z^m G(z) - z^m g_0 - \cdots - z g_{m-1}

Initial value theorem: g0=lim⁡z→∞G(z)g_0 = \lim_{z \to \infty} G(z)

Final value theorem: lim⁡k→∞gk=lim⁡z→1(z−1)G(z)\lim_{k \to \infty} g_k = \lim_{z \to 1} (z-1) G(z)

(valid if poles lie inside unit circle)

5. Laplace Transforms

xˉ(s)=∫0∞x(t)e−st dt\bar{x}(s) = \int_0^\infty x(t) e^{-st} \, dt

Initial value: x(0+)=lim⁡s→∞sxˉ(s)x(0^+) = \lim_{s \to \infty} s\bar{x}(s)

Final value: lim⁡t→∞x(t)=lim⁡s→0sxˉ(s)\lim_{t \to \infty} x(t) = \lim_{s \to 0} s\bar{x}(s)

6. Probability Fundamentals

Discrete Random Variables

E[X]=∑xx p(x)\mathbb{E}[X] = \sum_x x \, p(x)

Var[X]=E[X2]−E[X]2\mathrm{Var}[X] = \mathbb{E}[X^2] - \mathbb{E}[X]^2

Continuous Random Variables

E[X]=∫−∞∞x f(x) dx\mathbb{E}[X] = \int_{-\infty}^\infty x \, f(x) \, dx

Bayes’ Rule

p(x∣y)=p(y∣x)p(x)p(y)p(x|y) = \frac{p(y|x) p(x)}{p(y)}

7. Gaussian Distribution

Univariate

f(x)=1σ2πe−(x−μ)2/(2σ2)f(x) = \frac{1}{\sigma\sqrt{2\pi}} e^{-(x-\mu)^2/(2\sigma^2)}

Standard Form

If X∼N(μ,σ2)X \sim \mathcal{N}(\mu, \sigma^2), then: Y=X−μσ∼N(0,1)Y = \frac{X - \mu}{\sigma} \sim \mathcal{N}(0,1)

Cumulative distribution: Φ(z)=12π∫−∞ze−x2/2 dx\Phi(z) = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^z e^{-x^2/2} \, dx

Selected quantiles: Φ(1.96)=0.975,Φ(2.576)=0.995\Phi(1.96) = 0.975, \quad \Phi(2.576) = 0.995

Multivariate Gaussian

N(x;μ,Σ)=1(2π)D∣Σ∣exp⁡(−12(x−μ)TΣ−1(x−μ))\mathcal{N}(x; \mu, \Sigma) = \frac{1}{\sqrt{(2\pi)^D |\Sigma|}} \exp\left(-\frac{1}{2}(x-\mu)^T \Sigma^{-1} (x-\mu)\right)

Differential entropy: h(X)=12log⁡((2πe)D∣Σ∣)h(X) = \frac{1}{2} \log\left((2\pi e)^D |\Sigma|\right)

KL divergence: KL(p∥q)=12(log⁡∣Σ2∣∣Σ1∣−D+tr(Σ2−1Σ1)+(μ1−μ2)TΣ2−1(μ1−μ2))\mathrm{KL}(p \| q) = \frac{1}{2}\left(\log\frac{|\Sigma_2|}{|\Sigma_1|} - D + \mathrm{tr}(\Sigma_2^{-1}\Sigma_1) + (\mu_1-\mu_2)^T\Sigma_2^{-1}(\mu_1-\mu_2)\right)

8. Information Theory

Entropy

H(X)=∑xP(x)log⁡1P(x)H(X) = \sum_x P(x) \log \frac{1}{P(x)}

Mutual Information

I(X;Y)=H(X)−H(X∣Y)=D(PXY∥PXPY)I(X;Y) = H(X) - H(X|Y) = D(P_{XY} \| P_X P_Y)

Differential Entropy

h(X)=∫p(x)log⁡1p(x) dxh(X) = \int p(x) \log \frac{1}{p(x)} \, dx

Key Inequalities

Data-processing inequality: I(X;Y)≥I(X;Z)I(X;Y) \ge I(X;Z)

Fano’s inequality: 1+Pelog⁡∣X∣≥H(X∣Y)1 + P_e \log|\mathcal{X}| \ge H(X|Y)

9. Special Functions

Error Function

erf(x)=2π∫0xe−u2 du\mathrm{erf}(x) = \frac{2}{\sqrt{\pi}} \int_0^x e^{-u^2} \, du

Gamma Function

Γ(t)=∫0∞xt−1e−x dx\Gamma(t) = \int_0^\infty x^{t-1} e^{-x} \, dx

Gaussian Integral

∫−∞∞e−λu2 du=πλ\int_{-\infty}^{\infty} e^{-\lambda u^2} \, du = \sqrt{\frac{\pi}{\lambda}}

10. Numerical Methods

Newton-Raphson

xn+1=xn−f(xn)f′(xn)x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)}

Trapezium Rule

∫aby dx≈h2[y0+2(y1+⋯+yn−1)+yn]\int_a^b y \, dx \approx \frac{h}{2}[y_0 + 2(y_1 + \cdots + y_{n-1}) + y_n]

Forward Euler

un+1=un+Δt f(un,tn)u_{n+1} = u_n + \Delta t \, f(u_n, t_n)

Sources

  • Mathematics Data Book (2017 Edition), Cambridge University Engineering Department
  • Information Data Book (2017 Edition, revised 2019 & 2021), Cambridge University Engineering Department