Mathematical Foundations Reference

Mathematical Foundations Reference

This document consolidates transforms, probability, and statistics content from the Mathematics and Information databooks.

1. Fourier Series

Periodic function with period TT:

f(t)=d+∑n=1∞(ancos⁡2πntT+bnsin⁡2πntT)f(t) = d + \sum_{n=1}^\infty \left( a_n \cos\frac{2\pi nt}{T} + b_n \sin\frac{2\pi nt}{T} \right)

Complex Form

f(t)=∑n=−∞∞cneinω0tf(t) = \sum_{n=-\infty}^\infty c_n e^{in\omega_0 t}

where ω0=2π/T\omega_0 = 2\pi/T.

2. Fourier Transforms

Let ω=2πf\omega = 2\pi f.

Definition

G(ω)=∫−∞∞g(t)e−jωt dtG(\omega) = \int_{-\infty}^{\infty} g(t) e^{-j\omega t} \, dt

g(t)=12π∫−∞∞G(ω)ejωt dωg(t) = \frac{1}{2\pi} \int_{-\infty}^{\infty} G(\omega) e^{j\omega t} \, d\omega

Common Transform Pairs

Time domain g(t)g(t)Frequency domain G(ω)G(\omega)
DC level: 112πδ(ω)2\pi \delta(\omega)
Unit step: u(t)u(t)πδ(ω)+1jω\pi\delta(\omega) + \frac{1}{j\omega}
Complex exponential: ejω0te^{j\omega_0 t}2πδ(ω−ω0)2\pi\delta(\omega - \omega_0)
cos⁡(ω0t)\cos(\omega_0 t)π[δ(ω−ω0)+δ(ω+ω0)]\pi[\delta(\omega-\omega_0)+\delta(\omega+\omega_0)]
sin⁡(ω0t)\sin(\omega_0 t)πj[δ(ω−ω0)−δ(ω+ω0)]\frac{\pi}{j}[\delta(\omega-\omega_0)-\delta(\omega+\omega_0)]
Impulse train: ∑nδ(t−nT)\sum_n \delta(t-nT)2πT∑mδ(ω−2πmT)\frac{2\pi}{T}\sum_m \delta(\omega-\frac{2\pi m}{T})
Rectangular pulse∝sinc(ωb/2)\propto \mathrm{sinc}(\omega b/2)
Triangular pulse∝sinc2(ωb/2)\propto \mathrm{sinc}^2(\omega b/2)

where sinc(x)=sin⁡xx\mathrm{sinc}(x) = \frac{\sin x}{x}.

Properties

PropertyTime domainFrequency domain
Time shiftg(t−t0)g(t-t_0)e−jωt0G(ω)e^{-j\omega t_0} G(\omega)
Frequency shiftejω0tg(t)e^{j\omega_0 t} g(t)G(ω−ω0)G(\omega-\omega_0)
Differentiationdngdtn\frac{d^n g}{dt^n}(jω)nG(ω)(j\omega)^n G(\omega)
Convolutiong1(t)∗g2(t)g_1(t) * g_2(t)G1(ω)G2(ω)G_1(\omega) G_2(\omega)
Multiplicationg1(t)g2(t)g_1(t) g_2(t)12π(G1∗G2)(ω)\frac{1}{2\pi}(G_1 * G_2)(\omega)

Duality: If g(t)↔p(ω)g(t) \leftrightarrow p(\omega), then p(t)↔2πg(−ω)p(t) \leftrightarrow 2\pi g(-\omega).

Parseval’s Theorem: ∫−∞∞∣g(t)∣2 dt=12π∫−∞∞∣G(ω)∣2 dω\int_{-\infty}^{\infty} |g(t)|^2 \, dt = \frac{1}{2\pi} \int_{-\infty}^{\infty} |G(\omega)|^2 \, d\omega

3. Discrete Fourier Transform (DFT)

For xnx_n, n=0,…,N−1n = 0, \ldots, N-1:

Xk=∑n=0N−1xne−i2πkn/N,0≤k≤N−1X_k = \sum_{n=0}^{N-1} x_n e^{-i2\pi kn/N}, \quad 0 \le k \le N-1

Inverse: xn=1N∑k=0N−1Xkei2πkn/Nx_n = \frac{1}{N} \sum_{k=0}^{N-1} X_k e^{i2\pi kn/N}

Sampling Interpretation

  • Sampling frequency: f0f_0
  • Total time: T=N/f0T = N/f_0
  • Frequency resolution: 1/T1/T
  • Nyquist frequency: f0/2f_0/2

4. Z-Transforms

For sequence gkg_k:

G(z)=∑k=0∞gkz−kG(z) = \sum_{k=0}^{\infty} g_k z^{-k}

Key Properties

Time shift: gk+m⟷zmG(z)−zmg0−⋯−zgm−1g_{k+m} \longleftrightarrow z^m G(z) - z^m g_0 - \cdots - z g_{m-1}

Initial value theorem: g0=lim⁡z→∞G(z)g_0 = \lim_{z \to \infty} G(z)

Final value theorem: lim⁡k→∞gk=lim⁡z→1(z−1)G(z)\lim_{k \to \infty} g_k = \lim_{z \to 1} (z-1) G(z)

(valid if poles lie inside unit circle)

5. Laplace Transforms

xˉ(s)=∫0∞x(t)e−st dt\bar{x}(s) = \int_0^\infty x(t) e^{-st} \, dt

Initial value: x(0+)=lim⁡s→∞sxˉ(s)x(0^+) = \lim_{s \to \infty} s\bar{x}(s)

Final value: lim⁡t→∞x(t)=lim⁡s→0sxˉ(s)\lim_{t \to \infty} x(t) = \lim_{s \to 0} s\bar{x}(s)

6. Probability Fundamentals

Discrete Random Variables

E[X]=∑xx p(x)\mathbb{E}[X] = \sum_x x \, p(x)

Var[X]=E[X2]−E[X]2\mathrm{Var}[X] = \mathbb{E}[X^2] - \mathbb{E}[X]^2

Continuous Random Variables

E[X]=∫−∞∞x f(x) dx\mathbb{E}[X] = \int_{-\infty}^\infty x \, f(x) \, dx

Bayes’ Rule

p(x∣y)=p(y∣x)p(x)p(y)p(x|y) = \frac{p(y|x) p(x)}{p(y)}

7. Gaussian Distribution

Univariate

f(x)=1σ2πe−(x−μ)2/(2σ2)f(x) = \frac{1}{\sigma\sqrt{2\pi}} e^{-(x-\mu)^2/(2\sigma^2)}

Standard Form

If X∼N(μ,σ2)X \sim \mathcal{N}(\mu, \sigma^2), then: Y=X−μσ∼N(0,1)Y = \frac{X - \mu}{\sigma} \sim \mathcal{N}(0,1)

Cumulative distribution: Φ(z)=12π∫−∞ze−x2/2 dx\Phi(z) = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^z e^{-x^2/2} \, dx

Selected quantiles: Φ(1.96)=0.975,Φ(2.576)=0.995\Phi(1.96) = 0.975, \quad \Phi(2.576) = 0.995

Multivariate Gaussian

N(x;μ,Σ)=1(2π)D∣Σ∣exp⁡(−12(x−μ)TΣ−1(x−μ))\mathcal{N}(x; \mu, \Sigma) = \frac{1}{\sqrt{(2\pi)^D |\Sigma|}} \exp\left(-\frac{1}{2}(x-\mu)^T \Sigma^{-1} (x-\mu)\right)

Differential entropy: h(X)=12log⁡((2πe)D∣Σ∣)h(X) = \frac{1}{2} \log\left((2\pi e)^D |\Sigma|\right)

KL divergence: KL(p∥q)=12(log⁡∣Σ2∣∣Σ1∣−D+tr(Σ2−1Σ1)+(μ1−μ2)TΣ2−1(μ1−μ2))\mathrm{KL}(p \| q) = \frac{1}{2}\left(\log\frac{|\Sigma_2|}{|\Sigma_1|} - D + \mathrm{tr}(\Sigma_2^{-1}\Sigma_1) + (\mu_1-\mu_2)^T\Sigma_2^{-1}(\mu_1-\mu_2)\right)

8. Information Theory

Entropy

H(X)=∑xP(x)log⁡1P(x)H(X) = \sum_x P(x) \log \frac{1}{P(x)}

Mutual Information

I(X;Y)=H(X)−H(X∣Y)=D(PXY∥PXPY)I(X;Y) = H(X) - H(X|Y) = D(P_{XY} \| P_X P_Y)

Differential Entropy

h(X)=∫p(x)log⁡1p(x) dxh(X) = \int p(x) \log \frac{1}{p(x)} \, dx

Key Inequalities

Data-processing inequality: I(X;Y)≥I(X;Z)I(X;Y) \ge I(X;Z)

Fano’s inequality: 1+Pelog⁡∣X∣≥H(X∣Y)1 + P_e \log|\mathcal{X}| \ge H(X|Y)

9. Special Functions

Error Function

erf(x)=2π∫0xe−u2 du\mathrm{erf}(x) = \frac{2}{\sqrt{\pi}} \int_0^x e^{-u^2} \, du

Gamma Function

Γ(t)=∫0∞xt−1e−x dx\Gamma(t) = \int_0^\infty x^{t-1} e^{-x} \, dx

Gaussian Integral

∫−∞∞e−λu2 du=πλ\int_{-\infty}^{\infty} e^{-\lambda u^2} \, du = \sqrt{\frac{\pi}{\lambda}}

10. Numerical Methods

Newton-Raphson

xn+1=xn−f(xn)f′(xn)x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)}

Trapezium Rule

∫aby dx≈h2[y0+2(y1+⋯+yn−1)+yn]\int_a^b y \, dx \approx \frac{h}{2}[y_0 + 2(y_1 + \cdots + y_{n-1}) + y_n]

Forward Euler

un+1=un+Δt f(un,tn)u_{n+1} = u_n + \Delta t \, f(u_n, t_n)

Sources

  • Mathematics Data Book (2017 Edition), Cambridge University Engineering Department
  • Information Data Book (2017 Edition, revised 2019 & 2021), Cambridge University Engineering Department