Part I: Ensembles | Chapter 2

The Canonical Ensemble

The Boltzmann distribution, partition function, Helmholtz free energy, energy fluctuations, and the equipartition theorem

Historical Context

The canonical ensemble describes a system in thermal equilibrium with a heat bath at temperature \(T\). Gibbs introduced this framework in 1902, naming it “canonical” to signify its central role in statistical mechanics. The partition function \(Z\) (from German “Zustandssumme,” meaning sum over states) became the master key from which all thermodynamic quantities can be derived.

Boltzmann had earlier derived the exponential energy distribution in his kinetic theory work of the 1870s, but it was Gibbs who elevated it to a general principle applicable to any system in thermal contact with a reservoir.

1. The Boltzmann Distribution

System + Heat Bath (Canonical Ensemble)Isolated total system: E_tot = E_S + E_R = constSystem SEnergy E_SE_S << E_totp_i = e^(-βE_i)/ZReservoir RE_R = E_tot - E_ST = const(very large)δQEnergy exchange
Figure. The canonical ensemble: a small system S in thermal contact with a large heat bath R at temperature T. Energy flows freely between them while the total is conserved.

Derivation 1: From the Microcanonical Ensemble

Consider a small system S in contact with a large reservoir R. The total system S+R is isolated with energy \(E_{\text{tot}}\). When S is in microstate \(i\) with energy \(E_i\), the reservoir has energy \(E_{\text{tot}} - E_i\). By the fundamental postulate:

\[p_i \propto \Omega_R(E_{\text{tot}} - E_i)\]

Since \(E_i \ll E_{\text{tot}}\), we Taylor expand the entropy of the reservoir:

\[S_R(E_{\text{tot}} - E_i) \approx S_R(E_{\text{tot}}) - E_i \frac{\partial S_R}{\partial E}\bigg|_{E_{\text{tot}}} = S_R(E_{\text{tot}}) - \frac{E_i}{T}\]

Exponentiating: \(\Omega_R \propto e^{S_R/k_B} \propto e^{-E_i/(k_BT)}\), giving:

\[\boxed{p_i = \frac{e^{-\beta E_i}}{Z}, \qquad Z = \sum_i e^{-\beta E_i}, \qquad \beta = \frac{1}{k_BT}}\]

Derivation 2: Maximum Entropy with Constrained Energy

Alternatively, maximize the Gibbs entropy \(S = -k_B \sum_i p_i \ln p_i\) subject to two constraints:

\[\sum_i p_i = 1, \qquad \sum_i p_i E_i = \langle E \rangle\]

Using Lagrange multipliers \(\alpha\) and \(\beta\):

\[\frac{\partial}{\partial p_j}\left[-k_B \sum_i p_i \ln p_i - \alpha\sum_i p_i - \beta \sum_i p_i E_i\right] = 0\]
\[-k_B(\ln p_j + 1) - \alpha - \beta E_j = 0 \implies p_j = Ce^{-\beta E_j/k_B}\]

The multiplier \(\beta\) is identified as \(1/(k_BT)\) by comparison with thermodynamics.

2. The Partition Function

The partition function \(Z\) encodes all thermodynamic information about the system:

Thermodynamic Relations from Z

Helmholtz free energy: \(F = -k_BT \ln Z\)

Average energy: \(\langle E \rangle = -\frac{\partial \ln Z}{\partial \beta}\)

Entropy: \(S = k_B(\ln Z + \beta \langle E \rangle) = -\frac{\partial F}{\partial T}\bigg|_V\)

Pressure: \(P = -\frac{\partial F}{\partial V}\bigg|_T = k_BT \frac{\partial \ln Z}{\partial V}\bigg|_T\)

Heat capacity: \(C_V = \frac{\partial \langle E \rangle}{\partial T}\bigg|_V = k_B\beta^2 \frac{\partial^2 \ln Z}{\partial \beta^2}\)

Derivation 3: Free Energy from Z

The Gibbs entropy in the canonical ensemble is:

\[S = -k_B \sum_i p_i \ln p_i = -k_B \sum_i p_i(-\beta E_i - \ln Z)\]
\[= k_B\beta\langle E\rangle + k_B \ln Z = \frac{\langle E\rangle}{T} + k_B \ln Z\]

Rearranging: \(\langle E\rangle - TS = -k_BT\ln Z\). Since the Helmholtz free energy is \(F = E - TS\):

\[\boxed{F = -k_BT \ln Z}\]

3. Energy Fluctuations

Derivation 4: Fluctuation-Response Relation

The variance of energy in the canonical ensemble connects to the heat capacity:

\[\langle E^2 \rangle - \langle E \rangle^2 = \frac{\partial^2 \ln Z}{\partial \beta^2} = -\frac{\partial \langle E \rangle}{\partial \beta}\]

Since \(\partial/\partial\beta = -k_BT^2 \partial/\partial T\):

\[\boxed{\langle (\Delta E)^2 \rangle = k_BT^2 C_V}\]

This is a profound result: microscopic fluctuations are directly related to a macroscopic response function. For an ideal gas, \(C_V = \frac{3}{2}Nk_B\), so:

\[\frac{\sqrt{\langle(\Delta E)^2\rangle}}{\langle E \rangle} = \sqrt{\frac{2}{3N}} \sim \frac{1}{\sqrt{N}}\]

Physical Significance

The \(1/\sqrt{N}\) scaling means that for macroscopic systems, the canonical ensemble effectively selects a single energy, recovering the microcanonical result. This is ensemble equivalence in action. Deviations from this scaling near phase transitions (where \(C_V\) diverges) signal the breakdown of simple thermodynamic behavior.

4. The Equipartition Theorem

Derivation 5: General Proof

For a classical system with Hamiltonian \(H(q_1,\ldots,q_f,p_1,\ldots,p_f)\), consider the average:

\[\left\langle x_i \frac{\partial H}{\partial x_j}\right\rangle = \frac{\int x_i \frac{\partial H}{\partial x_j} e^{-\beta H} \prod_k dx_k}{\int e^{-\beta H} \prod_k dx_k}\]

Integrating by parts in \(x_j\) (assuming the Boltzmann factor vanishes at boundaries):

\[\left\langle x_i \frac{\partial H}{\partial x_j}\right\rangle = \frac{1}{\beta}\delta_{ij} = k_BT\,\delta_{ij}\]

For a quadratic degree of freedom \(H = \alpha x_i^2 + \ldots\):

\[\langle \alpha x_i^2 \rangle = \frac{1}{2}k_BT\]

Each quadratic term in the Hamiltonian contributes \(\frac{1}{2}k_BT\) to the average energy. For a monatomic ideal gas with \(H = \sum_i p_i^2/(2m)\), there are \(3N\) quadratic terms, giving \(\langle E \rangle = \frac{3}{2}Nk_BT\).

5. Applications

Quantum Harmonic Oscillator

For a single oscillator with energies \(E_n = \hbar\omega(n + 1/2)\):

\[Z_1 = \sum_{n=0}^{\infty} e^{-\beta\hbar\omega(n+1/2)} = \frac{e^{-\beta\hbar\omega/2}}{1 - e^{-\beta\hbar\omega}} = \frac{1}{2\sinh(\beta\hbar\omega/2)}\]
\[\langle E \rangle = \frac{\hbar\omega}{2} + \frac{\hbar\omega}{e^{\beta\hbar\omega} - 1} = \frac{\hbar\omega}{2}\coth\left(\frac{\beta\hbar\omega}{2}\right)\]

At high temperature \(k_BT \gg \hbar\omega\): \(\langle E \rangle \to k_BT\)(classical equipartition). At low temperature: \(\langle E \rangle \to \hbar\omega/2\)(zero-point energy only).

Paramagnet in a Field

For \(N\) spin-1/2 particles in a magnetic field \(B\), each spin has energy \(\pm\mu_B B\). The single-spin partition function is:

\[z = e^{\beta\mu_B B} + e^{-\beta\mu_B B} = 2\cosh(\beta\mu_B B)\]

For \(N\) independent spins: \(Z = z^N\). The magnetization is:

\[M = N\mu_B\tanh(\beta\mu_B B)\]

Ideal Gas Partition Function

For \(N\) identical classical particles in volume \(V\):

\[Z = \frac{1}{N!}\left(\frac{V}{\lambda_{dB}^3}\right)^N, \qquad \lambda_{dB} = \sqrt{\frac{2\pi\hbar^2}{mk_BT}}\]

From \(F = -k_BT\ln Z\) one recovers the ideal gas law \(PV = Nk_BT\)and the Sackur-Tetrode entropy.

6. Classical Partition Function

For a classical system with \(f\) degrees of freedom, the sum becomes an integral:

\[Z = \frac{1}{N!h^f}\int e^{-\beta H(\mathbf{q},\mathbf{p})} \prod_{i=1}^{f} dq_i\,dp_i\]

The factor \(h^f\) ensures dimensional consistency and provides the correct classical limit of the quantum partition function. For separable Hamiltonians\(H = \sum_i h_i\), the partition function factorizes: \(Z = \prod_i z_i\).

7. Connection to the Density of States

The partition function can be written as a Laplace transform of the density of states:

\[Z(\beta) = \int_0^{\infty} g(E) e^{-\beta E}\,dE\]

where \(g(E) = d\Omega/dE\) is the density of states. This establishes a deep connection between the microcanonical and canonical descriptions: \(Z\)is the Laplace transform of \(\Omega(E)\).

The inverse transform recovers \(g(E)\), and in the thermodynamic limit, the integral is dominated by a saddle point at \(E = \langle E \rangle\), confirming ensemble equivalence.

8. Computational Exploration

This simulation demonstrates the Boltzmann distribution, partition function calculations, energy fluctuations, and the breakdown of equipartition at low temperatures.

Canonical Ensemble: Boltzmann Distribution, Oscillator, Fluctuations, and Paramagnet

Python
script.py108 lines

Click Run to execute the Python code

Code will be executed with Python 3 on the server

9. Summary and Key Results

Core Formulas

  • Boltzmann: \(p_i = e^{-\beta E_i}/Z\)
  • Free energy: \(F = -k_BT\ln Z\)
  • Fluctuations: \(\langle(\Delta E)^2\rangle = k_BT^2 C_V\)
  • Equipartition: \(\langle\alpha x^2\rangle = \frac{1}{2}k_BT\)

Physical Insights

  • Z encodes all equilibrium thermodynamics
  • Fluctuations vanish as \(1/\sqrt{N}\) for large systems
  • Equipartition fails when \(k_BT \ll \hbar\omega\)
  • Canonical = microcanonical in thermodynamic limit

Practice Problems

Problem 1:Derive the partition function $Z$ for $N$ harmonic oscillators and show $\langle E\rangle = NkT$.

Solution:

1. A single classical harmonic oscillator has Hamiltonian $H = p^2/(2m) + \tfrac{1}{2}m\omega^2 x^2$. The single-particle partition function is:

$$z_1 = \frac{1}{h}\int_{-\infty}^{\infty}dp\int_{-\infty}^{\infty}dx\;\exp\!\left[-\beta\left(\frac{p^2}{2m} + \frac{1}{2}m\omega^2 x^2\right)\right]$$

2. Both integrals are Gaussian. The momentum integral gives $\sqrt{2\pi m/\beta}$ and the position integral gives $\sqrt{2\pi/(\beta m\omega^2)}$:

$$z_1 = \frac{1}{h}\cdot\sqrt{\frac{2\pi m}{\beta}}\cdot\sqrt{\frac{2\pi}{\beta m\omega^2}} = \frac{1}{\beta\hbar\omega} = \frac{k_BT}{\hbar\omega}$$

3. For $N$ independent oscillators: $Z = z_1^N = (k_BT/\hbar\omega)^N$.

4. The average energy is:

$$\langle E\rangle = -\frac{\partial\ln Z}{\partial\beta} = -\frac{\partial}{\partial\beta}\left(-N\ln\beta + \text{const}\right) = \frac{N}{\beta}$$

5. Therefore:

$$\boxed{\langle E\rangle = Nk_BT}$$

Each oscillator has 2 quadratic degrees of freedom, contributing $\tfrac{1}{2}k_BT$ each, consistent with the equipartition theorem.

Problem 2:Calculate the energy fluctuations $\langle(\Delta E)^2\rangle = kT^2 C_V$ for the canonical ensemble.

Solution:

1. Start from the definition of the average energy:

$$\langle E\rangle = \frac{\sum_n E_n e^{-\beta E_n}}{Z} = -\frac{1}{Z}\frac{\partial Z}{\partial\beta} = -\frac{\partial\ln Z}{\partial\beta}$$

2. Compute $\langle E^2\rangle$ similarly:

$$\langle E^2\rangle = \frac{1}{Z}\sum_n E_n^2 e^{-\beta E_n} = \frac{1}{Z}\frac{\partial^2 Z}{\partial\beta^2}$$

3. The variance is:

$$\langle(\Delta E)^2\rangle = \langle E^2\rangle - \langle E\rangle^2 = \frac{\partial^2\ln Z}{\partial\beta^2} = -\frac{\partial\langle E\rangle}{\partial\beta}$$

4. Convert the $\beta$-derivative to a $T$-derivative using $\partial/\partial\beta = -k_BT^2\,\partial/\partial T$:

$$\langle(\Delta E)^2\rangle = k_BT^2\frac{\partial\langle E\rangle}{\partial T}\bigg|_V = k_BT^2 C_V$$

5. Therefore:

$$\boxed{\langle(\Delta E)^2\rangle = k_BT^2 C_V}$$

This fluctuation-dissipation relation connects microscopic fluctuations to the macroscopic response function $C_V$.

Problem 3:Show that the canonical entropy $S = -k\sum p_n\ln p_n$ reduces to $S = k\ln\Omega$ in the microcanonical limit.

Solution:

1. In the canonical ensemble, $p_n = e^{-\beta E_n}/Z$. The Gibbs entropy is:

$$S = -k_B\sum_n p_n\ln p_n = -k_B\sum_n p_n(-\beta E_n - \ln Z)$$

2. This gives $S = k_B\beta\langle E\rangle + k_B\ln Z$.

3. In the microcanonical limit, energy fluctuations vanish ($\Delta E/E \sim 1/\sqrt{N} \to 0$). The partition function is dominated by states in a narrow energy shell:

$$Z \approx \Omega(E^*)\,e^{-\beta E^*}\,\delta E$$

where $E^* = \langle E\rangle$ is the most probable energy.

4. Substituting: $\ln Z \approx \ln\Omega(E^*) - \beta E^* + \ln(\delta E)$. Therefore:

$$S = k_B\beta E^* + k_B\ln\Omega(E^*) - k_B\beta E^* + k_B\ln(\delta E)$$

5. The $\ln(\delta E)$ term is sub-extensive (it does not grow with $N$), so in the thermodynamic limit:

$$\boxed{S = k_B\ln\Omega(E^*)}$$

This demonstrates ensemble equivalence: the canonical and microcanonical entropies agree in the thermodynamic limit.

Problem 4:Compute $Z$ for a two-level system (energies $0$ and $\varepsilon$) and plot $C_V(T)$ showing the Schottky anomaly.

Solution:

1. The partition function for a single two-level system is:

$$Z = e^{-\beta\cdot 0} + e^{-\beta\varepsilon} = 1 + e^{-\beta\varepsilon}$$

2. The average energy is:

$$\langle E\rangle = -\frac{\partial\ln Z}{\partial\beta} = \frac{\varepsilon\,e^{-\beta\varepsilon}}{1 + e^{-\beta\varepsilon}} = \frac{\varepsilon}{e^{\beta\varepsilon} + 1}$$

3. The heat capacity is $C_V = \partial\langle E\rangle/\partial T$:

$$\boxed{C_V = k_B\left(\frac{\varepsilon}{k_BT}\right)^2\frac{e^{\varepsilon/(k_BT)}}{\left(e^{\varepsilon/(k_BT)} + 1\right)^2}}$$

4. Behavior: $C_V \to 0$ as $T\to 0$ (exponentially frozen out) and $C_V \to 0$ as $T\to\infty$ (both levels equally populated, no further energy absorption).

5. The Schottky anomaly is the peak in $C_V$ at $T^* \approx 0.42\,\varepsilon/k_B$, with maximum value $C_V^{\max} \approx 0.44\,k_B$. This characteristic bump is a signature of systems with a finite number of energy levels and is observed experimentally in paramagnetic salts.

Problem 5:Derive the Helmholtz free energy $F = -kT\ln Z$ and show that $S = -(\partial F/\partial T)_V$.

Solution:

1. Start from the Gibbs entropy with $p_n = e^{-\beta E_n}/Z$:

$$S = -k_B\sum_n p_n\ln p_n = k_B\sum_n p_n(\beta E_n + \ln Z) = \frac{\langle E\rangle}{T} + k_B\ln Z$$

2. Rearranging: $\langle E\rangle - TS = -k_BT\ln Z$. Since the Helmholtz free energy is defined as $F = E - TS$:

$$\boxed{F = -k_BT\ln Z}$$

3. Now differentiate $F$ with respect to $T$ at constant $V$:

$$\frac{\partial F}{\partial T}\bigg|_V = -k_B\ln Z - k_BT\frac{\partial\ln Z}{\partial T}\bigg|_V$$

4. Using $\partial\ln Z/\partial T = \beta\langle E\rangle/T = \langle E\rangle/(k_BT^2)$:

$$\frac{\partial F}{\partial T}\bigg|_V = -k_B\ln Z - \frac{\langle E\rangle}{T} = -S$$

5. Therefore:

$$\boxed{S = -\frac{\partial F}{\partial T}\bigg|_V}$$

This is a standard thermodynamic identity, now derived from the statistical mechanical partition function, confirming the consistency of the canonical ensemble with thermodynamics.

← Microcanonical EnsembleGrand Canonical Ensemble →
Rate this chapter: