Part I: Ensembles | Chapter 1

The Microcanonical Ensemble

Entropy as the logarithm of the number of microstates, equal a priori probability, and the foundation of statistical mechanics for isolated systems

Historical Context

Ludwig Boltzmann, working in the 1870s, forged the revolutionary connection between the macroscopic quantity entropy and the microscopic count of accessible states. His tombstone in Vienna bears the inscription \(S = k \ln W\), arguably the most profound equation in all of physics. J. Willard Gibbs independently developed the ensemble formalism in his 1902 masterwork “Elementary Principles in Statistical Mechanics,” coining the term “microcanonical ensemble” for an isolated system at fixed energy.

Max Planck later refined Boltzmann's formula and introduced the constant \(k_B\)that now bears Boltzmann's name. The microcanonical ensemble remains the conceptual bedrock upon which all of statistical mechanics is built.

1. The Fundamental Postulate of Statistical Mechanics

Consider an isolated system with fixed energy \(E\), volume \(V\), and particle number \(N\). The fundamental postulate (equal a priori probability) states:

Equal A Priori Probability Postulate

For an isolated system in equilibrium, all accessible microstates consistent with the macroscopic constraints are equally probable. If \(\Omega(E, V, N)\) is the total number of microstates with energy in \([E, E + \delta E]\), then each microstate has probability:

\[p_i = \frac{1}{\Omega(E, V, N)}\]

This postulate cannot be derived from mechanics alone; it is an additional axiom motivated by ergodic theory and time-reversal symmetry. It implies that the system spends equal time in each accessible microstate over long observation periods.

The density matrix for the microcanonical ensemble is:

\[\hat{\rho}_{\text{mc}} = \frac{1}{\Omega} \sum_{i: E \leq E_i \leq E + \delta E} |i\rangle \langle i|\]

where the sum runs over all eigenstates \(|i\rangle\) of the Hamiltonian with energies in the specified shell.

2. Boltzmann Entropy: \(S = k_B \ln \Omega\)

Figure. The microcanonical ensemble: accessible microstates lie in a thin energy shell [E, E+δE] in phase space. The number of states Ω(E) determines the entropy.

Derivation 1: Entropy from Information Theory

The Shannon entropy for any probability distribution \(\{p_i\}\) is:

\[S = -k_B \sum_i p_i \ln p_i\]

For the microcanonical ensemble where \(p_i = 1/\Omega\) for all accessible states:

\[S = -k_B \sum_{i=1}^{\Omega} \frac{1}{\Omega} \ln \frac{1}{\Omega} = -k_B \cdot \Omega \cdot \frac{1}{\Omega} \cdot (-\ln \Omega) = k_B \ln \Omega\]

Derivation 2: Entropy Maximization

We can derive the equal probability postulate by maximizing entropy subject to the normalization constraint \(\sum_i p_i = 1\). Using Lagrange multipliers:

\[\frac{\partial}{\partial p_j}\left[-k_B \sum_i p_i \ln p_i - \lambda \left(\sum_i p_i - 1\right)\right] = 0\]

\[-k_B(\ln p_j + 1) - \lambda = 0 \implies p_j = e^{-1 - \lambda/k_B} = \text{const}\]

Since all probabilities are equal and must sum to 1, we get \(p_j = 1/\Omega\). This proves that the uniform distribution maximizes entropy among all distributions over \(\Omega\) states.

Key Insight: Additivity of Entropy

For two independent subsystems A and B, the total number of microstates is\(\Omega_{AB} = \Omega_A \cdot \Omega_B\). The logarithm converts this product into a sum:

\[S_{AB} = k_B \ln(\Omega_A \cdot \Omega_B) = k_B \ln \Omega_A + k_B \ln \Omega_B = S_A + S_B\]

This extensivity is the reason Boltzmann chose the logarithm. It ensures that entropy is an extensive (additive) thermodynamic quantity.

3. The Ideal Gas: Counting Microstates

Derivation 3: Volume of a Hypersphere in Phase Space

For \(N\) non-interacting classical particles in a box of volume \(V\), the phase space is \(6N\)-dimensional. The constraint \(\sum_{i=1}^{N} \frac{p_i^2}{2m} = E\)defines a hypersphere of radius \(R = \sqrt{2mE}\) in the \(3N\)-dimensional momentum space. The volume of a \(d\)-dimensional sphere of radius \(R\) is:

\[V_d(R) = \frac{\pi^{d/2}}{\Gamma(d/2 + 1)} R^d\]

For our \(3N\)-dimensional momentum sphere with \(d = 3N\):

\[\Phi(E) = V^N \cdot \frac{\pi^{3N/2}}{\Gamma(3N/2 + 1)} (2mE)^{3N/2}\]

where \(\Phi(E)\) is the total phase space volume up to energy \(E\). The number of microstates in the energy shell is:

\[\Omega(E) = \frac{1}{N! h^{3N}} \frac{d\Phi}{dE} \delta E = \frac{1}{N! h^{3N}} \cdot V^N \cdot \frac{2\pi^{3N/2}}{\Gamma(3N/2)} (2mE)^{3N/2 - 1} \cdot m \cdot \delta E\]

The factor \(1/N!\) accounts for the indistinguishability of identical particles (Gibbs correction), and \(h^{3N}\) sets the quantum of phase space volume.

Derivation 4: The Sackur-Tetrode Equation

Taking the logarithm and using Stirling's approximation \(\ln N! \approx N \ln N - N\):

\[S = k_B \ln \Omega \approx N k_B \left[\ln\left(\frac{V}{N}\left(\frac{4\pi m E}{3 N h^2}\right)^{3/2}\right) + \frac{5}{2}\right]\]

This is the celebrated Sackur-Tetrode equation. Using\(E = \frac{3}{2}Nk_BT\), we can rewrite it as:

\[S = Nk_B\left[\frac{5}{2} + \ln\left(\frac{V}{N\lambda_{dB}^3}\right)\right]\]

where \(\lambda_{dB} = h/\sqrt{2\pi m k_B T}\) is the thermal de Broglie wavelength. This result was verified experimentally for noble gases with remarkable precision.

4. Temperature and the Laws of Thermodynamics

Derivation 5: Temperature from Entropy

The microcanonical definition of temperature emerges from requiring thermal equilibrium between two subsystems A and B in thermal contact, with fixed total energy \(E = E_A + E_B\):

\[\Omega_{\text{total}} = \Omega_A(E_A) \cdot \Omega_B(E - E_A)\]

Maximizing \(\Omega_{\text{total}}\) (or equivalently \(S_{\text{total}}\)) with respect to \(E_A\):

\[\frac{\partial S_A}{\partial E_A}\bigg|_{V,N} = \frac{\partial S_B}{\partial E_B}\bigg|_{V,N}\]

This motivates the definition of temperature:

\[\frac{1}{T} \equiv \frac{\partial S}{\partial E}\bigg|_{V,N}\]

Similarly, pressure and chemical potential emerge as:

\[\frac{P}{T} = \frac{\partial S}{\partial V}\bigg|_{E,N}, \qquad \frac{\mu}{T} = -\frac{\partial S}{\partial N}\bigg|_{E,V}\]

The Fundamental Thermodynamic Relation

Combining these definitions gives the fundamental relation:

\[dS = \frac{1}{T}dE + \frac{P}{T}dV - \frac{\mu}{T}dN\]

or equivalently \(dE = TdS - PdV + \mu dN\), which is the first law of thermodynamics.

5. Applications and Examples

Two-Level System

Consider \(N\) independent spins, each with energy 0 or \(\epsilon\). If the total energy is \(E = n\epsilon\) (i.e., \(n\) spins are excited), the number of microstates is:

\[\Omega = \binom{N}{n} = \frac{N!}{n!(N-n)!}\]

The entropy per particle in the thermodynamic limit using Stirling's approximation:

\[\frac{S}{Nk_B} = -x\ln x - (1-x)\ln(1-x), \qquad x = n/N = E/(N\epsilon)\]

The temperature is:

\[\frac{1}{T} = \frac{\partial S}{\partial E} = \frac{k_B}{\epsilon}\ln\left(\frac{1-x}{x}\right)\]

Remarkably, when \(x > 1/2\) (more than half the spins excited), the temperature becomes negative. This is a genuine physical effect observed in nuclear spin systems (Purcell and Pound, 1951).

Einstein Solid

For \(N\) quantum harmonic oscillators sharing \(q\) energy quanta, the number of microstates is the “stars and bars” combinatorial result:

\[\Omega(N, q) = \binom{q + N - 1}{q} = \frac{(q + N - 1)!}{q!(N-1)!}\]

Harmonic Oscillators and Equipartition

For \(N\) classical harmonic oscillators with total energy \(E\), the phase space volume leads to:

\[\Omega \propto E^N \implies S = Nk_B \ln E + \text{const}\]

Taking the temperature derivative: \(1/T = Nk_B/E\), giving \(E = Nk_BT\), which is the classical equipartition result of \(k_BT/2\) per quadratic degree of freedom (here \(2N\) quadratic degrees).

6. The Gibbs Paradox

Without the \(1/N!\) factor, the entropy of an ideal gas would not be extensive. Mixing two identical gases by removing a partition would yield an entropy increase:

\[\Delta S_{\text{incorrect}} = 2Nk_B \ln 2 \neq 0\]

The Gibbs correction \(\Omega \to \Omega/N!\) resolves this paradox by accounting for the indistinguishability of identical particles. With this correction:

\[\Delta S_{\text{correct}} = 0\]

This was one of the earliest hints that identical particles require fundamentally different counting than distinguishable objects, foreshadowing quantum mechanics.

7. Equivalence of Ensembles

In the thermodynamic limit \(N \to \infty\), all three major ensembles (microcanonical, canonical, grand canonical) give identical results for thermodynamic quantities. This is because the energy distribution in the canonical ensemble becomes sharply peaked around \(\langle E \rangle\) with relative fluctuations:

\[\frac{\Delta E}{\langle E \rangle} = \frac{\sqrt{k_B T^2 C_V}}{\langle E \rangle} \sim \frac{1}{\sqrt{N}}\]

For macroscopic systems with \(N \sim 10^{23}\), this ratio is negligible (\(\sim 10^{-12}\)), making the ensembles effectively equivalent.

8. Computational Exploration

The following simulation explores the microcanonical ensemble for a two-level system and the ideal gas, computing the entropy, temperature, and demonstrating the approach to thermodynamic equilibrium.

Microcanonical Ensemble: Two-Level System, Einstein Solid, and Thermal Equilibrium

Python

script.py155 lines

import numpy as np
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
from math import lgamma, log, exp

# ============================================================
# Microcanonical Ensemble: Two-Level System and Ideal Gas
# ============================================================

fig, axes = plt.subplots(2, 2, figsize=(12, 10))
fig.suptitle('Microcanonical Ensemble Explorations', fontsize=16, fontweight='bold', color='white')
fig.patch.set_facecolor('#0a0a0a')

for ax in axes.flat:
    ax.set_facecolor('#1a0a0a')
    ax.tick_params(colors='white')
    ax.xaxis.label.set_color('white')
    ax.yaxis.label.set_color('white')
    ax.title.set_color('#fca5a5')
    for spine in ax.spines.values():
        spine.set_color('#333')

# --- Panel 1: Two-level system entropy ---
ax1 = axes[0, 0]
N_vals = [20, 50, 200, 1000]
for N in N_vals:
    x_arr = np.linspace(0.001, 0.999, 500)
    # Exact ln(Omega) using Stirling
    s_arr = []
    for x in x_arr:
        n = int(x * N)
        if n < 1:
            n = 1
        if n >= N:
            n = N - 1
        ln_omega = lgamma(N + 1) - lgamma(n + 1) - lgamma(N - n + 1)
        s_arr.append(ln_omega / N)
    ax1.plot(x_arr, s_arr, label='N = ' + str(N), linewidth=1.5)

# Thermodynamic limit
x_cont = np.linspace(0.001, 0.999, 500)
s_cont = -x_cont * np.log(x_cont) - (1 - x_cont) * np.log(1 - x_cont)
ax1.plot(x_cont, s_cont, 'w--', linewidth=2, label='Thermo limit', alpha=0.7)
ax1.set_xlabel('Energy fraction x = E/(N*eps)')
ax1.set_ylabel('S / (N * k_B)')
ax1.set_title('Two-Level System Entropy')
ax1.legend(fontsize=8, facecolor='#1a0a0a', edgecolor='#333', labelcolor='white')

# --- Panel 2: Temperature of two-level system ---
ax2 = axes[0, 1]
x_t = np.linspace(0.01, 0.99, 500)
# beta * epsilon = ln((1-x)/x)
beta_eps = np.log((1 - x_t) / x_t)
kT_over_eps = np.where(np.abs(beta_eps) > 1e-10, 1.0 / beta_eps, np.inf)
ax2.plot(x_t, kT_over_eps, color='#f87171', linewidth=2)
ax2.axhline(y=0, color='white', linestyle=':', alpha=0.3)
ax2.axvline(x=0.5, color='#fbbf24', linestyle='--', alpha=0.5, label='x = 0.5 (T -> inf)')
ax2.set_xlabel('Energy fraction x')
ax2.set_ylabel('k_B T / eps')
ax2.set_title('Temperature (Negative T for x > 0.5)')
ax2.set_ylim(-5, 5)
ax2.legend(fontsize=8, facecolor='#1a0a0a', edgecolor='#333', labelcolor='white')

# --- Panel 3: Einstein solid microstates ---
ax3 = axes[1, 0]
N_osc = 50
q_vals = np.arange(1, 200)
ln_omega_exact = []
for q in q_vals:
    ln_omega_exact.append(lgamma(q + N_osc) - lgamma(q + 1) - lgamma(N_osc))
ax3.plot(q_vals, np.array(ln_omega_exact) / N_osc, color='#fb923c', linewidth=2, label='Exact')

# High-T limit: S/N ~ ln(q/N) + 1
q_ht = q_vals[q_vals > N_osc]
s_ht = np.log(q_ht / N_osc) + 1
ax3.plot(q_ht, s_ht, 'w--', linewidth=1.5, alpha=0.6, label='High-T approx')
ax3.set_xlabel('Energy quanta q')
ax3.set_ylabel('S / (N * k_B)')
ax3.set_title('Einstein Solid (N = 50 oscillators)')
ax3.legend(fontsize=8, facecolor='#1a0a0a', edgecolor='#333', labelcolor='white')

# --- Panel 4: Equilibration of two systems ---
ax4 = axes[1, 1]
# Two Einstein solids exchanging energy
N_A, N_B = 50, 100
q_total = 200
q_A_range = np.arange(0, q_total + 1)
ln_omega_total = []
for q_A in q_A_range:
    q_B = q_total - q_A
    ln_oA = lgamma(q_A + N_A) - lgamma(q_A + 1) - lgamma(N_A) if q_A >= 0 else -1e10
    ln_oB = lgamma(q_B + N_B) - lgamma(q_B + 1) - lgamma(N_B) if q_B >= 0 else -1e10
    ln_omega_total.append(ln_oA + ln_oB)

ln_omega_total = np.array(ln_omega_total)
# Normalize to get probability
ln_max = np.max(ln_omega_total)
prob = np.exp(ln_omega_total - ln_max)
prob = prob / np.sum(prob)

ax4.plot(q_A_range, prob, color='#a78bfa', linewidth=2)
peak_q = q_A_range[np.argmax(prob)]
ax4.axvline(x=peak_q, color='#fbbf24', linestyle='--', alpha=0.7,
            label='Peak at q_A = ' + str(peak_q))

# Expected from equipartition: q_A/N_A = q_B/N_B => q_A = q_total * N_A/(N_A+N_B)
q_expected = q_total * N_A / (N_A + N_B)
ax4.axvline(x=q_expected, color='#34d399', linestyle=':', alpha=0.7,
            label='Equipartition: ' + str(int(q_expected)))

ax4.set_xlabel('Energy quanta in system A')
ax4.set_ylabel('Probability')
ax4.set_title('Thermal Equilibrium (N_A=50, N_B=100)')
ax4.legend(fontsize=8, facecolor='#1a0a0a', edgecolor='#333', labelcolor='white')

plt.tight_layout()
plt.savefig('output.png', dpi=150, bbox_inches='tight', facecolor='#0a0a0a')
plt.close()
print("Plot saved to output.png")

# Print numerical results
print("\n" + "=" * 60)
print("MICROCANONICAL ENSEMBLE: NUMERICAL RESULTS")
print("=" * 60)

print("\n--- Two-Level System (N = 1000) ---")
for x in [0.1, 0.25, 0.5, 0.75, 0.9]:
    n = int(x * 1000)
    ln_o = lgamma(1001) - lgamma(n + 1) - lgamma(1000 - n + 1)
    s_per_n = ln_o / 1000
    if abs(x - 0.5) > 0.001:
        beta_e = log((1 - x) / x)
        kT_e = 1.0 / beta_e
    else:
        kT_e = float('inf')
    print("  x = " + str(x) + ": S/(Nk) = " + str(round(s_per_n, 6)) + ", kT/eps = " + str(round(kT_e, 4) if kT_e != float('inf') else 'inf'))

print("\n--- Einstein Solid (N = 50) ---")
for q in [10, 50, 100, 500]:
    ln_o = lgamma(q + 50) - lgamma(q + 1) - lgamma(50)
    s_per_n = ln_o / 50
    # Temperature: 1/T = dS/dE ~ [S(q) - S(q-1)] / eps
    ln_o_m1 = lgamma(q - 1 + 50) - lgamma(q) - lgamma(50)
    beta_numerical = (ln_o - ln_o_m1)
    kT_numerical = 1.0 / beta_numerical if abs(beta_numerical) > 1e-10 else float('inf')
    print("  q = " + str(q) + ": S/(Nk) = " + str(round(s_per_n, 4)) + ", kT/eps = " + str(round(kT_numerical, 4)))

print("\n--- Equilibrium of Two Einstein Solids ---")
print("  N_A = 50, N_B = 100, q_total = 200")
print("  Most probable q_A = " + str(peak_q))
print("  Equipartition prediction = " + str(round(q_expected, 1)))
print("  Probability width (std) = " + str(round(np.sqrt(np.sum(prob * (q_A_range - peak_q)**2)), 2)))
print("\n" + "=" * 60)

Click Run to execute the Python code

Code will be executed with Python 3 on the server

9. Summary and Key Results

Core Formulas

Boltzmann entropy: \(S = k_B \ln \Omega\)
Equal probability: \(p_i = 1/\Omega\)
Temperature: \(1/T = \partial S/\partial E\)
Sackur-Tetrode: \(S = Nk_B[\frac{5}{2} + \ln(V/N\lambda_{dB}^3)]\)

Physical Insights

Entropy maximization implies equal a priori probability
The \(1/N!\) Gibbs factor resolves the mixing paradox
Negative temperatures arise in bounded energy systems
Ensembles become equivalent as \(N \to \infty\)

← Course Overview Canonical Ensemble →