Ensemble equivalence

What you will learn

• Statistical Ensembles: Understanding the role of ensembles in statistical mechanics and how they describe macroscopic systems using probability distributions.

• Microcanonical Ensemble (NVE): Definition, characteristics, and assumptions—fixed energy, volume, and particle number; statistical weight of microstates.

• Equivalence of Ensembles: Conditions under which the microcanonical and canonical ensembles give equivalent thermodynamic predictions in the thermodynamic limit.

• Canonical Ensemble (NVT): Definition and significance—fixed temperature, volume, and particle number; derivation of the Boltzmann factor.

• Partition Function (\(Z\)): Role of the partition function in statistical mechanics, its connection to thermodynamic properties (free energy, entropy, internal energy).

• Boltzmann Distribution: Probability of a system being in a given energy state at equilibrium, leading to macroscopic observables.

• Thermodynamic Connections: How ensemble averages link to macroscopic properties such as energy, entropy, and pressure.

• Fluctuations and Large System Limits: How energy fluctuations in the canonical ensemble become negligible for large systems, reinforcing ensemble equivalence.

Microcanonical Ensemble (NVE)

• A collection of all possible microscopic arrangements consistend with an equilibrium thermodynamic state is called statistical ensemble.

• Ensemble defines sample space over microstates over which we define micro and macro-state probabilities

• Consider an isolated fluid system with \(N=const\), \(V=const\) and \(E=const\). This is called a microcanonical ensemble

• In the absence of any physical constraints, no micro state is more probable than any other. This is known as “principle of equal a priory probability”.

A priori equal probability

\[p(E) = \frac{1}{\Omega(N, V, E)}\]

• P(E) Probability of any microstate in the system with energy \(E\)

Boltzmann equation

\[S = k_B log \Omega\]

• \(\Omega\) is the number of micro-states consistent with a macrostate of our system.

• \(S(N, V, E)\) is the entropy of an isolated system (we are in the NVE).

• \(k_B =1.380649\cdot 10^{-23} J/K\), Boltzmann’s constant

Applications of NVE

Two-state partciles

Cnsider a simple two-level system where lower level has energy \(0\) and upper level \(\epsilon\). Determine how the fraction of excited states \(f=\frac{n}{N}\) changes with temperature.

Solution

• The total energy when we have a \(n\) number of particles in upper level is:

\[E = n\epsilon\]

• Number of microstaates is all possible ways of arranging N number of particles given fixed energy \(E\)

\[\Omega = \frac{N!}{(N-n)! n!} = \frac{N!}{(N-n)! n!}\]

• Taking log of binomial (see random walk section) we get expression of entropy

\[S = k_B log \Omega = N k_B \Big [- \frac{n}{N}log \frac{n}{N} - (1-\frac{n}{N})log (1-\frac{n}{N}) \Big]\]

• Since we have entropy as a function of energy \(n=E/\epsilon\) we can use thermodynamic relations to obain expression for how fraction of excited states changes with temperature:

\[\frac{1}{T} = \frac{\partial S(n)}{\partial n} \frac{\partial n}{\partial E}= \frac{k_B}{\epsilon}\frac{\partial log\Omega(n)}{\partial n} = \frac{k_B}{\epsilon}log \frac{N-n}{N}\]

\[\frac{n}{N} = \frac{1}{1+e^{\beta \epsilon}}\]

\[E = n\epsilon = \frac{N\epsilon}{1+e^{\beta \epsilon}}\]

Random polymer chain

• Consider 1D polymer where \(N\) monomers are randomly oriended with \(+l\) and \(-l\) orientation along the \(x\) axis

Exchanging Energy

• Consider a small system in thermal contact with a large reservoir at temperature \( T \).

• The total system (system + reservoir) is isolated, with a fixed at \( E_t \) hence we have a microcanonical ensemble.

• The total number of microstates for the combined system is:

  \[ \Omega_t(E_t) = \sum_E \Omega(E) \Omega_r(E_t - E) \]

  where:

  • \( \Omega(E) \) is the number of microstates available to the system when it has energy \( E \).

  • \( \Omega_r(E_t - E) \) is the number of microstates available to the reservoir when the system has energy \( E \).

• The probability of the system being in a macrostate that is, having energy \(E\) is:

  \[ p(E) = \frac{\Omega(E) \Omega_r(E_t - E)}{\sum_E \Omega(E) \Omega_r(E_t - E)} \]

• The system is more likely to be in a macrostate \( E \) if the reservoir has many ways to accommodate the remaining energy \( E_t - E \).

Constant temperature ensemble (NVT)

• Since the reservoir is much larger than the system \( E \ll E_t \), its entropy or log of microstates is a smooth function of energy:

  \[log\Omega_r(E_t-E) \approx log\Omega_r(E_t) - \frac{\partial log \Omega_r}{\partial E}E = const - \beta E\]

• First factor is constnatn independent of energy and the reservoir influence is now only described by the coefficient which is related to temperature!

\[\beta = \frac{1}{k_B}\frac{dS}{dE}=\frac{1}{k_BT}\]

• We find that number of microstates of large reservoir decrease exponentially when system borrows energy \(E\)

\[ \Omega_r(E_t - E) \sim e^{- \beta E} \]

Boltzmann distribution (NVT)

Probability of a Macrostate

\[ p(E) = \frac{\Omega(E) e^{-\beta E}}{Z} \]

Probability of a Microstate

\[ p_i = \frac{e^{-\beta E_i}}{Z} \]

Parition Function

\[ Z = \sum_E \Omega(E) e^{-\beta E} = \sum_i e^{-\beta E_i} \]

• The partition function \(Z\) keeps the normalization of the ratio and is a sum over all micro or macrostates weighted by exponential Boltzman factor.

MaxEnt Derivation of NVT

We can derive the Botlzman distribution by Maximizing Entropy with a constraint placed on fixed average energy \(\langle E\rangle =U\)

Partition Functions and Thermodynamic limit

\[Z = \sum_E \Omega(E) e^{-\beta E} = \sum_E e^{-\beta (E-TS)}\]

• The number of states grows exponentially with system size \( N \), \( \Omega(E) = e^{\frac{S}{k_B}} \sim e^{N}\) while the Boltzmann factor decays exponentially with energy \(e^{-\beta E} \sim e^{-N}\)

• These competing exponential behaviors determine the dominant contribution to the partition function.

• In the thermodynamic limit (large \( N \) and \( V \)), only energies that significantly contribute to \( Z \) survive. This allows rewriting the integral as:

\[ Z \approx e^{\min_E [-\beta (E - TS)]} = e^{-\beta(U - TS)} = e^{-\beta F} \]

where:

• \( U = \langle E \rangle \) is the average energy, with fluctuations of order \( O(N^{1/2}) \).

• \( F = U - TS \) is the Helmholtz free energy.

Examples of using NVT

Two-state partciles (NVT)

Cnsider a simple two-level system where lower level has energy \(0\) and upper level \(\epsilon\). Determine how the fraction of excited states \(f=\frac{n}{N}\) changes with temperature.

Solution

• Solving a two-state particle system in an NVT ensemble is much easier because the partition function decouples into single particle contributions.

\[Z = \sum^{n=N}_{n=0} e^{-\beta E_n} = (1+e^{-\beta \epsilon})^N\]

\[F= -k_B T log Z = -k_BT N log(1+e^{-\beta \epsilon})\]

\[\langle E \rangle = \frac{\partial log Z}{\partial (-\beta)} = \frac{N\epsilon}{1+e^{\beta \epsilon}}\]

Free energy and macrostate probabilities

1. Microstates: The relative population of microstates is dictated by the ratio of Boltzmann weights which depends on energy difference \(\Delta E\)

\[p_1 = \frac{e^{-\beta E_1}}{Z} = \frac{e^{-\beta E_1}}{e^{-\beta F}}\]

\[{\frac{p_2}{p_1} = e^{-\beta (E_2-E_1)}}\]

2. Macrostates: Probability of macrostates with energy \(E_A\) is obtained by summing over all microstates with energy E_A or simply by multiplying by \(\Omega_A\). The latter is related to entropy, which ends up turning the numerator into the free energy of a macrostate \(A\): \(F_A = E_A-TS_A\)

\[p_A =\frac{\Omega (E_A) e^{-\beta E_A}}{Z_A} = \frac{e^{-\beta F_A}}{e^{-\beta F}}\]

• The relative population of macrostates is dictated by the ratio of entropic term times Boltzmann weights which depends on free energy difference \(\Delta F\)

\[{\frac{p_B}{p_A} = e^{-\beta (F_B-F_A)}}\]

Derivation of Average Energy and Fluctuations

• The probability of the system being in microstate i is given by the Boltzmann distribution:

\[ P_i = \frac{e^{-\beta E_i}}{Z}, \quad Z = \sum_i e^{-\beta E_i} \]

\[ \langle E \rangle = \sum_i P_i E_i = \frac{1}{Z} \sum_i E_i e^{-\beta E_i} \]

• The ensemble average energy can be related to the derivative of log of partition function:

\[ \langle E \rangle= -\frac{1}{Z} \frac{\partial Z}{\partial \beta} = -\frac{\partial \ln Z}{\partial \beta} \]

Moments of Energy in NVT ensemble

\[ \langle E\rangle = -\frac{\partial \ln Z}{\partial \beta} \]

\[ \sigma_E^2 = \langle E^2 \rangle - \langle E \rangle^2 = \frac{\partial^2 \ln Z}{\partial \beta^2} \]

Fluctuation-response and ensemble equivalence

• We can now show that heat capaicty is related to energy fluctuations a result known as fluctuation-response theorem

\[ C_V = \left( \frac{\partial \langle E \rangle}{\partial T} \right)_V = - k_B \beta^2 \frac{\partial^2 \ln Z}{\partial \beta^2} \]

Fluctuation-Response Theorem

\[\sigma_E^2 = k_B T^2 C_V \]

• Relative energy fluctuations scale as:

  \[ \frac{\sigma_E}{\langle E \rangle} = \frac{(k_B T^2 C_V)^{1/2}}{\langle E \rangle} \sim O(N^{-1/2}) \]

• In the thermodynamic limit \(N\rightarrow \infty \), fluctuations become negligible, justifying ensemble equivalence.

More examples

Temperature dependence of magnetization (NVE)

Consider a set of \(N= N_{\uparrow} + N_{\downarrow}\) spins pointing up and down in an external magnetic field \(B\) with energy \(\epsilon = \pm \mu B\) where \(\mu\) magnetic moment of the spin. DefiningDefine \(M = N_{\uparrow} - N_{\downarrow}\) as the overall magnetization and \(m = M/n\) magnetization per spin. Using NVE ensemble of spins with fixed energy \(E\) find the temperature dependence of \(m(T)\).

• We start by writing down the number of microstates for a given \(E\) we need to find a number of ways to partition \(N_{\uparrow}\) and \(N_{\downarrow}\) spins.

\[\Omega(N, N_{\uparrow}) = \frac{N!}{N_{\uparrow}! N_{\downarrow}!} = \frac{N!}{[1/2(N+M)!] [1/2(N-M)!]}\]

\[{\Omega (E,N)= \frac{N!}{\Big[\frac{1}{2}(N-E/B\mu)\Big]! \Big[\frac{1}{2}(N+E/B\mu)\Big]!}}\]

\[S = Nk_B \Big[log 2 - \frac{1}{2}log(1-m^2)-\frac{m}{2} log \frac{1+m}{1-m} \Big]\]

• Find temperature dependence by using \(\frac{1}{T} = \frac{\partial S}{\partial E}|_N\)

\[U = -\mu N B tanh \Big(\frac{\mu B}{k_B T} \Big)\]

• magnetization per spin \(m=-U/\mu NB\) is given by:

\[m = tanh \Big(\frac{\mu B}{k_B T} \Big)\]

Temperature dependence of magnetization (NVT)

Solve the same problem but now using NVT ensemble of spins coupled to a heat bath at fixed \(T\)t emperature instead

Partition function of a single spin

\[Z_1 = \sum_{s_1} e^{\beta \mu B s_1} = e^{\beta \mu B} + e^{-\beta \mu B} = 2 cosh(\beta \mu B)\]

Partition function of N spins

\[Z =Z_1^N\]

Free energy

\[ F(T, B) = - \beta^{-1} log Z = - N \beta^{-1} log \Big [ 2 cosh (\beta \mu B ) \Big ]\]

Entropy

\[S = - \Big(\frac{\partial F}{\partial T} \Big)_B = k_B log \Big [ 2 cosh (\beta \mu H ) \Big ] - k_B (\beta \mu B)\cdot tanh (\beta \mu B)\]

Magnetization

Magnetization \(M\) (extensive quantity) or magnetization per particle \(m=M/N\) (intensive quantity) is given as another free energy derivative:

\[M = - \Big(\frac{\partial F}{\partial B} \Big)_T = \mu tanh \Big(\beta \mu B \Big)\]

Magnetizatic susceptibility

In the context of paramagnet, we have another response function in the form of magnetic susceptibility.

\[\chi(T, B) = \Big(\frac{\partial m}{\partial B} \Big)_T = \mu^2 \beta B cosh^{-2} (\beta \mu B)\]

which leads to a well-known Curie Law

\[\chi(T, B=0) = \frac{\mu^2 }{k_B T}\]

Finally, as a consistency check we can combine entropy and free energy expressions to obtain internal energy:

\[U = F+TS = -\mu B tanh(\beta \mu B)\]

Problems

Shottky defects

Schotky defects are vacancies in a lattice of atoms. Creating a single vacancy costs an energy \(\epsilon\). Consider a lattice with \(N\) atoms and \(n\) vacacnies. In this model the total energy is solely a function of defects: \(E=n\epsilon\)

• Write down number of states and compute the entropy via Boltzmann formula. Plot number of states as a function of energy. You can use log of number of states for plotting.

• Compute how the temperature would affect the fraction of vacancies on the lattice. Plot fraction of vacancies as a function of temperature.

• How would the total energy depend on temperature \(T\). Derive expression for the high temeprature limit (\(\frac{\epsilon}{k_b T} \gg 1\)).

• Plot total energy as a function of temperature E(T)

Lattice gas

Consider a lattice gas of N particles distributed among V cells (with \(N\geq V\)). Suppose that each cell may be either empty or occupied by a single particle. The number of microscopic states of this syste will be given by:

\[\Omega (N, V) = \frac{V!}{N! (V-N)!}\]

• Obtain an expression for the entropy per particle \(s(v)=\frac{1}{N} \cdot S(N,V)\) where \(v=\frac{V}{N}\).

• From this simple fundamental equation, obtain an expression of equation of state \(p/T\).

• • Write an expansion of \(p/T\) in terms of density \(1/v\). Show that the first term gives Boyle law of ideal gases.

• Sketch a graph of \(\mu/T\), where \(\mu(\rho)\) is a chemical potential as a function of density. Comment on \(\rho\rightarrow 0\) and \(\rho\rightarrow 1\) limits.

Polymer Elasticity

Solve the problem 2.7 from the book.