# Introduction to Mathematical Physics/Statistical physics/Entropy maximalization

In general, a system is described by two types of variables. External variables ${\displaystyle y^{i}}$ whose values are fixed at ${\displaystyle y_{j}}$ by the exterior and internal variables ${\displaystyle X^{i}}$ that are free to fluctuate, only their mean being fixed to ${\displaystyle {\bar {X^{i}}}}$. Problem to solve is thus the following:

Problem:

Find distribution probability ${\displaystyle P_{l}}$ over the states ${\displaystyle (l)}$ of the considered system that maximizes the entropy

${\displaystyle S=-k_{B}\sum P_{l}\ln(P_{l})}$

and that verifies following constraints:

${\displaystyle {\begin{matrix}\sum X_{l}^{i}P_{l}&=&{\bar {X^{i}}}\\\sum P_{l}&=&1\end{matrix}}}$

Entropy functional maximization is done using Lagrange multipliers technique. Result is:

${\displaystyle P_{l}={\frac {1}{Z}}e^{-\lambda _{1}X_{l}^{1}-\lambda _{2}X_{l}^{2}-...}}$

where function ${\displaystyle Z}$, called partition function, \index{partition function} is defined by:

${\displaystyle Z=\sum _{(l)}e^{-\lambda _{1}X_{l}^{1}-\lambda _{2}X_{l}^{2}-...}}$

Numbers ${\displaystyle \lambda _{i}}$ are the Lagrange multipliers of the maximization problem considered.

Example:

In the case where energy is free to fluctuate around a fixed average, Lagrange multiplier is:

${\displaystyle \beta ={\frac {1}{k_{B}T}}}$

where ${\displaystyle T}$ is temperature.\index{temperature} We thus have a mathematical definition of temperature.

Example:

In the case where the number of particles is free to fluctuate around a fixed average, associated Lagrange multiplier is noted ${\displaystyle \beta \mu }$ where ${\displaystyle \mu }$ is called the chemical potential.

Relations on means[1] that:

${\displaystyle S/k=\sum \lambda _{i}{\bar {X^{i}}}+\ln(Z)}$

This relation that binds ${\displaystyle L}$ to ${\displaystyle S}$ is called a {\bf Legendre transform}.\index{Legendre transformation} ${\displaystyle L}$ is function of the ${\displaystyle y^{i}}$'s and ${\displaystyle \lambda _{j}}$'s, ${\displaystyle S}$ is a function of the ${\displaystyle y^{i}}$'s and ${\displaystyle {\bar {X^{j}}}}$'s.

1. They are used to determine Lagrange multipliers ${\displaystyle \lambda _{i}}$ from associated means ${\displaystyle {\bar {X_{i}}}}$} can be written as:
${\displaystyle -{\frac {\partial }{\partial \lambda _{i}}}\ln Z(y,\lambda _{1},\lambda _{2},...)={\bar {X_{i}}}}$

It is useful to define a function ${\displaystyle L}$ by:

${\displaystyle L=\ln Z(y,\lambda _{1},\lambda _{2},...)}$

It can be shown\footnote{ By definition

${\displaystyle S=-k_{B}\sum P_{l}\ln(P_{l})}$

thus

${\displaystyle S/k=-\sum P_{l}\ln({\frac {1}{Z}}e^{-\lambda _{1}X_{l}^{1}-\lambda _{2}X_{l}^{2}-...})}$
${\displaystyle S/k=1\ln Z+\lambda _{1}{\bar {X_{l}^{1}}}+\lambda _{2}{\bar {X_{l}^{2}}}+...}$