Detailed Fluctuation Theorems: A Unifying Perspective

The discovery of different fluctuation theorems (FTs) over the last two decades constitutes a major progress in nonequilibrium physics [1,2,3,4,5,6]. These relations are exact constraints that some fluctuating quantities satisfy arbitrarily far from equilibrium. They have been verified experimentally in many different contexts, ranging from biophysics to electronic circuits [7]. However, they come in different forms: detailed fluctuation theorems (DFTs) or integral fluctuation theorems (IFTs), and concern various types of quantities. Understanding how they are related and to what extent they involve mathematical quantities or interesting physical observables can be challenging.

The aim of this paper is to provide a simple yet elegant method to identify a class of finite-time DFTs for time-inhomogeneous Markovian jump processes. The method is based on splitting the entropy production (EP) in three contributions by introducing a reference probability mass function (PMF). The latter is parametrized by the time-dependent driving protocol, which renders the dynamics time-inhomogeneous. The first contribution quantifies the EP as if the system were in the reference PMF, the second the extent to which the reference PMF changes with the driving protocol, and the last the mismatch between the actual and the reference PMF. We show that when the system is initially prepared in the reference PMF, the joint probability distribution for the first two terms always satisfies a DFT. We then show that various known DFTs can be immediately recovered as special cases. We emphasize at which level our results make contact with physics and also clarify the nontrivial connection between DFTs and EP fluctuations. Our EP splitting is also shown to be connected to information theory. Indeed, it can be used to derive a generalized Landauer principle identifying the minimal cost needed to move the actual PMF away from the reference PMF. While unifying, we emphasize that our approach by no means encompasses all previously derived FTs and that other FT generalizations have been made (e.g., [5,8,9,10,11]).

The plan of this paper is as follows. Time-inhomogeneous Markov jump processes are introduced in Section 2. Our main results are presented in Section 3: We first introduce the EP as a quantifier of detailed balance breaking, and we then show that by choosing a reference PMF, a splitting of the EP ensues. This enables us to identify the fluctuating quantities satisfying a DFT and an IFT when the system is initially prepared in the reference PMF. While IFTs hold for arbitrary reference PMFs, DFTs require reference PMFs to be solely determined by the driving protocol encoding the time dependence of the rates. The EP decomposition is also shown to lead to a generalized Landauer principle. The remaining sections are devoted to selecting specific reference PMFs and showing that they give rise to interesting mathematics or physics: In Section 4 the steady-state PMF of the Markov jump process is chosen, giving rise to the adiabatic–nonadiabatic split of the EP [12]. In Section 5 the equilibrium PMF of a spanning tree of the graph defined by the Markov jump process is chosen, and gives rise to a cycle–cocycle decomposition of the EP [13]. Physics is introduced in Section 6, and the properties that the Markov jump process must satisfy to describe the thermodynamics of an open system are described. In Section 7 the microcanonical distribution is chosen as the reference PMF, leading to the splitting of the EP into system and reservoir entropy change. Finally, in Section 8, the generalized Gibbs equilibrium PMF is chosen as a reference and leads to a conservative–nonconservative splitting of the EP [14]. Conclusions are finally drawn, and some technical proofs are discussed in the appendices.

We introduce time-inhomogeneous Markovian jump processes and set the notation.

We consider an externally driven open system described by a finite number of states, which we label by n. Allowed transitions between pairs of states are identified by directed edges, where the label $\nu$ indexes different transitions between the same pair of states (e.g., transitions due to different reservoirs). The evolution in time of the probability of finding the system in the state n, $p_n\equiv p_n\left(t\right)$, is ruled by the master equation (ME): where the elements of the rate matrix are represented as

The latter is written in terms of stochastic transition rates, $\left\{w_e\right\}$, and the functions which map each transition to the state from which it originates (origin) and to which it leads (target), respectively. The off-diagonal entries of the rate matrix (the first term in brackets) give the probability per unit time to transition from m to n. The diagonal ones (second term in brackets) are the escape rates denoting the probability per unit time of leaving the state m. For thermodynamic consistency, we assume that each transition $e\equiv (nm,\nu )$ is reversible, namely if $w_e$ is finite, the corresponding backward transition $-e\equiv (mn,\nu )$ is allowed and additionally has a finite rate $w_{-e}$. For simplicity, we also assume that the rate matrix is irreducible at all times, so that the stochastic dynamics is ensured to be ergodic. The Markov jump process is said to be time-inhomogeneous when the transition rates depend on time. The driving protocol value ${\pi }_t$ determines the values of all rates at time t, $\left\{w_e\equiv w_e\left({\pi }_t\right)\right\}$.

The ME (2) can be rewritten as a continuity equation: where we introduced the averaged transition probability fluxes, and the incidence matrix D, which couples each transition to the pair of states that it connects, and hence encodes the network topology. On the graph identified by the vertices $\left\{n\right\}$ and the edges $\left\{e\right\}$, it can be viewed as a (negative) divergence operator when acting on edge-space vectors—as in the ME (5)—or as a gradient operator when acting on vertex-space vectors. It satisfies the symmetry $D_{-e}^n=-D_e^n$.

This section constitutes the core of the paper. The main results are presented in their most general form.

We now provide a first instance of reference PMF based on the fixed point of the Markov jump process.

The Perron–Frobenius theorem ensures that the ME (5) has, at all times, a unique instantaneous steady-state PMF

When using this PMF as the reference, $p_n^{\mathrm{ref}}$ = $p_n^{\mathrm{ss}}$, we recover the adiabatic–nonadiabatic EP rate decomposition [12,16,21,22,23,24]. More specifically, the nonconservative term gives the adiabatic contribution which is zero only if the steady state satisfies detailed balance, and the conservative term gives the nonadiabatic contribution which characterizes transient and driving effects. A specific feature of this decomposition is that both terms are non-negative, as proved in Appendix C: $\langle {\underset{\Sigma }{\dot{}}}_{\mathrm{nc}}\rangle \ge 0$ and $\langle {\underset{\Sigma }{\dot{}}}_{\mathrm{c}}\rangle \ge 0$. In turn, the nonadiabatic contribution decomposes into a relative entropy term and a driving one.

Provided that the forward and backward processes start in the steady state corresponding to the initial value of the respective protocol, the general DFT and IFT derived in Equation (32) and Equation (34) hold for the adiabatic and driving contributions of the adiabatic–nonadiabatic EP decomposition [12,21].

In detailed-balanced systems, the adiabatic contribution is vanishing, $\langle {\underset{\Sigma }{\dot{}}}_{\mathrm{a}}\rangle =0$, and we obtain a FT for the sole driving contribution:

The celebrated Crooks’ DFT [25,26,27] and Jarzynski’s IFT [28] are of this type.

We proceed by providing a second instance of reference PMF based on the equilibrium PMF for a spanning tree of the graph defined by the incidence matrix of the Markov jump process.

We partition the edges of the graph into two disjoint subsets: $\mathcal{T}$ and ${\mathcal{T}}^{*}$. The former identifies a spanning tree, namely a minimal subset of paired edges, $(e,-e)$, that connects all states. These edges are called cochords. All the other edges form ${\mathcal{T}}^{*}$, and are called chords. Equivalently, $\mathcal{T}$ is a maximal subset of edges that does not enclose any cycle—the trivial loops composed by forward and backward transitions, $(e,-e)$, are not regarded as cycles. The graph obtained by combining $\mathcal{T}$ and $e\in {\mathcal{T}}^{*}$ identifies one and only one cycle, denoted by ${\mathcal{C}}_e$, for $e\in {\mathcal{T}}^{*}$. Algebraically, cycles are characterized as: where $\left\{{\mathcal{C}}_e^{e^{\prime }}\right\}$, for $e\in {\mathcal{T}}^{*}$, represent the vectors in the edge space whose entries are all zero except for those corresponding to the edges of the cycle, which are equal to one.

We now note that if $\mathcal{T}$ were the sole allowed transitions, the PMF defined as follows would be an equilibrium steady state [15]: where $Z={\sum }_m{\prod }_{e\in {\mathcal{T}}_m}{w}_e$ is a normalization factor, and ${\mathcal{T}}_n$ denotes the spanning tree rooted in n, namely the set of edges of $\mathcal{T}$ that are oriented towards the state n. Indeed, $p_n^{\mathrm{st}}$ would satisfy the property of detailed balance, Equation (9):

We now pick this equilibrium PMF as a reference for our EP decomposition, $p_n^{\mathrm{ref}}=p_n^{\mathrm{st}}$. However, in order to derive the specific expressions for $\langle {\underset{\Sigma }{\dot{}}}_{\mathrm{nc}}\rangle$ and $\langle {\underset{\Sigma }{\dot{}}}_{\mathrm{c}}\rangle$, the following result is necessary: the edge probability fluxes can be decomposed as where $\left\{{\mathcal{E}}_e\right\}$ denotes the canonical basis of the edge vector space: ${\mathcal{E}}_e^{e^{\prime }}={\delta }_e^{e^{\prime }}$ [31]. Algebraically, this decomposition hinges on the fact that the set ${\left\{{\mathcal{C}}_e\right\}}_{e\in {\mathcal{T}}^{*}}\cup {\left\{{\mathcal{E}}_e\right\}}_{e\in \mathcal{T}}$ is a basis of the edge vector space [13]. Note that for $e\in {\mathcal{T}}^{*}$, the only nonvanishing contribution in Equation (51) comes from the cycle identified by e, and hence $\langle j^e\rangle =\langle {\mathcal{J}}_e\rangle$. The coefficients $\left\{\langle {\mathcal{J}}_e\rangle \right\}$ are called cocycle fluxes for the cochords, $e\in \mathcal{T}$, and cycle fluxes for the chords, $e\in {\mathcal{T}}^{*}$. They can be understood as follows [13]: removing a pair of edges, e and $-e$, from the spanning tree ($e,-e\in \mathcal{T}$) disconnects two blocks of states. The cocycle flux $\left\{\langle {\mathcal{J}}_e\rangle \right\}$ of that edge is the probability flowing from the block identified by the origin of e, $\mathfrak{o}\left(e\right)$, to that identified by the target of e, $\mathfrak{t}\left(e\right)$. Instead, the cycle flux $\left\{\langle {\mathcal{J}}_e\rangle \right\}$ of an edge, $e\in {\mathcal{T}}^{*}$, quantifies the probability flowing along the cycle formed by adding that edge to the spanning tree. Graphical illustrations of cocycle and cycle currents, $\langle {\mathcal{J}}^e\rangle -\langle {\mathcal{J}}^{-e}\rangle$, can be found in Reference [13].

We can now proceed with our main task. Using Equations (48) and (49), we verify that where is the cycle affinity related to ${\mathcal{C}}_e$. It follows that from which the nonconservative contribution readily follows:

In the last equality, we used the property of cycle fluxes discussed after Equation (51). Hence, the nonconservative contribution accounts for the dissipation along network cycles. In turn, combining Equation (16) with Equations (51) and (52), one obtains the conservative contribution which accounts for the dissipation along cocycles. Using these last two results, the EP decomposition (14) becomes the cycle–cocycle decomposition found in Reference [13]:

As for all decompositions, the conservative contribution—here the cocycle one—vanishes at steady state in the absence of driving. The cycle contribution instead disappears in detailed-balanced systems, when all the cycle affinities vanish. This statement is indeed the Kolmogorov criterion for detailed balance [32,33].

The fluxes decomposition Equation (51) is also valid at the trajectory level, where the cycle and cocycle fluxes become fluctuating instantaneous fluxes, $\{{\mathcal{J}}_e\}$. Obviously, the same holds true for the cycle–cocycle EP decomposition. Therefore, if the system is in an equilibrium PMF of type (49) at the beginning of the forward and the backward process, a DFT and an IFT hold by applying Equations (32) and (34). Note that the fluctuating quantity appearing in the DFT, ${\Sigma }_{\mathrm{d}}+{\Sigma }_{\mathrm{nc}}$, can be interpreted as the EP of the extended process in which, at time t, the driving is stopped, all transitions in ${\mathcal{T}}^{*}$ are shut down, and the system is allowed to relax to equilibrium—which is the initial PMF of the backward process.

It is worth mentioning that one can easily extend the formulation of our DFT by considering the joint probability distribution for each subcontribution of ${\Sigma }_{\mathrm{d}}$ and ${\Sigma }_{\mathrm{na}}$ antisymmetrical under time reversal. This can be shown using either the proof in Appendix B [16], or that in Appendix A [14]. In the case of the cycle–cocycle decomposition, it would lead to which is a generalization of the DFT derived in Reference [34] to time-inhomogeneous systems. In turn, the latter is a generalization of the steady-state DFT derived by Andrieux and Gaspard in Reference [35] to finite times.

The results obtained until this point are mathematical and have a priori no connection to physics. We now specify the conditions under which a Markov jump process describes the dynamics of an open physical system in contact with multiple reservoirs. This will enable us to introduce physically motivated decompositions and derive DFTs with a clear thermodynamic interpretation.

Each system state, n, is now characterized by given values of some system quantities, $\left\{X_n^{\kappa }\right\}$, for $\kappa =1,\dots ,{\mathsf{N}}_{\kappa }$, which include the internal energy, $E_n$, and possibly additional ones (see Table 2 for some examples). These must be regarded as globally conserved quantities, as their change in the system is always balanced by an opposite change in the reservoirs. When labeling the reservoirs with $\left\{r\right\}$, for $r=1,\dots ,{\mathsf{N}}_{\mathrm{r}}$, the balance equation for $X^{\kappa }$ along the transition e can be written as:

The lhs is the overall change in the system, whereas ${\delta }_{\mathrm{i}}{X}_e^{\kappa }$ denotes the changes due to internal transformations (e.g., chemical reactions [36,37]), and $\delta X_e^{(\kappa ,r)}$ quantifies the amount of $X^{\kappa }$ supplied by the reservoir r to the system along the transition e. For the purposes of our discussion, we introduce the index $y=(\kappa ,r)$—i.e., the conserved quantity $X^{\kappa }$ exchanged with the reservoir r—and define the matrix $\delta X$ whose entries are $\{\delta X_e^y\equiv \delta X_e^{(\kappa ,r)}\}$. All indices used in the following discussion are summarized in Table 3. Microscopic reversibility requires that $\delta X_e^y=-\delta X_{-e}^y$. Note that more than one reservoir may be involved in each transition (see Figure 4).

In addition to the trivial set of conserved quantities $\left\{X^{\kappa }\right\}$, the system may be characterized by some additional ones, which are specific for each system. We now sketch the systematic procedure to identify these quantities and the corresponding conservation laws [14,38]. Algebraically, conservation laws can be identified as a maximal set of independent vectors in the y-space, $\{{\ell }^{\lambda }\}$, for $\lambda =1,\dots ,{\mathsf{N}}_{\lambda }$, such that

Indeed, the quantities $\left\{{\ell }_y^{\lambda }\delta X_e^y\right\}$, for $\lambda =1,\dots ,{\mathsf{N}}_{\lambda }$, are combinations of exchange contributions $\left\{\delta X_e^y\right\}$, for $y=1,\dots ,{\mathsf{N}}_{\lambda }$, which vanish along all cycles. They must therefore identify some state variables, $\{L^{\lambda }\}$, for $\lambda =1,\dots ,{\mathsf{N}}_{\mathrm{y}}$, in the same way curl-free vector fields are conservative and identify scalar potentials:

This equation can be regarded as the balance equation for the conserved quantities. In the absence of internal transformations, ${\delta }_{\mathrm{i}}{X}_e^{\kappa }$, trivial conservation laws correspond to ${\ell }_y^{\kappa }\equiv {\ell }_{({\kappa }^{\prime },r)}^{\kappa }={\delta }_{{\kappa }^{\prime }}^{\kappa }$, so that the balance Equations (62) are recovered. Notice that each $L^{\lambda }$ is defined up to a reference value.

Each reservoir r is characterized by a set of entropic intensive fields conjugated to the exchange of the system quantities $\left\{X^{\kappa }\right\}$, $\left\{f_{(\kappa ,r)}\right\}$ for $\kappa =1,\dots ,{\mathsf{N}}_{\kappa }$ (e.g., [39] § 2–3). A short list of $X^{\kappa }$–$f_{(\kappa ,r)}$ conjugated pairs is reported in Table 2. The thermodynamic consistency of the stochastic dynamics is ensured by the local detailed balance,

It relates the log ratio of the forward and backward transition rates to the entropy change in the reservoirs resulting from the transfer of system quantities during that transition. This entropy change is evaluated using equilibrium thermodynamics (in the reservoirs), and reads $\left\{\delta S_e^{\mathrm{r}}=-f_y\delta X_e^y\right\}$. The second term on the rhs is the internal entropy change occurring during the transition, as $S_n$ quantifies the internal entropy of the state n. This term can be seen as the outcome of a coarse-graining procedure over a finer description in which multiple states with the same system quantities are collected in one single n [40]. Using Equation (65), the affinities, Equation (11), can be rewritten as:

This relation shows that the affinity is the entropy change in all reservoirs plus the system entropy change. In other words, while Equation (64) characterizes the balance of the conserved quantities along the transitions, Equation (66) characterizes the corresponding lack of balance for entropy, namely the second law.

As for the transition rates, the changes in time of the internal entropy S, the conserved quantities $\left\{X^{\kappa }\right\}$ (hence $\left\{\delta X_e^y\right\}$), and their conjugated fields $\left\{f_y\right\}$, are all encoded in the protocol function ${\pi }_t$. Physically, this modeling describes the two possible ways of controlling a system: either through $\left\{X^{\kappa }\right\}$ or S which characterize the system states, or through $\left\{f_y\right\}$ which characterize the properties of the reservoirs.

We start by considering a microcanonical PMF as reference: where is the Boltzmann’s equilibrium entropy. With this choice, the reference affinities become sums of entropy changes in the reservoirs and hence the nonconservative contribution becomes the rate of entropy change in all reservoirs

For the conservative contribution, one instead obtains:

Using Equation (17), it can be rewritten in terms of the Gibbs–Shannon entropy, and the Boltzmann entropy. Indeed, and so that

The conservative contribution thus contains changes in the system entropy caused by the dynamics and the external drive.

The EP decomposition (14) with Equations (76) and (81) is thus the well-known system–reservoir decomposition (i.e., the traditional entropy balance). Since the same decomposition holds at the trajectory level, if the initial PMF of the forward and backward processes are microcanonical, the DFT and IFT hold by applying Equations (32) and (34). When the driving does not affect the internal entropy of the system states $\left\{S_n\right\}$, the DFT and IFT hold for the reservoir entropy alone. Finally, the fluctuating quantity appearing in the DFT, ${\Sigma }_{\mathrm{d}}+{\Sigma }_{\mathrm{nc}}$, can be interpreted as the EP of the extended process in which, at time t, the driving is stopped, all temperatures are raised to infinity, ${\beta }_r\to 0$, and the system is allowed to relax to equilibrium—the initial PMF of the backward process.

We now turn to a reference PMF which accounts for conservation laws: the generalized Gibbs PMF.

To characterize this PMFs, we observe that since $\{{\mathit{\ell }}^{\lambda }\}$ are linearly independent (otherwise we would have linearly dependent conserved quantities), one can always identify a set of y’s, denoted by $\{y_{\mathrm{p}}\}$, such that the matrix whose rows are $\{{\ell }_{y_{\mathrm{p}}}^{\lambda }\}$, for $\lambda =1,\dots ,{\mathsf{N}}_{\lambda }$, is nonsingular. We denote by $\{{\overline{\ell }}_{\lambda }^{y_{\mathrm{p}}}\}$ for $\lambda =1,\dots ,{\mathsf{N}}_{\lambda }$, the columns of the inverse matrix. All other y’s are denoted by $\{y_{\mathrm{f}}\}$. Using the splitting $\{y_{\mathrm{p}}\}$–$\{y_{\mathrm{f}}\}$ and the properties of $\{{\ell }_{y_{\mathrm{p}}}^{\lambda }\}$, in combination with the balance equation for conserved quantities, Equation (64), the local detailed balance (65) can be decomposed as where are the system-specific intensive fields conjugated to the conserved quantities, and are differences of intensive fields called nonconservative fundamental forces. Indeed, these nonconservative forces are responsible for breaking detailed balance. When they all vanish, ${\mathcal{F}}_{y_{\mathrm{f}}}=0$ for all $y_{\mathrm{f}}$, the system is indeed detailed balanced and the PMF with ${\mathsf{\Phi }}_{\mathrm{gg}}:=ln{\sum }_{n}exp\left\{S_n-F_{\lambda }{L}_n^{\lambda }\right\}$, satisfies the detailed balance property (9). The potential corresponding to Equation (85), ${\psi }_n^{\mathrm{gg}}$, is minus the Massieu potential which is constructed by using all conservation laws (e.g., [39] §§ 5-4 and 19-1, [44] § 3.13). Choosing the PMF (85) as a reference, $p_n^{\mathrm{ref}}=p_n^{\mathrm{gg}}$, the reference affinity straightforwardly ensues from Equation (82),

Hence, where are the fundamental currents conjugated to the forces. For the conservative contribution, one obtains