Dynamic moral hazard without commitment

We study a discrete-time model of repeated moral hazard without commitment. In every period, a principal finances a project, choosing the scale of the project and a contingent payment plan for an agent, who has the opportunity to appropriate the returns of a successful project unbeknownst the principal. The absence of commitment is reflected both in the solution concept (perfect Bayesian equilibrium) and in the ability of the principal to freely revise the project’s scale from one period to the next. We show that removing commitment from the equilibrium concept is relatively innocuous—if the players are sufficiently patient, there are equilibria with payoffs low enough to effectively endow the players with the requisite commitment, within the confines of perfect Bayesian equilibrium. In contrast, the frictionless choice of scale has a significant effect on the project’s dynamics. Starting from the principal’s favorite equilibrium, the optimal contract eventually converges to the repetition of the stage-game Nash equilibrium, operating the project at maximum scale and compensating the agent (only) via immediate payments.

1. Another difference with the literature is that we do not need to assume that the principal and agent have different discount factors.

2. Hence, messages are not “cheap.” Adding messages by the agent to this binary disclosure decision would not affect the analysis: the agent would have to be indifferent over messages, and from the principal’s point of view such messages wouldn’t achieve anything she cannot achieve with the public randomization device.

3. That is, for each \(h_P^{n}\in H_P^{n}\), \(\sigma _{P}^{n}(h_P^{n})\) is a probability distribution over \([0,1] \times \mathbf {R}_+^2\), and the probability \(\sigma _{P}^{n}(\cdot )[A]\) assigned to any Borel set \(A\subset [0,1] \times \mathbf {R}_+^2\) is a measurable function of \(h_P^{n}\), and similarly for \(\sigma _{A}^{n}\).

4. The stage game is a game of imperfect information, since the principal does not observe Nature’s draw, but nonetheless subgame perfection implies sequential rationality, and hence a subgame-perfect equilibrium is also (part of) a perfect Bayesian equilibrium.

5. This does not imply, however, that non-stationary outcome paths achieving the same payoffs cannot be constructed. Equilibrium payoff vectors for which \(W >W^*\) can be achieved in various ways, as there is some leeway in specifying the timing of payments.

6. Because the inequality must bind (as is shown below), so that the functional equation can be rewritten as an unconstrained one (see Sect. 3.4.2), S is the unique solution of this equation, by Blackwell’s theorem.

7. It is here that we use the left inequality in (3). If this inequality fails, we will sometimes have \(z^S>0\).

8. The variables q and \(\underline{q}\) appear in the principal’s objective with weights \(p\pi -c\) and (after using the incentive constraint to eliminate \(\underline{z}^F\)) \((1-p)(p(1-k)\pi -c)\), respectively. They appear in the agent’s value with weights \(pk\pi \) and \(Dpk\pi \). The inequality \(D>1-p\) is a sufficient condition for the agent’s tradeoff between q and \(\underline{q}\) to place a higher premium on the first period than does the principal’s.

9. While renegotiation-proofness has been introduced for games with complete information, and so refers to subgame-perfect equilibrium, we note that the perfect Bayesian equilibria we consider are recursive as well, given that the agent’s strategy does not depend on his past private information. Hence, continuation equilibria are well-defined.

10. Recall that it exists if we assume \(p(1-k)\pi -c \le \frac{\delta p}{1-\delta (1-p)}(p\pi -c)\).

11. To see this, note that this equation is homogeneous in W; hence, working with \(\tilde{S}(W)/W\), and normalizing it by a constant to cancel the constant in the equation, we are left with a right-hand side that suggests taking an exponential, to “pull out” the multiplicative constant that appears in the argument of \(\tilde{S}\). Hence the logarithm.

12. Uniqueness of the solution of this equation follows from value iteration, since it is a contraction on the domain of functions for which the constraint is satisfied.

13. \(\underline{W}=W^*\) if and only if the constraint cannot be satisfied for any \(W<W^*\).

14. Note that we have already taken into account in the range of possible values of q (by requiring \(q\le w/(1-\delta )\)) the constraint that \(w^F \ge 0\).

15. See, e.g., Leoni (2009), Exercise 3.20.

16. Unlike \(s_{n+1}'\), \(s_{n+1}''\) is not continuously differentiable. Hence, \(\int _{w'}^{w''}s_{n+1}'''(w)dw \ne s_{n+1}''(w'')-s_{n+1}''(w')\) for some values of \(w',w''\), so that assuming \(s_n''' \ge 0\) as the induction hypothesis is not enough for the proof.

17. One might in each of the following cases wonder whether these adjustments preserve the feasibility of \((W^S,V^S)\). In each case, the resulting values remain within the convex hull of four payoff vectors, namely \((\underline{W}<\underline{V})\), \((\underline{V}-\underline{W},\underline{W})\), \((W^*,V^*)\), and the original values of \((W^S,V^S)\).

Authors and Affiliations

1. Department of Economics, Yale University, New Haven, CT, 06520, USA

   Johannes Hörner & Larry Samuelson

Corresponding author

Correspondence to Larry Samuelson.

We thank Bruno Biais and Thomas Mariotti for helpful conversations and comments and we thank Yi Chen for valuable research assistance. We thank the National Science Foundation (SES-1153893 and SES-1459158, Samuelson) for financial support.

Appendix: Proofs

1.1 Proof of Lemma 1

[1.1] If \(p(1-k)\pi -c>0\), then the repeated game has an equilibrium, which repeats the stage-game Nash equilibrium in each period, that achieves the efficient surplus (cf. Proposition 1.1). It is then immediate that every constrained efficient equilibrium gives this sum of payoffs, which can only be achieved by setting \(q=1\) in every period.

[1.2] Let \((q, z^F, z^S)\) identify the scale of the project and the payments to the agent (in the event of a reported failure and success) in the first period. Let \((W^S,V^S)\) and \((W^F,V^F)\) denote continuation payoffs in the event of a (reported) success and failure. A constrained efficient equilibrium outcome solves the maximization problem

subject to the constraints that \(W^S\), \(V^S\), \(W^F\), and \(V^F\) are equilibrium payoffs and that W and V are nonnegative, where \(\mathbf 1_S\) is an indicator for the event that the agent reports a success.

Because \(p(1-k)\pi -c>0\), we know that this problem has a positive solution, with a value equal to \(p\pi -c\). This value in turn can be generated only if \(q=1\) and \(\mathbf 1_S=1\), so that we can rewrite the problem solved by a constrained efficient equilibrium outcome as

subject to the constraints that \(W^S\), \(V^S\), \(W^F\), and \(V^F\) are equilibrium payoffs and that W and V are nonnegative. Now note that it has no effect on the objective, and preserves the constraints, and hence sacrifices no generality, to assume that \(z^F=0\). Hence, a constrained efficient equilibrium outcome must solve the maximization problem

subject to the constraints that \(W^S\), \(V^S\), \(W^F\), and \(V^F\) are equilibrium payoffs and that W and V are nonnegative. We can rearrange the incentive constraint to give \((1-\delta )pz^S \ge (1-\delta )pk\pi +\delta p(W^F-W^S)\), and inserting into the expression for W,

Now let \(\underline{W}\) be the lowest constrained-efficient payoff for the agent. Then \(W^F\ge \underline{W}\), and hence

holds for every agent payoff consistent with a constrained-efficient equilibrium, including \(\underline{W}\), which gives \(\underline{W} \ge (1-\delta )pk\pi +\delta \underline{W}\) and hence \(\underline{W}\ge pk\pi \).

[1.3] We again consider the maximization problem given by (7)–(11), subject to the constraints that \(W^F\), \(V^S\), \(W^F\), and \(V^F\) are equilibrium payoffs and that W and V are nonnegative. Assume this problem has a positive solution, with value \(\hat{S}\).

First suppose \(q=0\). Then the sum of continuation payoffs is positive (since the problem has a positive solution), and hence a higher sum of payoffs can be achieved by simply beginning in the first period with an equilibrium whose payoffs are the maximum of \(W^F+V^F\) and \(W^S+V^S\). We can thus assume \(q>0\).

Now suppose that \(q>0\) and \(\mathbf 1_S = 0\). Then it has no effect on continuation payoffs and preserves the constraints, and hence sacrifices no generality, to assume that \(W^S=W^F\) (simply choose \((W^S,V^S)\) and \((W^F,V^F)\) to both equal whichever of these pairs has the larger sum) and \(z^S=z^F=0\). But then the sum of payoffs can be increased (by \(p(1-k)\pi \)) while preserving continuation values and increasing \(z^S\) to \(qk\pi \) and setting \(\mathbf 1_S=1\). Hence, we can assume that \(\mathbf 1_S = 1\). There is then no loss of generality in assuming that \(z_S\) is set so that the incentive constraint binds. This gives the problem

subject to the constraint that \(W^S\), \(V^S\), \(W^F\), and \(V^F\) are equilibrium payoffs and that W and V are nonnegative. Solving the incentive constraint gives \((1-\delta )pz^S = (1-\delta )qpk\pi +\delta p(W^F-W^S)\), and inserting into the expressions for W and V, we have

subject to the familiar constraints. Recalling that \(\hat{S}\) is the solution to the surplus maximization problem, we have

where \(\underline{W}\) is the smallest equilibrium payoff for the agent. An upper bound on the principal’s equilibrium payoff is then given by \(\overline{V}\), where

for some \(q\in [0,1]\). Since \({p(1-k)\pi -c}<0\), this gives \(\overline{V}=0\), completing the proof. \(\square \)

1.2 Proof of Lemma 5

To simplify the notation, let \(w:=W/W^*\) and \(s(w):=S(W)/(p \pi -c)\). Hence, we are interested in the behavior of the function s(w) on \(w\in [0,1]\) (corresponding to the behavior of S(W) on \([0,W^*])\). By the principle of optimality, the function s is the solution to the problem \(\mathcal {P}\): solve for \(s:\mathbf {R}_+ \rightarrow \mathbf {R}_+\), given by

with boundary conditions \(s(0)=0\), \(s(w)=1\) for \(w \ge 1\).^{Footnote 14} By Blackwell’s theorem, this equation admits exactly one solution (which must be non-negative, as the right-hand side maps non-negative functions into non-negative ones).

1. [5.1]

   First, we show that s has an infinite right derivative at \(w=0\). To see that \(\lim _{w \downarrow 0} \frac{s(w)}{w} =+\infty \), consider the strategy that sets q so that \(w^F=w\) as long as \(w \in (0,1)\), that is, the strategy that sets \(q=w\) (until \(w \ge 1\), if ever, in which case the optimal strategy is followed, for a payoff of 1). This gives us a lower bound on the value, call it \(\tilde{s}\). This lower bound satisfies the recursion

   $$\begin{aligned} \tilde{s}(w)=\frac{1-\delta }{1-\delta +\delta p}w+\frac{\delta p}{1-\delta +\delta p}\tilde{s}\left( \frac{1+(1-\delta )\frac{1-p}{p}}{\delta }w\right) . \end{aligned}$$

Fixing w, we can compute the number n of successive successes required to push the resulting continuation payoff for the agent above 1, namely, the lowest integer such that

or

Plugging back into the recursion for \(\tilde{s}\), we obtain

and it is readily verified that the right-hand tends to infinity as \(w \rightarrow 0\), yielding the result.

Next, we show that s(w) has a zero left derivative at \(w=1\). Since the function s is constant to the right of \(w=1\), this ensures that s is differentiable at 1. To show that \(\lim _{w \uparrow 1} \frac{s(1)-s(w)}{1-w} =0\), we proceed as for the previous claim, but use another strategy for this end. Consider the strategy which, starting from \(w \in (\delta -(1-\delta )\frac{1-p}{p},1)\), sets \(q=1\) as long as the continuation payoff lies in this interval, and reverts to the optimal strategy as soon as this interval is exited. Note that, by choice of this interval, a single success yields \(w^S \ge 1\) independently of w within this interval. We derive a lower bound \(\tilde{s}\) for the value by considering that the payoff is 0 if this interval is ever exited from below. Starting from a given w within this interval, it takes n successive failures for this happen, where n is given by

We can solve the recursion that characterizes the lower bound and check that the loss \((s(1)-\tilde{s}(w))/(1-w)\) is given by

as was to be shown.

We now argue that s is differentiable on (0, 1) (and hence also at \(w=1\), given that we have just shown that the left derivative at 1 is 0, matching the right derivative over the range of values of \(w \ge 1\) over which the surplus is constant). Fix \(w \in (0,1)\), and correspondingly \(w^S,w^F\), and the equilibrium q at this value of w. Consider the following strategy available to the principal. At the beginning of the period, the principal uses the public randomization device to flip a coin. With probability \(\varepsilon ^2\), where \(\varepsilon \) is a small number, not necessarily positive, no funds are advanced (\(q=0\)), no payment is made, and we move to the next period with continuation promise \(w^S\). With probability

no funds are advanced, no payment is made, and we move to the next period with the agent’s payoff unchanged at w. We note that, because \(q>0\) and so \(\delta w^S-w>0\), we have \(w-\delta w^F>0\). Finally, with the remaining probability, play proceeds as in the equilibrium given w. First, note that because the coin flip is publicly observable, the incentive compatibility condition does not change: it only applies in the last of the three events, and conditional on this event, everything remains unchanged. Given this policy, the utility of the agent, denoted by \(w_{\varepsilon }\), satisfies

and it is easily verified that \(w_\varepsilon \lessgtr w\) iff \(\varepsilon \lessgtr 0\). Consider applying this strategy (parametrized by \(\varepsilon \)) at \(w_\varepsilon \) in some neighborhood \((w-\bar{\varepsilon },w+\bar{\varepsilon })\). We obtain a surplus \(\tilde{s}\) that is (i) below s (since this cannot outperform the best strategy), (ii) continuously differentiable in \(\varepsilon \), (iii) coincides with s at w. Because s is weakly concave, it follows as in Benveniste and Scheinkman (1979) that s is continuously differentiable.

1. [5.2]

   We show that for any \(w \in (0,1)\), we have \(s'(w^F)\ne s'(w^S)\). There are two cases. First, suppose that \(q <1\). Then the first-order condition must hold (recall that \(q>0\)), namely

   $$\begin{aligned} 1+(1-p)(s'(w^S)-s'(w^F))=0, \end{aligned}$$

from which it immediately follows that \(s'(w^F)\ne s'(w^S)\). Suppose instead that \(q=1\), and \(s'(w^F)= s'(w^S)=:\alpha \). We note that, because \(q=1\) and \(w<1\), we must have \(w^F<w<w^S\). We also note that, given that \(q=1\), for any \(\hat{w} \in [w^F,w]\), it holds that \(\hat{w}^S \le w^S\), and so since \(\hat{w}^S \ge \hat{w}\), we have also \(s'(\hat{w})=s'(\hat{w}^S)=\alpha \). From the envelope theorem, at \(\hat{w}\), we have \(s'(\hat{w})=ps'(\hat{w}^S)+(1-p)s'(\hat{w}^F)\), and so it follows that also \(s'(\hat{w}^F)=\alpha \). Hence, from the first-order condition, \(\hat{q}\), the scale chosen at \(\hat{w}\) must also be equal to 1. Hence \(\hat{w}^F<w^F\). In fact, because we can take \(\hat{w}=w^F\), it follows that \(s'((w^F)^F)=\alpha \), with \(q(w)=1\), \(q(w^F)=1\), and this argument can be repeated. Given that \(q(w)=q(w^F)=q((w^F)^F)=\cdots =1\), the sequence \((w^F,(w^F)^F,\ldots )\) eventually hits 0, and so the slope must be \(\alpha \) at arbitrarily small values of w, a contradiction to our finding that \(s'(0)=\infty \).

1. [5.3]

   This in turn allows us to argue that \(q>w\). Suppose that \(q \le w\). Then \(w^F \ge w\). But then, by familiar reasoning, \(s'(w^F)=s'(w^S)=s'(w)\), contradicting our finding that these derivatives must be different.

2. [5.4]

   The function s is strictly concave on (0, 1). Suppose not, and consider an interval \([w_1,w_2]\) of maximal length \(w_2-w_1=:\lambda >0\) over which it is affine. Because of \(s'(w^F)\ne s'(w^S)\), we must have \( w_1^F \le w_1\), \( w_2^F \le w_1\), and similarly \( w_2^S \ge w_2\), \( w_1^S \ge w_2\). It is readily verified from the formulas for \(w_k^S,w_k^F\) that

   $$\begin{aligned} \max \{|w_1^F-w_2^F| ,|w_1^S-w_2^S|\}>\lambda . \end{aligned}$$

Hence, by definition, s is not affine over one of the two intervals \([\min \{w_1^F,w_2^F\},\max \{w_1^F,w_2^F\}]\), \([\min \{w_1^S,w_2^S\},\max \{w_1^S,w_2^S\}]\). Consider now the strategy that at \(w=(w_1+w_2)/2\), picks \(q=(q_1+q_2)/2\) (where \(q_k\) is the scale chosen at \(w_k\)), and continuation payoffs \(w^F=(w_1^F+w_2^F)/2\), \(w^S=(w_1^S+w_2^S)/2\), and reverts back to optimal play thereafter. Because the payoff in the first period is an average of the payoffs starting at \(w_1\) and \(w_2\), and because the continuation payoff is at least as much as the corresponding average (with one strict inequality, corresponding to the outcome that takes the continuation payoff into the interval over which s is not linear), this gives a payoff that is strictly higher than the average over the payoffs starting at \(w_1\) and \(w_2\).

1. [5.5]

   Finally, because s is strictly concave and differentiable on (0, 1), with \(s'(w)\) being an average of \(s'(w^F)\) and \(s'(w^S)\), and \(w^F \ne w^S\), it follows that \(w^F<w<w^S\) for every \(w \in (0,1)\). \(\square \)

1.3 Proof of Proposition 3

We maintain the normalizations introduced in Sect. 5.2.

Because \(s'\) is strictly decreasing, it is differentiable almost everywhere. In addition, \(s'\) is continuous and \(\{w:s'(w)=\infty \}\) is a zero-measure set (namely \(\{0\}\)), and hence \(s'\) is absolutely continuous.^{Footnote 15} If \(q \in (0,1)\) is optimal at w, it must hold that

(with the obvious notation \(w^S(q),w^F(q)\)), and we note that there can be at most one solution to such an equation, since \(w^F\) is decreasing and \(w^S\) increasing in q. By the same reasoning, if a solution \(q \in (0,1)\) exists, it cannot be that \(q=1\) is also optimal, as this would require \(\int _{w^F(q)}^{w^S(q)}s''(w){\mathrm{{d}}}w \ge -\frac{1-\delta }{1-p}= \int _{w^F(1)}^{w^S(1)}s''(w){\mathrm{{d}}}w\). Hence, the (upper hemicontinuous) correspondence \(q:[0,1] \rightrightarrows [0,1]\) mapping w into the set of maximizers of s is a continuous function. Then we can decompose the interval [0, 1] into a partition of intervals \(\{I_k\}\), \(\{J_k\}\), with \(q<1\) on each \(I_k\), and \(q=1\) on each \(J_k\), and it follows from Santos (1991), for instance, that q is differentiable on the interior of each \(I_k\).

We already know that there exists \(a > 1-\delta \) such that \(q \in (0,1)\) for all \(w \in (0,a)\) (this is because we must have \(w^F>0\)). Similarly, there exists \(b>0\) such that \(q=1\) for all \(w \in [1-b,1]\). Indeed, note that, with \(q=1\), the first-order condition for any w such that \(w^S \ge 1\) becomes \(1-\delta -(1-p)s'(w^F) \ge 0\), which is true for w close enough to 1, because \(\lim _{w \rightarrow 1}w^F=1\) and Lemma 5.1 (establishing the zero derivative at \(s=1\)).

It remains to show that the partition \({I_k}\), \(J_k\) has a simple structure: \(q<1\) if and only if \(w \le \hat{w}\), for some \(\hat{w} \in (0,1)\). This will be established by value iteration, making the following induction hypothesis. Let there be given a function \(s_n\) on (0, 1), with the following properties: it is continuously differentiable, with \(s'_n(w)>0\) for all \(w \in [0,1]\), with \(s'_n\) being almost everywhere twice differentiable, with \(s''_n(w)<0\), and \(s'''_n(w)>-s''_n(w)\), \(s_n(0)=0\), \(s_n(1)=1\), \(\lim _{w \rightarrow 0}s_n'(w)=+\infty \), \(\lim _{w \rightarrow 1}s_n'(w)=0\) (and we extend it to \(s_n(w)=1\) for \(w \ge 1\)). We establish that these properties also hold for \(s_{n+1}\), the function resulting from maximizing over q the payoff from \(\mathcal {P}\), taking \(s_n\) as the continuation payoff. Let \(q_{n+1}\) be the maximizing correspondence (a function, by the same arguments as before).

We divide the analysis according to whether \(q=q_{n+1}(w)=1\) or \(q<1\)—that is, we consider in turn open intervals of values of w for which \(q=1\) and then \(q<1\). That there is such partition follows from arguments entirely analogous to the previous ones, and omitted (as is the proof that \(s'_{n+1}\) is continuously differentiable, and that \(\lim _{w \rightarrow 0}s_{n+1}'(w)=+\infty \), \(\lim _{w \rightarrow 1}s_{n+1}'(w)=0\)).

First, suppose that \(q=1\) (which enters the definition of \(w^S\) and \(w^F\)). Then we have from the envelope theorem,

and, obviously, since \(q=1\) identically on such an (open) interval, we have more generally, for \(k=1,2,3\),

where \(f^{(k)}\) refers to the kth derivative of f (and differentiability follows from the differentiability of \(s^{(k)}_n\)). So, in particular,

Consider now an interval over which \(q<1\) (obviously, q need not be constant). The necessary first-order condition with respect to q gives

and differentiating with respect to w (differentiability of \(q'_{n+1}\) follows from the implicit function theorem), we get

The envelope theorem gives

which implies \(s'_{n+1}>0\). This equation holding identically, we differentiate again with respect to w (not forgetting to insert \(q'_{n+1}(w)\)) to get

yielding \(s''_{n+1} <0\). We repeat this exercise once more, to get

It remains to show that

which will follow if we show

By the induction hypothesis,

hence it suffices to show that

or

which is equivalent to

or

which holds trivially. Hence,

where we use concavity and the continuous differentiability of \(s'\).^{Footnote 16} It follows that \(q_{n+1}\) is continuously increasing on some interval \([0,\hat{w}]\), and equal to 1 on \([\hat{w},1]\).

By value iteration then, we have that \(s^n \rightarrow s\), and \(q^n\) (which converges along some subsequence, by Helly’s selection theorem, given that \(q^n\) is monotone) converges to a policy q that is optimal with an infinite horizon (see, e.g., Schäl 1975). In particular, it follows that q is non-decreasing on some \([0,\hat{w}]\), and equal to 1 on \([\hat{w},1]\). By our prior results, we know that it must be continuous.

1.4 Calculations, Remark 2

Consider a continuation history beginning with value W, and with the next two periods referred to as periods 1 and 2. Recalling that

we can write

where \(\overline{q}_2\) is the second period value of q following a success in period 1, and \(\underline{q}_2\) the value following a failure, and where \(W^{SF}\) (for example) is the continuation value following first a success and then a failure. Now suppose we contemplate adjustments in the values of \(q_1\) and \(q_2\) satisfying

Then this adjustment preserves the principal’s expected payoff in periods 1 and 2, and also preserves the values of \(W^{SS}\) and \(W^{FF}\). The effect of this adjustment on S(W) then arises entirely out of its effect on \(W^{SF}\) and \(W^{FS}\). Here, we have

Hence, increasing \(q_1\) gives us an increase in \(W^{SF}\) and a like-sized decrease in \(W^{FS}\), with a decrease in \(q_1\) having the reverse effect. Because S(W) is concave, and since a success followed by failure and a failure followed by success are equiprobable, it is always advantageous to adjust \(q_1\) (with the corresponding adjustments in \(\underline{q}_2\) and \(\overline{q}_2\)) so as to push \(W^{SF}\) and \(W^{FS}\) closer together. Now notice the following string of equivalent statements:

Hence, in the absence of any constraints, we must have

with

corresponding to pressure to increase \(q_1\) and

corresponding to pressure to decrease \(q_1\).

Now suppose \(q_1=1\). Then surely we must have

since the left side is necessarily negative and the right side positive. Then optimality calls for a value of \(q_1<1\), a contradiction, in the absence of some constraint.

Hence, if \(q_1\), \(\underline{q}_2\) and \(\overline{q}_2\) are all interior, then

which gives \(W^{SF}=W^{FS}\), which suffices for the result.

1.5 Proof of Proposition 4

We maintain the normalizations introduced in Sect. 5.2.

Monotonicity in \(\delta \) for fixed \(W>0\). First, we claim that \(\delta '>\delta \) implies that \(s_{\delta '} \ge s_{\delta }\). Let \(q=q_{\delta }(w)\) be the maximizer given \(\delta \). Note that, given \(\delta '\), we can set \(q'=\frac{1-\delta }{1-\delta '}q\) (a feasible policy, as we have \(w^F_{\delta '}(q')>0\) if and only if \(w^F_{\delta }(q)>0\)). Because the Bellman operator is monotone, consider applying \(q'\) with a continuation payoff \(s=s_\delta \): we claim that the resulting payoff exceeds s(w), so that the fixed point \(s_{\delta '}\) lies above \(s_\delta \), strictly for \(w \in (0,1)\). Indeed, because \(s(0)=0\), and s is strictly concave, we have that, for \(\alpha >1\), \(w \in (0,1)\),

hence, for \(w'=w+q(1-\delta )(1-p)/p=w+q'(1-\delta ')(1-p)/p\), as well as \(w'=w-q(1-\delta )=w-q'(1-\delta ')\),

and the result follows from

Limits of S and q as \(\delta \rightarrow 1\) for fixed \(W>0\). Next, we show that \(\lim _{\delta \rightarrow 1}s_\delta (w)=1\) for all \(w >0\). Consider the strategy that sets \(q(w)=1\) for all \(w>1-\delta \), and \(q(w)=w/(1-\delta )\) otherwise. Note that, because the payoff from this strategy is non-decreasing in w, we have that, for \(w >1-\delta \),

and we note that the process that takes value \(w+\frac{(1-\delta )(1-p)/p}{\delta }\) with probability p and takes value \(w-\frac{(1-\delta )}{\delta }\) with probability \(1-p\) is a martingale (for \(w>1-\delta \)). Hence, by Azuma–Hoeffding’s inequality (see, for instance, Alon and Spencer 2000), starting from w, the probability that the process hits \(\{w:w \le 1-\delta \}\) in exactly N steps is majorized by

so that the loss from this policy (relative to 1) is majorized by

which tends to 0 as \(\delta \rightarrow 1\).

Finally, we note that this also implies that the optimal policy converges pointwise to 1. Given concavity and convergence of w to 1, it follows from the first-order condition

that, fixing \(w'\), and picking \(\delta \) high enough so that necessarily \(s'(w^S)-s'(w^F)>-1/(1-p)\) for all \(w>w'\), \(q(w)=1\). \(\square \)

1.6 Calculations, Section 4.2

We first argue that the lowest equilibrium payoffs \((\underline{W},\underline{V})\) can be jointly obtained and lie on the function h. Suppose they are not jointly obtained. Then we have an equilibrium with payoffs \((\underline{W},\tilde{V})\) that achieves the lowest equilibrium payoff for the agent and for which \(\tilde{V}\) is the minimum payoff for the principal consistent with the agent receiving \(\underline{W}\), but which features \(\tilde{V}>\underline{V}\). Hence, there exist feasible continuation payoffs \((W^F,V^F)\) and \((W^S,V^S)\) such that the values \((\underline{W},\tilde{V})\) satisfy

Suppose \(\mathbf 1_S=1\). Then we can rewrite this as

We now make use of the fact that (by assumption) the final constraint does not bind. First, we must have \(z^F=0\), since reducing \(z^F\) reduces \(\underline{W}\) while preserving all of the constraints, contradicting that \(\underline{W}\) is the agent’s lowest equilibrium payoff. Next, \(W^F=\underline{W}\). Otherwise, we could replace \((W^F,V^F)\) with \((\lambda \underline{W}+(1-\lambda )W^F, \lambda \tilde{V}+(1-\lambda )V^F)\), which preserves the constraints and the feasibility of continuation payoffs, increasing \(\lambda \) and hence decreasing \(W^F\) until either the final constraint binds or \(W^F=\underline{W}\). Next, the agent’s incentive constraint must bind, since otherwise we can reduce \(z^S\), possibly until it hits zero, and then replace \((W^S,V^S)\) by \((\lambda \underline{W}+(1-\lambda )W^S, \lambda \tilde{V}+(1-\lambda )V^S)\), again in the process preserving feasibility and reducing \(W^S\) and continuing until either the final constraint binds or the agent’s incentive constraint binds. Hence, we have

Now suppose \(W^S>\underline{W}\). Then we can decrease \(W^S\), preserving feasibility by replacing \((W^S,V^S)\) with \((\lambda \underline{W}+(1-\lambda )W^S, \lambda \tilde{V}+(1-\lambda )V^S)\), and also increasing \(z^S\) so as to preserve both the incentive constraint and the agent’s payoff, continuing until either the final constraint binds or \(W^S=\underline{W}\). Our problem is then

Now a reduction in q and a proportional reduction in \(z^S\) gives us an equilibrium with a smaller value for the agent, a contradiction to \(\underline{W}\) being the lowest equilibrium payoff for the agent, unless \(q=0\). But then we have \(\underline{W}=0\), also a contradiction.

Suppose instead \(\mathbf 1_S=0\). Then we can rewrite this as

We can take \(z^S=q=0\) and can let the continuation payoffs following a success equal those following a failure, thereby setting \(W^S\) equal to \(W^F\) and hence eliminating the agent’s incentive constraint, making this

But then either the final constraint binds or, by replacing \((W^F,V^F)\) with \((\lambda \underline{W}+(1-\lambda )W^F, \lambda \tilde{V}+(1-\lambda )V^F)\), we can take \(W^F=\underline{W}\), giving \(\underline{W}=0\), a contradiction.

We thus know that there exists a payoff \((\underline{W},\underline{V})\), drawn from the function \(V=h(W)\), that is the lowest equilibrium payoff for each player. From our previous reasoning, we have that we can take \(z^F=0\), that the function \(\underline{V}=h(\underline{W})\) binds, and that the agent’s incentive constraint binds. The outcome of an equilibrium with \(\mathbf 1_S=0\) can be duplicated by an equilibrium with \(\mathbf 1_S=1\) and \(q=0\), and so \((\underline{W},\underline{V})\) solves

The current-period expected payoff is given by \((pz^S, p(qk\pi -z^S))=:\Theta \), and we have the convex combination

We now note that^{Footnote 17}

• If \(pqk\pi >h(0)\), then we can adjust \(z^S\), with a corresponding adjustment in \((W^S,V^W)\) to preserve (12), noting that this also necessarily preserves the incentive constraint, until we have \(z^S=h(pqk\pi )\), i.e., until \(\Theta \) lies on the function h.

• \(pqk\pi \le h(0)\), then we can adjust \(z^S\), with a corresponding adjustment in \((W^S,V^W)\) to preserve (12), noting that this also necessarily preserves the incentive constraint, until we have \(z^S=0\).

Hence, we can assume that \(\Theta \) is drawn either from the function h or from the line segment connecting the original to the vertical intercept of h. We can then note that if \(\Theta \) is drawn from the graph of the function h, it must be that \(z^S=0\). If not, we can preserve the continuation payoffs \((W^S,W^F)\) and replace \(\Theta \) with the vertical intercept of h, with the resulting solution to (12) allowing us to reduce the putative lowest equilibrium payoffs, a contradiction. Hence, we can assume that \(\Theta \) is drawn from the line segment connecting the origin with the vertical intercept of h.

For large values of \(\delta \), we can now derive an upper bound on the lowest equilibrium payoffs. Suppose we set \(\Theta \) equal to the vertical intercept of h. Then from (12), the agent’s incentive constraint, and the equation for h, we have, respectively

The third equation fixes the value of q. Given this, the first two equations jointly determine \(\overline{W}\) and \(W^S\). The resulting solution for the value of \(\overline{W}\), with the accompanying value is an upper bound on the value of the lowest equilibrium payoffs, since we have constructed this value subject to the restriction that \(\Theta \) be given by the vertical intercept of the function h.

As \(\delta \) declines to zero, value of \(W^S\) associated with this upper bound explodes. Hence, there exists value \(\underline{\delta }\) at which the attendant value of \(W^S\) hits \(pk\pi \). Above this value of \(\delta \) our current construction will yield an equilibrium, and hence the upper bound no longer applies.

We can show that if \(\delta \) is sufficiently small but nonzero, then the lowest equilibrium payoffs are \((W^*,V^*)\). Substituting the agent’s incentive constraint into the expression for \(\underline{W}\), we have \(\underline{W} \ge (1-\delta ) qpk\pi +\delta W^F\), which implies that \(qpk\pi \le \underline{W}\), and hence

Feasibility requires

Both sides of this weak inequality are linear and increasing in \(\underline{W}\). The relationship holds with equality if \(\underline{W}= pk\pi \). We are interested in its smallest solution. This smallest solution will be \(pk\pi \) if the slope of the left side falls short of that of the right, i.e., if

which gives

This will clearly be satisfied, and hence \((\underline{W},\underline{V}) = (W^*,V^*)\), for sufficiently small but positive \(\delta \).

We thus have that for all

the lowest equilibrium payoffs are \((W^*,V^*)\). This is a sufficient but not necessary condition for lowest equilibrium payoffs to be \((W^*,V^*)\), since our feasibility calculation assumed that the continuation payoffs could achieve the efficient surplus, which is an overestimate.

Finally, when it is nontrivial, what determines the location of the \((\underline{W},\underline{V})\) on the function h? It will be the solution to the program