Transversality Conditions in Infinite Horizon Models

International Finance Discussion Papers Number 172

January 1981

TRANSVERSALITY CONDITIONS IN INFINITE HORIZON MODELS

by

Jo Anna Gray and Steven W. Salant

NOTE: International Finance Discussion Papers are preliminary materials circulated to stimulate discussion and critical comment. References in publications to International Finance Discussion Papers (other than an acknowledgment by a writer that he has had access to unpublished material) should be cleared with the author or authors.

Transversality Conditions in Infinite Horizon Models by Jo Anna Gray and Stephen W. Salant*

I. Introduction

The transversality condition associated with finite horizon maximization problems is well-known. It is sometimes believed that the analogous condition for infinite horizon problems can be obtained simply by replacing evaluations at the terminal time with the limit of these quantities as time tends to infinity. That this belief is incorrect has been shown by Hubert Halkin (1974), who produced an example where an optimal solution to an infinite horizon problem violated the limiting extension of the standard condition.

In this paper we develop a new transversality condition which must be satisfied by all optimal programs in the particular class of infinite horizon problems to which the Halkin example belongs. In addition, the approach used in deriving this new condition is used to show that the limiting extension of the standard transversality condition is necessary for a second class of problems to which the Halkin example does not belong.

To facilitate the reading of this paper -- and the writing of it -we conduct the analysis in discrete time. Our aim is to make the new condition accessible to the many applied economists who, like ourselves,

*Board of Governors of the Federal Reserve System and the Federal Trade Commission, respectively.

are users of control theory but whose knowledge of the pertinent techniques may extend no further than the treatments of Dorfman [1969] or Arrow and Kurz [1970]. We are aware that several economists -- beginning with Weitzman [1973] -- have shown that the limiting extension of the standard transversality condition is necessary in some classes of problems. But these analyses neither address the problem considered here nor mention the example formulated by Halkin.2/ Furthermore, many of the results of

this literature are so technical as to be inaccessible to the wide audience which needs to use them, a situation we find regrettable.

The paper is organized as follows: In section II we review and interpret the necessary conditions for an optimum in a finite horizon, discrete-time problem. Among the conditions is the wel1-known transversality condition for finite horizon problems. Halkin's problem is introduced as an example of an infinite horizon problem whose optimal solution violates the limiting extension of this transversality condition. In section III we develop a new condition that is necessary for the class of problems to which the Halkin example belongs. Here we follow a time honored tradition of examining the marginal gain due to feasible perturbations of a proposed program. We show that even if an infinite horizon program satisfies the Kuhn-Tucker conditions, it may nonetheless be

2/

improved upon by feasible arbitrage schemes of a particular type.—

1/In contrast, Halkin's example and the unsolved problem it raises is discussed in Halkin (1974), Arrow and Kurz (1970, p.46), and Takayama (1974, p. 625). The problem is also alluded to in Dixit (1976, p. 117).

2/In most of the more technical literature, feasible variations in a proposed program are referred to as perturbations. In keeping with our perspective as economists, however, we have chosen to refer to these variations as arbitrage schemes, or arbitrages, throughout the remainder of the paper.

Our new condition insures that the marginal gain from such variations is zero. We then return to the Halkin example and verify that all the optimal solutions considered by Halkin do, indeed, satisfy our new condition. In section IV it is shown that the approach of examining the consequences of a particular type of arbitrage in order to derive a transversality condition for infinite horizon problems can be usefully extended to problems in which only the state variable is restricted. For this class of problems the appropriate transversality condition does turn

out to be the limiting extension of the finite horizon condition.

II. The Kuhn-Tucker Conditions and the Halkin Example

In this section we review the development of the necessary conditions for an optimum in a finite-horizon, discrete-time problem in which the value of the control variable is restricted. The transversality condition emerges simply as one of the Kuhn-Tucker conditions. The other conditions are interpreted as ruling out arbitrages reversed after a finite interval. Attention then turns to the infinite horizon problem. If a program is optimal in infinite time, no finite segment of it can be dominated by a new segment over the same time interval which begins and ends with the same respective values for the state variable. This implies that the condition ruling out arbitrages reversed after a finite interval must be satisfied -for all such intervals -- by any program optimal over an infinite horizon. However, such a program may violate the limiting extension of the standard transversality condition, as we learn at the end of the section from

consideration of Halkin's example.

-4-

Consider the following problem:2/ T t

Maximize Z=xr g V[c(t),m(t)] (1) t=]

subject to: m(1) =m (2) m(t+1) = f[c(t), m(t)] c(t) > 2(t) u(t) > c(t) for t=1,2,...T.

Here 6 iS a positive discount factor ~/ c(t) is the control variable, and m(t) is the state variable. Note that although the control variable is restricted, the state is not.

The problem under consideration is a standard non-linear programming problem.—/ There are 2T+] variables to choose (c(t), m(t), for t=1,T and m(T+1)). Assuming the feasible set is not empty and that f[.,.] is continuous, it can be established that (2) describes a non-empty, compact set. Then assuming that V[.,.] is continuous, a solution to the programming problem must exist, and we can turn to its characterization.

All functions are assumed differentiable.

3/A bar over a variable indicates that the variable is exogenous.

4/Since what we will be deriving are conditions which any program must satisfy if it is optimal, we have imposed no restrictions on the size of the discount factor. The reader is reminded, however, that an optimal solution to an infinite horizon problem may not always exist.

5/For an extensive treatment of such problems, see Zangwill (1969).

Define the Lagrangean

L = t

ium oa

. a° { VEc(t) sm(t)] + a(t) CfLe(t) ,m(t)] - m(t+1)] +

a(t)Lu(t) - c(t)] + p(t)[c(t) - 2(t)]} + 2(0)(m - m(0))

Then, provided the constraint functions (2) meet certain regularity

conditions ,©/ the Kuhn-Tucker theorem implies the existence of 3T+1 numbers (a(t), p(t), and A(t) for t=1,T and A(0)) which satisfy the

following conditions at an optimal solution:/ - at _ 7 Lect) = 8 LV et) + Mt) fee) - a(t) + p(t)] = 0 for t=1,T (3)

—~ ot -14_ 7 Lact) = 8& UVacey * A(t) Facey - A(t-1e = 0 for t=1,T (4) L = -a(T)e! = 0 (5) m(T+1) Lo(t) > 0, a(t) > 0, with complimentary slackness, for t=1,T (6)

o(t) > 0, p(t) > 0, with complimentary slackness, for t=1,T (7)

6/See Takayama, p. 474-480 for a careful treatment of an optimal growth problem in discrete time. Conditions under which the Weierstrass Theorem and the rank constraint qualification can be applied are discussed.

7/In the remainder of the paper, the notation x (t) Stands for the partial derivative of x with respect to the scat ar variable y(t).

8/ L = 0 for t=0,T. (8) A(t)

The resulting conditions are analogous to the first-order conditions of continous-time control problems. Equation (3) indicates how the control variable (c(t)) should be set given the co-state variable (A(t)), the state variable (m(t)), and the multipliers associated with the control (given by (6) and (7)); the state variable then evolves according to the transition equation (recovered in (8)), while the co-state variable follows (4). The final value of the co-state variable is determined by (5). Since the value of the state variable at T+] does not, by assumption, affect the value of the program, e!x(T) = 0. Equation (5) is the transversality condition for this "free-endpoint" problem.2/

The Kuhn-Tucker conditions can usefully be regarded as restrictions which must hold if various feasible arbitrages are of no value. Suppose in a feasible program that at dates t* and t' (t' greater than t*) consumption is interior. Then it is feasible to vary c(t*) in either direction, leave intermediate c(t) unaltered, and then vary c(t') so that m(t'+1) and the

subsequent program are unaltered. The resulting change in the value of the

8/In contrast to a(t) or p(t), there is no restriction on the sign of A(t) in (8) since this multiplier is associated with an equality constraint.

9/If, instead, the terminal state variable is constrained to be m(T+1) > b, the transversality condition replacing (5) can also be derived using the Kuhn-Tucker theorem; it is 4(T) > 0, m(T+1)-b > 0, with complimentary slackness.

program is denoted dZ/dc(t*), and can be readily computed by making use of the well-known result that the derivative of the sum of a finite number

of functions equals the sum of the derivatives of each function: 10/

dZ/dce(t*) = A(t*,t') + B(t*,t') (9) where A(t#st') = 8 Ve (pay +O Vn(eeeny fetes) * t' ¢t weteeg & Y(t) fe(te) fm(te41) fm(te+2) ***m(t-1) and

* ' t B(t*,t') = -6° eae tx) fm(tee1) 'm(t*+2) °° T(t)”

A(t*,t') represents the gains and losses of initially increasing the control variable in period t* and then returning the control to its original path until period t'. The first term in the expression defining A(t*,t') gives the gain in the objective function resulting directly from the increase in the control variable in the initial period t*. The second term reflects the losses associated with the consequent change in the state variable in the following period. Since the control variable is returned to its original path in period t*+1, gains and losses in subsequent periods result only from the cumulative effects on the state of the change in its value in period t*+1. These are captured by the sum which is the third

term in the expression defining A(t*,t'). In period t', however, the

10/See appendix A for details and footnote 13 for a demonstration that the proposition is not, in general, true if the sum is an infinite series.

control variable must again deviate from its original path, this time by the amount necessary to return the state variable to its original path. The additional cost incurred in terminating the arbitrage is simply the discounted value of this last change in the control variable. It is given by B(t*,t'). Distinguishing B(t*,t'), which represents the cost of terminating or "reversing" the arbitrage, from A(t*,t'), which represents the cost of "maintaining" the arbitrage, will prove extremely useful in the analysis of the next section.

It is easily shown that if the proposed program satisfies the Kuhn-Tucker conditions, the arbitrage proposed above will have no value;

that is A(t*,t') + B(t*,t') = 0. (10)

Hence, any feasible arbitrage initiated at t* and reversed at t' is unprofitable if the program is optimal. Moreover, the Kuhn-Tucker

conditions imply that the following generalization of (10) holds even if

c(t') is at a boundary :22/

A(t*,t') + B'(t*,t') = 0 (10') where

tak ety cot ayn B'(t*,t') = Bp A(t MEF (ee) fnceee) fm(tee2) oe f(t yd

and A(t*,t') is defined by (9).

11/See appendix A for details.

Similar conclusions must hold for infinite horizon problems.

Consider a feasible program [m(1), m(2),....; c(1), c(2),-.-.]. For any t* such that c(t*) is interior to the control set, A(t*,t')+B'(t*,t')=0 if the program is optimal. To prove this, we could consider maximizing over

the subinterval from t* to t' with initial condition m(t*) and terminal condition m(t'+l) given by the infinite horizon problem. The necessary conditions for this “clamped end-point" finite horizon problem may be derived as before from the Kuhn-Tucker theorem and will once again include (3), (4), (6), (7), and (8). These conditions must hold if the infinite horizon program is optimal; otherwise it could be dominated by an alternate program that differs from the proposed program only between t* and t'.

It might also be thought that a limiting version of equation (5) must hold if an infinite horizon program is optimal. There are, however, well-known cases in which the limiting extension of this transversality condition is violated by optimal solutions in infinite horizon problems.

An example of such a case has been devised by Hubert Halkin. The example, as it appears in Takayama and Arrow and Kurz, is cast in continuous time. Since the analysis of this paper is in discrete time, we will work with the discrete time analog of Halkin's example. In our notation, the problem posed by Halkin is to maximize an objective function

of the form

Z= ¢ [l-m(t)Je(t), (11) t=l

subject to the following constraints:

- 10 -

m(1) = 0

m(t+1) = m(t) + [l-m(t)]e(t), (12) c(t) > -1

1 > c(t) for t=1,2,....

By substituting equation (12), the transition equation, into equation

(11), the latter may be rewritten as

The value of the objective function in this problem is simply the limiting value of the state variable. The solution to the problem may now be determined by inspection: Equation (12), the transition equation, tells us that the maximum achievable value of the state variable under the restrictions of this problem is one. Accordingly, the maximum value of the objective function is unity. Any path of the control variable that generates a limiting value of the state variable of one is, therefore, an optimal path. There are an infinite number of such solutions. Halkin restricts his attention to those in which the control assumes a constant value, denoted c, between zero and one. As long as c exceeds zero but is less than one, the state variable will grow monotonically over tine, approaching an asymptotic value of unity as time tends to infinity. There

are, then, a continuum of solutions to this problem of the form

-ll-

c(t) =c for t=1,2,.... where (14) O<c<l.

All these solutions violate the usual transversality condition.

The value of A(t), the shadow price of a unit of the state variable, is -l for all t along any of these solution paths 22/ Hence, the limit of

A(t) as t tends to infinity is -1. The limiting extension of the standard condition requires that this limit be zero.

The Halkin example demonstrates the evident absence of a transversality condition among the necessary conditions for an optimum in infinite horizon problems in which the value of the control variable is restricted. In section III we will develop a new transversality condition

that is necessary for an optimum in this class of problems.

III. A New Condition In this section we employ additional arbitrage arguments to derive a

new transversality condition that must hold for optimal programs in the

12/ In the Halkin example g is equal to unity. Accordingly, the limiting extension of equation (5) becomes

lim -A(t) = 0.

t +@ To verify that A(t) = -1 for all t, solve equation (3) for A(t):

For the Halkin example, V =f y=i-m(t) for all t, giving A(t) = -1

for all t. c(t) c(t

-12-

class of infinite horizon problems described in section II. The new condition insures the absence of certain "unreversed" arbitrage opportunities, just as the Kuhn-Tucker conditions (excluding (5)) insure the absence of certain "reversed" arbitrage opportunities. It is shown that in finite horizon problems our new condition is redundant since it must be satisfied by any program which meets the usual transversality condition. But in infinite horizon problems the usual condition need not hold and our new condition takes its place. It is found that the optimal solutions to the Halkin example do, indeed, satisfy the new condition. The infinite horizon version of our problem is given by (1'), which

replaces (1), and (2'), which replaces (2):

Maximize Z(o) = i Bp V(c(t), m(t)) (1')

subject to: m(1l) =m (2')

u(t) > c(t), for t=1,2,....

From section II we know that an optimal solution to this problem in which.

c(t*) is interior must satisfy equation (10'). It follows that any feasible arbitrage initiated at t* and reversed at t' is unprofitable. We turn now to an alternative "“unreversed" arbitrage and the added condition it

implies.

-13-

Consider again a feasible program [m(1), m(2),...; c(1), c(2),...]. For any t* such that c(t*) is interior to the control set, it is possible to vary c(t*) in either direction and leave all subsequent c(t) unchanged. In this new arbitrage, consumption is not altered after t* in order to restore the state variable to its original path. This is what we mean by an “unreversed" arbitrage. Our new condition rules out marginal gains for this unreversed arbitrage.

The infinite horizon specification of our problem creates some special difficulties for the derivation of this new condition. As c(t*) is varied, holding constant consumption at other dates, the infinite sequence of the state variables at dates subsequent to t* is altered. This sequence can, however, be written as a function of c(t*) using the transition equation and the (given) values of the controls associated with the original program. Consequently, the discounted value of the program given in (1') can be considered simply as a function of the single instrument c(t*). But even if we were to assume that this function is well-defined (converges pointwise) for 2(t*) < c(t*) < u(t*), this would not be sufficient to insure that the value of the program is a differentiable, or even a continuous, function of c(t*). Moreover, even if it is differentiable (and hence continuous) in some interval, its derivative might differ from the

infinite sum of derivatives. For, alas, the derivative of a sum is not,

- 14 -

in general, equal to the sum of the derivatives when the sums are infinite.22/

To insure that the derivative of the limit function given by (1') can be computed by summing up the derivatives of each of its terms, we make two additional assumptions: that the discounted value of the program exists for the specified c(t*) and that the series of changes in subsequent

discounted utilities converges uniformly on some open interval when

considered as a function of c(t*). These assumptions are sufficient to

13/ There exist examples of infinite series for which it is true that the derivative of the limit of the sum is not equal to the limit of the sum of the derivatives of the individual terms. One such example is found in Goldberg (1976):

Let f (x) = & h(x,t), where 0 < x <1 ” t=1 n and F(x) = £ h(x,t) = x" ne

t=1

Here f_(x) is the limiting value of the series, and F(x) is the nth partial sum. Note that

f (x) = _lim f_(x) = 0 for all x. r} n+» © n It follows that the derivative of the infinite sum is zero for al] permissible values of x.

Suppose we tried to calculate this derivative of the limit of the series by summing up the derivatives of each term -- as we do in the text. The derivative of any partial sum is given by

t _ n-] F(x) =x. The limit of this derivative as n tends to infinity is given by . ' . | for x =] nl im, FAC) ~ 0 for x <1

Thus, for x=1, the derivative of the limit of the infinite series is not equal to the limit of the sum of its derivatives. Additional assumptions are required to eliminate such cases.

-15-

guarantee that the derivative of Z(@) with respect to c(t*) is equal to the sum of the derivatives of its individual terms £4/ The optimal

solutions to the Halkin example discussed in the preceding section, as well as the solutions to other problems of interest, satisfy these assumptions. With the preceding discussion as background, we can now formally derive our new condition which any optimal solution must meet.

Define the infinite series of changes in discounted utilities which result from an unreversed arbitrage as A(t*). Then A(t*) can be easily computed: t*+]

* = t* A(t*) = Bo Voces) +B Victeer) fects) * a5)

ro opty

t=t*42

m(t) Po(t*) fm(te+) fin(t*42) °° fn(te1)°

A comparison of (10') and (15) confirms the intuitively obvious. The costs associated with an unreversed, infinite horizon arbitrage differ from those associated with a reversed, finite horizon arbitrage in only two ways: The cost of maintaining the arbitrage becomes an infinite sum, and the cost of terminating the arbitrage is never paid. Accordingly,

A(t*) = lim A(t*,t'). (16)

t'+a

If the proposed program is optimal, and if the additional assumptions discussed above are met, then A(t*) must be zero. That is,

_—_—_

14/See appendix B for the relevant theorems and references.

- 16 -

lim A(t*,t') = 0. (17)

t'+ © Moreover, taking the limit of (10') as t' tends to infinity gives

lim [A(t*,t') + B'(t*,t')] = 0. (18)

t'+ 0 Equation (17) and (18) together imply

lim B'(t*,t') = 0 (19)

t'+o

Using the definition of B'(t*,t') from (10'), it is useful to rewrite (19) as

im B* a(t") Efe (es) facemer) fm(tee2) oe fmt) = (20)

tire

Our new necessary condition may now be stated as follows:

If a program is optimal, then for every t* for which consumption is interior, it is true that either (i) the program satisfies the transversality condition given by (20), or (ii) the value function given by (1') is not defined at c(t*), or (iii) the series of derivatives given in (15), when expressed as a function of c(t*), is not uniformly convergent .L2/

For purposes of comparison, note that the limiting extension of the

standard transversality condition is

lim gt a(t") = 0. (5')

t'>

From (20) and (5') we see that our new condition involves the product of

15/Using the transition equation, (15) can be rewritten as a function of c(t*). See appendix B for details.

-17 -

the standard condition and terms that represent the cumulative change in the state variable generated by an unreversed arbitrage over an infinite horizon. Since the cumulative change in the state over an infinite horizon may be unbounded, it is evident that (5') does not necessarily imply (20). Further, since the change in the state over an infinite horizon may approach zero, (20) may be satisfied even if (5') is not. Thus, we have derived a new necessary condition for an optimum that is independent of the limiting extension of the usual finite horizon condition.

It should be noted that in finite horizon problems our new transversality condition is implied by the standard condition and, accordingly, is redundant. For finite horizon problems, we know that the standard transversality condition, (5) is necessary for an optimum. It is possible to derive an additional transversality condition, analogous to (20), by considering, as before, the consequences of an unreversed arbitrage. The resulting condition is once again a product involving the standard transversality condition and terms representing the cumulative

change in the state variable over the horizon of the problem: . oT ; B(t*.T) = BAT), (Ex) Fnceeer) fmteee) ee Fe(T)] = 0 (21)

Since the cumulative change in the state is finite for any finite T, (21) is necessarily satisfied whenever the standard condition, 6!y(T)=0, is satisfied. Hence, for finite horizon problems, our new transversality condition is redundant. In infinite horizon problems, on the other hand, we have shown that the limiting extension of the standard condition is

invalid, and our new condition, (20), takes its place.

- 18 -

It is easily verified that the optimal solutions to the Halkin example considered in the preceding section satisfy our new condition. In section II it was shown that eX a(t) is equal to -1 for all t in

Halkin's problem. The cumulative change in the state is given by

we Foe) fim(tety) fm(tet2) oo fm(t') = (22) lim = [l-m(t*) ][1-c(t*+1) J[1-c(t*+2) ]...£1-c(t')]. t'+e

Since c(t) is a constant in the open interval between zero and one, the limiting value of (22) is zero. It follows that (20) is satisfied by all the optimal programs considered by Halkin. Thus, although the limiting extension of the usual condition is violated in Halkin's example, our new condition is not.

In this section we have derived a new transversality condition that must be met by optimal programs in infinite horizon problems in which the value of the control variable may be restricted but the value of the state is not. This new condition is not necessarily implied by the limiting extension of the usual finite horizon transversality condition, which can be violated by optimal programs in this class of problems. The Halkin example, which belongs to this class of problems and which has opt imal solutions that violate the limiting extension of the standard condition, was reconsidered. It was shown that these optimal solutions to the Halkin

example do, indeed, satisfy the new condition.

- 19 -

IV. Further Results

In this section we argue that the approach employed in section III to derive a transversality condition for the class of problems to which the Halkin example belongs can be usefully extended to other classes of problems. In particular, it is possible to generate a transversality condition for infinite horizon problems in which the state variable, rather than the control, is restricted. The analysis centers once again on the consequences of an unreversed arbitrage scheme. The particulars of the scheme, however, do differ from those of the scheme employed in section III. For problems in which only the state variable is constrained, we find that the appropriate transversality condition is the limiting extension of the standard finite horizon condition. The section, and the paper, conclude with a discussion of the economic intuition behind our results and the circumstances under which a transversality condition may fail, as the standard condition does in the Halkin example.

Consider the following problem:

Maximize Z =,f, e°V[c(t), m(t)] (24)

subject to: m(1l) =m (25) m(t+1) = f[c(t), m(t)] m(t) > x(t) u(t) >m(t) for t = 1,2,...

The functions that restrict the state variable, u(t) and e(t), are themselves bounded functions. There are no other constraints on the

problem. In particular, the control variable is unrestricted.

- 20 -

By following a line of reasoning similar to that employed in section II, a set of necessary conditions analogous to those of section II can be developed. The conditions include all the Kuhn-Tucker conditions for the finite horizon version of the problem, except for the transversality condition. These conditions are well-known and will not be reviewed here. As before, however, it will be useful to interpret the Kuhn-Tucker conditions as insuring that certain feasible arbitrages are of no value. Suppose in a feasible program the state variable is interior over an interval running from t* to t'. Then it is feasible to alter m(t*+1) in either direction, maintain the same deviation of m(t) from its original path through period t', and then return the state to its original value in period t'+] so that m(t'+1) and the subsequent program are unaltered.16/ This can be achieved through the appropriate manipulation of the control variable in periods t* through t'. As before the change in the value of the program resulting from this arbitrage is

easily computed and must be zero if the program is optimal:

dZ/dm(t*+1) = C(t*,t') + D'(t*,t') = 0 (26) where c(t*,t') = pV a ettV y+ V crin(t) fee] , c(t*) t=t*4+] m(t) c(t) Fo(t) c(t*)

and

D'(t*,t') = “6° Alt )Focpsy:

16/We assume that F(t) is not zero for t > t*.

- 21 -

C(t*,t') may be interpreted as the cost of maintaining the arbitrage through period t', while D'(t*,t') represents the cost of terminating or reversing the arbitrage. As before it is easily shown that if the proposed program satisfies the Kuhn-Tucker conditions, it also satisfies (26).

To derive a transversality condition for this problem, we consider the consequences of the unreversed version of the arbitrage described above. If we repeat the approach of section III, we find that the following must be true for an optimal program: If the unreversed arbitrage described above is feasible, then either the transversality condition for the problem must be satisfied -- that is, the limiting value of D'(t*,t') must be zero --, or the value function Z is not defined at m(t*+1), or the sum of the derivatives of the terms composing Z does not converge uniformly when expressed as a function of m(t*+1) 2

The arbitrage will be feasible if it is possible to maintain some constant deviation of the state variable from its original path. Two situations in which the arbitrage is infeasible may be distinguished. Examples of each are illustrated in figures 1 and 2.

In figure 1, the state variable asymptotically approaches one of its bounds as time goes to infinity. In this case it is impossible to generate a constant "epsilon" increase in the value of the state variable at all points in time without violating the restrictions on the state variable. There will always be a sufficiently large t for which the deviation of the

original value of the state variable from its bound is less than any

17/We are substituting out of c(t), which requires that f{c(t) ,m(t)] be invertible in its first argument for t > t*. See also footnote 16.

- 22 -

g(t)

Fig. | Fig. 2

epsilon chosen. In figure 2, a limiting value of the state variable does not exist. The state variable exhibits oscillatory behavior, periodically taking on the value of one of its bounds, or approaching that bound asymtotically. In this case it is impossible to generate an epsilon decrease in the value of the state variable without violating the restrictions on the state variable. If either of these two situations

exist, it will be true that

lim [sup m(t') - u(t')]Linf m(t') - e(t')] = 0

L to

Here sup m(t'), or the supremum of m(t'), is defined to be the largest

value of m(t') that occurs for all t greater than t 28/ Similarly,

18/As such it is non-increasing and bounded below by the function 2(t). Since g(t) is itself bounded below by assumption, the limiting value of

sup m(t') exists. Similarly, it can be shown that inf m(t') has a limiting value.

- 23 -

inf m(t'), or the infimum of m(t'), is defined to be the smallest value of m(t') that occurs for all ta greater than t'. In the case in which the state variable has a limiting value, as illustrated in figure 1, this

condition reduces to

lim [m(t') - u(t')][m(t') - g(t')J = 0

t+ @

We may now formally state the transversality condition for the

infinite horizon problem studied in this section:

t

lim gt y(t") = 0 t'+ © or (27) lim ECsup m(t') - u(t')JLinf m(t') - 2(t')] = 0 t'+ 0

The full necessary condition is as follows:

If a program is optimal, it must be true that either (i) the transversality condition given by (27 is satisfied, or (ii) the objective function Z does not converge pointwise for the program (its limiting value does not exist), or (iii) the sum of the derivatives of the terms composing (26) does not converge uniformly when expressed as a function of m(t*+1).

For problems in which uniform convergence of the sum of the

derivatives can be assured, and in which the limiting value of the

- 24 -

objective function and the state variable exist, our necessary condition

reduces to

lim p° y(t!) = 0

t' +0 or (28) lim [m(t') - u(t')J{m(t') - 2(t')] = 0. . t! +0

It can be shown that equation (28) is, in turn, precisely the limiting extension of the transversality condition associated with the finite horizon version of this problem. Thus, we see that the limiting extension of the usual transversality condition for this class of problems is the condition that insures, under certain assumptions, the absence of a particular kind of feasible arbitrage opportunity.

To derive a transversality condition appropriate to a particular class of problems, we first must find an arbitrage scheme that is always feasible in that class of problems. This requirement is met by the two schemes employed in this paper. Note, however, that the arbitrage scheme of section III could not have been applied to the problem addressed earlier in this section. The arbitrage scheme of section III (referred to as scheme 1 from here on) involves a one time change in the control variable and a consequent alteration in the path of the state variable. The resulting path of the state could, in principle, take any form. Since the problem introduced at the beginning of this section is one in which the state variable is restricted, there is no guarantee that scheme 1 is feasible in this second class of problems. Similarly, the arbitrage scheme employed in this section (scheme 2) may not be feasible in the class of

problems we considered in section III, as we will illustrate shortly.

- 25 -

Hence, the transversality condition of section III may be violated by optimal solutions to problems in which only the state variable is restricted and is therefore inappropriate for such problems; similarly, the transversality condition of this section may be violated by optimal solutions to problems in which the control variable is restricted and is, likewise, inappropriate for such problems.

To lend concreteness to the discussion, consider one last time the Halkin example. Since the limiting extension of the standard transversality condition is violated by solutions which are nonetheless optimal, any arbitrage associated with that condition must be infeasible. This is, indeed, true in the case of the optimal solutions we have considered to the Halkin example. Recall that in the Halkin example, each constant consumption path between zero and one generates a value of the state variable and the objective function that approach unity as time goes to infinity. The arbitrage associated with the limiting extension of the usual finite horizon transversality condition (scheme 2) is one that requires a constant epsilon change in the value of the state variable at all times after some t*. For any epsilon, there must come a time when raising the state variable by that amount would imply setting the state at a value greater than unity. This can not be done by selecting values of the control variable that are less than one; it can only be achieved by setting consumption at a value greater than one, which violates the constraints imposed on the problem. The arbitrage is, then, infeasible.

The results of this section and the previous one suggest the following interpretation of transversality conditions in infinite horizon

problems: Transversality conditions, like other necessary conditions,

- 2% -

insure the absence of particular feasible arbitrage opportunities. If a transversality condition is violated by a proposed solution to a problem, then either the solution is sub-optimal or every arbitrage associated with

that transversality condition is infeasible.

- 27 -

Appendix A

Equation (9) of the text can be derived as follows: Differentiating

the objective function with respect to consumption in period t*, we get

dZ/dc(t*) = A(t*,t') + B(t*,t') (A.1) where A(t*,t') = pV + t ty Cdm(t) /de(t*)] (t*, = 8 c(t*) bette] B m(t) m(t) /de(t*)

B(t*,t") = BY Vopr) Lde(t") /de(t*)].

By differentiating the transition equation, the following expressions for

dm(t)/dce(t*) and dc(t')/dc(t*) can be derived:

dm(t)/dce(t*) = f for t=t*+1

c(t*)

dm(t)/dc(t*) = f f

o(t*) fm(t¥41) hm(tee2) °F

=t* ' m(t-1) for tet*#2,.-.5¢

dc(t')/dc(t*)

i

Tey Fett) tect ty fincteer) ot fmt 1)

Substituting these expressions into (A.1), we arrive at equation (9). That any program satisfying the Kuhn-Tucker conditions also satisfies

(9) may be verified as follows: From (4) we know

m(t) + At) face) - \(t-1)g7) = 0 for t=t*#1,...,t'.

- 28 -

Since t runs from t*+1 to t', this forms a set of (t'-t*) equations. For

each t in the interval (t*+1,t'), multiply the corresponding equation by

t Bf a(te+1) hm(t*42) °° fimn(t-1)°

(We have adopted the convention that when t-1 < t*+1, the product following

at in the above expression is equal to one.) Add together all the

resulting equations and simplify to get

t* tho, -B° rA(t*) - B’ A(t )fmn(t*+1) fm(te42) °°° fm(t 1-1) (A.2) t' t cette: © V(t) fines) fn(ets2y*fin(t-1) *

By assumption, c(t*) and c(t') are interior, even if consumption at other dates is not. For t = t* or t', then, equations (3), (6), and (7) imply that

A(t) = “Ve(t) Foc) . (A.3)

Substituting this into (A.2), we can verify that A(t*,t') + B(t*,t') = 0, or that equation (10) is satisfied. It is also useful to derive an expression more general than (10) which takes account of the possibility that c(t') is at a boundary. Since c(t*) is still -- by assumption -interior, we can use (A.3) to eliminate A(t*) in (A.2) above. What results

from the substitution is (10') of the text.

- 29 -

Appendix B

The purpose of this appendix is to state formally the restrictions mentioned in the text which insure that the derivative of the limit function of an infinite series equals the limit of the sum of the derivatives of each term in the series. Consider a feasible program [c(1), c(2),+.063 m(1), m(2),.-.J. Choose any period t* such that g(t*) < c(t*) < u(t*). If an alternative feasible consumption is chosen at t* while consumption levels at other dates are unchanged, then the state variable from t*+1 onwards can be written as

m(t*+1) = f[c(t*) ,m(t*)] = m[t*+1,c(t*)],

m(t*+2) = f[e(t*+1) ,f£c(t*) m(t*)]] = m[t*+2,c(t*)],

etc. Note that from t*+2 on, the state variable is a composite function of c(t*). Hence, the value of the entire program can be expressed as a

function of c(t*):

t*-1 Z£c(t*)] = aa B V[Ec(t) ,m(t) ]

+ 8° VEc(t*) ,m(t*)] (B.1)

+ x peVEc(t) m[t,c(t*) J] t=t*+1

We assume first that the infinite sum given in (B.1) converges for c(t*) = c(t*). Further, we note that the first t*-1 terms of this sum are independent of c(t*). Consider the remaining terms. Define h fe(t*) J

to be the nth term in this series (n=0,1,...). Since

- 30 -

g(t*) < c(t*) < u(t*), there exists an open interval around c(t*) which contains feasible consumption points. Moreover, h [e(t*) J is differentiable on this open interval since it is the result of

the composition of a finite number of differentiable functions.

(See Marsden, p. 168.) If we make the second assumption that

thi[c(t*) J converges uniformly to glc(t*)] on this open interval, then Z'[c(t*)] exists and equals th Le(t*) ] for c(t*) in the open interval

around c(t*). For details, see Apostel (1957, p.403).

- 31 -

References

Apostol, T. Mathematical Analysis. Reading, Mass.: Addison-Wesley Publishing Co., Inc., 1957.

Arrow, K.d., and Kurz, M. Public Investment, the Rate of Return, and

Optimal Fiscal Policy. Baltimore, Md.: Johns Hopkins Press, 1970.

Dixit, A.K. Optimization in Economic Theory. London: Oxford University Press, 19/76.

Dorfman, R. "An Economic Interpretation of Optimal Control Theory." American Economic Review, LIX(5) (December 1969), pp. 817-831.

Goldberg, R.R. Methods of Real Analysis. New York: John Wiley and Sons, 1976.

Halkin, H. "Necessary Conditions for Optimal Control Problems with

Infinite Horizons." Econometrica, Vol. 42, No. 2 (March, 1974), pp. 267-272.

Marsden, J. Elementary Classical Analysis. San Francisco: W. H. Freeman and Company, 1974.

Takayama, A. Mathematical Economics. Hinsdale, I1].: Dryden Press, 1974.

Weitzman, M.L. “Duality Theory for Infinite Horizon Convex Models." Management Science, 19 (7) (March 1973), pp. 783-789.

Zangwill, W. Nonlinear Programming: A Unified Approach. Englewood Cliffs, N.J.: Prentice Hall, 1969.