Revisiting Risky Money

Finance and Economics Discussion Series Federal Reserve Board, Washington, D.C. ISSN 1936-2854 (Print) ISSN 2767-3898 (Online) Revisiting Risky Money Travis D. Nesmith 2024-090 Please cite this paper as: Nesmith, Travis D. (2024). “Revisiting Risky Money,” Finance and Economics Discussion Series 2024-090. Washington: Board of Governors of the Federal Reserve System, https://doi.org/10.17016/FEDS.2024.090. NOTE: Staff working papers in the Finance and Economics Discussion Series (FEDS) are preliminary materials circulated to stimulate discussion and critical comment. The analysis and conclusions set forth are those of the authors and do not indicate concurrence by other members of the research staff or the Board of Governors. References in publications to the Finance and Economics Discussion Series (other than acknowledgement) should be cleared with the author(s) to protect the tentative character of these papers.

Revisiting Risky Money TravisD.Nesmith∗ QuantitativeRiskAnalysis BoardofGovernorsoftheFederalReserveSystem November9,2024 Abstract Risk was first incorporated into monetary aggregation over thirty-five years ago, using a stochastic version of the workhorse money-in-the-utility-function model. Nevertheless,themathematicalfoundationsofthisstochasticmodelremainshaky. To firm the foundations, this paper employs a slightly richer probability concept than standard Borel-measurability, which enables me to prove the existence of a well-behavedsolutionandtoderivestochasticEulerequations. Thismeasurability approachislong-establishedalbeitlesscommonineconomics,possiblybecausethe derivationofstochasticEulerequationsisnew. Importantly,theproblem’seconomics arenotrestrictedbytheapproach. Consequently,theresultsprovidefirmfootingfor thegrowingmonetaryaggregationunderriskliterature,whichintegratesmonetary and finance theory. As crypto-currencies and stable coins garner more attention, solidifyingthefoundationsofriskymoneybecomesmorecritical. Themethodalso supportsderivingstochasticEulerequationsforanydynamiceconomicsproblemthat featurescontemporaneousuncertaintyaboutprices,includingassetpricingmodels likecapmandstochasticconsumerchoicemodels. Keywords: money;risk;monetaryaggregation;assetpricing;dynamic programming;stochasticmodeling;uncertainty;Eulerequations JEL:c61;c62;d81;d84;e40;g12 ∗20thandCSts.,NW,MailStop188,Washington,DC20551;E-mail:travis.d.nesmith@frb.gov IwishtothankfirstWilliamA.BarnettandthenRichardA.Anderson,JaneM.Binner,W.ErwinDiewert,Fredj Jawadi,BarryE.Jones,DavidMillsJr.,HeinzSchättler,StevenE.ShrevenumerouscolleaguesattheBoard andparticipantsatthe8thannualconferenceoftheSocietyforEconomicMeasurementformanyproductive discussions. Nevertheless,theviewspresentedaresolelymyownanddonotnecessarilyrepresentthoseof theFederalReserveBoardoritsstaff.Anyremainingerrorsaremysoleresponsibility. 1

1 Introduction Theclamoraroundcrypto-currenciesandstablecoinsmightgivetheimpressionthatthe topic of risky money is relatively new. In fact, over thirty-five years ago Poterba and Rotemberg(1987)raisedtheissueandconstructedamonetaryaggregateaccountingforthe factthatmonetaryassetsmayearnvariableinterestandsuchinterestratesarerisky. Ina seriesofpapers,Barnettandhisco-authorsdevelopedthistheoryofmonetaryaggregation underrisk(Barnettetal.,1991,Barnett,1995,Barnettetal.,1997,2000,BarnettandLiu, 2000,BarnettandWu,2005,Barnettetal.,2021). Othercontributionstoboththetheory and practice of including risk assets in monetary aggregates include Drake et al. (1999, 2003),ElgerandBinner(2004),Binneretal.(2018), andSerletisandXu(2024).1 Likein the deterministic theory, for example in Barnett (1980) and Anderson et al. (1997), the money-in-the-utilityfunctionmodel—nowdefinedtobestochastic—istheworkhorseof risky monetary aggregation, because it implicitly subsumes any reason to hold money withinthemodel. Thisliteratureextendsaggregationtocontemporaneouslyriskyassets, and at the same time generalizes the well-known Capital Asset Pricing Model (capm).2 Morerecently,theabilitytoaddressriskhasenabledcreditcards,whichareanimportant sourceofliquidity,tobeincludedinmonetaryaggregatesforthefirsttime(BarnettandLiu, 2019,BarnettandSu,2019,2020,Barnettetal.,2023,BarnettandPark,2023,2024a,b). For China,theaggregatescanbeextendedtoothersourcesofconsumercredit(Barnettetal., 2022). Yemba(2022)usesrisk-adjustedmonetaryusercoststostudydollarization. Finally, Duanetal.(2023)usethetheorytoincludegreenbondswithinmonetaryaggregates,which couldbeofgrowingimportance. In addition to its workhorse role in risky monetary aggregation, many other monetaryandfinancialmodels—rangingfromcash-in-advancetoassetpricingmodels—are specialcasesofstochasticmoney-in-the-utility-function(smiuf)models.3 Forexample, stochastic decision problems that include monetary or financial assets have long been usedtoaddressavarietyoftopicsincludingassetpricing(Townsend,1987,Hansenand Singleton,1983,LucasJr.,1990,Finnetal.,1990,Bohn,1991,BansalandColemanII,1996), currencysubstitutionandexchangerates(Imrohoroglu,1994,BasakandGallmeyer,1999), intertemporalsubstitution(DutkowskyandDunsky,1996),monetaryaggregation(Poterba andRotemberg,1987,Barnettetal.,1997,BarnettandSerletis,2000,BarnettandWu,2005), moneydemand(Holman,1998,BarnettandXu,1998,ChoiandOh,2003),optimalmonetary policy(Chang,1998,Charietal.,1998,BoyleandPeterson,1995,CalvoandVegh,1995, Dupor,2003,Canzonerietal.,2006),andpricedynamics(LucasJr.,1978,DenHaan,1990, 1Restrepo-Tobón(2015)arguesthataccountingforriskisnotcriticalformonetaryaggregatesastherisk adjustmenttomonetaryusercostsissmall;however,Binneretal.(2018)findsthatriskierassetsshouldbe incorporatedinpermissiblemonetaryaggregatesandthattheadjustmentislargerifforecastedreturnson riskyassetsareused. 2SeeKaratzasandShreve(1998)foratextbooktreatmentofcapmandaroadmapfortheasset-pricing literature. 3Deterministicmoney-in-the-utility-functionmodels,cash-in-advancemodels,andothertransactioncost modelsofmoneyarefunctionallyequivalent(Feenstra,1986). 2

Matsuyama, 1990, 1991, Hodrick et al., 1991). In each case, the usefulness of the model dependsonthederivationofthestochasticEulerequationsthatcharacterizethemodel’s solution. However, these stochastic Euler equations are usually just assumed to exist. Consequently,thevalidityandapplicabilityofthesemodels’resultsisdifficulttoascertain. Forthesmiufmodelandothernestedmodels,assumingtheexistence,uniqueness,and otherpropertiesofsolutions,includingderivingstochasticEulerequations,isnotimmaterial. Solvingstochasticdynamicmodelsisnotnecessarilystraightforwardmathematically. Thesituationissimilartotheinitialdevelopmentofstochasticoptimalgrowthmodels, wheresolutionswereassumedtoexist,butinthatcaseeffortsweremadequicklytoput themodelsonfirmerfooting.4 Thispapershoresupthemathematicalfoundationofthe smiufmodel,andthereforetheresearchthatreliesoniteitherdirectlyorthroughtheuse ofanestedmodel. Withtheon-goingdevelopmentofcrypto-currencies,strengtheningthe foundationsofhowtoapproachriskymoneyisparticularlytimely. Specifically, this paper shows that under weak measurability conditions an unique solutiontothesmiufmodelthatsatisfiesastochasticversionofBellman’sequationexists. ThesemeasurabilityconditionsfollowBertsekasandShreve(1978),whichprovidesstochasticdynamicprogramming(dp)resultsthataremoregeneralthatthemeasure-theoretic approachthatisstandardinadvancedeconomicapproaches(StokeyandLucasJr.,1989). These results are achieved by enriching the probability model underlying expectation formation. Theconditionsaremuchweakerthanthenormalassumptionsemployedineconomics,bothmathematicallyandeconomically. Theconditionsareweakermathematically, becausetheresultsareachievedbylookingforsolutionstotheprobleminarichersetof candidatesolutions,sothestandardapproachisenrichedratherthanrestricted. Perhaps surprisingly,theresultsfromdeterministicdparegenerallystillobtainableinthisricher spaceofsolutioncandidates. Theseresults—theexistenceanduniquenessofasolution—havegenerallybeenassumed inthesmiufliterature,despitethefactthatnothingintheproblemspecificationwould ensure such results hold. The one exception is the recent paper (Barnett et al., 2021); however, that paper of necessity restricts the state space to be finite, which is a strong assumption. Suchastrongmathematicalassumptionimposesrestrictionsontheeconomics oftheproblem. Thisrestrictivenessisnotunique;theusualadditionalassumptionsmade tosolvestochasticdynamiceconomicmodels—typicallyboundednessandcompactness,or morerecentmonotonicityconditions—alsoconstraintheobjectivefunctionandthusthe economics. Suchrestrictionsmaybeacceptableinmanyapplications,buttheobjectofthe smiufliteratureistosupportaggregationoverrisky“money”. Limitsontheeconomicrange ofthemodel,therefore,arenottrivialbecausethisobjectiveispredicatedonapproximating unknownaggregatorfunctions. Consequently,thereisapremiumplacedonthegenerality oftheresults. Sothefactthatconditionsassumedhereareeconomicallytrivial,anddonot constraintheeconomicapplicability,iscritical. Despitetheadvantages,thisapproachdevelopedbyBertsekasandShreve(1978)has notbeenwidelyusedineconomics,likelybecauseeconomicapplicationsgenerallyrequire 4Forarecentreviewofthemathematicalgroundingofoptimalgrowthmodels,seeSpearandYoung(2017). 3

theadditionalstepofderivingstochasticEulerequationsandsuchaderivationwasnot available. Thispaperprovesthatunderthericherdefinitionofmeasurabilitythevalue functioninheritsdifferentiabilityfromtheobjectivefunction,whichallowsthederivationof stochasticEulerequations. Becausetheconditionsareweak,andeconomicallyinnocuous, theresultsbroadentherangeofstochasticeconomicmodelsbothsolvablebydp,andwhose solutioncanbecharacterizedbydynamicfirstorderconditions. Toborrowathemefrom LjungqvistandSargent(2004,pg.16,19–20),theresultscontributetothe“imperialism” ofdp,whichhasallowedmoreandmoredynamiceconomicproblemstobeformulated recursivelyandsolvedthroughdp;specifically,thispaper,althoughfocusingonasingle, albeitgeneral,modelalsoprovidesageneralapparatusforderivingthefirstorderconditions forrecursiveeconomicmodelsunderuncertaintywithoutimposingstrongrestrictions. Returningtothespecificsmiufmodel,thestochasticEulerequationsderiveddepend onatrade-offbetweenanasset’srateofreturn,risk,andliquidity,insteadofdependingon justtheusualmonetarytrade-offbetweenreturnandliquidity.5 Thethree-waytrade-off generalizes much of the voluminous asset-pricing literature in finance, where only the two-dimensional trade-off between risk and return is considered.6 In particular, these stochasticEulerequationsgeneralizethecapmfirst-orderconditions,whereinthetrade-off withliquidityisignored(Barnettetal.,1997). Becauseofthegeneralityofboththesmiuf modelandthemathematicalmethods,thispaperreinforcesthevalidityofpreviousresults in both monetary and financial models, which depended on the (assumed) existence of stochasticEulerequations.7 Theorganizationofthispaperisasfollows. Section2presentsthedynamicdecision problem. Section3laysoutthemeasure-theoreticapparatususedtomodelexpectations. Section4discussesmeasurableselection,whichisnecessaryforthedpresults. Section5 reviewsthestochasticdpapproachtosolvingthedecisionproblem. Conditionsguaranteeingtheexistenceofanoptimalplanthatsatisfiestheprincipalofoptimalityaredeveloped. The optimal plan is shown to be stationary, non-random, and (semi-) Markov. Section 6provesthatdifferentiabilityofthesolutionfollowsfromdifferentiabilityoftheutility function. Thisresultcombinedwiththeresultsintheprevioussectionformallysupports thederivationofthestochasticEulerequations. Thelastsectionprovidesashortconclusion. Proofsarecollectedintheappendixexceptfortheproofthatthedifferentiabilityis inheritedbythevaluefunction,whichisnovelandtheextension. 5Theimportanceofthislatertrade-offhaslongbeenrecognized(Tobin,1958). 6AnexceptionisBansalandColemanII(1996),whouseliquiditytoexplaintheequitypremiumpuzzle (MehraandPrescott,1985). 7Althoughthisdiscussionemphasizesmonetaryandfinancialmodelsduetothecentralimportanceofrisk insuchmodels,theresultsalsoapplytootherdynamiceconomicmodelsthatincorporatecontemporaneous uncertainty,suchasmodelswithsearchcostsorinformationrestrictions. 4

2 Household Decision Problem The model is based on the infinite-horizon stochastic household decision problems in Barnett (1995) and Barnett et al. (1997).8 The model is general in that preferences are definedoveranarbitrary(finite)numberofassetsandgoods,buttheformoftheutility functionisnotspecified. Resultsprovenforthismodel,willholdformorerestrictivesmiuf models.9 Following the earlier papers, 𝑆(𝑠), the consumption possibility set for any period 𝑠 ∈ {𝑡,𝑡 +1,…,∞},isdefinedtobe 𝑛 𝑘 ∑ 𝑝 𝑐 ≤ ∑ [(1+𝜌 )𝑝∗ 𝑎 −𝑝∗𝑎 ] 𝑆(𝑠) = {(a ,𝐴 ,c ) ∈ 𝑌 | 𝑗𝑠 𝑗𝑠 𝑖,𝑠−1 𝑠−1 𝑖,𝑠−1 𝑠 𝑖𝑠 } (1) 𝑠 𝑠 𝑠 𝑗=1 𝑖=1 +(1+𝑅 )𝑝∗ 𝐴 −𝑝∗𝐴 +I 𝑠−1 𝑠−1 𝑠−1 𝑠 𝑠 𝑠 where,foreachperiod𝑠,a = (𝑎 ,…,𝑎 )isa𝑘-dimensionalvectorofplannedrealasset 𝑠 1𝑠 𝑘𝑠 balanceswhereeachelement𝑎 hasanominalholding-periodyieldof𝜌 ,𝐴 isplanned 𝑖𝑠 𝑖𝑠 𝑠 holdingsofthebenchmarkassetwhichhasanexpectednominalholdingperiodyieldof 𝑅 ,c = (𝑐 ,…,𝑐 )isa𝑛-dimensionalvectorofplannedrealconsumptionofnon-durable 𝑠 𝑠 1𝑠 𝑛𝑠 goodsandserviceswhereeachelement𝑐 hasapriceof𝑝 ,I isthenominalvalueofincome 𝑗𝑠 𝑗𝑠 𝑠 fromallothersources,whichisnon-negative,and𝑝∗ isatruecost-of-livingindexdefined 𝑠 as a function over some non-empty subset of the 𝑝 .10 Prices, including the aggregate 𝑗𝑠 price,𝑝∗,andratesofreturnarestochasticprocesses. 𝑌isthefeasiblesetℝ𝑘+1×𝐶,where 𝑠 𝐶 is the household survival set—a subset of the 𝑛-dimensional non-negative Euclidean orthant—andℝhastheusualmeaning;notethattheconstructionallowsforshort-selling asassetholdingscanbenegative. For the stochastic processes, the information set must be specified. It is assumed thatcurrentpricesandthebenchmarkrateofreturnareknownatthebeginningofeach periodandcurrentinterestonallotherassetsisrealizedattheendofeachperiod. More specifically,forall𝑖,𝑗and𝑠,𝑝 ,𝑝∗,𝑅 (and𝑅 )and𝜌 areknownatthebeginningof 𝑗𝑠 𝑠 𝑠 𝑠−1 𝑖,𝑠−1 period𝑠,while𝜌 isnotknownuntiltheendofperiod𝑠. Sincetheirreturnsareunknown 𝑖𝑠 inthecurrentperiod,theassets,a ,arerisky,while𝐴 istherisk-freeasset. Despitethe 𝑠 𝑠 uncertainty, the constraint contains only known variables in the current period, so the consumercansatisfy(1)withcertainty. Againfollowingthepriorliterature, the(representative)consumermaximizestheir intertemporally-additiveutility,solvingattime𝑡, ∞ 𝑠−𝑡 1 sup{𝑢(a,c)+𝐸 [ ∑ ( ) 𝑢(a ,c )]} (2) 𝑡 𝑡 𝑡 𝑠 𝑠 1+𝜉 𝑠=𝑡+1 8Earlierversionsofthismodel,bothdeterministicandstochastic,areinthepaperscollectedinBarnettand Serletis(2000). 9Similarresultstotheonesdevelopedhere,wouldapplytofinitehorizonversions,exceptthattheoptimal policywouldbetime-varying. 10Theassumptionthatatruecost-of-livingindexexistsistrivial,because,thelimitingcaseisasingletonso that𝑝∗=𝑝 ,where𝑝 denotesthepriceofanumérairegoodorservice.Notethatthepreviousliterature 𝑠 𝑗∗𝑠 𝑗∗𝑠 denotedthevectorofgoodsandservicesbyx. 𝑡 5

subjectto(a ,c ,𝐴 ) ∈ 𝑆(𝑠)forall𝑠 ∈ {𝑡,𝑡 +1,…,∞}andthetransversalitycondition 𝑠 𝑠 𝑠 𝑠−𝑡 1 limsup𝐸[( ) 𝐴 ] = 0, (3) 𝑡 𝑠 1+𝜉 𝑠→∞ where the operator 𝐸[ ] denotes expectations formed on the basis of the information 𝑡 available at time 𝑡, and 0 < 𝜉 < ∞ is the subjective rate of time preference.11 The form of the utility function, 𝑢(⋅), is unspecified, but assumed to satisfy standard regularity conditions. Further,thetransversalityconditionrulesoutunboundedborrowingatthe benchmarkrateofreturn. Thisdecisionproblemresemblesastandardmoney-in-the-utilityfunctionmodel,butinthisstochasticdecisionproblemtheassetscanbeeithermonetary orfinancialassets. Thedeterministicversionofthisdecisionproblemhasbeenthecoremodelunderlying monetaryaggregationsinceBarnett(1977,1980): SeeAndersonetal.(1997)forasummary. Itisalsothefoundationofmostofthestochasticmoney-in-the-utilityfunctionmodelscited previously. However,BarnettandWu(2005)extendedthemodeltorelaxtheassumption of intertemporal additive separability and Barnett and Su (2019) used this result in an application incorporating credit cards.12 This extension, which includes lagged terms in the utility function, still assumes the existence, uniqueness and differentiability of a solution;theresultsderivedinthispapercouldreadilyincorporatethisextension. Barnett etal.(2021)considersrecursiveutility,whichlikelywouldnotbecoveredbytheresults here;therelationofthatpapertothisonewillbediscussedinthenextsections. 3 Expectations and Dynamic Programming Thegoalistobeprovethatthedecisionproblemdefinedbyequations(1)–(3)hasaunique solution and to characterize the solution. As the model is recursive, dp is the natural candidate solution method.13 dp is especially appealing for stochastic problems where statesofthedynamicsystemareuncertain,becausethedpsolutiondeterminesoptimal controlfunctionsdefinedforalladmissiblestates. But,forastochasticmodel,caremust betakentoensurethatdpisfeasible. BertsekasandShreve(1978,Chapter1)andStokey andLucasJr.(1989,Chapter9)discusstherequirementsforimplementingstochasticdpin moredetail. Solvingtheproblemrequiresthatakeygapisaddressed. Thedecisionproblemdefined in equation (2) is not fully specified, as the expectations operator 𝐸[ ] is not formally 𝑡 defined. This present lack of definition is intentional as it reflects the smiuf literature; inparticular,Barnett(1995),Barnettetal.(1997)andthesubsequentliteratureusingthis 11SeeDrouhin(2020)fordiscussionofconstanttimepreferencesandmethodstorelaxit. 12Althoughtheresultsarecharacterizedasrelaxingintertemporalseparability,therelaxationisactually oftheadditivestructure. Gorman’soverlappingtheoremwouldimplythatthecurrentperiodisstillweakly separable(Nesmith,2007). 13TheseminalbookisBellman(1957). OtherreferencetextsfordpincludeSargent(1987),Stokeyand LucasJr.(1989),Bertsekas(2000),andLjungqvistandSargent(2004). 6

model,withtheexceptionofBarnettetal.(2021),neverformallydefinethe(conditional) expectationsoperator. Furtherstructureisneededbeforedporanyothersolutiontechnique canbeapplied. Infact,thespecificationofexpectationsisintertwinedwithwhetherand howtheproblemcanbesolved. Thespecificationofexpectationsalsoaffectshowgeneral the problem is economically. In the rest of this section, different specifications will be discussed,lookingparticularlyattheimplicationsfordpandtheeconomicrestrictiveness. First, approaches that limit how risk or uncertainty enters the problem will be briefly discussed. Then,thenextsubsectionwillcoverthemoregeneralspecification. Contemporaneouscertainty: One simple avenue to solve the model using dp is to specify that all current state variables are known with perfect certainty and only futurestatesareuncertain. Inthatcase,deterministicdpresultsareapplicable. It mightappearthattheproblemgivenby(2)isdeterministicinthecurrentperiod. But, thisisnotthecase;allowingassetreturnstoberiskyintroducescontemporaneous uncertaintyasthestatevariableinthenextperiodisstochastic. Ifcontemporaneous certaintywasassumed,itwouldimplythatallassetreturnsarerisk-free. Suchan assumptionisnotappropriateforfinancialmodelsgenerally,andisnonsensicalfor theliteratureexpresslyseekingtoextendmonetaryaggregationtoriskyassets. This illustrates,inaparticularlystrongfashion,howassumptionsmadetosolveaproblem canaffecttheeconomicsoftheproblem. Finiteorcountablesupport: Alternatively,thestochasticprocessescouldbeassumedto havefiniteorcountablesupport. Suchanassumptionwouldimplythatexpectations areweightedsums,ratherthanintegralsagainstsomemeasure. Thisspecification wouldimplythattherearenoissueswithmeasurabilityinapplyingthedpalgorithm. Consequently,thetheabilitytosolvetheproblemdependsonlyontheproblem’s structureandtheresultsfromdeterministicdpwouldapply(BertsekasandShreve, 1978,pp.6). Suchanapproachisnotuncommonineconomics;forexample,muchof theexpositioninLjungqvistandSargent(2004)restrictsthestochasticprocessestobe Markovchains,sothedeterministicresultscarryover.14 Barnettetal.(2021)explicitly assumes Choquet expectations; making this assumption (implicitly) restricts the underlyingstochasticprocessesbecauseChoquetexpectationsareonlydefinedfor finiteorcountableprobabilities. However,assumingcountabilityisstrongerthan thenormineconomicsandfinance,particularlywhenriskandfinancialdecisionmakingarethefocus. Forexample,assumingcountabilitydoesnotmeetAliprantis andBorder’s(2006)dictumthat“[t]hestudyoffinancialmarketsrequiresmodels that are both stochastic and dynamic, so there is a double imperative for infinite dimensionalmodels.” Certaintyequivalence: Anothercommonapproachistoassumecertaintyequivalence hold. Going back to Simon (1956) and Theil (1957), the certainty equivalent approachconsistsofreplacingstochasticprocesseswithadeterministicfunctionofthe 14Inothercases,LjungqvistandSargent(2004)allowsinfinitesupport,butassumeseitheri.i.d. or(conditional)Gaussianity. 7

process—mostcommonlytheircurrent(conditional)expectations—andsolvingthe resultingdeterministicdynamicprogram;seeVandeWaterandWillems(1981)fora generalformulationforbothdiscreteandcontinuousproblems. Undercertainconditions,thesolutionforthestochasticproblemwillbethesameasthisdeterministic version. However,theconditionsarerestrictive. First,certaintyequivalencerequires thatpastandpresentdecisionsdonotaffecteitherthestochasticprocessesthemselvesortheinformationthatthedecision-makerwouldhaveabouttheprocesses goingforward(Duchan,1974). Second,certaintyequivalenceeffectivelyeliminates therelevanceofrisk;asParra-Alvarezetal.(2021)comment,“ifriskmatters,breaking certainty equivalence is desired in order to account for the effects of risk.” More explicitly,underperfectcertainty: a.)decisionsareunaffectedbyriskthatdoesnot changethedeterministicfunctionoftheprocesses,and,b.)counterfactually,assets donotexhibitanyriskpremia(Fernández-Villaverdeetal.,2016). Toillustratethe firstpointmoreconcretely,undercertaintyequivalence,iftheconditionalexpectationisused,thenthechoicesmadewillnotdependatallonvarianceoranyother higher-momentofthestochasticprocess. Theimplicationisthattheagentwould takenoprecautionaryresponsestohedgeagainstriskineithersavingsorinvestment decisions.15. Fernández-Villaverdeetal.(2016)pointoutthattheresultinglackof anyprecautionaryresponsetoriskbiasesanywelfareanalysis. Itisnotsurprising thatthesecondpointfollowsasassumptionsaffectingwelfarealsoaffectstructural assetpricing(AlvarezandJermann,2004). Thegeneralrelationwithriskisveryrelevanthere. Andtheseissuespersisteven if certainty equivalence is only used as a linear approximation around a steady state(seeDíaz-Giménez,2001,Aruobaetal.,2006). Buttheassumptionsneededto ensurecertaintyequivalenceholdshavefurtherimplicationsfortheeconomicsof thedecisionproblem. Certaintyequivalenceholdsgenerallyiftheobjectivefunction is quadratic and the dynamics and constraints are linear (Amman, 1996). With additionalassumptionsontheconditionalexpectationofthestochasticprocesses, certaintyequivalenceholdsundersuchaLinear-Quadraticframeworkwithavariety ofassumptionsonthedisturbances,startingwiththeclassicassumptionofadditive Gaussian noise used in Simon (1956), which is still common (see Ljungqvist and Sargent,2004,pp.113-115),andundervariousextensions(e.g.SpeyerandGustafson, 1974,AkashiandNose,1975,VandeWaterandWillems,1981,Andersonetal.,1996, DerpichandYüksel,2023,amongmanyothers). Restrictingtheobjectivefunctionto haveaquadraticformandthedynamicstobelinearareclearlystrongassumptions. Forutilityfunctions,thequadraticassumptionhasverystrongandcounterfactual implications(Pratt,1964,JappelliandPistaferri,2017);furthermore,fixingtheutility functiontoaparticularformisinconsistentwithhowconsumeraggregationtheory generallyproceeds. 15SeeLjungqvistandSargent(2004,pp.115)andAruobaetal.(2006) 8

Althougheachoftheseapproachesfindsuses,eachlimitshowriskaffectsthedecision andhasfurtherimpactsontheeconomics. Perhapsasaconsequence,itiscommontouse theframeworkfirstintroducedbyBlackwell(1965)andassumethatthedisturbancespaces areBorelspaces. 3.1 Measurability Modelingriskbyassumingthatthedisturbancesaregeneratedfromaprobabilityspace ismoregeneralthanthespecificationsdiscussedsofar. Inthiscontext,Borelspacesare arichprobabilitymodelandassumingaBorelspacestructurearguablyisthenormfor stochastic decision problems, particularly in economics following Stokey and Lucas Jr. (1989). FromassumingaBorelspacestructure,itfollowsthatthedecisionandconstraint spacesfortheinfinite-horizonproblemarealsoBorelspaces. Asdiscussedabove,since theunderlyingprobabilityspaceisassumedtobeuncountable,theexpectationoperator becomesanintegralagainstsomeprobabilitymeasure. Consequentlyinthiscase,applying dphingesonmeasurabilityissues. Unfortunately,Borel-measurabilityisnotnecessarily maintainedunderthedpalgorithm. Atahighlevel,ifexpectationsaredefinedonBorel spaces,applyingdptosolvethemodelrequireseitherjustassumingmeasurableselection isfeasibleoraddingvariouscompactnessand(semi-)continuityassumptions. Theselater assumptions are mathematically strong. In addition, compactness and the necessary continuityassumptionsimplicitlyconstraintheeconomics. It is worth noting that the results generally available after assuming compactness andcontinuitymaystillbelacking. Asarelevantexample,assumingcompactnessrules out classical stochastic Linear-Quadratic control problems (Bertsekas and Shreve, 1978, pp. 12); the reason is that in the classic specification of such problems, the choices can influencethedistributionofthestochasticstates16 Theinabilitytohandlemodelswhere currentdecisionsaffecteitherstochasticdistributionsortheavailableinformationaboutthe distributionsisageneralrestriction. Therefore,usingthisapproachtostochasticdecision problemsarguablycouldproduceresultssubjecttoakindof‘Lucascritique’becausethe potential for choices to change risk has to be ruled out a priori. After discussing issues withcompactnessandcontinuityinthenexttwosubsections,thefeasibilityofmeasurable selectionisaddressedinthefollowingsectionbyusingaricherprobabilitystructurethan Borelspaces. 3.1.1 Compactness Compactnessisnaturalstartingplacewhenlookingtoextendresultsfromdeterministic or countable contexts. As Tao (2008, pp. 168) details, in metric or probability spaces compactnessisa“powerfulproperty”thatimpliestheinfinitedimensionalspaceis“almostfinite”inthatsetsinthespaceexhibitpropertiessimilartofinitesets(SeealsoAliprantis andBorder,2006,pp.37-41). Infinite-dimensionalspaces,compactnessisequivalentto being closed and bounded; compactness immediately extends the Weierstrass theorem 16Asaresult,certaintyequivalencealsodoesnothold. 9

ontheexistenceofoptimumsforcontinuousfunctionsoverclosedandboundedsetsto infinite dimensional spaces. But, as Luenberger (1969, pp. 39) states, “the restriction to compactsetsissosevereininfinite-dimensionalnormedspaces”thattheextensionapplies toonly“theminorityofoptimizationproblems.” Giventhemathematicalstrengthoftheassumption,itisnotsurprisingthatcompactness alsoaffectstheeconomicsofadecisionproblem. Ifeithertheobjectivefunctionitselfor thestatespaceisassumedtobecompact,optimalsolutionswillbebounded.17 Butsuch assumptionsruleoutcertainformulationsofeconomicproblems. ThiscouldincludeLinear- Quadraticdecisionproblemswherecertaintyequivalencedoesnothold. Butmorebasically, manyrelativelycommoneconomicdynamicproblemsareunbounded,includingconstant returns to scale growth or more generally any growth problem where the technology permitssustainedgrowthovertheinfinitehorizon(StokeyandLucasJr.,1989,Takayama, 1985,pp.87;92–84andpp.577–578respectively). Solvingsuchproblems,eitherinvolves imposing boundedness on the economics or seeking a method that does not assume compactness. Compactnesshasimplicationsfortheeconomicsbeyondjustboundedness. Compactnesslimitsthefeasiblesetandrulesoutcertaininvestmentstrategies. Forexample,Page andWooders(1996)connectscompactnesstonounboundedarbitrage. Rulingoutarbitrage may be sensible, but it is better to be aware of doing so rather than implicitly doing so unawares. Gutiérrez(2009)andAndrikopoulos(2013)detailevenmorenuancedconnectionsbetweencompactnessandpreferences. Solvingastochasticdynamicprogramming problemhingesonmeasurabilityissues. Compactnessissuchastrongassumptionthatit cansolvethem,butitisnotasolutiontailoredtotheproblem;itisnotsurprisingthatit hasfarreachingimplicationsbeyondmeasurability. 3.1.2 Continuity Despiteitsstrength,compactnessisnotsufficient. Certaincontinuityassumptions,orat leastsemi-continuityassumptions,muststillbelayeredontopofit. Importantly,thesetypes ofcontinuityassumptionsmustbemadenotabouttheutilityfunctionorobjectivefunction moregenerally,butratheronthevaluefunctionwhoseexistenceisnotevencertain. Such assumptionsarbitrarilyrestrictthevaluefunction. Assuming(semi-)continuityofthevalue functionismuchstrongerthanjustassumingpreferencesare(semi-)continuous,because (semi-)continuityoftheutilityfunctiondoesnotgenerallyimply(semi-)continuityofthe valuefunction. Furthermore,inthecontextofassetdecisions,imposingcontinuityofthe valuefunctionrulesoutavarietyofinvestmentstrategiesorsolutionswhichwouldinvolve switchingorotherdiscontinuousbehavior. 17Theseassumptionsdonotnecessarilymeanrestrictingtofinitesolutionsasacompactificationthereal numberscouldbeusedtoincludeinfiniteoutcomes. 10

4 Measurable Selection Thechallengewithtakingameasuretheoryapproachtostochasticdpisthatintegration againstameasureisnotwell-definedforallfunctions.18 Therefore,themeasurabilityof functionswillbecentraltotheabilitytoderiveastochasticdpsolution. Furthermore,the measurablespacecannotbearbitrary. Insteadofassumingcompactnessandthenecessaryandarbitraryancillarycontinuity assumptions,asolutionissoughtwithinarichersetoffunctions: universally-measurable functions. Relaxing the measurability restrictions, which expands the class of possible solutions,ismoreappealingthanimposingstrongrestrictionsthatlimittheclassofpossible solutions. Section5.3discussesthesetrade-offsfurther. Thefollowingstandarddefinitionwillbeusedrepeatedly: Definition1(Measurablefunction). Let𝑋and𝑌betopologicalspacesandlet𝔉 beany 𝑋 𝜎-algebraon𝑋andlet𝔅 betheBorel𝜎-algebraon𝑌. Afunction𝑓∶ 𝑋 → 𝑌is𝔉-measurable 𝑌 if𝑓−1(𝐵) ∈ 𝔉 ∀𝐵 ∈ 𝔅 . 𝑋 𝑌 Transitionfunctionsarecommonlyusedtoincorporatestochasticshocksintoafunctionalequation(seeStokeyandLucasJr.,1989,pg.212). Thecurrenttreatmentissimilar, exceptinsteadofbeginningwithtransitionfunctions,stochastickernelsareemployed. Definition2(StochasticKernel). Let𝑋and𝑌beBorelspaceswith𝔅 denotingtheBorel 𝑌 𝜎-algebra. Let𝑃(𝑌)denotethespaceofprobabilitymeasureson(𝑌,𝔅 ). Astochastickernel, 𝑌 𝑞(𝑑𝑦|𝑥), on 𝑌 given 𝑋 is a collection of probability measures in 𝑃(𝑌) parameterized by 𝑥 ∈ 𝑋. If 𝔉 is a 𝜎-algebra on 𝑋, and 𝛾−1(𝔅 ) ⊂ 𝔉 where 𝛾∶ 𝑋 → 𝑃(𝑌) is defined by 𝑃(𝑌) 𝛾(𝑥) = 𝑞(𝑑𝑦|𝑥), then 𝑞(𝑑𝑦|𝑥) is 𝔉-measurable. If 𝛾 is continuous, 𝑞(𝑑𝑦|𝑥) is said to be continuous. AstochastickernelisaspecialcaseofaregularconditionalprobabilityforaMarkov process,definedinShiryaev(1996,Definition6,pg.226). Inabstractterms,if𝑥istakento representthestateofthesystemattime𝑡andthesystemisMarkovian,theconditional expectation operator applied to an element of 𝑌 can, formally be viewed as an integral againstthestochastickernel,i.e. 𝐸 [𝑓(𝑥,𝑦)] = ∫𝑓(𝑥,𝑦)𝑞(𝑑𝑦|𝑥)(Shiryaev,1996,Theorem 𝑡 3, pg. 226). As this definition requires measurability of the function, a measurability assumption is necessary to define expectations, even before a solution is sought. If a function,definedontheCartesianproductof𝑋and𝑌,isBorel-measurableandthestochastic kernel is Borel-measurable, then ∫𝑓(𝑥,𝑦)𝑞(𝑑𝑦|𝑥) is Borel-measurable (Bertsekas and Shreve,1978,Proposition7.29,pg.144). Theimplicationisthattheconditionalexpectation is Borel-measurable. Integration defined this way operates linearly and obeys classical 18Theouterintegral,whichiswell-definedforanyfunction,couldbeused.However,notonlyisthereno uniquedefinitionfortheouterintegral,butitisnotalinearoperator. Consequently,expectationswouldnot beadditive. Recursivemethods,suchasdp,generallyrequireadditivitysothattheoverallproblemcanbe brokenintosmallersub-problems.Barnettetal.(2021)relaxesadditivity,butonlybyrestrictingthestatespace tobefinite. Theouterintegralapproachmightcouldprovideageneralizationoftheirresults,butitisnot clearifthesolutionwouldexistandsatisfytheBellmanequationornotinaninfinitestatespace. 11

convergencetheorems. Also,theintegralisequaltotheappropriateiteratedintegralon productspaces. Thesestatementswillbeclarifiedinthenextsection. Asthestatespacefortheproblemhasnotbeendefined,itmayappearthatthestochastic kernelislimitedinthatitonlyrepresentsconditionalexpectationsforMarkoviansystems. Infact,non-MarkovianprocessescanalwaysbereformulatedasMarkovianbyexpanding thestatespace. Inthesequel,thestatespacefortheproblemwillbeformulatedsothat the process is Markovian. Borel measurability, by itself, is not adequate to prove the existence of the solution. Bertsekas and Shreve (1978) address this problem through a richerconceptofmeasurability. Theirapparatusincludesuppersemianalyticfunctions andmeasurabilitywithregardtotheuniversal𝜎-algebra. Definitionsandrelevantresults arepresentedbelow;proofscanbefoundinAliprantisandBorder(2006)andBertsekas andShreve(1978).19 Thekeyrelationisthattheuniversal𝜎-algebraincludestheanalytic 𝜎-algebra,whichinturnincludestheBorel𝜎-algebra. ThisimpliesthatallBorel-measurable functionsareanalyticallymeasurable,andthatallanalyticallymeasurablefunctionsare universallymeasurable. Thusthemovetouniversalmeasurabilityis,relativetotheBorel model,relaxingaconstraint, insteadofimposingarestriction. Bymovingtouniversalmeasurability, the information set is enriched and the measurability assumptions are technicallyrelaxed. Sucharelaxationdoesnodamagetotheeconomicsbehindthemodel. Theresultsarestandardandtheproofsareomitted. Borelmeasurabilityisnotadequatefordp,becausetheorthogonalprojectionofaBorel set is not necessarily Borel measurable Specifically, if 𝑓∶ 𝑋 ×𝑌 → ℝ∗, where ℝ∗ is the extended real numbers, is given and 𝑓∗∶ 𝑋 → ℝ∗ is defined by 𝑓∗(𝑥) = sup 𝑓(𝑥,𝑦) 𝑦∈𝑌 thenforeach𝑐 ∈ ℝ,definetheset {𝑥 ∈ 𝑋|𝑓∗(𝑥) > 𝑐} = proj ({(𝑥,𝑦) ∈ 𝑋 ×𝑌|𝑓(𝑥,𝑦) > 𝑐}) (4) 𝑋 whereproj ( )istheprojectionmappingfrom𝑋 ×𝑌onto𝑋. If𝑓( )isBorel-measurable 𝑋 then {(𝑥,𝑦) ∈ 𝑋 ×𝑌|𝑓(𝑥,𝑦) > 𝑐} (5) isBorel-measurable,but{𝑥 ∈ 𝑋|𝑓∗(𝑥) > 𝑐}maynotbeBorel-measurable(Blackwell,1965, Davidson,1994,StokeyandLucasJr.,1989). Thedpalgorithmrepeatedlyimplementssuchprojections,sotheconditionalexpectation of functions like 𝑓∗( ) will need to be evaluated, requiring that the function is measurable. Thismeasurabilityrequirementleadstothedefinitionofanalyticsets, the analytic𝜎-algebra,andanalyticmeasurability: Definition3(Analyticsets). Asubset𝐴ofaBorelspace𝑋isanalyticifthereexistsaBorel space𝑌andBorelsubset𝐵of𝑋 ×𝑌suchthat𝐴 = proj (𝐵). The𝜎-algebrageneratedby 𝑋 theanalyticsetsof𝑋isreferredtoastheanalytic𝜎-algebra,denotedby𝒜 ,andfunctions 𝑋 thataremeasurablewithrespecttoitarecalledanalyticallymeasurable. 19FurtherreferencesincludeDavidson(1994),Dudley(1989),Lang(1993),Shiryaev(1996)andStokeyand LucasJr.(1989). 12

Davidson (1994, pg. 329) refers to analytic sets as “nearly” measurable because, for anymeasurablespaceandanymeasure,𝜇,onthatspace,theanalyticsetsaremeasurable underthecompletionofthemeasure. Thecompletionofaspacewithrespecttoameasure involvessetting𝜇(𝐸) = 𝜇(𝐴)foranyset𝐸suchthat𝐴 ⊂ 𝐸 ⊂ 𝐵whenever𝜇(𝐴) = 𝜇(𝐵). Effectively,thisassignsmeasurezerotoallsubsetsofmeasurezerosets(Davidson,1994, pg.39). Analyticsetsaddresstheproblemofmeasurableselectionwithinadynamicprogram, because, if 𝑋 and 𝑌 are Borel spaces, and, if 𝐴 ⊂ 𝑋 is analytic and 𝑓∶ 𝑋 → 𝑌 is Borelmeasurable,then𝑓(𝐴)isanalytic. Thisimpliesthatif𝐵 ⊂ 𝑋 ×𝑌isanalyticthenproj (𝐵) 𝑋 isalsoanalytic. Analyticsetsarethesmallestgroupsofsetssuchthattheprojectionofa Borelsetisamemberofthegroup(AliprantisandBorder,2006). Analyticsetsareusedto defineuppersemianalyticfunctionsasfollows: Definition4. Let𝑋beaBorelspaceandlet𝑓∶ 𝑋 → ℝ∗ beafunction. Then𝑓( )isupper semianalyticif{𝑥 ∈ 𝑋|𝑓(𝑥) > 𝑐}isanalytic ∀𝑐 ∈ ℝ. Thefollowingresultiskeyfortheapplicationofthedpalgorithm: Lemma5. Let𝑋and𝑌beBorelspaces,andlet𝑓∶ 𝑋 ×𝑌 → ℝ∗ beuppersemianalytic,then 𝑓∗∶ 𝑋 → ℝ∗ definedby𝑓∗(𝑥) = sup 𝑓(𝑥,𝑦)isuppersemianalytic. 𝑦∈𝑌 Twoimportantpropertiesofuppersemianalyticfunctionsarethatthesumofsuch functionsremainsuppersemianalytic,andif𝑓∶ 𝑋 → ℝ∗isuppersemianalyticand𝑔 ∶ 𝑌 → 𝑋isBorelmeasurable,thenthecomposition𝑓 ∘𝑔isuppersemianalytic. Mostimportantly, theintegralofaboundeduppersemianalyticfunctionagainstastochasticintegralisupper semianalytic. Thisisstatedasalemma: Lemma6. Let𝑋and𝑌beBorelspacesandlet𝑓∶ 𝑋×𝑌 → ℝ∗beauppersemianalyticfunction eitherboundedaboveorboundedbelow. Let𝑞(𝑑𝑦|𝑥)beaBorel-measurablestochastickernel on𝑌given𝑋. Then𝑔∶ 𝑋 → ℝ∗ definedby𝑔(𝑥) = ∫𝑓(𝑥,𝑦)𝑞(𝑑𝑦|𝑥)isuppersemianalytic. 𝑌 Semianalyticfunctionshaveonerelevantlimitation. Iftwofunctionsareanalytically measurable,theircompositionisnotnecessarilyanalyticallymeasurable. Thisdifficulty canbeovercomemovingtothestillricheruniversallymeasurable𝜎-algebra:20 Definition7(Universal𝜎-algebra). Let𝑋beaBorelspace,𝑃(𝑋)bethesetofprobability measureson𝑋,andlet𝔅 (𝜇)denotethecompletionof𝔅 withrespecttotheprobability 𝑋 𝑋 measure𝜇 ∈ 𝑃(𝑋). Theuniversal 𝜎-algebra 𝔘 isdefinedby 𝑋 𝔘 = ⋂ 𝔅 (𝜇). (6) 𝑋 𝜇∈𝑃(𝑋) 𝑋 If 𝐴 ∈ 𝔘 , 𝐴 is called universally measurable, and functions that are measurable with 𝑋 respectto𝔘 arecalleduniversallymeasurable. 𝑋 20Theslightlytighter,butlessintuitive,𝜎-algebraoflimitmeasurablesetswouldbesufficient.Again,moving tothelargerclassdoesnotimposeanyrestrictions. 13

Theuniversally-measurable𝜎-algebraisthecompletionoftheBorel𝜎-algebrawith respecttoeveryBorelmeasure. Consequently,itdoesnotdependonanyspecificBorel measure. NotethateveryBorelsubsetofaBorelspace𝑋isalsoananalyticsubsetof𝑋, which implies that the 𝜎-algebra generated by the analytic sets is larger than the Borel 𝜎-algebra. Thefactthatanalyticsetsaremeasurableunderthecompletionofanymeasure impliesthattheyareuniversally-measurable,so𝔅 ⊆ 𝒜 ⊆ 𝔘 . 𝑋 𝑋 𝑋 Universal measurability enables the stochastic dp recursion to be implemented. Of course, if a stochastic kernel is Borel-measurable, it is universally measurable. With universallymeasurable stochastickernels, integrationoperates linearly, obeysclassical convergence theorems, and iterates on product spaces. This is stated formally as the followingtheorem: Theorem8. Let𝑋 ,𝑋 ,…beasequenceofBorelspaces,𝑌 = 𝑋 ×⋯×𝑋 ,and 𝑌 = 𝑋 ×𝑋 ×⋯. 1 2 𝑛 1 𝑛 1 2 Let 𝜇 ∈ 𝑃(𝑋 ) be given and, for 𝑛 = 1,2,…, let 𝑞 (𝑑𝑥 |𝑦 ) be a universally measurable 1 𝑛 𝑛+1 𝑛 stochastickernelon𝑋 given𝑌 . Thenfor𝑛 = 2,3,…,thereexistuniqueprobabilitymeasures 𝑛+1 𝑛 𝑟 ∈ 𝑃(𝑌 )suchthat∀𝑋 ∈ 𝔅 ,…,𝑋 ∈ 𝔅 𝑛 𝑛 1 𝑋 𝑛 𝑋 1 𝑛 𝑟 (𝑋 ∩𝑋 ∩⋯∩𝑋 ) = ⋯ 𝑞 (𝑋 |𝑥 ,…,𝑥 ) 𝑛 1 2 𝑛 ∫ ∫ ∫ 𝑛−1 𝑛 1 𝑛−1 𝑋 𝑋 𝑋 1 2 𝑛 ×𝑞 (𝑑𝑥 |𝑥 ,…,𝑥 )…×𝑞 (𝑑𝑥 |𝑥 )𝜇(𝑑𝑥 ) (7) 𝑛−2 𝑛−1 1 𝑛−2 1 2 1 1 If𝑓∶ 𝑌 → ℝ∗ isuniversallymeasurable,andtheintegraliswell-defined,21 then 𝑛 𝑓𝑑𝑟 = ⋯ 𝑓(𝑥 ,…,𝑥 )𝑞 (𝑑𝑥 |𝑥 ,…,𝑥 ) ∫ 𝑛 ∫ ∫ ∫ 1 𝑛 𝑛−1 𝑛 1 𝑛−1 𝑌 𝑋 𝑋 𝑋 𝑛 1 2 𝑛 ×𝑞 (𝑑𝑥 | 𝑥 ,…,𝑥 )×⋯×𝑞 (𝑑𝑥 |𝑥 )𝜇(𝑑𝑥 ). (8) 𝑛−2 𝑛−1 1 𝑛−2 1 2 1 1 Therefurtherexistsauniqueprobabilitymeasure𝑟 ∈ 𝑃(𝑌)suchthatforeach𝑛themarginal of 𝑟on𝑌 is𝑟 . 𝑛 𝑛 Theformaldefinitionoftheconditionalexpectationsoperatoris,therefore,theintegral ofthefunctionversus𝑟 or𝑟. Thisdefinitionallowsuniversallymeasurableselection: 𝑛 Theorem9(MeasurableSelection). Let𝑋and𝑌beBorelspaces,𝐷 ∈ 𝑋 ×𝑌beananalyticset suchthat𝐷 = {𝑦 |(𝑥,𝑦) ∈ 𝐷},and𝑓∶ 𝐷 → ℝ∗ beanuppersemianalyticfunction. Define 𝑥 𝑓∗∶ proj (𝐷) → ℝ∗ by 𝑋 𝑓∗(𝑥) = sup 𝑓(𝑥,𝑦). (9) 𝑦∈𝐷 𝑥 Thentheset 𝐼 = {𝑥 ∈ proj (𝐷) | forsome𝑦 ∈ 𝐷 ,𝑓(𝑥,𝑦 ) = 𝑓∗(𝑥)}isuniversallymeasur- 𝑋 𝑥 𝑥 𝑥 able,andforevery𝜖 > 0,thereexistsauniversallymeasurablefunction𝜙∶ proj (𝐷) → 𝑌 𝑋 21Theintegraliswell-definedifeitherthepositiveornegativepartsofthefunctionarefinite.Suchafunction willbecalledintegrable. 14

suchthat𝐺𝑟(𝜙) ⊂ 𝐷andforall𝑥 ∈ proj (𝐷)either 𝑋 𝑓[𝑥,𝜙(𝑥)] = 𝑓∗(𝑥) if 𝑥 ∈ 𝐼 (10) or 𝑓∗(𝑥)−𝜖 if 𝑥 ∉ 𝐼and𝑓∗(𝑥) < ∞ 𝑓[𝑥,𝜙(𝑥)] ≥ { (11) 1/𝜖 if 𝑥 ∉ 𝐼and𝑓∗(𝑥) = ∞ TheselectorobtainedinTheorem9,𝑓[𝑥,𝜙(𝑥)]isuniversallymeasurable. Ifthefunction 𝜙( ) is restricted to be analytically measurable, then 𝐼 is empty and (11) holds. In this case, the selector is not necessarily universally measurable. For Borel-measurable functions𝜙( ),theanalyticresultdoesnotholduniformlyin𝑥. Thestrongresultgiven by(10)isonlyavailableforuniversallymeasurablefunctions. Similarly,strongresultsare availableforBorel-measurablefunctionsifsignificantlystrongerregularityassumptions aremaintained.22 Theweakerregularityconditionsareappealing,astheyallowasolution withoutimposingrestrictionsontheeconomicsoftheproblem. 5 Stochastic Dynamic Programming There are several issues with simply applying dp to the smiuf decision problem. First, thefunctionalformoftheutilityfunctionisnotspecified,asdoingsowouldrestrictthe applicabilityoftheresults. Second,aspreviouslydiscussed,thereareanumberoftechnical difficulties in applying dp methods in a general stochastic setting. Section 3’s measure theorywasdevelopedtoovercomethesedifficulties.23 Threetasksarerepeatedlyperformedinthedprecursion. First,aconditionalexpectation is evaluated. Second, the supremum of an extended real-valued function in two (vector-valued) variables, the state and the control, is found over the set of admissible controlvalues. Finally,aselectorwhichmapseachstatetoacontrolthat(nearly)achieves the supremum in the second step is chosen. Each of these steps involves mathematical challengesinthestochasticcontext. Anespeciallyimportanttechnicalconcernisthatthe measurabilityassumptionsnotbedestroyedbyanyofthethreesteps. Thefirstandsecondstepsrequirethattheexpectationoperatorcanbeiteratedand interchanged with the supremum operator. As shown in Section 4, these requirements are met by the integral definition of the expectations operator, for either the Borel- or universally-measurablespecifications. Steptwoencountersaproblemwithmeasurability, becauseoftheissuewithprojectionsofBorelsetsalsodiscussedintheprevioussection. Analytic-measurabilityissufficienttoaddressthisparticularproblem,butsuchmeasurabilityisnotnecessarilypreservedbythecompositionoftwofunctions. Usingsemianalytic 22Forexample,StokeyandLucasJr.(1989)assumethat𝐷iscompact,and𝑓isuppersemicontinuous. 23Forfurtherreferencestodpinmeasurespaces,seeBlackwell(1965,1970),Strauch(1966),Hinderer(1970), Blackwelletal.(1974),DynkinandJuskevic(1975). 15

functionsandassuminguniversal-measurabilitysolvesthisproblem. Universalmeasurabilityalsoallowsmeasurableselectionundermildregularityconditions,addressingthethird stepwithoutassumingthatpolicyiscompact-valuedandupperhemi-continuousasisdone inStokeyandLucasJr.(1989). Asallthreestepsofthedpalgorithmcanbeimplemented for the stochastic problem, the existence of an optimal or nearly optimal program can be proven. Furthermore, the principal of optimality holds for the (near) optimal value function. Toshowthattheseresultsareapplicabletosmiufmodels,theproblemlaidoutin(2) is mapped into Bertsekas and Shreve’s general stochastic dp model. One adjustment is necessaryasBertsekasandShrevedefinelowersemianalyticfunctionsratherthanupper semianalytic functions, because their exposition addresses the finding the infinum of a function. Thisdifferencerequirescarefuladjustmentoftheirregularityconditions. 5.1 GeneralFramework FollowingBertsekasandShreve(1978,pg.188-189),thegeneralinfinitehorizonmodelis definedasfollows: Definition10(StochasticOptimalControlModel). Ainfinitehorizonstochasticoptimal controlmodel isaneight-tuple(𝑋,𝑌,𝑆,𝑍,𝑞,𝑓,𝛽,𝑔)where: 𝑋 Statespace: anon-emptyBorelspace; 𝑌 Controlspace: anon-emptyBorelspace; 𝑆 Controlconstraint: afunctionfrom𝑋tothesetofnon-emptysubsetsof𝑌. ThesetΓ = {(𝑥,𝑦)|𝑥 ∈ 𝑋,𝑦 ∈ 𝑆(𝑥)}isassumedtobeanalyticin𝑋 ×𝑌; 𝑍 Disturbancespace: anon-emptyBorelspace; 𝑞(𝑑𝑧|𝑥,𝑦)]Disturbancekernel: aBorel-measurablestochastickernelon𝑍given𝑋 ×𝑌; 𝑓 Systemfunction: aBorel-measurablefunctionfrom𝑋 ×𝑌 ×𝑍to𝑋; 𝛽 Discountfactor: apositiverealnumber;and 𝑔 One-stagevaluefunction: anuppersemianalyticfunctionfromΓtoℝ∗. Thefilteredprobabilityspaceusedinthestochasticoptimalcontrolmodelconsistsof fourelements: 1)the(Cartesian)productofthedisturbancespacewiththeinfiniteproduct of the state and control spaces, 𝑍 ×(Π∞ (𝑋 ×𝑌)); 2) a 𝜎-algebra (generally universally 𝑖=𝑡 𝑖 measurable)onthatproductspace;3)theprobabilitymeasuredefinedinTheorem8on page 14; and, 4) the filtration defined by the restriction to the product of the state and controlspacesthathavealreadyoccurred,(Π𝑠−1(𝑋 ×𝑌))×𝑋 whereitisunderstoodthat 𝑖=𝑡 𝑖 𝑖 𝑠 eachspaceisacopyoftherespectivespace. Establishingtheexistenceofasolutiontoastochasticoptimalcontrolmodelmeans establishingtheexistenceofanoptimalpolicyfortheproblem. Specifically,thefollowing definitionsfromBertsekasandShreve(1978)areused: Definition 11 (Policy). A policy is a sequence 𝜙 = (𝜙,𝜙 ,…) such that, for each 𝑠 ∈ 𝑡 𝑡+1 {𝑡,𝑡 +1,…}, 𝜙 (𝑑𝑦 |𝑥,𝑦,…,𝑦 ,𝑥 ) (12) 𝑠 𝑠 𝑡 𝑡 𝑠−1 𝑠 16

isauniversallymeasurablestochastickernelon𝑌,given𝑋 ×𝑌 ×⋯×𝑌 ×𝑋satisfying 𝜙 (𝑆(𝑥 )|𝑥,𝑦,…,𝑦 ,𝑥 ) = 1, (13) 𝑠 𝑠 𝑡 𝑡 𝑠−1 𝑠 for every (𝑥,𝑦,…,𝑦 ,𝑥 ). If for every 𝑠, 𝜙 is parameterized by only 𝑥 , then 𝜙 is a 𝑡 𝑡 𝑠−1 𝑠 𝑠 𝑠 𝑠 Markov policy. Alternatively, if for every 𝑠, 𝜙 is parameterized by only (𝑥,𝑥 ), then 𝜙 𝑠 𝑡 𝑠 𝑠 is a semi-Markov policy. The set of all Markov policies, Φ, is contained in the set of all semi-Markov policies, Φ′. If for each 𝑠 and (𝑥,𝑦,…,𝑦 ,𝑥 ), 𝜙 (𝑑𝑦 | 𝑥,𝑦,…,𝑦 ,𝑥 ) 𝑡 𝑡 𝑠−1 𝑠 𝑠 𝑠 𝑡 𝑡 𝑠−1 𝑠 assignsmassonetosomeelementof𝑌,𝜙isnon-randomized. If𝜙isaMarkovpolicyofthe form𝜙 = (𝜙,𝜙,𝜙,…),itiscalledstationary. 𝑡 𝑡 𝑡 Definition 12 (Value Function). Let 𝜙 be a policy for the infinite horizon model. The (infinitehorizon)valuefunctioncorrespondingto 𝜙at 𝑥 ∈ 𝑋is ∞ 𝑉 (𝑥) = [∑𝛽𝑘𝑔(𝑥 ,𝑦 )]𝑑𝑟(𝜙,𝜇 ) 𝜙 ∫ 𝑘 𝑘 𝑥 𝑘=0 (14) ∞ = ∑[𝛽𝑘 𝑔(𝑥 ,𝑦 ) 𝑑𝑟 (𝜙,𝜇 )] ∫ 𝑘 𝑘 𝑘+𝑡 𝑥 𝑘=0 where,foreach𝜙 ∈ Φ′ and𝜇 ∈ 𝑃(𝑋),𝑟(𝜙,𝜇 )istheuniqueprobabilitymeasuredefinedin 𝑥 equation(7)and, forevery𝑘, the𝑟 (𝜙,𝜇 )istheappropriatemarginalmeasure.24 The 𝑘+𝑡 𝑥 (infinitehorizon)optimalvaluefunctionat𝑥 ∈ 𝑋is𝑉∗(𝑥) = sup 𝑉 (𝑥). 𝜙∈Φ′ 𝜙 Note that the optimal value function is defined over semi-Markov policies; this is withoutlossofgenerality. Furthermore,BertsekasandShreve(1978,pg.216)showthat theoptimalvaluecanbereachedbyonlyconsideringMarkovpolicies. Theadvantageof includingsemi-MarkovpoliciesisthattheoptimummayrequirearandomizedMarkov policy, but only need a non-randomized semi-Markov policy. Finally, the Jankov-von Neumanntheoremguaranteestheexistenceofatleastonenon-randomizedMarkovpolicy soΦandΦ′ arenon-empty. Thefollowingdefinesoptimalityforpolicies: Definition13(Optimalpolicies). If𝜖 > 0,thepolicy𝜙is𝜖-optimalif 𝑉∗(𝑥)−𝜖 if𝑉∗(𝑥) < ∞ 𝑉 (𝑥) ≥ { (15) 𝜙 1/𝜖 if𝑉∗(𝑥) = ∞ forevery𝑥 ∈ 𝑋. Thepolicy𝜙isoptimalif𝑉 (𝑥) = 𝑉∗(𝑥). 𝜙 Inthenextsubsection,equation(2)isrestatedinthisoptimalcontrolframework. The lastsubsectionaddresseswhatconditionsareneedtoguaranteetheexistenceofa(nearly) optimalpolicy. 24The interchange of the integral and the summation is justified by either the monotone or bounded convergencetheorems. 17

5.2 RestatingtheHouseholdProblem Embeddingthehouseholddecisionproblemintothisframeworkrequiresspecifyingthe stateandcontrolspaces. SincethespacesinthisproblemareallfiniteEuclideanspaces, thestateandcontrolspaceswillbeBorelnomatterhowdefined. Forautilityproblem, it is natural to generally define ‘prices’ as states and ‘quantities’ as controls, but there is no unique specification required for the dp algorithm. Also, if the utility function demonstratedhabitpersistenceasinBarnettandWu(2005),laggedconsumptionvariables wouldnaturallybestatevariablesinthecurrentstage. Definetheperiod𝑠statesbyx = (𝜶 ,𝝍 ),where𝜶 = (a ,𝐴 )and𝝍 denotesthe 𝑠 𝑠 𝑠 𝑠 𝑠−1 𝑠−1 𝑠 vector of interest rates, prices, and other income that were realized at the beginning of period𝑠,normalizedby𝑝∗, 𝑠 𝝍 = ⎛ ⎜(1+𝜌 ) 𝑝 𝑠 ∗ −1 ,…,(1+𝜌 ) 𝑝 𝑠 ∗ −1 ,(1+𝑅 ) 𝑝 𝑠 ∗ −1 , 𝑝 1,𝑠 ,…, 𝑝 𝑛,𝑠 , I 𝑠 ⎞ ⎟. (16) 𝑠 ⎜⏟⏟⏟⏟⏟⏟⏟⏟⏟ 1 ⏟ ,𝑠 ⏟ − ⏟⏟ 1 ⏟⏟⏟⏟⏟𝑝⏟𝑠 ∗ ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟ 𝑘 ⏟ , ⏟ 𝑠− ⏟⏟ 1 ⏟⏟⏟⏟⏟𝑝⏟𝑠 ∗ ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟ 𝑠 ⏟ − ⏟⏟ 1 ⏟⏟⏟⏟𝑝⏟⏟𝑠 ∗ ⏟⏟ ⏟⏟𝑝⏟𝑠⏟ ∗ ⏟⏟⏟⏟⏟⏟⏟𝑝⏟⏟𝑠 ∗ ⏟⏟ 𝑝 𝑠 ∗⎟ ⎝ ⎠ Interestrates Prices The period 𝑠 controls are defined to be y = (𝜽 ,c ) where 𝜽 = (a ,𝐴 ). The state space 𝑠 𝑠 𝑠 𝑠 𝑠 𝑠 𝑋 is (2𝑘 +𝑛+2)–dimensional Euclidean space, and the control space 𝑌 is a subset of (𝑘 +𝑛+1)–dimensionalEuclideanspace. Thisnotationisusefulinwhatfollows. Again notethat,for𝑠 < ∞,elementsof𝑎 and𝐴 maybenegative,sothatshort-sellingisallowed. 𝑠 𝑠 Thebudgetconstraintcanbeusedtoeliminateoneofthecontrols,becausetheconstraint will hold exactly at every time-period for any optimal solution. Therefore, a redundantcontrolhasbeenspecifiedandthesetofadmissiblecontrolsactuallylieina (𝑘 +𝑛)-dimensional linear subspace of 𝑌. Besides satisfying the budget constraint, the controlvariablesneedtobeinsidethesurvivalset. Byleavingintheredundantcontrol,it iseasiertoexplicitlyspecifythisconstraint. Thecontrolssetcanbewrittenasafunction ofonlyperiod𝑠statesandcontrolsas 𝑘+1 𝑘+1+𝑛 𝑆(x) = {y ∈ 𝑌 | ∑(𝑦 −𝜓𝑥)+ ∑ 𝜓𝑦 −𝜓 ≤ 0} (17) 𝑖 𝑖 𝑖 𝑗 𝑗 𝑘+𝑛+2 𝑖=1 𝑗=𝑘+2 wheretheperiodsubscript𝑠hasbeensuppressedandtheothersubscriptsdenotepositions intherespectivevector. Thefirstkeyassumptionis Criterion14. AssumethatΓ = {(𝑥,𝑦)|𝑥 ∈ 𝑋,𝑦 ∈ 𝑆(𝑥)}isanalyticin𝑋 ×𝑌. Thesystemfunctionhasarelativelysimpleform. Itisdefinedby x = (𝜶 ,𝝍 ) = 𝑓(x ,y ,z ) = (𝜽 ,𝝍 +z ) (18) 𝑠+1 𝑠+1 𝑠+1 𝑠 𝑠 𝑠 𝑠 𝑠 𝑠 Inwords,thefirstpartitionofthestatesevolvesaccordingtothesimplerule𝜶 = 𝜽 ,and 𝑠+1 𝑠 thesecondevolvesasastate-dependentstochasticprocess,accordingto𝝍 = 𝝍 +z .25 𝑠+1 𝑠 𝑠 Ifz wasapurewhitenoiseprocess,then𝝍 wouldbearandomwalk. 𝑠 𝐬 25ThiswouldhavetoslightlymodifiedtoaccountforthemodelinBarnettandWu(2005)thatincludes habitpersistence. 18

Thediscountfactorisdefinedby𝛽 = 1/(1+𝜉)andsatisfies0 < 𝛽 < 1. Theone-stage valuefunction𝑔(𝑥 ,𝑦 )issimplytheperiodutilityfunctionas𝑔(x ,y ) = 𝑔(y ) = 𝑢(a ,c ); 𝑠 𝑠 𝑠 𝑠 𝑠 𝑠 𝑠 itisafunctionofonlythecontrols.26 Theframeworkwouldallow𝑔( )tobetime-varying or to depend on the states, however, this would complicate the derivation of stochastic EulerequationsinSection6. Theremainingassumption,whichcompletesthemappingof thestochasticutilityproblemintothedpmodel,is: Criterion15. Assumethat𝑔(𝑦)isanuppersemianalyticfunctionfromΓtoℝ+. Theassumptionsaremathematicallymuchweakerthantheassumptionsusedinthe standardeconomictextStokeyandLucasJr.(1989). Moreimportantly,theassumptions are economically weaker; the key assumption is a richer model of uncertainty, rather than continuity restrictions on the value function, and compactness of constraint set. Furthermore,ratherthanassumethatmeasurableselectionisfeasible,orequivalentlyrather thanjustassumethemeasurabilitychallengeswithapplyingdpdon’texist,measurable selectioncanbeshowntobefeasibleunderthisricherdefinitionofuncertainty. 5.3 ExistenceofaSolutionandthePrincipalofOptimality Theexistenceofasolutiontothehouseholddecisionproblemorequivalentlytheexistence of a (nearly) optimal policy can now be proved. First, as it is easier, the optimal value functionisshowntosatisfyastochasticversionofBellman’sequationandthePrincipalof Optimality. Theresultismostcleanlystatedusingthefollowingdefinition: Definition16(Statetransitionkernel). Thestatetransitionkernelon𝑋given𝑋 ×𝑌is definedby 𝑡(𝐵|𝑥,𝑦) = 𝑞({𝑧|𝑓(𝑥,𝑦,𝑧) ∈ 𝐵}|𝑥,𝑦) = 𝑞(𝑓−1(𝐵) |𝑥,𝑦). (19) (𝑥,𝑦) Thus,𝑡(𝐵|𝑥,𝑦)istheprobabilitythatthestateattime(𝑠+1)isin𝐵giventhatthestateat time𝑠is𝑥andthecontrolis𝑦. Notethat𝑡(𝑑𝑥′|𝑥,𝑦)inheritsthemeasurabilityproperties ofthestochastickernel. Thenthefollowingmappinghelpstostateresultsconcisely: Definition17. Let𝑉∶ 𝑋 → ℝ∗ beuniversallymeasurable. Definetheoperator𝑇by 𝑇(𝑉) = sup {𝑔(𝑦)+𝛽 𝑉(𝑥′) 𝑡(𝑑𝑥′ | 𝑥,𝑦)}. (20) ∫ 𝑦∈𝑆(𝑥) 𝑋 Severallemmascharacterizetheoptimalpolicies. Thefollowinglemmashowsthatthe optimalvaluefunctionfortheproblemsatisfiesafunctionalrecursionthatisastochastic versionofBellman’sequation. 26Recallthatthereisaredundantcontrol. 19

Lemma18. Theoptimalvaluefunction𝑉∗(𝑥)satisfies𝑉∗ = 𝑇(𝑉∗)forevery𝑥 ∈ 𝑋. Thisnecessityresultimpliesthatanoptimalpolicyisafixedpointofthemappingthat isimplicitlydefinedinthelemma. Thefollowingsufficiencyresultimpliesthatastochastic versionofBellman’sprincipalofoptimalityholdsforstationarypolicies. Lemma19(PrincipalofOptimality). Let𝜙 = (𝜙,𝜙,…)beastationarypolicy. Thenthepolicy isoptimalif f 𝑉 = 𝑇(𝑉 )forevery𝑥 ∈ 𝑋. 𝜙 𝜙 Beforeexaminingexistenceofoptimalpolicies,notethatthemeasurabilityassumptions alreadyimplytheexistenceofan𝜖-optimalpolicy. FromProposition9.20inBertsekasand Shreve(1978,pg.239),thenon-negativityoftheutilityfunctionisenoughtoassertthe existenceofan𝜖-optimalpolicyusingsimilarargumentsasinthepreviouslemmas. Lemma20. Foreach𝜖 > 0,thereexistsan𝜖-optimalnon-randomizedsemi-Markovpolicy fortheinfinitehorizonproblem. Ifforeach𝑥 ∈ 𝑋thereexistsapolicyfortheinfinitehorizon problem,whichisoptimalat𝑥,thenthereexistsasemi-Markov(randomized)optimalpolicy. Thefactthatexistenceofanyoptimalpolicyissufficientfortheexistenceofasemi- Markovrandomizedoptimalpolicyisimportant. Theprimaryconcerniswiththefirst periodreturnorutilityfunction. Fortheinitialperiod,thesemi-Markov𝜖-optimalpolicy is Markov as clearly 𝜙(𝑑𝑦 |𝑥,𝑥) = 𝜙(𝑑𝑦 |𝑥). If 𝜖-optimality is judged to be sufficient, 𝑡 𝑠 𝑡 𝑡 𝑡 𝑠 𝑡 ∗ ∗ thensimplyuse𝑔(𝜙 )where𝜙 isthefirstelementoftheoptimalpolicy. Theprincipalof 𝑡 𝑡 optimalitywouldonlyholdapproximately,however. Similarly,ifanoptimalpolicydoes actuallyexist,therandomnessisnotanissueastheoptimalpolicyisnon-randominthe firstelement. Inthatcase,theprincipalofoptimalitymaynotholdasequation(2)isonly guaranteedtoholdforstationarypolicies. Consequently,minimaladditionalassumptions areuseful. Beforemakingtheseadditionalassumptions,definethedpalgorithmasfollows: Definition21(DynamicProgrammingAlgorithm). Thealgorithmisdefinedrecursively by 𝑉 (𝑥) = 0 ∀𝑥 ∈ 𝑋 (21) 0 𝑉 (𝑥) = 𝑇(𝑉 (𝑥)) ∀𝑥 ∈ 𝑋, 𝑘 = 0,1,… (22) 𝑘+1 𝑘 Proposition9.14ofBertsekasandShreve(1978)impliesthatthealgorithmconverges fortheproblemasstatedinthefollowinglemma. Lemma22. 𝑉 = 𝑉∗ ∞ Unfortunately,theconvergenceisnotnecessarilyuniformin𝑥. Additionally,itisnot possibletosynthesizetheoptimalpolicyfromthealgorithm,asisthecasefordeterministic problems, because 𝑉 , while universally measurable, is not necessarily semianalytic for 𝑘 all𝑘. Theregularityassumptionsarestrengthenedbyimposingamildboundednessassumption. 20

Criterion23(Boundedness). Assumethat∀𝑖and∀𝑠 ∈ {𝑡,𝑡 +1,…},𝜓 > 0. Furtherassume 𝑖,𝑠 thatthesinglestageutilityfunctioncontainsnopointsofglobalsatiation. AssumingCriterion23leadstostrongerresults. First,underthisboundednesscondition, thedpalgorithmconvergesuniformlyforanyinitialuppersemianalyticfunctionnotjust zero. Furthermore, necessary and sufficient conditions for the existence of an optimal policyareavailable. Notethattheconditionwouldbeviolatedbyquadraticutilityfunctions thathavea“blisspoint”(JappelliandPistaferri,2017). Lemma 24 (Existence). Assume Criterion 23 holds. Then for each 𝜖 > 0, there exists an 𝜖-optimalnon-randomizedstationaryMarkovpolicy. Ifforeach𝑥 ∈ 𝑋thereexistsapolicy fortheinfinitehorizonproblem,whichisoptimalat𝑥,thenthereexistsanuniqueoptimal non-randomizedstationarypolicy. Furthermore,thereisanoptimalpolicyifandonlyiffor each𝑥 ∈ 𝑋thesupremumin sup {𝑔(𝑦)+𝛽 𝑉∗(𝑥′) 𝑡(𝑑𝑥′ | 𝑥,𝑦)} (23) 𝑋 𝑦∈𝑆(𝑥) isachieved. CombiningthislemmawithLemma19impliesthatthereexistsanoptimalpolicyif andonlyifthereexistsastationarypolicysuchthat𝑉 = 𝑇(𝑉 )forevery𝑥 ∈ 𝑋. Itisclear 𝜙 𝜙 thatthe𝜖-optimalnon-randomizedstationaryMarkovpolicyistheuniversallymeasurable selectorfromTheorem9onpage14. Ifuniversallymeasurableselectionisassumedtobe possible(i.e. theset𝐼definedinTheorem9istheentireset𝐷),thenthesupremumwill beachieved. ThisassumptionisweakerthanrequiringthatBorel-measurableselection is possible, as in Stokey and Lucas Jr. (1989). The assumption is also weaker than the regularity conditions needed to solve semicontinuous models in Bertsekas and Shreve (1978),whichhaveBorel-measurableoptimalplans. Thefollowinglemma,whichfollows fromProposition9.17ofBertsekasandShreve(1978)suppliesasufficientconditionforthe supremumtobeachieved. Lemma25. UnderCriterion23,ifthereexistsanon-negativeinteger𝑘suchthatforeach 𝑥 ∈ 𝑋,𝜆 ∈ ℝ,and𝑘 ≥ 𝑘,theset 𝑆 (𝑥,𝜆) = {𝑦 ∈ 𝑆(𝑥) | 𝑔(𝑦)+𝛽 𝑉 (𝑥′) 𝑡(𝑑𝑥′|𝑥,𝑦) ≥ 𝜆} (24) 𝑘 ∫ 𝑘 iscompactin𝑌thenthereexistsanon-randomizedoptimalstationarypolicyfortheinfinite horizonproblem. ThisisaweakerconditionthanassumingthattheconstraintsetsarecompactorthatΓ isupperhemi-continuous. Ifthesupremumin(23)isachievedfortheinitialstate𝑥 ∈ 𝑋, 𝑡 theboundednessassumptionimpliesthatauniquestationarynon-randomMarkovoptimal planexists. 21

6 Stochastic Euler Equations Inthissection,Eulerequationsforthestochasticdecisionarederived. Althoughnecessary and sufficient conditions for the optimum to exist have been established, the stronger characterization given by Euler equations is often needed and is always useful. The usefulnessofstochasticEulerequationsisdiscussedinStokeyandLucasJr.(1989,pg.280- 283). Bellman’sequation,whichhasbeenshowntohold,canbeusedtoderivestochastic Euler equations. Of course, the optimal value function needs to be differentiable. In addition,itmustbepossibletointerchangetheorderofintegration. Theinterchangeis possible,forexample,ifeachpartialderivativeof𝑉isabsolutelyintegrable(Blumeetal., 1982,Lang,1993,StokeyandLucasJr.,1989,Theorem9.10,pg.266-257). Inthepresent case, it is sufficient to show that the value function is differentiable on an open subset containing𝑥,becauseofCriterion23. Thenthevaluefunctionmeetstheconditionsin 𝑡 Mattner(2001),particularlythelocallyboundedassumption,andtheinterchangeisvalid.27 Theenvelopetheoremthenimpliesthefollowingtworesults. First,theoptimalsolution inheritsdifferentiability. Second,thestochasticEulerequationsproposedinBarnettetal. (1997)canbederived. Moregenerally,thisderivationdemonstratesthatstochasticEuler equationscanbederivedfortheclassofmodels. Theorem26(DifferentiabilityoftheValueFunction). If 𝑈( )isconcaveanddifferentiable, thenthevaluefunctionisdifferentiable. Proof. Let𝜙denotetheoptimalstationarynon-randomMarkovpolicy. Notethatattime𝑠, 𝜙isafunctionof𝑥 . Tosimplifynotation,let𝑉(𝑥) = 𝑉 (𝑥). Bellman’sequationimplies 𝑠 𝜙 𝑉(𝑥) = 𝑔(𝜙(𝑥))+𝛽 𝑉[𝑓(𝜙(𝑥),𝑧)] 𝑝(𝑑𝑧|𝑥,𝑦) (25) ∫ 𝑍 holdsforany𝑥. Note𝑥 = 𝑓(𝑦,𝑧), sothevaluefunctionwithintheintegralisbeing 𝑡 𝑡+1 𝑡 𝑡 evaluatedoneperiodintothefuture. Let𝑥0 denotetheactualinitialstate. For𝑥 ∈ 𝑁(𝑥0), where𝑁(𝑥0)isaneighborhoodof𝑥0,define 𝐽(𝑥) = 𝑔(𝑥,𝜙(𝑥0))+𝛽 𝑉[𝑓(𝜙(𝑥0),𝑧)] 𝑝(𝑑𝑧|𝑥,𝑦). (26) ∫ 𝑍 Inwords,𝐽(𝑥)isthevaluefunctionwiththepolicyconstrainedtobetheoptimalpolicy for𝑥0. Clearly,𝐽(𝑥0) = 𝑉 (𝑥0)and,∀𝑥 ∈ 𝑁(𝑥0),𝐽(𝑥) ≤ 𝑉 (𝑥)because𝜙(𝑥0)isnotthe 𝜙 𝜙 optimalpolicyfor𝑥 ≠ 𝑥0. Iftheoriginalutilityfunction𝑈( )isconcaveanddifferentiable thensois𝑢( )andtherefore𝑔( ). Thisassumptionimpliesthat𝐽(𝑥)isalsoconcaveand differentiable. TheenvelopetheoremfromBenvenisteandScheinkman(1979)combined 27Thetheoremactuallyappliestoholomorphicfunctions,buttheproofcanbereadilyadaptedforfirst-order (real)differentiablefunction. 22

withthefactthatthepolicy𝜙isoptimaluniformlyin𝑥thenimpliesthat𝑉 (𝑥)isdiffer- 𝜙 entiable for all 𝑥 ∈ int(𝑋), As prices and rates of return are assumed to be larger than zero,theonlyinitialconditionsforwhich𝑉 (𝑥)isnotdifferentiableareinfinite(positive 𝜙 ornegative)initialassetendowments,whichcanbeexcluded. Theorem27(StochasticEulerEquations). If 𝑈( )isconcaveanddifferentiable. Thenthe stochasticEulerequationsfor (2)are, 𝜕𝑢(𝑎 𝑡 ∗,𝑐 𝑡 ∗) 𝜕𝑢(𝑎 𝑡 ∗,𝑐 𝑡 ∗) 1 𝑝 𝑡 ∗ 𝜕𝑢(𝑎 𝑡 ∗ +1 ,𝑐 𝑡 ∗ +1 ) = − 𝐸 [(1+𝜌 ) ] (27) 𝜕𝑎 𝜕𝑐′ 1+𝜉 𝑡 𝑖,𝑡 𝑝∗ 𝜕𝑐′ 𝑖 𝑡+1 and 𝜕𝑢(𝑎 𝑡 ∗,𝑐 𝑡 ∗) 1 𝑝 𝑡 ∗ 𝜕𝑢(𝑎 𝑡 ∗ +1 ,𝑐 𝑡 ∗ +1 ) = 𝐸 [(1+𝑅) ] (28) 𝜕𝑐′ 1+𝜉 𝑡 𝑡 𝑝∗ 𝜕𝑐′ 𝑡+1 where𝑎∗ and𝑐∗ arethecontrolsspecifiedbythenon-randomizedoptimalstationarypolicy 𝑡 𝑡 and𝑐′ isanarbitrarynuméraire. Tobeclear,theseEulerequationsarenotnew,havingbeenderivedinBarnett(1995) andBarnettetal.(1997). However,thederivationhereformallyestablishestheirvalidity underweakmeasurabilityconditions. Incontrast,priorderivationsassumedthatBellman’s equationappliedandthatitcouldbedifferentiated. AsdiscussedextensivelyinBertsekas andShreve(1978),assumingthatBellman’sequationappliestostochasticproblemsisa materialassumption. FurtherthedifferentiabilityofastochasticBellman’sequationhad notbeenpreviouslyestablished. TheimplicationsofthesestochasticEulerequationshavebeenextensivelydiscussed andextendedinthemonetaryaggregationliteraturecited. Briefly,toprovidesomeintuition ontheirmeaning,substitutefrom(28)into(27). Thelinearityoftheconditionalexpectations operatorimpliedbythelinearityofthewell-definedintegral,produces 𝜕𝑢(𝑎 𝑡 ∗,𝑐 𝑡 ∗)/𝜕𝑎 𝑖 = 1+ 1 𝜉 𝐸 𝑡 [(𝑅 𝑡 −𝜌 𝑖,𝑡 ) 𝑝 𝑝 ∗ 𝑡 ∗ 𝑢 𝑐′(𝑎 𝑡 ∗ +1 ,𝑐 𝑡 ∗ +1 )] (29) 𝑡+1 where 𝑢 𝑐′(𝑎 𝑡 ∗ +1 ,𝑐 𝑡 ∗ +1 ) = 𝜕𝑢(𝑎 𝑡 ∗ +1 ,𝑐 𝑡 ∗ +1 )/𝜕𝑐′. The first order condition for a simple utility maximization problem for consumption goods is 𝜕𝑢(𝑐)/𝜕𝑐 𝑖 = 𝑝 𝑖.28 Similarly, the right- 𝜕𝑢(𝑐)/𝜕𝑐 𝑝 𝑗 𝑗 handsideof(29)definestherelevantinformationforassets’relativeprices;itimpliesa generalized user cost for risky assets (Barnett et al., 1997). From these stochastic Euler equations,therelativepricesortheusercostsforassetsgenerallydependonatrade-off betweenanasset’srateofreturn,risk,andtheliquidity,ormonetaryservices,theasset providesintermsofitscontributiontoutility. Thethree-waytrade-offgeneralizesmuch 28Thereare,ofcourse,𝑛otherequationsfordifferentiationwithrespecttoelementsof𝑐.Theseequations aresimplerinthattheyarenon-stochastic. 23

of the voluminous asset-pricing literature in finance, where only the two-dimensional trade-offbetweenriskandreturnisconsidered. Theresultsin(29)defineanassetpricing rulethatisastrictgeneralizationoftheconsumptioncapmassetpricingrule. AsshownbyBarnettetal.(1997),thetrade-offsimplifiestoatrade-offbetweenliquidity andexpectedreturnsunderriskneutrality,whichmirrorsthedeterministiccase. Ifthe marginalutilityofconsumptiondefinedinthenuméraire𝑢 𝑐′(𝑎 𝑡 ∗ +1 ,𝑐 𝑡 ∗ +1 )isindependentof thenominalinterestratesthen 𝜕𝑢(𝑎 𝑡 ∗,𝑐 𝑡 ∗)/𝜕𝑎 𝑖 = 1+ 1 𝜉 𝐸 𝑡 [(𝑅 𝑡 −𝜌 𝑖,𝑡 ) 𝑝 𝑝 ∗ 𝑡 ∗ ]𝐸 𝑡 [𝑢 𝑐′(𝑎 𝑡 ∗ +1 ,𝑐 𝑡 ∗ +1 )] (30) 𝑡+1 andtheratiobetweenanytwoassetswouldbe 𝜕𝑢(𝑎 𝑡 ∗,𝑐 𝑡 ∗)/𝜕𝑎 𝑖 (𝑅 𝑡 −𝜌 𝑖,𝑡 ) = 𝐸 [ ]. (31) 𝜕𝑢(𝑎 𝑡 ∗,𝑐 𝑡 ∗)/𝜕𝑎 𝑗 𝑡 (𝑅 𝑡 −𝜌 𝑗,𝑡 ) Thisratioisthesameasthedeterministiccase,exceptthatitdependsonexpectedvalues. Fromtheearlierdiscussion,undercertaintyequivalencesolvingadeterministicproblem using expected values replicates the stochastic solution. Therefore, imposing certainty equivalenceimpliestheindependenceofmarginalconsumptionfrominterestrates,which illustrates how certainty equivalence makes risk largely inconsequential for stochastic decisionproblems.29 7 Conclusion LjungqvistandSargent(2004,pg.19)arguethatincreasingtherangeofproblemsamenable torecursivetechniqueshasbeenoneofthekeyadvancesinmacroeconomictheory. They refer to the “art” of choosing the right state variables so that a problem can be solved through recursive techniques as crucial to this advance (pg. 16). This paper takes a differentapproachtowardsthesamegoal;thearthasbeenindefiningthecharacteristics of the state and control spaces. The choice of space and the subsequent measurability assumptionsallowsmiufmodelstobesolvedthroughthedprecursion. Theresultsmirror those that are available for deterministic dynamic problems: an unique solution exists thatcanbedifferentiatedtoderive(stochastic)Eulerequations. Theresultsfirmupthe mathematicalfoundationsofthemonetaryaggregationliteratureandotherresearchthat employssmiufmodelsorstochasticmodelsthatarespecialcasesofthesmiufmodel,such asconsumptioncapm. Themethodusedinthispaperrequiresregularityconditionsthatarelessrestrictive thanotherapproaches. Evenmoreimportantly,theregularityconditionsdonotrestrictthe economicsoftheproblem. Consequently,theresultsarebroadlyapplicabletomonetaryand 29Asquadraticutilitygenerallyexhibitsincreasingriskaversion(JappelliandPistaferri,2017), further researchisneededtoexploretherelationshipbetweenriskneutralityandcertaintyequivalenceinthisdecision problem,aseitherassumptionresultsinequation(31)holding. 24

financialmodels,particularlythemanymodelswheretheexistenceofasolutionwasjust assumed. Theapproachtomodelinguncertaintycanbeappliedtootherstochasticeconomic models, but applying risk to financial assets is the clear starting point for introducing contemporaneousuncertainty. ExtendingthemethodtoderivestochasticEulerequations increases the utility of Bertsekas and Shreve’s measurabilty approach to stochastic dp. Consequently,moremonetaryandfinancialmodels,whereriskplaysacentralrole,canbe solvedwithoutimposingrestrictiveconditions. Furthermore,theapproachcanbeapplied broadlytootherdecisionproblemswithcontemporaneousuncertainty: forexample,utility maximizationproblemswherethecurrentpricesofgoodsandservicesareuncertain,due tosearchcostsorsomeotherinformationalfriction. The smiuf problem integrates monetary and finance models, containing important examplesfromeachasspecialcases. Furtherworkonintegratingaspectsoffinanceinto modelsofmoneycouldaddressboththefactthattechnologicalandtheoreticaladvances havebeensteadilyincreasingtheliquidityofriskyassetsandthefactthereislittleconsensus onhowtomodelrisk. Inparticular,thedevelopmentofcrypto-currenciesandstablecoins makes being able to model risky money even more important. Consequently, formally establishing the underpinnings of the expected utility framework, which is the most commonlyusedapproach,iscritical. Theresearchcouldalsobenefitfromfurtherwork, likeinBarnettetal.(2021),toadaptsmiufmodelsforalternativemodelsofrisk,particularly iftheexpectedutilityframeworkseemslessappropriatetonewdevelopmentsindigital money. References Akashi,Hajime,andKazuoNose.1975,“Oncertaintyequivalenceofstochasticoptimal controlproblem.”Int.J.Control,21(5): 857–863.doi:10.1080/00207177508922040. Aliprantis,CharalambosD.,andKimC.Border.2006,InfiniteDimensionalAnalysis: AHitchhiker’sGuide.HeidelbergandNewYork: Springer,3edition.doi:10.1007/3-540- 29587-9. Alvarez,Fernando,andUrban J.Jermann.2004,“Usingassetpricestomeasurethecost ofbusinessCycles.”J.Polit.Econ.,112(6): 1223–1256.doi:10.1086/424738. Amman,Hans.1996,“Numericalmethodsforlinear-quadraticmodels.”InHandbookof ComputationalEconomics,volume1,chapter13,587–618,Elsevier.doi:10.1016/S1574- 0021(96)01015-5. Anderson,EvanW.,EllenR.McGrattan,LarsPeterHansen,andThomasJ.Sargent. 1996,“Mechanicsofformingandestimatingdynamiclineareconomies.”InHandbook ofComputationalEconomics,volume1,chapter4,171–252,Elsevier.doi:10.1016/S1574- 0021(96)01006-4. 25

Anderson, Richard G., Barry E. Jones, and Travis D. Nesmith. 1997, “Monetary aggregationtheoryandstatisticalindexnumbers.”Fed.ReserveBankSt.LouisRev.,79(1): 31.doi:10.20955/r.79.31-52. Andrikopoulos,Athanasios.2013,“Compactnessinthechoiceandgametheories: A characterizationofrationality.”Econ.TheoryBull.,1: 105–110.doi:10.1007/s40505-013- 0002-8. Aruoba,S.Borağan,JesúsFernández-Villaverde,andJuanF.Rubio-Ramírez.2006, “Comparing solution methods for dynamic equilibrium economies.” J. Econ. Dynam. Control,30(12): 2477–2508.doi:10.1016/j.jedc.2005.07.008. Bansal,Ravi,andWilburJohnColemanII.1996,“Amonetaryexplanationoftheequity premium,termpremium,andrisk-freeratepuzzles.”J.Polit.Economy,104(6): 1135–71. doi:10.1086/262056. Barnett,WilliamA.1977,“PollakandWachteronthehouseholdproductionfunction approach.”J.Polit.Economy,85(5): 1073–1082.doi:10.1086/260623. Barnett, William A. 1980, “Economic monetary aggregates: An application of index number and aggregation theory.” J. Econometrics, 14(1): 11–48. doi:10.1016/0304- 4076(80)90070-6. Barnett,WilliamA.1995,“Exactaggregationunderrisk.”InSocialchoice,welfare,and ethics: Proc.EighthInt.Symp.Econ.TheoryEconometrics,eds.WilliamA.Barnett,Hervé Moulin, Maurice Salles, and Norman J. Schofield, 353–74, International Symposia in EconomicTheoryandEconometrics,CambridgeUniversityPress. Barnett,WilliamA.,MarcelleChauvet,DaniloLeiva-Leon,andLitingSu.2023,“The credit-card-servicesaugmentedDivisiamonetaryaggregates.”J.Money,Credit,Banking. doi:10.1111/jmcb.13088. Barnett,WilliamA.,QingHan,andJianboZhang.2021,“Monetaryservicesaggregationunderuncertainty: AbehavioraleconomicsextensionusingChoquetexpectation.” JEconBehavOrgan,182: 437–447.doi:10.1016/j.jebo.2019.03.026. Barnett,WilliamA.,KunHe,andJingtongHe.2022,“Consumptionloanaugmented DivisiamonetaryindexandChinamonetaryaggregation.”JRiskFinanManage.,15(10). doi:10.3390/jrfm15100447. Barnett, William A., Melvin J. Hinich, and Piyu Yue. 1991, “Monitoring monetary aggregatesunderriskaversion.”InMonetaryPolicyonthe75th AnniversaryoftheFederal ReserveSystem: ProceedingsoftheFourteenthAnnualEconomicPolicyConferenceofthe FederalReserveBankofSt.Louis,ed.MichaelT.Belongia,189–222.doi:10.1007/978-94- 011-3888-8_10. 26

Barnett, William A., Melvin J. Hinich, and Piyu Yue. 2000, “The exact theoretical rational expectations monetary aggregate.” Macroecon. Dynam., 4(2): 197–221. doi:10.1017/S1365100500015030. Barnett,WilliamA.,andJinanLiu.2019,“Usercostofcreditcardservicesunderriskwith intertemporalnonseparability.”J.Finan.Stab.,42: 18–35.doi:10.1016/j.jfs.2019.05.005. Barnett, William A., and Yi Liu. 2000, “Beyond the risk neutral utility function.” In DivisiaMonetaryAggregates: TheoryandPractice,eds.MichaelT.BelongiaandJaneE. Binner,volume1,chapter1,11–27,London: Palgrave.doi:10.1057/9780230288232. Barnett, William A., Yi Liu, and Mark Jensen. 1997, “CAPM risk adjustment for exact aggregation over financial assets.” Macroecon. Dynam., 1(2): 485–512. doi:10.1017/S136510059700308. Barnett,WilliamA.,andHyunPark.2024a,“Havecreditcardservicesbecomeimportant tomonetaryaggregation? AnapplicationofsignrestrictedBayesianVAR.”J.Appl.Econ., 27(1): 2321422.doi:10.1080/15140326.2024.2321422. Barnett, William A., and Hyun Park. 2024b, “Nonseparability of credit card services within Divisia monetary aggregates.” Macroecon. Dyn., 1–28. doi:10.1017/S1365100524000245. Barnett,WilliamA.,andSoheePark.2023,“Forecastinginflationandoutputgrowth withcredit-card-augmentedDivisiamonetaryaggregates.”J.Forecast.,42(2): 331–346. doi:10.1002/for.2910. Barnett, William A., and Apostolos Serletis, eds. 2000, The Theory of Monetary Aggregation,ContributionstoEconomicAnalysis,volume245.Amsterdam;NewYorkand Oxford: ElsevierScience,North-Holland.doi:10.1108/S0573-8555. Barnett, William A., and Liting Su. 2019, “Risk adjustment of the creditcard augmented Divisia monetary aggregates.” Macroecon. Dynam., 23: 90–114. doi:10.1017/S1365100518000160. Barnett,WilliamA.,andLitingSu.2020,“Financialfirmproductionofinsidemonetary andcreditcardservices: Anaggregationtheoreticapproach.”Macroecon.Dynam.,24(1): 130–160.doi:10.1017/S1365100518000767. Barnett,WilliamA.,andShuWu.2005,“Onusercostsofriskymonetaryassets.”Ann. Finance,1(1): 35–50.doi:10.1007/s10436-004-0003-6. Barnett, William A., and Haiyang Xu. 1998, “Stochastic volatility in interest rates and nonlinearity in velocity.” Int. J. Syst. Sci., 29(11): 1189–1201. doi:10.1080/00207729808929608. 27

Basak, Suleyman, and Michael Gallmeyer. 1999, “Currency prices, the nominal exchangerate,andsecuritypricesinatwo-countrydynamicmonetaryequilibrium.”Math. Finance,9(1): 1–30.doi:10.1111/1467-9965.00061. Bellman,RichardErnest.1957,DynamicProgramming.Princeton: PrincetonUniversity Press. Benveniste,L.M.,andJ.A.Scheinkman.1979,“Onthedifferentiabilityofthevaluefunctionindynamicmodelsofeconomics.”Econometrica,47(3): 727–32.doi:10.2307/1910417. Bertsekas, Dimitri P. 2000, Dynamic Programming and Optimal Control, volume 1-2. Belmont,Massachusetts: AthenaScientific,4edition. Bertsekas, Dimitri P., and Steven E. Shreve. 1978, Stochastic Optimal Control: The Discrete-TimeCase.NewYork: AcademicPress,Inc. Binner, Jane M., Sajid Chaudhry, Logan Kelly, and James L. Swofford. 2018, ““Risky” monetary aggregates for the UK and US.” J. Int. Money Financ., 89: 127–138. doi:10.1016/j.jimonfin.2018.08.015. Blackwell, D., D. Freedman, and M. Orkin. 1974, “The optimal reward operator in dynamicprogramming.”Ann.Prob.,2(5): 926–941.doi:10.1214/aop/1176996558. Blackwell,David.1965,“Discounteddynamicprogramming.”Ann.Math.Statist.,36(1): 226–235,URLhttps://www.jstor.org/stable/2238089. Blackwell,David.1970,“Onstationarypolicies(withdiscussion).”J.Roy.StatisticalSociety, Ser.A,133: 33–37.doi:10.2307/2343810. Blume, Lawrence, David Easley, and Maureen O’Hara. 1982, “Characterization of optimal plans for stochastic dynamic programs.” J. Econ. Theory, 28(2): 221–34. doi:10.1016/0022-0531(82)90059-X. Bohn,Henning.1991,“Oncash-in-advancemodelsofmoneydemandandassetpricing.” J.Money,Credit,Banking,23(2): 224–42.doi:10.2307/1992778. Boyle,GlennW.,andJamesD.Peterson.1995,“Monetarypolicy,aggregateuncertainty, andthestockmarket.”J.Money,Credit,Banking,27(2): 570.doi:10.2307/2077884. Calvo,GuillermoA.,andCarlosA.Vegh.1995,“Fightinginflationwithhighinterest rates: The small open economy case under flexible prices.” J. Money, Credit, Banking, 27(1): 49–66.doi:10.2307/2077850. Canzoneri,MatthewB.,RobertE.Cumby,andBehzadT.Diba.2006,“HowdomonetaryandfiscalpolicyinteractintheEuropeanMonetaryUnion?” InNBERInternational SeminaronMacroeconomics2004,eds.RichardH.Clarida,JeffreyA.Frankel,Francesco Giavazzi, and Kenneth D. West, MIT Press, URL http://www.jstor.org/ stable/40215051. 28

Chang,Roberto.1998,“Crediblemonetarypolicyinaninfinitehorizonmodel: Recursive approaches.”J.Econ.Theory,81(2): 431–61.doi:10.1006/jeth.1998.2395. Chari, V. V., Lawrence J. Christiano, and Martin Eichenbaum. 1998, “Expectation trapsanddiscretion.”J.Econ.Theory,81(2): 462–92.doi:10.1006/jeth.1998.2421. Choi,WoonGyu,andSeonghwanOh.2003,“Amoneydemandfunctionwithoutput uncertainty,monetaryuncertainty,andfinancialinnovations.”J.Money,Credit,Banking, 35(5): 685–709,URLhttps://www.jstor.org/stable/3649823. Davidson,James.1994,StochasticLimitTheory: AnIntroductionforEconometricians.New York: OxfordUniversityPress. DenHaan,WouterJ.1990,“TheoptimalinflationpathinaSidrauski-typemodelwith uncertainty.”J.Monet.Econ.,25(3): 389–409.doi:10.1016/0304-3932(90)90060-H. Derpich, Milan S., and Serdar Yüksel. 2023, “Dual effect, certainty equivalence, and separationrevisited: Acounterexampleandarelaxedcharacterizationforoptimality.” IEEETrans.Autom.Control,68(2): 1259–1266.doi:10.1109/TAC.2022.3151189. Díaz-Giménez, Javier. 2001, “Linear quadratic approximations: An introduction.” In ComputationalMethodsfortheStudyofDynamicEconomies,chapter2,13–29,Oxford UniversityPress.doi:10.1093/0199248273.003.0002. Drake, Leigh, Adrian R. Fleissig, and Andy Mullineux. 1999, “Are “Risky Assets” substitutes for “Monetary Assets”?” Econ. Inq., 37(3): 510–526. doi:10.1111/j.1465- 7295.1999.tb01445.x. Drake, Leigh, Andy Mullineux, and Juda Agung. 2003, “Incorporating risky assets in Divisia monetary aggregates.” Bul. Ekon. Monet. dan Perbank., 3(1): 98–120. doi:10.21098/bemp.v3i1.289. Drouhin,Nicolas.2020,“Non-stationaryadditiveutilityandtimeconsistency.”J.Math. Econ.,86: 1–14.doi:10.1016/j.jmateco.2019.10.005. Duan,Rongrui,JaneM.Binner,JamesJ.Swofford,andAnandadeepMandal.2023, “Anewgreenmeasureofmoney: Theoryandpractice.”Technicalreport,Universityof Birmingham. Duchan, Alan I. 1974, “A clarification and a new proof of the certainty equivalence theorem.”Int.Econ.Rev.,15(1): 216–224.doi:10.2307/2526101. Dudley,RichardM.1989,RealAnalysisandProbability.TheWadsworth&Books/Cole mathematicsseries,PacificGrove,Calif.: Wadsworth&Brooks/Cole,AdvancedBooks& Software. Dupor,Bill.2003,“Optimalrandommonetarypolicywithnominalrigidity.”J.Econ.Theory, 112(1): 66–78.doi:10.1016/S0022-0531(03)00100-5. 29

Dutkowsky, Donald H., and Robert M. Dunsky. 1996, “Intertemporal substitution, money, and aggregate labor supply.” J. Money, Credit, Banking, 28(2): 216–32. doi:10.2307/2078024. Dynkin,E.B.,andA.A.Juskevic.1975,ControlledMarkovProcessesandtheirApplications. Moscow: Springler-Verlag. Elger,Thomas,andJaneM.Binner.2004,“TheUKhouseholdsectordemandforrisky money.”Cont.Macroecon.,4(1): 1–20.doi:10.2202/1534-5998.1136. Feenstra,RobertC.1986,“Functionalequivalencebetweenliquiditycostsandtheutility ofmoney.”J.Monet.Econ.,17(2): 271.doi:10.1016/0304-3932(86)90032-2. Fernández-Villaverde,J.,J.F.Rubio-Ramírez,andF.Schorfheide.2016,“Solution andestimationmethodsforDSGEmodels.”InHandbookofMacroeconomics,eds.JohnB. Taylor and Harald Uhlig, volume 2, chapter 9, 527–724, Elsevier. doi:10.1016/bs.hesmac.2016.03.006. Finn,MaryG.,DennisL.Hoffman,andDonE.Schlagenhauf.1990,“Intertemporal asset-pricingrelationshipsinbarterandmonetaryeconomies: Anempiricalanalysis.”J. Monet.Econ.,25(3): 431–51.doi:10.1016/0304-3932(90)90062-9. Gutiérrez,JoséManuel.2009,“Acharacterizationofcompactnessthroughpreferences.” Math.SocialSci.,57(1): 131–133.doi:10.1016/j.mathsocsci.2008.08.002. Hansen, Lars Peter, and Kenneth J. Singleton. 1983, “Stochastic consumption, risk aversion,andthetemporalbehaviorofassetreturns.”J.Polit.Economy,91(2): 249–65, URLhttps://www.jstor.org/stable/1832056. Hinderer,K.1970,FoundationsofNonstationaryDynamicProgrammingwithDiscreteTime Parameter.NewYork: Springer-Verlag. Hodrick, Robert J., Narayana R. Kocherlakota, and Deborah Lucas. 1991, “The variability of velocity in cash-in-advance models.” J. Polit. Economy, 99(2): 358–84. doi:10.1086/261754. Holman, Jill A. 1998, “GMM estimation of a money-in-the-utility-function model: The implications of functional forms.” J. Money, Credit, Banking, 30(4): 679–698. doi:10.2307/2601124. Imrohoroglu, Selahattin. 1994, “GMM estimates of currency substitution between the Canadian dollar and the U.S. dollar.” J. Money, Credit, Banking, 26(4): 792–807. doi:10.2307/2077947. Jappelli,Tullio,andLuigiPistaferri.2017,“Thecertaintyequivalencemodel.”InThe EconomicsofConsumption: TheoryandEvidence,chapter4,65–81,OxfordUniversity Press.doi:10.1093/acprof:oso/9780199383146.003.0004. 30

Karatzas,Ioannis,andStevenE.Shreve.1998,MethodsofMathematicalFinance.New YorkandHeidelberg: Springer. Lang,Serge.1993,RealandFunctionalAnalysis.NewYork: Springer. Ljungqvist,Lars,andThomasJ.Sargent.2004,RecursiveMacroeconomicTheory.CambridgeandLondon: MITPress,2nd edition. Lucas Jr., Robert E. 1978, “Asset prices in an exchange economy.” Econometrica, 46(6): 1429–45.doi:10.2307/1913837. LucasJr.,RobertE.1990,“Liquidityandinterestrates.”J.Econ.Theory,50(2): 237–264. doi:10.1016/0022-0531(90)90001-Z. Luenberger,DavidG.1969,OptimizationbyVectorSpaceMethods.WileyProfessional, Wiley-Interscience. Matsuyama,Kiminori.1990,“Sunspotequilibria(rationalbubbles)inamodelofMoneyin-the-Utility-Function.”J.Monet.Econ.,25(1): 137–44.doi:10.1016/0304-3932(90)90049-A. Matsuyama,Kiminori.1991,“Endogenouspricefluctuationsinanoptimizingmodelofa monetaryeconomy.”Econometrica,59(6): 1617–31.doi:10.2307/2938282. Mattner, Lutz. 2001, “Complex differentiation under the integral.” Nieuw Arch. Wisk., 2(1): 32–35, URL http://www.math.leidenuniv.nl/~naw/serie5/ deel02/mrt2001/pdf/mattner.pdf. Mehra, Rajnish, and Edward C. Prescott. 1985, “The equity premium: A puzzle.” J. Monet.Econ.,15(2): 145–61.doi:10.1016/0304-3932(85)90061-3. Nesmith, Travis D. 2007, “Rational seasonality.” In Functional Structure Inference, eds. William A. Barnett and Apostolos Serletis, International Symposia in Economic TheoryandEconometrics,volume18,227–255,Leeds: EmeraldGroupPublishingLimited. doi:10.1016/S1571-0386(07)18011-X. Page,FrankH.,andMyrnaHoltzWooders.1996,“Anecessaryandsufficientcondition forthecompactnessofindividuallyrationalandfeasibleoutcomesandtheexistenceof anequilibrium.”Econ.Letters,52(2): 153–162.doi:10.1016/S0165-1765(96)00855-5. Parra-Alvarez,JuanCarlos,HamzaPolattimur,andOlafPosch.2021,“Riskmatters: Breakingcertaintyequivalenceinlinearapproximations.”J.Econ.Dynam.Control,133: 104248.doi:10.1016/j.jedc.2021.104248. Poterba,JamesM.,andJulioJ.Rotemberg.1987,“Moneyintheutilityfunction: An empiricalimplementation.”InNewapproachestomonetaryeconomics: Proc.SecondInt. Symp.Econ.TheoryEconometrics,219–40,InternationalSymposiainEconomicTheory andEconometrics.doi:10.1017/CBO9780511759628.007. 31

Pratt,JohnW.1964,“Riskaversioninthesmallandinthelarge.”Econometrica,32(1/2): 122–136.doi:10.2307/1913738. Restrepo-Tobón, Diego A. 2015, “Evidence that risk adjustment is unnecessary in estimates of the user cost of money.” Ecos De Economía, 19(41): 50–70. doi:10.17230/ecos.2015.41.3. Sargent,ThomasJ.1987,DynamicMacroeconomicTheory.Cambridge,Mass.andLondon: HarvardUniversityPress. Serletis,Apostolos,andLiboXu.2024,“Thecredit-augmentedDivisiaaggregatesand themonetarybusinesscycle.”Macroecon.Dynam.,1–40.doi:10.1017/S1365100523000627. Shiryaev,A.N.1996,Probability.NewYork: Springer. Simon, Herbert A. 1956, “Dynamic programming under uncertainty with a quadratic criterionfunction.”Econometrica,24(1): 74–81.doi:10.2307/1905261. Spear, S., and W Young. 2017, “Generalizations of optimal growth theory: Stochastic models, mathematics, and metasynthesis.” Macroecon. Dynam., 21(2): 515–544. doi:10.1017/S1365100515000590. Speyer,JasonL.,andDonaldE.Gustafson.1974,“Stochasticoptimalcontroloflinear dynamicsystems.”AIAAJ.,12(8): 1013–1020.doi:10.2514/3.49403. Stokey, Nancy L., and Robert E. Lucas Jr. 1989, Recursive Methods in Economic Dynamics. Cambridge, Mass. and London: Harvard University Press. doi:https://doi.org/10.2307/j.ctvjnrt76. Strauch, Ralph E. 1966, “Negative dynamic programming.” Ann. Math. Statist., 37(4): 871–890.doi:10.1214/aoms/1177699369. Takayama, Akira. 1985, Mathematical Economics. Cambridge University Press, 2 edition, URL https://EconPapers.repec.org/RePEc:cup:cbooks: 9780521314985. Tao,Terence.2008,“Compactnessandcompactification.”InThePrincetonCompanionto Mathematics,ed.TimothyGowers,Princeton,NJ: PrincetonUniversityPress. Theil,Henri.1957,“Anoteoncertaintyequivalenceindynamicplanning.”Econometrica, 25(2): 346–349.doi:10.2307/1910260. Tobin,James.1958,“Liquiditypreferenceasbehaviortowardsrisk.”Rev.Econ.Stud.,25(2): 65–86.doi:10.2307/2296205. Townsend,RobertM.1987,“Asset-returnanomaliesinamonetaryeconomy.”J.Econ. Theory,41(2): 219–47.doi:10.1016/0022-0531(87)90018-4. 32

Van de Water, Henk, and Jan C. Willems. 1981, “The certainty equivalence property in stochastic control theory.” IEEE Trans. Autom. Control, 26(5): 1080–1087. doi:10.1109/TAC.1981.1102781. Yemba, Boniface P. 2022, “User cost of foreign monetary assets under dollarization.” Financ.Res.Lett.,49: 103023.doi:10.1016/j.frl.2022.103023. Appendix A.1 ProofofLemma18 Proof. Notethat𝑔(𝑦)isuppersemianalyticandnon-negative. Thisimpliesthat−𝑔(𝑦)is lowersemianalyticandnon-positive. Also ∞ 𝑉̃ (𝑥) = [∑𝛽𝑘(−𝑔(𝑥 ,𝑦 ))]𝑑𝑟(𝜙,𝜇 ) = −𝑉 (𝑥) (32) 𝜙 ∫ 𝑘+𝑡 𝑘+𝑡 𝑥 𝜙 𝑘=0 and𝑉̃∗(𝑥) = inf 𝑉̃ (𝑥) = −𝑉∗(𝑥). ThenfromProposition9.8inBertsekasandShreve 𝜙∈Φ (1978,pg.225), 𝑉̃∗(𝑥) = inf{−𝑔(𝑦)+𝛽 𝑉̃∗(𝑥′)𝑡(𝑑𝑥′ | 𝑥,𝑦)} (33) ∫ 𝑦∈𝑌 𝑋 since−𝑔(𝑦)satisfiestheirassumptionlabeled(N)onpage214. Takingthenegativeofeach sideimpliestheresultasthenegationcanbetakeninsidetheintegral. A.2 ProofofLemma19 Proof. Followingthesameargumentationasinthepreviouslemma,giventhepropertiesof 𝑔(𝑦)theresultholdsfor𝑉̃ (𝑥)byProposition9.13inBertsekasandShreve(1978,pg.228), 𝜙 whichimpliestheresult. A.3 ProofofLemma24 Proof. Criterion23andtheTransversalityconditiongivenin(3)implythat𝑔(𝑦)isbounded over Γ so that ∃𝑏 such that ∀(𝑥,𝑦) ∈ Γ, 𝑔(𝑦) < 𝑏. Since 𝑔(𝑦) ≥ 0, obviously 𝑔(𝑦) > −𝑏. Alsorecallthat𝛽 = 1/(1+𝜉)sothat0 < 𝛽 < 1. Thisimpliesthattheproblemsatisfies theassumptionlabeled(D)inBertsekasandShreve(1978,pg.214). Thelemmafollows fromProposition9.19,Proposition9.12andCorollary9.12.1inBertsekasandShreve(1978, pg.228). 33

A.4 ProofofTheorem27 Proof. UsingthenotationfromtheproofofTheorem26,theenvelopetheoremimpliesthat 𝜕𝑔(𝑥,𝜙(𝑥)) 𝜕𝑔(𝜙(𝑥)) 𝑉 (𝑥0) = 𝐽 (𝑥0) = | = | (34) 𝑥 𝑥 𝜕𝑥 𝜕𝑥 𝑥=𝑥0 𝑥=𝑥0 where𝑉 (𝑥)isavector-valuedfunctionwhose𝑖𝑡ℎ elementisgivenby 𝑥 𝜕𝑔(𝑥,𝜙(𝑥))/𝜕𝑥. (35) 𝑖 Thedifferentiabilityofthevaluefunctioncombinedwiththeabilitytointerchangedifferentiationwithintegrationforthestochasticintegral,implythatthenecessaryconditions for𝜙(𝑥0)tobeoptimalare 𝜕𝑉 (𝑥) 𝑦 | = 0 (36) 𝜕𝑦 𝑦=𝜙(𝑥0) where𝜕𝑉/𝜕𝑦isavector-valuedfunctionwhose𝑖𝑡ℎ elementisgivenby𝜕𝑉/𝜕𝑦. Itfollows 𝑖 thatthestochasticEulerequationis 𝜕𝑔(𝑦) ′ 𝜕𝑓(𝑦,𝑧) +𝛽 𝑉 [𝑓(𝑦,𝑧)] 𝑝(𝑑𝑧 | 𝑥,𝑦)| = 0 (37) ∫ 𝑦 𝜕𝑦 𝜕𝑦 𝑍 𝑦=𝜙(𝑥) where 𝜕𝑔/𝜕𝑦 is a 𝑘 +𝑛+1 vector-valued function whose 𝑖𝑡ℎ element 𝜕𝑔/𝜕𝑦, 𝜕𝑓/𝜕𝑦 is a 𝑖 𝑘+𝑛+1by2𝑘+𝑛+2matrixwith𝑖,𝑗element𝜕𝑓/𝜕𝑦. Equation(34)canbeusedtoreplace 𝑗 𝑖 theunknownvaluefunction,sothat(37)becomes 𝜕𝑔(𝑦) 𝜕𝑔(𝑦)𝜕𝑓(𝑦,𝑧) +𝛽 𝑝(𝑑𝑧 | 𝑥,𝑦)| = 0. (38) ∫ 𝜕𝑦 𝜕𝑥 𝜕𝑦 𝑍 𝑦=𝜙(𝑥) Thesimpleformofthesystemequationimpliesthat 𝜕𝑓/𝜕𝑦 = [𝐼 0 ], (39) (𝑘+𝑛+1)×(𝑘+𝑛+1) (𝑘+𝑛+1)×(𝑘+1) sothat(38)becomes, 𝜕𝑔(𝑦) 𝜕𝑔(𝑦) +𝛽 𝑝(𝑑𝑧 | 𝑥,𝑦)| = 0. (40) ∫ 𝜕𝑦 𝜕𝑥 𝑍 𝑦=𝜙(𝑥) Usingthefactthatthereisaredundantcontrolattheoptimum,anelementcanbeeliminated from𝜙(𝑥). Inparticular,chooseanarbitraryelementof𝑐. Denotethisnuméraireelement by𝑐′,andtheremaining𝑛−1elementsof𝑐by𝑐 . Assumewithoutlossofgeneralitythat − 𝑝′ = 𝑝∗ so that 𝜓′ = 1 where 𝜓′ is the element of 𝜓 that coincides with 𝑐′. To further 34

simplify notation, let 𝑦∗ denote 𝑦 evaluated at the optimum at time 𝑠. Then using the 𝑠 obviousnotation,𝑔(𝑦∗) = 𝑔(𝜃∗,𝑐∗,𝑐′∗)and(40)impliesthat,for𝑖 ∈ {1,…,𝑘}, 𝑠 𝑠 − 𝜕𝑔(𝜃∗,𝑐∗ ,𝑐′∗) 𝜕𝑔(𝜃∗,𝑐∗ ,𝑐′∗) 𝑡 −𝑡 𝑡 𝑡 −𝑡 𝑡 − 𝜕𝜃 𝜕𝑐′ 𝑖 𝜕𝑔(𝜃∗ ,𝑐∗ ,𝑐′∗ ) 𝑡+1 −𝑡+1 𝑡+1 ∗ +𝛽 (𝜓 ) 𝑝(𝑑𝑧|𝑥,𝑦 ) = 0. ∫ 𝑖,𝑡+1 𝜕𝑐′ 𝑍 Also,takingthederivativewithregardsto𝑦 (thebenchmarkasset)implies 𝑘+1 𝜕𝑔(𝜃∗,𝑐∗ ,𝑐′∗) 𝜕𝑔(𝜃∗ ,𝑐∗ ,𝑐′∗ ) − 𝑡 −𝑡 𝑡 +𝛽 (𝜓 ) 𝑡+1 −𝑡+1 𝑡+1 𝑝(𝑑𝑧 | 𝑥,𝑦∗) = 0. (41) 𝜕𝑐′ ∫ 𝑘+1,𝑡+1 𝜕𝑐′ 𝑍 Substitutingtheoriginalnotationprovestheresult. 35