How Well Does Economic Uncertainty Forecast Economic Activity?
Abstract
Despite the enormous reach and influence of the literature on economic and economic policy uncertainty, one surprisingly under-researched topic has been the forecasting performance of economic uncertainty measures. We evaluate the ability of seven popular measures of uncertainty to forecast in-sample and out-of-sample over real and financial outcome variables. We also evaluate predictive content over different quantiles of the GDP growth distribution. Real-time data and estimation considerations are highly consequential, and we devote considerable attention to them. Four main findings emerge. First, there is some explanatory power in all uncertainty measures, with relatively good performance by macroeconomic uncertainty (Jurado, Ludvigson, and Ng, 2015). Second, macro uncertainty has additional predictive content over the widely-used excess bond premium of (Gilchrist and Zakrajsek, 2012) and the National Financial Conditions Index. Third, quantile regressions for GDP growth indicate strong predictive power, especially at the lower ends of the distribution, for all uncertainty measures except the VIX. Finally, we construct new real-time versions of both macroeconomic and financial uncertainty and compare them to their ex-post counterparts used in the literature. Real-time uncertainty measures have comparatively poor forecasting performance, even to the point of overturning some of the conclusions that emerge from using ex-post uncertainty measures. Accessible materials (.zip)
Finance and Economics Discussion Series Divisions of Research & Statistics and Monetary Affairs Federal Reserve Board, Washington, D.C. How Well Does Economic Uncertainty Forecast Economic Activity? John Rogers and Jiawen Xu 2019-085 Please cite this paper as: Rogers, John, andJiawenXu(2019). “HowWellDoesEconomicUncertaintyForecastEconomic Activity?,” Finance and Economics Discussion Series 2019-085. Washington: Board of Governors of the Federal Reserve System, https://doi.org/10.17016/FEDS.2019.085. NOTE: Staff working papers in the Finance and Economics Discussion Series (FEDS) are preliminary materials circulated to stimulate discussion and critical comment. The analysis and conclusions set forth are those of the authors and do not indicate concurrence by other members of the research staff or the Board of Governors. References in publications to the Finance and Economics Discussion Series (other than acknowledgement) should be cleared with the author(s) to protect the tentative character of these papers.
How Well Does Economic Uncertainty Forecast Economic Activity?∗ John Rogers Jiawen Xu International Finance Division Shanghai University of Federal Reserve Board Finance and Economics December 2019 ∗The views expressed here are solely our own and should not be interpreted as reflecting the viewsoftheBoardofGovernorsoftheFederalReserveSystemorofanyotherpersonassociated withtheFederalReserveSystem.
How Well Does Economic Uncertainty Forecast Economic Activity? Abstract Despite the enormous reach and influence of the literature on economic andeconomicpolicyuncertainty,onesurprisinglyunder-researchedtopichas beentheforecastingperformanceofeconomicuncertaintymeasures. Weevaluatetheabilityofsevenpopularmeasuresofuncertaintytoforecastin-sample and out-of-sample over real and financial outcome variables. We also evaluatepredictivecontentoverdifferentquantilesoftheGDPgrowthdistribution. Real-time data and estimation considerations are highly consequential, and we devote considerable attention to them. Four main findings emerge. First, there is some explanatory power in all uncertainty measures, with relatively goodperformancebymacroeconomicuncertainty(Juradoetal.(2015)). Second,macrouncertaintyhasadditionalpredictivecontentoverthewidely-used excess bond premium of Gilchrist and Zakrajsˇek (2012) and the National FinancialConditionsIndex(NFCI).Third,quantileregressionsforGDPgrowth indicate strong predictive power, especially at the lower ends of the distribution, for all uncertainty measures except the VIX. Finally, we construct new real-timeversionsofbothmacroeconomicandfinancialuncertaintyandcomparethemtotheirex-postcounterpartsusedintheliterature. Real-timeuncertaintymeasureshavecomparativelypoorforecastingperformance,eventothe point of overturning some of the conclusions that emerge from using ex-post uncertaintymeasures.
“It’sdifficulttomakepredictions,especiallyaboutthefuture.” —YogiBerra 1 Introduction Research on economic uncertainty over the last decade has been ubiquitous. As made plain from a glance at www.policyuncertainty.com, research on uncertainty is devoted to macroeconomic phenomenon such as inflation and GDP growth, microeconomic issues concerning firm-level investment and export market entry and exit,andfinancetopicssuchascorporatestrategyandequityreturns. Newmeasures reflectuncertaintyinthemindsofconsumers,traders,managers,andpolicymakers about possible futures, and cover events like terrorism, natural disasters, war and climate change. It is difficult to overstate the reach and influence of this literature. Asofthiswriting,Googlescholarcitationcountsforfourprominentarticlesinthis literature are approaching ten thousand (Bloom (2009), Baker et al. (2016), Bloom etal.(2007),andBloometal.(2018)). Surprisingly, little work has focused on the forecasting performance of the various measures of economic uncertainty.1 We fill that gap in the literature in this paper. We consider both in-sample and out-of-sample forecasting, both real and financial outcome variables, and sub-sample stability. We also devote attention to real-time considerations, and find that conclusions concerning forecasting performancedependsignificantlyonthem. Our measures of uncertainty, all for the U.S., sample from the different types thathaveemergedfromthislargeliterature: 1Werecentlybecameawareoftwoexceptionswrittenconcurrently: Hengge(2019)andKalamara etal. (2019). In addition, likeus, Jovanovic and Ma(2019) focus on quantiles of the output distribution,thoughwithamuchdifferentfocusthanours.Manypapersidentifyshockstomeasures ofuncertaintyinVARsandestimatetheirtransmissioneffectsin-sample. Inaddition,Caldaraetal. (2016) examine the interaction between economic uncertainty and financial conditions, also using VARs, while Leduc and Liu (2016) estimate VARs using alternative measures of uncertainty, and matchimpulseresponseswithaDSGEmodel. Theseestimationstrategiesarequitedifferentfrom theforecastingexercisesweperform. 1
Newspaper-based: economic policy uncertainty (EPU) from Baker et al. (2016) andmonetarypolicyuncertainty(MPU)fromHustedetal.(forthcoming); Regression-based: macroeconomic uncertainty (MU) from Jurado et al. (2015), andfinancialuncertainty(FU)fromLudvigsonetal.(forthcoming); Market-based: theVIXasinBloom(2009);and Survey-based: theconsumeruncertaintymeasureofLeducandLiu(2016)andthe professionalforecastersuncertaintyindexofRossiandSekhposyan(2015).2 3 Our measures are available at a monthly frequency with the exception of SPF uncertainty, which is quarterly. We benchmark the forecasting performance of the uncertainty measures by comparing it to the performance of the excess bond premium (EBP) of Gilchrist and Zakrajsˇek (2012) and the Chicago Fed’s National Financial Conditions Index (NFCI), which have been shown to have high predictive powerovermanymacroeconomicvariables.4 Weexaminethemarginalexplanatorypowerofuncertaintyoverabaselineforecastfromadynamicfactormodelofthetypeusedextensivelyintheliteraturewith success (Bai and Ng (2002)). We begin by casting a wide net, examining how well our uncertainty measures forecast each of 128 variables in the updated McCracken and Ng (2016) data set. We show that there is substantial explanatory power, both in-sample and out-of-sample, based on comparison of the baseline dynamic fac- 2TheLeducandLiu(2016)measureisconstructedfromthemonthlyMichiganSurveyquestion: “Speakingnowoftheautomobilemarket–doyouthinkthenext12monthsorsowillbeagoodtime orabadtimetobuyavehicle,suchasacar,pickup,vanorsportutilityvehicle?”andthefollow-up questionastowhy.TheLeduc-Liumeasureisthefractionofrespondentswhoreportthat“uncertain future”isareasonwhyitwillbeabadtimeoverthenext12months. 3The index Rossi and Sekhposyan (2015) propose is constructed from the Philadelphia Fed’s SurveyofProfessionalForecasters(SPF).TheirconstructcomparestheSPFrealizedforecasterror of real GDP growth with the SPF historical forecast error distribution. If the realization is in the tails of the distribution, it is deemed to be very difficult to predict from all available information, implyingthatthemacroeconomicenvironmentishighlyuncertainthen. 4The NFCI provides a comprehensive weekly update on U.S. financial conditions in money markets, debt and equity markets, and the traditional and “shadow” banking systems. The index is constructed to have an average value of zero and a standard deviation of one over a sample period extending back to 1971. Positive values of the NFCI have been historically associated with tighter-than-average financial conditions, while negative values have been historically associated withlooser-than-averagefinancialconditions. 2
tor model to the model augmented with one of the uncertainty measures. Macro uncertainty(MU)doesparticularlywell,onparwithEBPandNFCI.5 Following these initial explorations, we then examine whether uncertainty has any additional predictive content over the widely-used excess bond premium of Gilchrist and Zakrajsˇek (2012). We find that there is added predictability, though only for macro uncertainty. Adding MU to the regressions used by Gilchrist- Zakrajsek, we find that it has the expected sign and is statistically significant in regressions for employment, unemployment, industrial production, non-residential investment,andinventories,evencontrollingforEBP.Theotheruncertaintyproxies dopoorly,whilethepredictivecontentofEBPandNFCIremainshigh. Next we use quantile regressions to examine whether the forecasting performanceofuncertaintymeasuresvariesoverdifferentpartsoftheGDPgrowthdistribution. For example, does uncertainty forecast recessionary conditions better than expansions? There is good reason to expect that it might, in light of important recent work on “Growth at Risk” as well as our Figures 4 and 5, which we return to below.6 We find both in sample and out of sample that several measures of uncertainty show strong predictive power, especially at lower quintiles. In this exercise, MU out-performs all competitors, including EBP and the NFCI measure emphasizedbyAdrianetal.(2019)intheirexaminationofU.S.GDPgrowthquintiles. Finally, we demonstrate that real-time data construction and estimation issues are highly important for reaching conclusions about forecasting performance. Sev- 5Wealsoexaminedthevarianceriskpremium(Bollerslevetal.(2009))andameasureofequity returnskewnessacrossS&P500firms(RTFerreira(2018)). Furthermore,weexaminedbothEPU andallofitssub-indexes: monetarypolicy, fiscalpolicy, taxes, governmentspending, healthcare, nationalsecurity,entitlementprograms,regulation,financialregulation,tradepolicyandsovereign debt (currency crises). We find, but do not report, that uncertainty concerning monetary policy, regulation, and financial regulation have similar in-sample performance as does the general EPU index. Forout-of-sampleforecasting,sub-categoriessuchasmonetarypolicy,regulation,financial regulation,andtradepolicyperformevenbetterthanoverallEPU. 6Adrianetal.(2019)modelthedistributionoffutureU.S.GDPgrowthasafunctionofcurrent financial and economic conditions. They show that the estimated lower quantiles exhibit strong variationasafunctionofcurrentfinancialconditions,whiletheupperquantilesarestableovertime. Adrianetal.(2018)extendthisanalysisto11advancedand10emergingmarketeconomies. 3
eral of our uncertainty measures do not contain values that were, strictly speaking, available in real time. MU, FU, and EBP are all regression based. Their magnitudes are residuals derived through estimation using a full-sample-period data set. The NFCI, an index constructed from 46 weekly, 33 monthly, and 26 quarterly indicators,isalsosubjecttorevisions. Furthermore,thedatasetincludesmanyseries that are themselves continuously revised. This is true of GDP growth, of course, implying that the forecast errors in the SPF are also not strictly-speaking real-time measures. RossiandSekhposyan(2015)provideareal-timeversionoftheiruncertaintymeasure,whichweexamineaswell. The importance of these real-time considerations is foreshadowed in figures 1 and 2. In Figure 1, we display the Jurado-Ludvigson-Ng measure of macroeconomic uncertainty along with our recalculation of that series using a real-time data set and rolling estimation window, as explained in section 6. Both series are scaled such that the index equals 1.0 in January 2000. Notice that real-time macro uncertainty fell between 2000 and 20008, while ex-post uncertainty rose. In JLN’s original series, uncertainty peaks at a level nearly 80% above the starting point, but with our real-time series that rise is greatly attenuated, only about 40% above starting point. In Figure 2 we display quantiles 1 through 5 of the distribution of GDPgrowthagainstmacroeconomicuncertainty. Allseriesarenormalizedtohave a mean of zero and standard deviation equal to 1.0. We display these quantiles for the real-time estimates of the two series (in blue) and for the ex-post measures (in red). The first red bar on the left, for example, displays the average level of expost macro uncertainty (vertical axis) when ex-post GDP growth was in its lowest quantile,andwhatthemeanGDPgrowthwasinthatquantile(horizontalaxis). Althoughforbothex-postandreal-timecasesuncertaintyismuchhigherinthelowest growthquantilethanthehigherones,theex-postGDPgrowthdistributionisnoticeably ”stretched”, with larger values at both the low end and high end, compared to the real-time quantiles. Furthermore, using the ex-post measures, the relationship between uncertainty and growth is monotonically negative for the first four quan- 4
tiles, but using the real-time counterparts gives rise to a see-saw pattern (down, up, down, up) across quantiles. What appears to be unusually high or low uncertainty (andgrowth)withthebenefitofhindsight,wasnotasevidentinrealtime. Figure1RealtimeMUv.s. Ex-postMU The figures suggest that considering how much uncertainty existed in real time versus how much is measured ex-post may affect forecasting performance significantly. Thenewspaper-basedEPUandMPUmeasures,aswellasthemarket-based measures, are closest to real-time series. We level the playing field in our forecast comparison exercises by using our newly-constructed real-time MU and FU measures, as well as the real-time SPF measure. We find that the real-time uncertainty measures, especially MU, fare much worse than their ex-post revised counterparts. This is arguably the main take-away of the paper. To rephrase the great Yankee catcher, we find that, ”Making predictions, even about the future, is less difficult whenyouobservepartofthatfuture.”7 7Related to this, in the forecasting exercises below, we use one month ahead MU and FU to forecast variables at time t+h for h=1,3,12. There is a potential “look-ahead bias” for the case of h=1, however, because the 1-step ahead MU and FU at time t contain information at t+1 by 5
Figure2Quantilesofreal-timeGDPgrowthwithreal-timeMUandex-postGDP growthwithex-postMU2002:I-2018:III In the next section, we further describe our data and the in-sample predictive exercises, and follow that with a description of the out-of-sample forecast tests. In section 3, we estimate the marginal predictive content of uncertainty and NFCI when added to the Gilchrist-Zakrajsek regressions. In section 4, we estimate quantile regressions that allow us to compare predictability across the GDP growth distribution. Thefinalsectionwedevotetocomparisonofthepredictivecontentofthe resultsabovetothoseusinguncertaintymeasuresbasedonreal-timevintagedata. construction. Hencetheimportanceofouranalysisofh>1andourrealtimeexercises. 6
2 Uncertainty Measures and their Predictive Power over a Large Macroeconomic Data Set 2.1 Racehorses: sevenuncertaintymeasuresplusEBPandNFCI InFigure3andTable1,respectively,wedepictourmeasuresofuncertaintyandthe correlationsamongthem. Noticethelargespikesaround2008-09inmostmeasures. CorrelationsaretypicallyquitelargeforallmeasuresexceptMPU.BothNFCIand EBP are highly correlated with most of the uncertainty measures.8 We begin with the “kitchen sink”, examining the predictive power of these uncertainty measures over the 128 monthly macroeconomic and financial time series from the (updated) datasetofMcCrackenandNg(2016). 2.2 In-sample predictive regression We define “predictability” of a particular uncertainty measure as its marginal contributiontothedynamicfactormodelrepresentedbyequation(1): ˆ y =α +φ y (L)y +βϕF(L)F +γ(cid:48)Z +ε y (1) i,t+h i i i,t i t i t i,t+h where y is the transformed variable of interest, one of the time series from the i,t McCrackenandNg(2016)dataset. Similarly,wetransformy ,theh-stepahead i,t+h ˆ forecast, also according to the McCracken-Ng code.9 The F are estimated factors t from the dynamic factor model, with the number of factors selected using the criteria of Bai and Ng (2002). Our benchmark, workhorse dynamic factor model is 8TheresultsforMPUsuggestatleastthattheFed’scommitmenttoazerointerestratepolicy andpre-announcedlarge-scaleassetpurchaseswereeffectiveinkeepinguncertaintyaboutmonetary policyfromexplodinginanenvironmentthatwasotherwiserepletewithuncertainty. 9Thisisanunbalancedmonthlydatasetspanning1959:1-2018:12. Weapplyspecifictransformationstotherawseriesbeforeestimationandconstructthefactorsaccordingtothetransformation code provided in the data file. For example, real personal income (RPI), the first variable in the monthly data set, is transformed by (cid:52)ln(x). y is defined as y = C(ln(x )−ln(x)), with t t+h t+h h t+h t C=1200 for monthly data andC=400 for quarterly data. For details, see the data appendix of McCrackenandNg(2016). 7
a formidable one, as the literature has shown it to have great forecasting success (Stock and Watson (2006) provide an early survey.). The Z term contains, altert nately,oneofthesevenuncertaintymeasuresdescribedabove,EBP,andNFCI.10 Thepredictiveregression(1)isestimatedbyOLS,with4lagsofy(cid:48) sand2lags i,t ˆ ofF.11 Thein-samplepredictivecontentoftheaforementioneduncertaintyindexes t is measured by the t-statistics of γ computed using HAC standard errors. Table 2 i summarizes the number of series with significant indexes for h = 1,3,12.12 Each column reports the number of significant series for different forecast horizons h. MU does well across all horizons, while EBP and NFCI also have good predictive content. EPU has relatively less predictive power than other indexes, but it does improveasthehorizonincreases. 2.3 Out-of-sample forecasting In our out-of-sample forecasting exercise we use data from 1990:1-1999:12 for insample estimation and model selection, and the rest of the data for out-of-sample forecast accuracy evaluation. We compute the h-step ahead mean squared forecast error(MSFE)foreachmodel j andseriesi. MSFEh = 1 T ∑ 2 −h (y − ∧ y j )2 i,j T −T −h+1 i,t+h i,t+h|t 2 1 t=T 1 ∧j where y is the h-step ahead forecast of y in model j computed using the dii,t+h|t i,t rectapproach. Parameterestimation,factorestimationandmodelselectionarefully recursive. The first simulated out of sample forecast is made in 1999:12. To construct this forecast, we use only data available from 1990:1. Thus regressions were 10Wealsocomputethefirstprincipalcomponentfromthesetofuncertaintymeasuresandlabel theresultingseriesPC1inthetables. Thisturnsoutnottohavemuchpredictivepowerandsowe donotfocusonitsperformance. 11Wealwayskeep4lagsofy ’sintheregressionandleaveoutthoseinsignificantregressorsin i,t F anditslag. Wereportt-statisticsofZ inthescreenedregression. t t 12The t-statistics for all of the 128 series are not reported due to space constraint but available uponrequest. 8
run for t = 1990:1,...,1999:12−h, then the values of the regressors at t = 1999:12 were used to forecasty . All parameters, factors, and so forth were then re- 1999:12+h estimated, information criteria were recomputed, and models were selected using datafrom1990:1through2000:1,andforecastsfromthesemodelswerethencomputedfory . Thefinalsimulatedoutofsampleforecastismadein2018:6−h 2000:1+h fory . 2018:6 Forecast accuracy is evaluated via the significance of Clark-West test statistics by comparing MSFEs of the competing model j with the benchmark model 0. ∧0 y is the h-step ahead forecast of y using the factor-based benchmark model i,t+h|t i,t (2).The competing model j is a nested model with additional uncertainty index j, j∈{EPU,MU,FU,MPU,VIX,CarU,EBP,NFCI}. ˆ y =α +φ y (L)y +βϕF(L)F +ε y (2) i,t+h i i i,t i t i,t+h We choose the same forecasting horizons as above for the in-sample predictive regressions (h = 1,3,12). In Table 3, we report the number of series with significantly smaller out-of-sample MSFE than the benchmark model.13 By analogy to Table 2, we summarize the number of significant out-of-sample MSFE, by column for the different horizons h. MU again does well, while EBP and NFCI also have strong forecasting power. They perform better than the benchmark in nearly half of the 128 series, an impressive finding in light of results in the literature that the factor-based or diffusion index forecasting model is difficult to beat empirically. As the forecasting horizon increases, EPU tends to perform better and can beat the benchmarkinapproximately1/3ofthe128series. Theconclusionsfromourkitchensinkanalysisthatevaluatesthemarginalperformance of each measure in isolation are that (a) all measures of uncertainty have some predictive content, both in-sample and out-of-sample, and (b) among the uncertaintymeasures,MUdoesbest,equivalenttoEBPandNFCI. 13Out-of-sampleMSFEfortheindividualseriesarenotreportedbutavailableuponrequest. 9
3 Marginal Predictability of Uncertainty over EBP Inthissection,weexamineifthereismarginalpredictivepowerofuncertaintyover EBP and NFCI in the Gilchrist-Zakrajsek (GZ) key regressions (their regression 2, Table 6) with a specific uncertainty measure added. We also add NFCI, which was not in the original GZ regressions, because of the strong evidence presented in Adrianetal.(2019)ofitspredictivepower,includingoverEBP. Thein-samplepredictiveregressionis: p ∧GZ (cid:53)hY =α+∑β (cid:53)Y +γ TS +γ RFF +γ S +γ EBP +γ NFCI +γ UI +ε t+h i t−i 1 t 2 t 3 t 4 t 5 t 6 t t+h i=1 where (cid:53)hY ≡ C ln( Y t+h), h ≥ 0 is the forecast horizon. Here TS denotes t+h h+1 Y t−1 t the “term spread”—defined as the difference between the three-month constantmaturity Treasury yield and the ten-year constant-maturity yield; RFF denotes the t real federal funds rate. The credit spread index is decomposed into two parts: a component that captures systematic movements in default risk of individual firms ∧GZ and a residual component—the excess bond premium, we denote S and EBP t t respectively. UI ∈{EPU,MU,FU,MPU,CarU,VIX} Thefullsampledataisfrom1990:1-2018:6. Thecompleteresultsareintables4 and 5, where we report the coefficients and t-statistics for the uncertainty measure, EBP, and NFCI (the other three variables noted above are included, as in Gilchrist and Zakrajsˇek (2012), but not reported in order to save space). TheY in monthly t regressionsareEMP,UERandIPM,representingprivatenon-farmpayrollemployment; civilian unemployment rate; and index of manufacturing industrial production. In Table 4, we see that NFCI has marginal predictive power over EBP for all threeseriesatallhorizons(h=1,3,12). Inaddition,notethatalloftheuncertainty measures except MU are insignificant for all forecasted series and at all horizons. InTable5,werunregression(2)usingquarterlydataforGDPanditsmaincomponents. Inthetable,C-D(C-NDS)ispersonalconsumptionexpendituresondurable (non-durable) goods; I-RES is residential investments; I-NRS is business fixed in- 10
vestment in structures. The full sample is from 1990:Q1 to 2018:Q2, and forecast horizon is 4 steps. Once again, we see that MU performs quite well, while the other measures of uncertainty have no marginal predictive content and sometimes enter with the wrong sign, as with the VIX. Macroeconomic uncertainty has an impressive degree of predictability for all components, frequencies, and prediction horizons. MUknocksoutthesignificanceofEBPinseveralcases.14 4 GDP Growth Distribution and Uncertainty In this section, we estimate quantile regressions to assess the correlation and predictive content of uncertainty indexes with GDP growth at different quantiles. In Figures 4 and 5 we display the unconditional correlations between GDP growth at differentquantileswithouruncertaintyindexes. Onthehorizontalaxis,wedisplay the average annualized quarterly GDP growth rates at τ=0.1,0.3,0.5,0.7,0.9; on the vertical axis, we show the mean value for each uncertainty index in those quarters when GDP growth is in that particular quantile. The figures show that when GDP growth is low and even negative (τ = 0.1), all uncertainty indexes are quite high, and conversely, when GDP growth is high, the uncertainty indexes are typicallylow. ThisnegativerelationshipismonotonicallysoforEPUandMU. Next, we further analyze whether uncertainty indexes provide additional predictive power, over factors estimated from large macro data set, for different parts of the growth distribution. In order to do so, we run predictive quantile regression of y on x , where x is a vector containing a constant, current and lagged values t+h t t ˆ ofy ,estimatedfactorsF,anduncertaintyindexes. Thequantilecoefficientsβ are t t τ chosentominimizethequantileweightedabsolutevalueoferrors: 14Husted, Rogers, and Sun (2019), show that MPU has strong predictive power for the crosssectionoffirm-levelinvestment. Othermeasuresofuncertaintymayalso. 11
ˆ T−h (cid:0) (cid:1) β =argmin ∑ τ·1 |y −x β |+(1−τ)·1 |y −x β | τ (y t+h ≥xtβ) t+h t τ (y t+h <xtβ) t+h t τ βτ ∈Rk t=1 where 1 denotes the indicator function. We use FRED-QD for factor estimation (.) in this section.15 There are in total 248 series, out of which 125 are used for factor estimation. We exclude EPU from the dataset for factor estimation, and so use 124 ˆ series for factor estimation. The F are estimated using the complete unbalanced t panelfrom1959:Ito2018:IV. InTable6,wereportthequantileregressioncoefficientsandt-statisticsforeach of the uncertainty indexes, including the quarterly SPF uncertainty now, at τ = 0.1,0.3,0.5,0.7,0.9 and for h = 1,4,8. Most uncertainty series are significantly andnegativelyrelatedto1-quarteror4-quarteraheadGDPgrowthrateatthelower quantiles. MU, EBP, and NFCI have the strongest negative relationships at the lowestquantiles,whileEPUandtheVIXhavetheweakest. 5 Summary of the Sub-sample Analysis Macroeconomic time series cover a long time span and when it comes to forecast evaluation, it is usually crucial to consider time variation in parameters. This often leads to improved performance in sub-samples (see Clements and Hendry (1999) and Hendry and Mizon (2005)). Stock and Watson (2009) split data into pre and post1984sub-samplesandfoundsubstantialin-samplepredictivefitimprovements insub-periodsafter1984. Inthissection,wediscusssub-sampleresultsbothbefore and after the 2008 beginning of the financial crisis. The results are reported in Appendixtables. In the “kitchen sink” analysis, EPU performs particularly well in the pre-2008 period but has less predictive content after 2008. Several other indexes perform 15FRED-QDcanalsobedownloadedathttp://research.stlouisfed.org/econ/mccraken/. Itisupdatedeveryquarter. 12
better in the post-2008 period, with the number of series with significant indexes evenincreasingastheforecasthorizonincreases. Whenh=12,MU,FUandNFCI are significant in over 75 out of 128 regressions. The out-of-sample results are mostly consistent with in-sample results: EPU performs better before 2008, as do FUandEBP.Thebestperformingindexespreandpost2008,respectively,areEPU and NFCI. EPU improves upon the benchmark in about 40 out of 128 series. EBP outperformsthebenchmarkin44outof128cases. We also examine the Gilchrist and Zakrajsˇek (2012) regressions for two subsamples: 1985:1-2007:12and2008:1-2018:12. EPUhassignificantpredictivepower and largely displaces that of EBP and NFCI especially for h=1,3 during 1985:1- 2007:12. WealsoreplicatetheGZTable7(quarterlyseries)forsub-samples1985:Q1- 2007:Q4 and 2008:Q1-2018:Q4. EPU is statistically significant and of the correct signforI-NRSinthepre-2008sub-period,butoveralldoesnotappeartohavemuch predictive content. The predictive power of NFCI decreases in this period. In the post-2008 crisis period, all three indexes lose their predictive power compared to thefullsample. We also estimate the quantile regressions over sub-samples. In general, the results are quite similar to the full sample results. Slight differences exist at h = 8. In-sample quantile predictive regression results during 1973:I-2007:IV show that EPU is positively related to GDP growth at quantiles lower than 0.5; EBP is positively related at the lowest quantile. MU and EBP at other quantiles are negatively related to GDP growth. Results for GDP growth during 2008:I-2018:IV indicate that MU, FU and EBP are all significantly and negatively related to GDP growth, especially at short or medium forecast horizons. Overall, the performance of MU isthebest. 13
6 Real-time Data Issues 6.1 Uncertainty in Real-time Our analysis above indicates that MU has strong predictive content, almost always betterthaninfluentialuncertaintymeasureslikeEPUandMPU.However,asnoted above, MU, FU, and EBP are unavailable in real time, as these series are residuals derived through estimation using a full-sample-period data set. Furthermore, the datasetunderlyingconstructionoftheseuncertaintymeasuresincludesmanyseries that are themselves continuously revised. The newspaper-based EPU and MPU measures, as well as the survey measures and market-based VIX, are closest to real-timeseries. Inthissection,weleveltheplayingfieldbyconstructingreal-time indexesforMUandFUandcomparingtheirperformancetotheothers. We begin by reconstructing MU from all vintages of the McCracken-Ng data set, beginning in 1999:08 and ending in 2019:01. Since financial data are never revised, we use only the one financial data set vintage updated to 2018:12.16 All macroandfinancialseriesexceptfor’MZMSL’,’DTCOLNVHFNM’,’DTCTHFNM’, ’INVEST’areusedforfactorestimation. Foreachvintage,weconstructabalanced panel starting from 1978:06 and ending in the corresponding month. Due to data availability, for vintages from 2004:01 and moving forward, we include 120 out of 132 macroeconomic series used in Jurado et al. (2015).17 We also exclude some series no longer reported in earlier vintages of FRED-MD.18 We use the Matlab 16ThankstoSaiMaforprovidingustheupdatedfinancialdatainLudvigsonetal.(forthcoming). 17We exclude ’HWI’, ’HWIURATIO’, ’NAPMPI’, ’NAPMEI’, ’NAPM’, ’NAPMNOI’, ’NAPMSDI’, ’NAPMII’, ’NAPMPRI’, ’VXOCLSx’, ’Agg wkly hours’, ’Currency’, ’ACOGNO’ fromtheoriginalrawdatasetforvariousreasons. ’VXOCLSx’isexcludedfrommacrodatasetbut includedinthefinancialdatasettocalculatefinancialuncertainty. Inhistoricalvintagedatabefore 2014:12,all’HWI’and”NAPM’relatedseriesarenotreported. Also’Aggwklyhours’and’Currency’arenotfoundintheFRED-MDdataset. Vintagedatafor’ACOGNO’startslate,in1992:02, sowedeleteittopreservealongpanel. 18Forvintagesfrom2003:12andgoingback,’DPCERA3M086SBEA’(Realpersonalconsumptionexpenditures)isremoved.Forvintagefrom2003:05andgoingback,’USTPU’(AllEmployees: Trade,Transportation Utilities)isremoved. Forvintagesfrom2002:11goingback,someseriesrelatedtoindustrialproductionsuchas’IPDCONGD’,’IPNCONGD’,’IPBUSEQ’,’IPDMAT’,’IPN- MAT’,’IPB51222S’,’IPFUELS’areremoved.Forvintagesfrom2000:07,seriesrelatedtopersonal 14
and R code posted on Serena Ng’s website to reconstruct the MU index. The estimation and construction procedures are repeated every month on a new vintage of data. We collect the last observation of each MU series, estimated vintage by vintagestartingwith1999:08,toformtherealtimeMUseries. InFigure6,weplot the 1-step, 3-step, and 12-step ahead real time MU (top panel) together with the ex-post MU updated in 2019:02 (bottom panel). The real-time MU series is much smootherthantheoriginalJLNmeasure. Whatappearstobehighuncertaintyabout the macroeconomy in hindsight, was not as apparent in real time. We display the analogousrealtimeandex-postFUseriesinFigure7. Next,wecomparethe”kitchensink”predictivecontentofreal-timeandex-post measures. In Table 7, we report comparison results of the in-sample (top panel) andout-of-sample(lowerpanel)exercises.19 Wereproducetheresultsforourother uncertaintymeasuresinthelowerpartofeachpanel,forcomparisonpurposes. Real timeMUandFUconsistentlyhavemuchweakerforecastingpowerrelativetotheir ex-post variants. Unlike the analysis of earlier sections, which documented the superior forecasting performance of the original, ex-post measures of MU and FU over alternatives like EPU, we find that the real-time variants of MU and FU do no betterthatthealternativemeasuresofuncertainty. 6.2 Forecasting real time macro data Aseparate,butrelated,questionconcernsforecastingreal-timeversionsoftheoutcomevariables. Asiswell-known,macroeconomictimeseriesareforeversubjectto revision. In Table 8 we perform the analogous (to table 7) comparison of real-time uncertainty and ex-post uncertainty, but now for forecasting real-time vintages of theseriesintheMcCracken-Ngdatabase. Wefirstconstructedareal-timebalanced consumption expenditure such as ’PCEPI’, ’DDURRG3M086SBEA’, ’DNDGRG3M086SBEA’, ’DSERRG3M086SBEA’areremoved. 19Forout-of-sampleforecasting,weusedatafrom1999:7-2007:12forin-sampleestimation.The restofthedataareusedforout-of-sampleforecastingevaluation. 15
panelofthemacroseriesinthedataset,overtheperiod1999:7-2018:12. Thisisthe firstannouncedvintagedataforeachmonth. Duetodataavailability,105seriesare includedinthereal-timepanel,insteadof128intheex-postpanel.20 Similartothe above exercises, we run both in-sample predictive regressions and out-of-sample forecasting for those real time macro series. The factors are extracted from the combinedreal-timemacroandfinancialdataset,alongwiththerelevantuncertainty indexessuchasEPU,real-timeMU/FUandex-postMU/FU.Theresults,presented in table 8, once again show a sharp deterioration in forecasting with real-time uncertaintymeasures. Finally, we analyze the in-sample predictability of uncertainty measures over the real-time GDP growth distribution. We carried out a similar factor-based quantile regression as in section 4, but now with factors extracted from our newlyconstructed real-time monthly macro dataset (converted to quarterly frequency). The results are presented in table 9. The top panel displays results for the distributionofreal-timeGDPgrowthquantileswhilethebottompanelisforex-postgrowth quantiles as the outcome variable.21 The results again reveal a decline in forecastingperformanceofreal-timeuncertaintymeasurescomparedtotheirex-postcounterparts. EPU, which is effectively a real-time measure, forecasts real-time GDP growth quantiles much better than it forecasts ex-post quantiles. In the real-time exerciseofthetoppanel,EPUhasmoreexplanatorypowerthaneitherreal-timeor ex-postMUandoftenoutperformsreal-timeFUandSPF. 20The series excluded from the real-time panel are ’DPCERA3M086SBEA’, ’IPD- CONGD’,’IPNCONGD’,’IPBUSEQ’,’IPDMAT’,’IPNMAT’,’IPB51222S’,’IPFUELS’,’HWI’, ’HWIURATIO’, ’USTPU’, ’ACOGNO’, ’WPSFD49207’, ’WPSFD49502’, ’WPSID61’, ’WP- SID62’, ’CUSR0000SAD’, ’CUSR0000SA0L2’, ’PCEPI’, ’DDURRG3M086SBEA’, ’DND- GRG3M086SBEA’,’DSERRG3M086SBEA’,’VXOCLSx’. 21The bottom panel is thus analogous to Table 6, but with a shorter sample due to the (un)availabilityofallvintages. 16
7 Conclusion As influential as the literature on economic, financial, and economic policy uncertaintyhasbeen,surprisinglylittleattentionhasbeendevotedtothepureforecasting performanceofuncertaintymeasures. Weevaluatetheabilityofsevenpopularmeasuresofuncertaintytoforecastin-sampleandout-of-sampleoverrealandfinancial outcomevariables. Wealsoassesssub-samplestabilityandexaminereal-timeconsiderations, as well as examining predictive content over different quantiles of the GDP growth distribution. We find some explanatory power in all uncertainty measures, especially macroeconomic uncertainty (Jurado, Ludvigson, and Ng (2015)). Both traditional regression analysis and quantile regressions for GDP growth indicate that most uncertainty measures have strong predictive power, especially at the lower ends of the growth distribution. A crucial take-away from our analysis is that real-time data considerations are very important in reaching conclusions about forecasting with uncertainty measures. We construct real-time versions of both macroeconomicandfinancialuncertainty,andshowthattheyhavepoorerforecastingperformancesthantheirex-postcounterparts. Ourpapersuggestsanaddendum to the famous quote of the former New York Yankees catcher: ”It is difficult to makepredictions,especiallyaboutthefuture,butlessdifficultwhenyouseepartof thefuturefirst.” References Adrian, Tobias, Federico Grinberg, Nellie Liang, and Sheheryar Malik, The termstructureofgrowth-at-risk,InternationalMonetaryFund,2018. , Nina Boyarchenko, and Domenico Giannone, “Vulnerable growth,” AmericanEconomicReview,2019,109(4),1263–89. 17
Bai, Jushan and Serena Ng, “Determining the number of factors in approximate factormodels,”Econometrica,2002,70(1),191–221. Baker, Scott R, Nicholas Bloom, and Steven J Davis, “Measuring economic policyuncertainty,”TheQuarterlyJournalofEconomics,2016,131(4),1593–1636. Bloom,Nicholas,“Theimpactofuncertaintyshocks,”Econometrica,2009,77(3), 623–685. , Max Floetotto, Nir Jaimovich, Itay Saporta-Eksten, and Stephen J Terry, “Reallyuncertainbusinesscycles,”Econometrica,2018,86(3),1031–1065. Bloom,Nick,StephenBond,andJohnVanReenen,“Uncertaintyandinvestment dynamics,”Thereviewofeconomicstudies,2007,74(2),391–415. Bollerslev, Tim, George Tauchen, and Hao Zhou, “Expected Stock Returns and VarianceRiskPremia,”ReviewofFinancialStudies,2009,22,4463–4492. Caldara, Dario, Christina Fuentes-Albero, Simon Gilchrist, and Egon Zakrajsek, “The macroeconomic impact of financial and uncertainty shocks,” EuropeanEconomicReview,2016,88,185–207. Clements, Mark P and David F Hendry, Forecasting Non-stationary Economic TimeSeries,MITPress,1999. Ferreira, Thiago RT, “Stock Market Cross-Sectional Skewness and Business CycleFluctuations,”InternationalFinanceDiscussionPapers,2018. Gilchrist,SimonandEgonZakrajsˇek,“Creditspreadsandbusinesscyclefluctuations,”AmericanEconomicReview,2012,102(4),1692–1720. Hendry,DavidFandGrahamEMizon,“Forecastinginthepresenceofstructural breaks and policy regime shifts,” in James H Stock and Donald W K Andrews, eds., Identification and Inference for Econometric Models: Essays in Honor of ThomasJ.Rothenberg,CambridgeUniversityPress,2005,pp.481—-502. 18
Hengge,Martina,“Uncertaintyasapredictorofeconomicactivity,”2019. Husted, Lucas, John Rogers, and Bo Sun, “Monetary Policy Uncertainty,” JournalofMonetaryEconomics,forthcoming. Jovanovic,BoyanandSaiMa,“UncertaintyandGrowthDisasters,”2019. Jurado, Kyle, Sydney C Ludvigson, and Serena Ng, “Measuring uncertainty,” AmericanEconomicReview,2015,105(3),1177–1216. Kalamara,Elena,SujitKapadia,Kapetanios,ChrisRedl,andArthurTurrell, “Making text count for macroeconomics: what newspaper text can tell us about theeconomy,”2019. Leduc, Sylvain and Zheng Liu, “Uncertainty shocks are aggregate demand shocks,”JournalofMonetaryEconomics,2016,82,20–35. Ludvigson,SydneyC,SaiMa,andSerenaNg,“Uncertaintyandbusinesscycles: exogenousimpulseorendogenousresponse?,”JournalofFinance,forthcoming. McCracken, Michael W and Serena Ng, “FRED-MD: A monthly database for macroeconomic research,” Journal of Business & Economic Statistics, 2016, 34 (4),574–589. Rossi, Barbara and Tatevik Sekhposyan, “Macroeconomic uncertainty indices based on nowcast and forecast error distributions,” American Economic Review, 2015,105(5),650–55. Stock, James H and Mark Watson, “Forecasting with many predictors,” HandbookofEconomicForecasting,2006,pp.515—-554. and , “Forecasting in dynamic factor models subject to structural instability,” The Methodology and Practice of Econometrics. A Festschrift in Honour of DavidF.Hendry,2009,173,205–261. 19
Table1CorrelationsamongUncertaintymeasures: 1990:1-2018:6 EPU MU FU MPU VIX CarU EBP NFCI EPU 1.00 0.27*** 0.33*** 0.47*** 0.41*** 0.57*** 0.38*** 0.35*** MU 1.00 0.68*** -0.09* 0.60*** 0.33*** 0.67*** 0.85*** FU 1.00 -0.02 0.85*** 0.23*** 0.73*** 0.77*** MPU 1.00 0.07 0.03 0.07 -0.11** VIX 1.00 0.26*** 0.68*** 0.77*** CarU 1.00 0.32*** 0.44*** EBP 1.00 0.76*** NFCI 1.00 Note: The table reports Pearson correlation coefficients between different uncertainty measures from 1990:1-2018:6. ***,**,* denote 1%, 5%, and 10% significance levels, respectively. 20
Table2Summarytableofin-samplepredictiveregressionresults h=1 h=3 h=12 EPU 21 22 33 MU 33 29 41 FU 33 27 30 MPU 20 15 13 VIX 21 12 18 CarU 24 27 26 EBP 38 47 51 NFCI 38 39 23 PC1 34 29 38 Note: Thistablereportsthenumberofseriesforwhichtheindexissignificantininthe predictive regression. We use the complete data span for each measure: EPU from 1985:1-2018:12; MU and FU 1960:7-2018:12; MPU 1985:1-2018:6; VIX 1990:1- 2018:12; CarU 1978:2-2018:12. EBP 1973:1-2018:12; NFCI 1971:1-2018:12. PC1 stands for the first principle component from a dataset containing all uncertainty measures from 1990:1-2018:6 except for EBP and NFCI. The factors are estimated using 128macroand147financialvariablesfrom1960:1-2018:12. Table3Summarytableofout-of-sampleforecasting h=1 h=3 h=12 EPU 35 44 45 MU 56 54 35 FU 34 43 20 MPU 26 14 25 VIX 38 29 28 CarU 34 44 49 EBP 48 45 50 NFCI 55 48 35 Note: ThistablereportsthenumberofserieswithsignificantlysmallerMSFErelative to the benchmark model, i.e. reject the Clark-West test at the 10% significance level. The pseudo out-of-sample forecasting values are computed from 2000:1 to 2018:6. Datafrom1990:1to1999:12areusedforin-sampleestimation. Theparameterestimation,modelselection,andlagordersareestimatedrecursively. Themultiplestepahead forecastsarecomputedusingthedirectapproach. 21
Table4Monthlyin-samplepredictiveregressioninGilchrist&Zakrajsek(2012): 1990:1-2018:6 h=1 h=3 h=12 EMP UER IPM EMP UER IPM EMP UER IPM EBP -0.38*** 10.42*** -3.47*** -0.47*** 11.14*** -2.87*** -0.56*** 7.93*** -1.48** (-2.61) (2.89) (-3.72) (-2.79) (3.53) (-3.24) (-2.40) (3.39) (-1.81) NFCI -0.67** 14.69** -3.70* -0.77** 14.32** -3.59* -0.75 10.70** -3.27 (-2.05) (2.07) (-1.61) (-1.91) (2.18) (-1.46) (-1.25) (1.83) (-1.28) EPU -0.002 0.04 -0.005 -0.001 0.003 -0.0002 0.002 -0.03 0.01 (-0.98) (1.03) (-0.37) (-0.48) (0.09) (-0.01) (0.79) (-0.90) (0.61) EBP -0.41*** 9.76*** -3.49*** -0.48*** 9.92*** -2.76*** -0.51** 6.45*** -1.24* (-2.64) (3.02) (-3.92) (-2.77) (3.59) (-3.11) (-2.08) (2.80) (-1.46) NFCI -0.36 4.07 -2.78* -0.39 4.39 -1.96 -0.24 2.62 -1.90 (-0.99) (0.52) (-1.41) (-0.91) (0.66) (-1.09) (-0.40) (0.45) (-0.79) MU -4.62*** 121.65*** -10.78 -5.45*** 106.66*** -17.46 -6.36*** 81.79*** -13.14* (-2.74) (3.86) (-0.81) (-2.62) (3.75) (-1.20) (-2.83) (3.25) (-1.29) EBP -0.46*** 12.31*** -4.45*** -0.54*** 12.49*** -3.24*** -0.60** 8.65*** -1.54* (-2.99) (3.55) (-4.16) (-3.00) (4.05) (-2.82) (-2.17) (3.23) (-1.59) NFCI -0.75** 16.65** -4.72** -0.84** 15.70*** -3.97** -0.80 11.42** -3.33 (-2.17) (2.20) (-2.12) (-2.03) (2.34) (-1.81) (-1.22) (1.85) (-1.18) FU 0.40 -10.41 7.28** 0.51 -12.25 2.99 0.80 -11.47 1.72 (0.74) (-0.84) (2.13) (0.74) (-1.10) (0.79) (0.57) (-0.88) (0.30) EBP -0.39*** 10.32*** -3.21*** -0.47*** 11.09*** -2.69*** -0.49** 7.37*** -1.11 (-2.57) (2.72) (-3.48) (-2.65) (3.38) (-2.92) (-1.96) (2.93) (-1.19) NFCI -0.73** 16.76*** -4.29** -0.81** 14.52** -3.86** -0.74* 10.27** -3.40* (-2.26) (2.41) (-1.94) (-2.06) (2.26) (-1.68) (-1.30) (1.89) (-1.42) MPU -0.001 0.03 -0.01** -0.001 0.003 -0.01 -0.001 0.002 -0.01 (-0.61) (1.10) (-1.81) (-0.63) (0.20) (-1.00) (-0.48) (0.13) (-1.02) EBP -0.49*** 13.06*** -4.20*** -0.61*** 13.23*** -3.59*** -0.76*** 9.85*** -2.29*** (-3.02) (3.36) (-4.23) (-3.25) (3.96) (-3.74) (-3.28) (4.53) (-3.00) NFCI -0.85** 19.01*** -4.86** -1.01** 18.19*** -4.77** -1.16** 14.68*** -4.69* (-2.31) (2.53) (-1.93) (-2.31) (2.70) (-1.78) (-1.70) (2.34) (-1.57) VIX 0.02* -0.56** 0.15* 0.03** -0.62*** 0.17* 0.06*** -0.73*** 0.22** (1.58) (-2.03) (1.53) (2.20) (-2.64) (1.64) (2.54) (-2.96) (1.95) EBP -0.42*** 11.65*** -3.57*** -0.49*** 11.43*** -2.82*** -0.47** 7.39*** -1.26* (-2.79) (3.20) (-4.03) (-2.89) (3.70) (-3.15) (-1.86) (3.04) (-1.52) NFCI -0.65** 10.30* -3.74* -0.77** 11.52* -4.44* -0.97* 10.67* -4.07* (-1.82) (1.31) (-1.51) (-1.72) (1.58) (-1.59) (-1.46) (1.64) (-1.48) CarU -0.02 1.38** -0.01 -02.0205 0.75* 0.21 0.08** -0.12 0.24** (-0.47) (2.09) (-0.07) (-0.12) (1.29) (0.99) (1.83) (-0.30) (1.66) Note: Thistablereportsthepredictiveregressioncoefficientsandt-statistics. Statisticalsignificanceatthe10%,5%and1%levelsaredenotedby*,**and***,respectively.
Table5QuarterlyIn-samplepredictiveregressioninGilchrist&Zakrajsek(2012): 1990:I-2018:II h=1 h=4 GDP C-D C-NDS I-RES I-NRS GDP C-D C-NDS I-RES I-NRS EBP -0.68* 1.46 -0.11 2.85 -4.40* -0.12 2.58*** -0.05 2.80* -7.43*** (-1.48) (0.83) (-0.36) (1.20) (-1.61) (-0.31) (2.47) (-0.15) (1.55) (-2.64) NFCI -2.56** -9.01** -0.88* -9.48** -5.34 -2.01** -6.30** -0.78* -1.69 -5.64* (-2.22) (-2.30) (-1.52) (-1.81) (-1.13) (-1.93) (-2.13) (-1.36) (-0.42) (-1.29) EPU 0.01 0.04** -0.002 0.09*** -0.02 0.01 0.02 0.001 0.09*** 0.001 (1.06) (1.84) (-0.52) (2.65) (-0.45) (1.16) (1.20) (0.14) (3.16) (0.04) EBP -0.45 2.87* -0.09 6.82*** -4.61** 0.06 3.43*** 0.04 6.57*** -7.17*** (-0.99) (1.47) (-0.31) (2.62) (-1.83) (0.15) (2.94) (0.16) (3.36) (-2.77) NFCI -1.53** -6.02* -0.49 -0.20 0.49 -1.01 -3.78 -0.24 7.33** 0.48 (-1.71) (-1.61) (-0.80) (-0.05) (0.08) (-1.03) (-1.24) (-0.36) (1.72) (0.09) MU -13.43** -33.53*** -4.38** -93.27*** -60.13** -13.41*** -34.43*** -5.76** -88.88*** -59.62*** (-2.20) (-2.48) (-1.89) (-3.32) (-1.92) (-5.03) (-3.99) (-2.12) (-3.42) (-2.51) EBP -0.88* 1.36 -0.40 3.29 -3.57 -0.39 1.43 -0.27 2.05 -5.73** (-1.40) (0.58) (-1.10) (1.28) (-1.25) (-0.78) (1.04) (-0.71) (0.92) (-2.01) NFCI -2.93*** -10.33*** -1.29** -8.83** -4.03 -2.44** -9.16*** -1.15** -3.80 -3.46 (-2.80) (-2.56) (-2.19) (-1.74) (-0.87) (-2.01) (-2.55) (-2.01) (-0.86) (-0.86) FU 2.47 9.01 2.25** 14.44* -13.59 2.68 13.46* 2.13* 27.40*** -17.48** (0.98) (1.02) (2.23) (1.45) (-1.26) (0.99) (1.63) (1.54) (2.59) (-1.98) EBP -0.51 2.36 -0.13 4.24** -3.54* 0.07 3.22*** 0.004 4.17** -6.49** (-1.06) (1.16) (-0.41) (1.65) (-1.33) (0.16) (2.46) (0.01) (1.88) (-2.23) NFCI -2.50** -8.05** -0.93* -5.27 -7.37** -2.05** -6.18** -0.82* 2.49 -6.74** (-2.08) (-2.00) (-1.53) (-0.90) (-1.82) (-1.82) (-1.93) (-1.37) (0.51) (-1.73) MPU -0.001 0.002 -0.001 0.01 -0.04* -0.002 -0.01 -0.001 0.01 -0.03* (-0.12) (0.11) (-0.38) (0.39) (-1.36) (-0.57) (-0.50) (-0.64) (0.50) (-1.42) EBP -1.23*** 0.49 -0.39* 3.28 -5.08** -0.62** 1.51* -0.31 3.06* -7.39*** (-2.51) (0.27) (-1.31) (1.26) (-1.82) (-1.77) (1.29) (-0.98) (1.36) (-2.59) NFCI -3.96*** -14.62*** -1.52*** -12.85** -6.16 -3.23*** -10.91*** -1.50*** -5.83 -5.62 (-3.45) (-3.27) (-2.77) (-1.97) (-1.26) (-3.11) (-3.21) (-3.13) (-1.16) (-1.28) VIX 0.19*** 0.62*** 0.08*** 0.66*** 0.08 0.16*** 0.47*** 0.09*** 0.73*** 0.0001 (3.81) (3.21) (4.15) (2.47) (0.34) (3.28) (3.04) (3.35) (3.12) (0.0008) EBP -0.52 2.31 -0.10 4.39** -4.76* 0.02 3.01*** 0.02 4.24** -7.23*** (-1.25) (1.28) (-0.40) (1.89) (-1.62) (0.04) (2.46) (0.09) (2.12) (-2.58) NFCI -2.28** -8.84** -0.59 -7.05 -4.95 -1.73* -5.96** -0.46 0.41 -4.39 (-1.80) (-2.09) (-0.95) (-1.21) (-0.90) (-1.57) (-1.99) (-0.75) (0.09) (-0.80) CarU -0.07 0.28 -0.11** 0.39 -0.17 -0.07 0.02 -0.12** 0.48 -0.28 (-0.53) (0.69) (-1.86) (0.53) (-0.28) (-0.74) (0.08) (-2.05) (0.76) (-0.51) EBP -0.79** 1.54 -0.19 3.09* -5.52** -0.12 2.78** 0.00 4.30** -8.38*** (-1.77) (0.85) (-0.63) (1.52) (-1.92) (-0.24) (2.13) (0.00) (2.09) (-2.83) NFCI -1.94* -6.54* -0.58 -3.26 -1.09 -1.63* -4.86* -0.69 1.31 -0.29 (-1.54) (-1.55) (-1.06) (-0.64) (-0.22) (-1.40) (-1.52) (-1.09) (0.35) (-0.07) SPF -0.69 2.75 -0.15 11.42* -14.50* 0.51 -0.29 0.12 6.54 -9.55* (-0.45) (0.61) (-0.24) (1.60) (-1.38) (0.31) (-0.09) (0.14) (1.07) (-1.54) Note: Thistablereportsthepredictiveregres2s3ioncoefficientsandt-statistics. Statisticalsignificanceatthe10%,5%and1%levelsaredenotedby*,**and***,respectively. SPFstandsfor theuncertaintyinSPFfour-quarters-aheadforecastsassociatedwithnewsoroutcomesthatare unexpectedlynegative.
Table6In-samplepredictivequantileregressionforGDPgrowth τ 0.1 0.3 0.5 0.7 0.9 h=1 EPU -0.55** -0.18 -0.06 -0.18 -0.07 (-1.71) (-0.95) (-0.71) (-0.99) (-0.51) MU -1.65*** -1.50*** -0.88*** -0.35* 0.60** (-4.99) (-5.54) (-3.92) (-1.35) (1.92) FU -0.43 -0.42** -0.42** -0.06 0.25 (-1.17) (-1.90) (-2.06) (-0.54) (0.86) EBP -0.94*** -0.76*** -0.83*** -0.45** -0.24 (-2.57) (-3.20) (-3.70) (-1.72) (-0.82) NFCI -1.24*** -1.09*** -0.67*** -0.48** -0.18 (-4.89) (-4.12) (-2.90) (-2.05) (-0.67) MPU -0.29 -0.23 -0.32* -0.31** -0.38** (-1.04) (-1.27) (-1.61) (-1.89) (-1.94) VIX -0.30 0.04 0.38** 0.54*** 0.34* (-1.03) (1.24) (1.73) (2.98) (1.46) CarU -0.53* -0.26 -0.24 -0.29 -0.54* (-1.30) (-0.94) (-1.10) (-1.10) (-1.54) SPF -0.33 -0.37* -0.51** -0.18 -0.18 (-1.03) (-1.30) (-1.99) (-0.95) (-0.81) h=4 EPU -0.04 -0.03 -0.05 -0.03 -0.04 (-1.27) (-0.74) (-0.54) (-0.63) (-0.64) MU -1.39*** -1.15*** -0.93*** -0.29* -0.23* (-5.89) (-7.55) (-5.42) (-1.46) (-1.28) FU -0.35** -0.11 -0.05 -0.08 -0.01 (-2.23) (-0.86) (-0.65) (-0.81) (-0.98) EBP -0.47*** -0.43*** -0.23 -0.01 -0.23* (-2.50) (-2.74) (-1.18) (-0.77) (-1.54) NCFI -1.00*** -0.62*** -0.49*** -0.02 -0.06 (-4.34) (-4.34) (-2.81) (-0.95) (-0.79) MPU -0.22* -0.20** -0.31*** -0.23** -0.18** (-1.41) (-1.74) (-3.68) (-2.11) (-1.71) VIX -0.05 -0.11 0.25** 0.31*** 0.19* (-0.76) (-1.12) (1.88) (2.89) (1.40) CarU 0.05 -0.04 -0.10 -0.08 -0.32* (0.85) (-0.66) (-0.82) (-0.67) (-1.62) SPF -0.10 -0.09 -0.38*** -0.36** -0.29** (-0.83) (-0.83) (-2.43) (-2.25) (-1.92) Note: This table reports the quantile regression coefficients and t-statistics for all uncertainty measures, adding one index to the benchmark model individually. The first five measures are from1973:I-2018:IV.MPUisfrom1985:I-2018:II.VIXarefrom1990:I-2018:II.CarUisfrom 1978:I-2018:IV. SPF stands for four-quarters-ahead uncertainty associated with news or outcomesthatareunexpectedlynegative. Statisticalsignificanceatthe10%,5%and1%levelsare denotedby*,**and***,respectively. 24
Table7Real-timeandex-postuncertaintypredictabilityfortheMcCracken-Ng databaseseries. In-sample h=1 h=3 h=12 realtimeMU 28 15 35 ex-postMU 46 37 58 realtimeFU 12 13 43 ex-postFU 29 39 63 EPU 10 20 50 MPU 20 15 15 VIX 36 22 42 CarU 23 32 37 EBP 55 52 69 NFCI 38 46 68 Out-of-sample h=1 h=3 h=12 realtimeMU 27 41 25 Ex-postMU 51 56 37 realtimeFU 30 32 45 Ex-postFU 41 60 50 EPU 35 21 35 MPU 17 11 21 VIX 37 27 37 CarU 23 29 30 EBP 46 41 48 NFCI 55 50 55 Note: Thetoppanelsummarizesthenumberofserieswithsignificantindexesfromin-sample predictive regressions, using data from 1999:7-2018:6. The bottom panel reports the number ofserieswithsignificantsmallerMSFErelativetothebenchmarkmodel,i.e. rejectClark-West testat10%significancelevel. Thepseudoout-of-sampleforecastingvaluesarecomputedfrom 2008:1to2018:6. Datastartingfrom1999:7to2007:12areusedforin-sampleestimation. 25
Table8Real-timeandex-postuncertaintypredictabilityforreal-time McCracken-Ngdatabase. In-sample h=1 h=3 h=12 EPU 11 20 47 realtimeMU 28 25 46 ex-postMU 39 42 55 realtimeFU 24 22 43 ex-postFU 25 30 57 MPU 7 3 16 VIX 38 28 48 CarU 14 20 42 EBP 36 39 57 NFCI 37 48 59 Out-of-sample h=1 h=3 h=12 EPU 15 25 46 realtimeMU 41 44 44 ex-postMU 46 52 55 realtimeFU 26 31 46 ex-postFU 40 48 59 MPU 27 22 20 VIX 32 35 51 CarU 17 25 42 EBP 40 52 58 NFCI 40 57 64 Note: The top panel of this table reports the number of series for which the index is significant in the predictive regression. The full sample is from 1999:9-2018:10. The factors are estimated using full sample data of 105 real time macro and 147 financial variables. The bottom panel reports the number of series with significantly smaller MSFE relative to the benchmark model, i.e. reject the Clark-West test at the 10% significance level. The pseudo out-of-sample forecasting values are computed from 2009:9to2018:6. Datafrom1999:9to2009:8areusedforin-sampleestimation. The parameterestimation,modelselection,andlagordersareestimatedrecursively. 26
Table9Real-timeandex-postGDPgrowthquantileregressions: in-sample prediction τ 0.1 0.3 0.5 0.7 0.9 Real-timeGDPGrowth EPU -0.99*** -0.98*** -0.60*** -0.74*** -0.40*** (-5.81) (-4.45) (-3.22) (-3.42) (-2.38) RealtimeMU -0.47* -0.59** -0.53** -0.19 -0.22 (-1.58) (-2.18) (-1.92) (-1.22) (-1.05) Ex-postMU -0.60*** -0.77*** -0.72*** -0.36* -0.01 (-2.47) (-2.62) (-2.52) (-1.48) (-0.64) RealtimeFU -0.79*** -0.88*** -1.05*** -1.08*** -0.66** (-2.48) (-2.45) (-3.18) (-3.32) (-2.32) Ex-postFU -1.73*** -1.62*** -0.71** -0.004 -0.37* (-4.65) (-5.20) (-2.07) (-0.47) (-1.56) RealtimeSPF -1.49*** -0.77*** -0.26 -0.28* 0.06 (-3.15) (-2.52) (-1.05) (-1.43) (0.52) Ex-postSPF -1.36*** -0.63** -0.38** -0.32* -0.25 (-3.33) (-1.91) (-1.70) (-1.47) (-0.99) Ex-postGDPGrowth EPU -0.21 -0.18 -0.13 -0.30* -0.11 (-1.25) (-1.18) (-0.95) (-1.61) (-0.93) RealtimeMU -0.65** -0.07 0.20 -0.12 -0.27 (-1.70) (-0.79) (0.83) (-0.59) (-1.21) Ex-postMU -0.33 -0.80* -0.40 -0.51* -0.33 (-1.00) (-1.63) (-1.00) (-1.33) (-0.94) RealtimeFU -0.39 -0.38* -0.75*** -0.21 -0.14 (-1.26) (-1.30) (-2.88) (-0.86) (-0.72) Ex-postFU -1.40*** -1.09*** -0.87*** -0.38 -0.46* (-3.44) (-3.62) (-2.67) (-1.09) (-1.42) RealtimeSPF -0.17 -0.29 -0.44** -0.55*** -0.41** (-1.26) (-1.25) (-1.66) (-2.37) (-1.75) Ex-postSPF -0.09 -0.42* -0.62** -0.66** -0.51** (-1.17) (-1.29) (-1.91) (-2.21) (-1.80) Note: This table reports the h=1 quantile regression coefficients and t-statistics for uncertainty measures, adding one index to the benchmark model individually. In the toppanel,thesampleisfrom2002:I-2018:III.Inthebottompanel,datafrom1999:III- 2018:IIIareusedforestimation. Statisticalsignificanceatthe10%,5%and1%levels aredenotedby*,**and***,respectively. 27
Figure3UncertaintyMeasures 28
Figure4BarchartofGDPgrowthwithuncertaintymeasures: Group1 1978:I-2018:III 29
Figure5BarchartofGDPgrowthwithuncertaintymeasures: Group2 1990:I-2018:II 30
Figure6RealtimeMUv.s. ex-postMU 31
Figure7realtimeFUv.s. ex-postFU 32
Figure8realtimev.s. ex-postuncertaintyinSPF 33
Cite this document
John Rogers and Jiawen Xu (2019). How Well Does Economic Uncertainty Forecast Economic Activity? (FEDS 2019-085). Board of Governors of the Federal Reserve System, Finance and Economics Discussion Series. https://whenthefedspeaks.com/doc/feds_2019-085
@techreport{wtfs_feds_2019_085,
author = {John Rogers and Jiawen Xu},
title = {How Well Does Economic Uncertainty Forecast Economic Activity?},
type = {Finance and Economics Discussion Series},
number = {2019-085},
institution = {Board of Governors of the Federal Reserve System},
year = {2019},
url = {https://whenthefedspeaks.com/doc/feds_2019-085},
abstract = {Despite the enormous reach and influence of the literature on economic and economic policy uncertainty, one surprisingly under-researched topic has been the forecasting performance of economic uncertainty measures. We evaluate the ability of seven popular measures of uncertainty to forecast in-sample and out-of-sample over real and financial outcome variables. We also evaluate predictive content over different quantiles of the GDP growth distribution. Real-time data and estimation considerations are highly consequential, and we devote considerable attention to them. Four main findings emerge. First, there is some explanatory power in all uncertainty measures, with relatively good performance by macroeconomic uncertainty (Jurado, Ludvigson, and Ng, 2015). Second, macro uncertainty has additional predictive content over the widely-used excess bond premium of (Gilchrist and Zakrajsek, 2012) and the National Financial Conditions Index. Third, quantile regressions for GDP growth indicate strong predictive power, especially at the lower ends of the distribution, for all uncertainty measures except the VIX. Finally, we construct new real-time versions of both macroeconomic and financial uncertainty and compare them to their ex-post counterparts used in the literature. Real-time uncertainty measures have comparatively poor forecasting performance, even to the point of overturning some of the conclusions that emerge from using ex-post uncertainty measures. Accessible materials (.zip)},
}