The Fed's Asymmetric Forecast Errors

Finance and Economics Discussion Series Divisions of Research & Statistics and Monetary Affairs Federal Reserve Board, Washington, D.C. The Fed’s Asymmetric Forecast Errors Andrew C. Chang 2018-026 Please cite this paper as: Chang, Andrew C. (2018). “The Fed’s Asymmetric Forecast Errors,” Finance and EconomicsDiscussionSeries2018-026. Washington: BoardofGovernorsoftheFederalReserve System, https://doi.org/10.17016/FEDS.2018.026. NOTE: Staff working papers in the Finance and Economics Discussion Series (FEDS) are preliminary materials circulated to stimulate discussion and critical comment. The analysis and conclusions set forth are those of the authors and do not indicate concurrence by other members of the research staff or the Board of Governors. References in publications to the Finance and Economics Discussion Series (other than acknowledgement) should be cleared with the author(s) to protect the tentative character of these papers.

The Fed’s Asymmetric Forecast Errors Andrew C. Chang* February 15, 2018 Abstract I show that the probability that the Board of Governors of the Federal Reserve Systemstaff’sforecasts(the“Greenbooks”)overpredictedquarterlyrealgrossdomestic product (GDP) growth depends on both the forecast horizon and also whether the forecasted quarter was above or below trend real GDP growth. For forecasted quarters that grew below trend, Greenbooks were much more likely to overpredict real GDP growth, with one-quarter ahead forecasts overpredicting real GDP growth more than 75% of the time, and this rate of overprediction was higher for further ahead forecasts. For forecasted quarters that grew above trend, Greenbooks were slightly more likely to underpredict real GDP growth, with one-quarter ahead forecasts underpredicting growth about 60% of the time. Unconditionally, on average, Greenbooks overpredicted real GDP growth. JEL Codes: C53; D23; E03; E17 Keywords: AsymmetricForecastErrors;FederalOpenMarketCommittee;Forecast Accuracy; Greenbook; Monetary Policy; Real-Time Data *Chang: Senior Economist, Division of Research and Statistics, Board of Governors of the Federal Reserve System. 20th St. NW and Constitution Ave., Washington DC 20551 USA. +1 (657) 464-3286. a.christopher.chang@gmail.com. https://sites.google.com/site/andrewchristopherchang/ †The views and opinions expressed here are ours and are not necessarily those of the Board of Governors of the Federal Reserve System. I thank Stephanie Aaronson, Aditya Aladangady, Felix Galbis-Reig, Paul Lengermann,JeremyB.Rudd,RiccardoTrezzi,andseminarparticipantsattheBoardforhelpfulcomments. I thank Emily G. Massaro for research assistance. Any errors are mine. 1

1 Introduction The Board of Governors of the Federal Reserve System staff (the “staff”) prepare a detailed projection of the US economy for meetings of the Federal Open Market Committee, called the “Greenbooks”. The academic literature has examined whether Greenbooks contain superiorpredictivepowercomparedtoprivateforecasters, time-seriesmodels, orthe“market consensus”.1 In this paper, I take a different view and ask one question that remains surprisinglyunanswered: whataretheconditionalprobabilitiesthatGreenbooksoverpredicted quarterly real gross domestic product (GDP) growth when the forecasted quarter ended up growing at either above or below trend? I depart from conventional measures of forecast accuracy, such as root mean squared error, because Greenbook forecasters may exhibit non-differentiable loss functions due to, among other things, psychological biases that affect their forecasts. These biases are well known in the psychological literature and they can affect master forecasters just as easily as novice psychics (Kahneman and Tversky, 1979). UsingGreenbookforecastsover54years,IfindtheprobabilitythatGreenbooksoverpredicted real GDP growth depends on both the forecast horizon and also whether the forecasted quarter grew above or below trend. Conditional on a forecasted quarter that grew below trend, Greenbooks were much more likely to overpredict real GDP growth, with one quarter ahead forecasts overpredicting real GDP growth more than 75% of the time. This rate of overprediction is higher for further ahead forecasts. Conditional on a forecasted quarter that grew above trend, Greenbooks were slightly more likely to underpredict real GDP growth, with one-quarter ahead forecasts underpredicting growth about 60% of the time. This rate of underprediction is about the same for further ahead forecasts. I cautiously interpret these results as Greenbooks having an asymmetric loss function for real GDP growth forecast errors. 1Examples include Faust and Wright (2009), Arai (2014), Ericsson et al. (2015), and Chang and Hanson (2016). 2

2 Data IuseGreenbookquarterlyrealGDPforecastsfrom1967to2011,thelastyearthatGreenbook data are available. I consider Greenbook forecasts from a one-quarter backcast to a fivequarteraheadforecast(thereforethelatestGreenbookinmydatasetcontainsaGDPforecast ofthefirstquarterof2013). ForactualGDP,IusetheBureauofEconomicAnalysis’s(BEA) first-release estimates. The BEA second-, third-, and October 2017 estimates all give similar results. All data come from the Federal Reserve Bank of Philadelphia real-time data center (Croushore and Stark, 1999).2 Following the recommendations of Chang and Li (2017, Forthcoming-a), replication files for this paper can be found on my website.3 3 Forecast Errors Figure 1 shows the probabilities that the Greenbooks overpredicted real GDP growth, given that the economy turned out to experience below-trend real GDP growth. I define trend as a 5-year moving average, but my results are similar using a 10-year moving average or looking at forecasts conditioned on being above or below zero real GDP growth. The horizontal axis of Figure 2 is the forecast horizon in quarters, where t = 0 indicates a “nowcast” of the current quarter, t = (cid:0)1 indicates a one-quarter “backcast”, and t = 1 indicates one-quarter ahead forecast. TheprobabilitythatGreenbooksoverpredictedrealGDPgrowth,giventhattheforecasted quarter turned out to experience below-trend GDP growth, rises monotonically with the forecast horizon. For current quarter “nowcasts”, given the current quarter ended up below trend, the Greenbook overpredicted GDP about 65% of the time. This probability rises 2Available at: https://www.philadelphiafed.org/research-and-data/real-time-center/real-time-data/ for GDP and https://www.philadelphiafed.org/research-and-data/real-time-center/greenbook-data for Greenbooks. Downloaded on November 28th, 2017. The BEA October 2017 estimates were the latest estimates available on this date. 3https://sites.google.com/site/andrewchristopherchang/research 3

monotonically to just below 90% for five-quarter ahead forecasts. Figure 2 shows the probabilities that Greenbooks overpredicted real GDP growth, given that the forecasted quarter turned out to be above trend. For current quarter “nowcasts”, this probability is about 40 percent, and is about stable throughout the rest of the forecast horizon. Importantly, the probabilities that Greenbooks overpredicted real GDP growth depend on whether the forecasted quarter grew above or below trend. Except for one-quarter “backcasts”,thisdifferenceinconditionalprobabilitiesisstatisticallysignificant. Thepattern is also stable across data vintages and robust to using only Greenbooks since 1990.4 4 Possible Explanations for Asymmetric Forecast Errors 4.1 Uninformed Greenbook Forecasts One hypothesis as to why the Greenbook forecasts exhibit asymmetric forecast errors around trend is that the Greenbooks are completely uninformed about the future state of the economy, so they forecast a naive trend growth rate. However, I do not find this hypothesis to be credible. Bythelawoftotalprobability,theunconditionalprobabilitythatGreenbooksoverpredicted real GDP growth can be decomposed into equation (1). Pr(Overprediction) = Pr(Overprediction j AboveTrendGrowth)(cid:3)Pr(AboveTrendGrowth) + Pr(Overprediction j BelowTrendGrowth)(cid:3)Pr(BelowTrendGrowth) (1) Assuming that Greenbooks had symmetric loss for unconditional overprediction (the outcome of equation (1)) then Greenbooks should have overpredicted fifty percent of the 4The fact that real GDP growth rates are skewed to the left does not affect my findings because I am countingwhetherforecastsareeitheraboveorbelowtrendandignoringthemagnitudeoftheforecasterror. 4

time. However, as shown in Figure 3, past “nowcasts” of the current quarter Greenbooks overpredicted more often than they underpredicted real GDP growth. Furthermore, if Greenbookswerealsocompletelyuninformedinadditiontohavingsymmetriclossforunconditional overprediction, then a reasonable forecasting strategy to ensure that the probability of overprediction is fifty percent would have been to forecast trend growth, as the probability of above trend growth is approximately equal to the probability of below trend growth of fiftypercent.5 ButthisstrategywouldimplythatPr(Overprediction j AboveTrendGrowth) would be close to zero and that Pr(Overprediction j BelowTrendGrowth) would be close to one,whichisnotwhatthedatainFigures1and2show. Therefore,completeuninformativeness cannot explain my findings. 4.2 Asymmetric GDP Source Data Quality A second hypothesis that could explain the Greenbook’s asymmetric forecast errors around trend is that the real-time data that the staff used to forecast real GDP growth were of different quality when the economy was above trend vs. when it was below trend. For example, suppose that when the economy was below trend that the data the staff used to forecast real GDP growth were of poorer quality relative to the quality of the data when the economy was above trend. If this data quality story was the case then, relatively speaking, thestaffhadapoorsignalaboutwhattheeconomywasdoingwhentheeconomywasgrowing below trend, so the natural response would have been to produce a more naive forecast that was closer to, though still below, trend. However, I also do not find this hypothesis to be credible. Although real-time data quality are unobservable, one way to infer real-time data quality is to look at BEA revisions to real GDP growth. The argument for studying revisions is that the BEA both receives and incorporates new source data into its GDP estimate well 5The probabilities of above trend growth and below trend growth are approximately equal regardless of using either a 5 year or 10 year moving average as trend, and also are approximately equal across data vintages. 5

after it publishes its first-release estimate of real GDP growth in the month after the quarter closes. BEA revisions to GDP occur years after the BEA publishes its first-release estimate (Landefeld, Seskin, and Fraumeni, 2008; Chang and Li, Forthcoming-b). If the source data that the BEA receives after it publishes its first-release estimate of real GDP growth improve on its earlier source data, and if this improvement in source data is different for when the economy is above vs. below trend, then we should expect revisions to GDP growth that are different depending on whether the economy was above or below trend. To check for different revisions, I compute revisions between the BEA first-release and October2017estimatesforbothwhenthefirst-releaseestimateswereaboveandbelowtrend. The average magnitude of these revisions are approximately the same for when the economy was above vs. below trend, which casts doubt on the asymmetric GDP source data quality hypothesis.6 4.3 Asymmetric Loss AthirdhypothesisforexplainingtheGreenbook’sasymmetricforecasterrorsisthatGreenbooks have an asymmetric loss function. I cannot test this hypothesis directly, but my negative results for uninformed forecasts in section 4.1 and for asymmetric GDP source data quality in section 4.2 leave asymmetric loss open as a possibility for explaining my findings. 5 Conclusion I document two findings about Greenbook forecasts. First, I find evidence that the probability that Greenbooks overpredict quarterly real GDP growth depends on whether the forecasted quarter grew either above or below trend. The probability of overprediction conditional on a quarter growing below trend is higher than the same probability conditional on the quarter growing above trend. This difference is 6Ialsocomputethedifferencebetweensecond-orthird-releaseestimatesandtheOctober2017estimates, which gives similar results. 6

statistically significant and could suggest that Greenbooks have an asymmetric loss function. Second, I find different forecast-horizon dependence of mistakes in Greenbook forecasts of real GDP growth depending on whether the economy was above or below trend. Over forecast horizons, the Greenbook probability of overpredicting real GDP growth conditional on the economy that was below trend increases with the forecast horizon, but for an economy that was above trend this probability of overprediction is about flat. A caveat to my results is that I treat the Greenbook forecasts as unconditional, as does most of the literature. However, Greenbook forecasts are conditioned on assumed monetary policy. Treating the forecasts as unconditional gives valid results when either the difference between the assumed policy and the Greenbook staff’s unconditional expectation for policy are close or the feedback from economic variables to policy is minimal (Faust and Wright, 2008). 7

Figure 1: The Probability that Greenbooks Overpredicted Real GDP Growth Given the Forecasted Quarter Grew Below Trend 90 80 Percent 70 60 50 −1 0 1 2 3 4 5 Forecast horizon (quarters) 8

Figure 2: The Probability that Greenbooks Overpredicted Real GDP Growth Given the Forecasted Quarter Grew Above Trend 60 50 Percent 40 30 20 −1 0 1 2 3 4 5 Forecast horizon (quarters) Description: The horizontal axis is the forecast horizon in quarters, where t = 0 indicates a “nowcast” of the current quarter. Actual real GDP growth is the BEA first-release. Figure 1 shows the probability that Greenbooks overpredicted real GDP growth given that the forecasted quarter grew below trend. Figure 2 shows the probability that Greenbooks overpredicted real GDP growth given that the forecasted quarter grew above trend. Trend is a five-year moving average of quarterly real GDP growth rates. Dashed blue lines are Clopper and Pearson (1934) 95% confidence intervals. Interpretation: Greenbooks overpredicted real GDP growth more frequently when the forecasted quarter grew below trend than when the forecasted quarter grew above trend, suggesting Greenbooks may have an asymmetric loss function. 9

Figure 3: The Unconditional Probability that Greenbooks Overpredicted Real GDP Growth 65 60 Percent 55 50 45 −1 0 1 2 3 4 5 Forecast horizon (quarters) Description: The horizontal axis is the forecast horizon in quarters, where t = 0 indicates a “nowcast” of the current quarter. Actual real GDP growth is the BEA first-release. Dashed blue lines are Clopper and Pearson (1934) 95% confidence intervals. Interpretation: On average, Greenbooks overpredicted real GDP growth for forecasts past the current quarter. 10

References Arai, Natsuki, “Using Forecast Evaluation to Improve the Accuracy of the Greenbook Forecast,” International Journal of Forecasting 30:1 (2014), 12-19. Chang, Andrew C., and Tyler J. Hanson, “The Accuracy of Forecasts Prepared for the Federal Open Market Committee,” Journal of Economics and Business 83 (2016), 23-43. http://dx.doi.org/10.1016/j.jeconbus.2015.12.001. Chang, Andrew C., and Phillip Li. “A Preanalysis Plan to Replicate Sixty Economics Papers that Worked Half of the Time,” American Economic Review 107:5 (2017), 60-64. Chang, Andrew C., and Phillip Li. “Is Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say “Often Not”,” Critical Finance Review (Forthcominga). Chang, Andrew C., and Phillip Li. “Measurement Error in Macroeconomic Data and Economics Research: Data Revisions, Gross Domestic Product, and Gross Domestic Income,” Economic Inquiry (Forthcoming-b). Clopper, Charles J., and Egon S. Pearson, “The Use of Confidence or Fiducial Limits Illustrated in the Case of the Binomial,” Biometrika 26:4 (1934), 404-413. Croushore, Dean, and Tom Stark, “A Real-Time Data Set for Macroeconomists,” Federal Reserve Bank of Philadelphia Working Paper 99-4 (1999). Ericsson, Neil R., Steadman B. Hood, Fred Joutz, Tara M. Sinclair, and Herman O. Stekler, “Time-dependent Bias in the Fed’s Greenbook Forecasts,” JSM Proceedings, Business and Economics Statistics Section, Alexandria, Virginia, 1568-1582. Faust, Jon, and Jonathan H. Wright, “Efficient Forecast Tests for Conditional Policy Forecasts,” Journal of Econometrics 146 (2008), 293-303. 11

Faust, Jon, and Jonathan H. Wright, “Comparing Greenbook and Reduced Form Forecasts Using a Large Realtime Dataset,” Journal of Business & Economic Statistics 27:4 (2009), 468-479. http://dx.doi.org/10.1198/jbes.2009.07214 Kahneman, Daniel, and Amos Tversky, “Intuitive Prediction: Biases and Corrective Procedures,” Management Science 12 (1979), 313-327. Landefeld, J. Steven, Eugene P. Seskin, and Barbara M. Fraumeni, “Taking the Pulse of the Economy: Measuring GDP,” Journal of Economic Perspectives 22:2 (2008), 193-216. 12