Eliciting GDP Forecasts from the FOMC's Minutes Around the Financial Crisis

K.7 Eliciting GDP Forecasts from the FOMC’s Minutes Around the Financial Crisis Neil R. Ericsson Please cite paper as: Ericsson, Neil R. (2015). Eliciting GDP Forecasts from the FOMC’s Minutes Around the Financial Crisis International Finance Discussion Papers 1152. http://dx.doi.org/10.17016/IFDP.2015.1152 International Finance Discussion Papers Board of Governors of the Federal Reserve System Number 1152 November 2015

Board of Governors of the Federal Reserve System International Finance Discussion Papers Number 1152 November 2015 Eliciting GDP Forecasts from the FOMC’s Minutes Around the Financial Crisis Neil R. Ericsson NOTE: International Finance Discussion Papers are preliminary materials circulated to stimulate discussion and critical comment. References to International Finance Discussion Papers (other than an acknowledgment that the writer has had access to unpublished material) should be cleared with the author or authors. Recent IFDPs are available on the Web at www.federalreserve.gov/pubs/ifdp/. This paper can be downloaded without charge from the Social Science Research Network electronic library at www.ssrn.com.

Eliciting GDP Forecasts from the FOMC’s Minutes Around the Financial Crisis Neil R. Ericsson ∗ Abstract: Stekler and Symington (2016) construct indexes that quantify the Federal Open Market Committee’s views about the U.S. economy, as expressed in the minutes of the FOMC’s meetings. These indexes provide insights on the FOMC’s deliberations, especially at the onset of the Great Recession. The current paper complements Stekler and Symington’s analysis by showing that their indexes reveal relatively minor bias in the FOMC’s views when the indexes are reinterpreted as forecasts. Additionally, these indexes provide a proximate mechanism for inferring the Fed staff’s Greenbook forecasts of the U.S. real GDP growth rate, years before the Greenbook’s public release. Keywords: Autometrics, bias, Fed, financial crisis, FOMC, forecasts, GDP, Great Recession, Greenbook, impulse indicator saturation, projections, Tealbook, United States. JEL classifications: E58, C53. Forthcoming in the International Journal of Forecasting. The author is a staff economist in the ∗ Division of International Finance, Board of Governors of the Federal Reserve System, Washington, DC 20551 USA, and a Research Professor of Economics, Department of Economics, The George Washington University, Washington, DC 20052 USA. He may be reached at ericsson@frb.gov and ericsson@gwu.edu. The views in this paper are solely the responsibility of the author and should not be interpreted as necessarily representing or reflecting the views of the Federal Open Market Committee, its principals, the Board of Governors of the Federal Reserve System, or of any other person associated with the Federal Reserve System. This paper uses publicly available information, andonlypubliclyavailableinformation. Itdoesnot useanyinternalorconfidentialFederalReserve Board information, either directly or indirectly. The author is grateful to David Hendry, Aaron Markiewitz, Jaime Marquez, Ellen Meade, J Seymour, Tara Sinclair, Herman Stekler, and Joyce Zickler for helpful discussions and comments. All numerical results were obtained using PcGive Version 14.0B3, Autometrics Version 1.5e, and Ox Professional Version 7.00 in 64-bit OxMetrics Version 7.00: see Doornik and Hendry (2013) and Doornik (2009).

1 Introduction Monetary policy decisions by the Fed’s Federal Open Market Committee (FOMC) have attracted considerable attention in recent years, especially with quantitative easing through large-scale asset purchases and with the introduction of forward guidance; see Bernanke (2012) and Yellen (2012) inter alia. The FOMC’s decisions are based in part on the Greenbook forecasts, which are economic forecasts produced by the Fed’s staff. Romer and Romer (2008), Sinclair, Joutz, and Stekler (2010), Nunes (2013), and Ericsson, Hood, Joutz, Sinclair, and Stekler (2013) inter alia have extensively analyzed the Greenbook forecasts; and Banternghansa and McCracken (2009) and Sheng (2015) have examined the FOMC participants’ own forecasts. SteklerandSymington(2016)proposeacreativeandinsightfulinnovationonsuch existingstudiesofU.S.monetarypolicy. SteklerandSymingtonemployatextualanalysis of the minutes of the FOMC meetings for 2006—2010, developing quantitative indexes that reflect the extent of optimism or pessimism expressed in the FOMC minutes themselves on the current and future outlook for the U.S. economy. Through theseindexesandaccompanyingresearch, SteklerandSymingtonprovidekeyinsights on the views of the FOMC, especially those held at the onset of the recent financial crisis and Great Recession. Specifically, Stekler and Symington find temporary inaccurate assessments by the FOMC of the economy at that time–in part due to inaccurate advance estimates of GDP growth. The current paper shows that Stekler and Symington’s indexes reveal even more. With few exceptions, Stekler and Symington’s indexes imply relatively minor bias in the FOMC’s views when the indexes are reinterpreted as forecasts. Furthermore, these indexes very closely track the Greenbook forecasts of the current-quarter and one-quarter-ahead U.S. real GDP growth rates. Stekler and Symington’s indexes thus provide a proximate mechanism for inferring these Greenbook forecasts, years in advanceofthepublicreleaseoftheGreenbook. TheminutesofanFOMCmeetingare published three weeks after the meeting itself, whereas the Greenbook is not released to the public until at least five years after it is presented to the FOMC. This paper is organized as follows. Section 2 describes Stekler and Symington’s indexes, the Greenbook forecasts, and the data being forecast. Section 3 discusses differentapproachestotestingforforecastbias, anditproposesimpulseindicatorsaturation as a generic test of potentially time-varying forecast bias. Section 4 presents evidence on forecast bias; and it constructs “post-casts” of the 2010 Greenbook forecasts, conditional on Stekler and Symington’s indexes. Section 5 concludes. Before proceeding, it is important to note that the presence of forecast bias (or lack thereof) is consequential, both economically and statistically. That said, the particular sense in which forecast bias is consequential depends in part on whether the Greenbook forecast and Stekler and Symington’s indexes are interpreted as “forecasts” or as “projections”, where “projections” are in the sense of being policy sim- 1

ulations conditional upon a certain set of assumptions. If they are interpreted qua forecasts, then forecast bias implies potential room for improvement in terms of standard performance measures, although forecast improvement may be feasible for unbiased forecasts as well. If the forecasts are interpreted qua projections, then forecast biasimplieslimitedusefulnessoftheforecastsasrepresentinginterestinghypothetical paths for economic policy. With that in mind, the Greenbook forecasts and Stekler and Symington’s indexes are always referred to as “forecasts” below, while recognizing that sometimes they may be more usefully viewed as projections. This broader usage of the term “forecast” is also in line with Clements and Hendry (2002, p. 2): “A forecast is any statement about the future”. 2 The FOMC Minutes Index, Forecasts, and Data This section describes the indexes constructed by Stekler and Symington (2016) from the minutes of the FOMC, some related forecasts, and the data on the U.S. real GDP growthratesbeingforecast. Theconstructionoftheseindexesmayinvolvetruncation and nonlinearities, so this section also discusses those issues. FOMC Minutes Index. Stekler and Symington (2016) (hereafter, S&S) employ a focused textual analysis of the minutes for the 40 FOMC meetings during 2006— 2010. From their textual analysis, S&S construct quantitative indexes that gauge the FOMC’sviewsonthecurrentandfuturestrengthoftheU.S.economy, asexpressedin the FOMC minutes themselves; see S&S (Sections 3.1.1, 3.1.2, and 4.1) in particular. S&S regard these indexes as “measuring the FOMC’s views about the state of the economy” (Section 3.1.1); and the indexes are scaled such that they correspond to GDP growth rates. To design their indexes, S&S examine certain sections of the minutes that discuss: (i) the current economic outlook, typically in a paragraph or paragraphs beginning “The information reviewed at the ... meeting suggested that ...”; and (ii) the future economic outlook, typically in a paragraph or paragraphs beginning “In their discussion of the economic situation and outlook, meeting participants ...”. In these sets of paragraphs, Stekler and Symington search for select keywords that characterize views on the outlook. Keywords range from “strong”, “robust”, “considerable”, “upbeat”, “brisk”, and “surge” for a very optimistic outlook to “recession”, “contraction”, and “sharp and widespread decline” for a very pessimistic one. S&S (Table 2) provide details. Fromthe frequencies of occurrence of the keywords, Stekler and Symington create two indexes, one for the current outlook and one for the future outlook. TheseindexesarecalledFOMCMinutesIndexes(orFMIs)below. S&Sthen 2

Table 1: A Schematic for the Generation of the FOMC Minutes Index. Information or Description Units, Range, Transformation or Source    Greenbook forecast ( ), and Numerical  0  certain qualitative and and quantitative information ( ) textual 0 |  =  (   ) (a) Writing of the FOMC minutes FOMC 1 0  0 0 | ↓  Text of the FOMC minutes Text 1 |  =  (  ) (b) Quantification of the FOMC minutes S&S (Table 3) 2 1 1 1 | ↓  Stekler and Symington’s score index [ 1+1] 2 − |  =  (  ) (c) Calibration of the score to S&S (Table 4)  2 2 2 | forecasts of the GDP growth rate | ↓  FOMC Minutes Index (FMI) [ 1%+4%]  − document properties of their indexes, including through comparisons with the Greenbook forecasts and with the forecasts from the Survey of Professional Forecasters (SPF). Section 4 below analyzes additional properties of S&S’s indexes. The empirical results in S&S and those below are subject to an important caveat. Specifically, the context of the paragraphs for (i) and (ii) in the FOMC minutes affects the interpretation of the FMIs. Starting with the January 2009 minutes, the paragraphs for (i) and (ii) respectively appear under separate sections titled: “Staff Review of the Economic Situation”, and “Participants’ Views on Current Conditions and the Economic Outlook”, or similar (italics added). Thus, the “current-outlook FMI” draws on text about the Fed staff’s views, whereas the “future-outlook FMI” ostensibly reflects the views of theFOMCparticipantsonbothcurrentconditionsandfutureoutlook. Thesenuances are germane to the interpretation of the FMIs when compared with (e.g.) the Fed staff’sGreenbookforecasts,asinS&S(Section4.1andFigure2). Section4.3discusses these subtleties further. See Danker and Luecke (2005) for a valuable perspective on the evolution of the FOMC’s minutes. Table 1 presents a schematic for the generation of the FMIs, formalizing the creation of the FOMC minutes and the derivation of the FMI therefrom, as follows. 3

(a) Writing of the FOMC minutes. The FOMC draws on the Greenbook forecast ( ), certain qualitative and quantitative information ( ), and possibly  0 additional information ( ) in writing the text of the FOMC minutes ( ). 0 1 That transformation is denoted  (   ). The subscript  is the time in- 0  0 0 dex that dates the FOMC meetings. Thevariable includesinformationonthehousingand financial markets; 0 and the Greenbook forecast itself draws on that information as well. This presence of  motivates in part S&S’s interest in ascertaining  ’s roles in 0 0 influencing the minutes  . The text of the minutes also provides the basis for 1 S&S’s assessment of the FOMC’s views on risk and uncertainty to the economy. (b) Quantification of the FOMC minutes as a score. S&S (Table 2) then analyze the text in the minutes of the FOMC meetings to create a score index ( ), as 2 given in S&S (Table 3). That quantification of the text may involve additional information ( ). The implied transformation is  (  ). 1 1 1 1 (c) Calibration of the score. Finally, S&S (Table 4) calibrate that score index to forecasts of the GDP growth rate, again possibly involving additional information ( ). The implied transformation is  (  ), and the resulting FMI 2 2 2 2 is denoted  . The variable being forecast is the U.S. real GDP growth rate,  denoted  .  The FMI thus is an implicit function of the Greenbook forecast, certain qualitative and quantitative information, and possible additional information involved in (a) the writing of the FOMC minutes, (b) the quantification of the FOMC minutes as a score, and (c) the calibration of that score to forecasts of the GDP growth rate. This description applies to the FMIs for both the current outlook and the future outlook, albeit with the time horizon (current or future) being implicit in the notation. Equation (1) articulates the implicit function for the FMI in three ways.  =  ( ( (   ) ) )  2 1 0  0 0 1 2 =  (     )   0 0 1 2   (1)  ≈ The first line in equation (1) expresses the FMI as an explicit repeated function of its information sources. The second line shows the implied implicit function  ( ).  · The third line indicates that the FMI may be approximated to some degree by the Greenbook forecast alone. Comparing the FMI directly with the Greenbook forecast is thus of interest; see Section 4.2. Equation (1) also clarifies that the FMI is a forecast in its own right. Hence, it can be evaluated as a forecast, such as with tests of unbiasedness and efficiency; seeSection3. Moreover, thethirdlineapproximatesthenonlinearfunction  (     ) by a simple linear one (= 0+1 ), per White (1980). Impulse   0 0 1 2  4

8 8 CCuurrrreenntt--oouuttllooookk FFMMII FFuuttuurree--oouuttllooookk FFMMII 6 CCuurrrreenntt--qquuaarrtteerr GGrreeeennbbooookk 6 OOnnee--qquuaarrtteerr--aahheeaadd GGrreeeennbbooookk 4 4 2 2 0 0 -2 -2 -4 -4 -6 -6 -8 -8 2006 2007 2008 2009 2010 2011 2006 2007 2008 2009 2010 2011 8 8 FFMMII--GGrreeeennbbooookk ddiiffffeerreennttiiaall ((hh==00)) FFMMII--GGrreeeennbbooookk ddiiffffeerreennttiiaall ((hh==11)) 6 6 4 4 2 2 0 0 -2 -2 -4 -4 -6 -6 -8 -8 2006 2007 2008 2009 2010 2011 2006 2007 2008 2009 2010 2011 Figure 1: The current-outlook and future-outlook FMIs, the Greenbook forecasts of the current-quarter and one-quarter-ahead U.S. real GDP growth rates, and the differentials between the FMI and Greenbook forecasts. indicatorsaturationcanhelpdeterminethedegreetowhichthatapproximationholds; see Section 3. Greenbook forecasts. Figure 1 compares the FMI with the Greenbook forecasts in a 2 2 panel of graphs, with each graph labeled by a suffix a, b, c, or d, as follows: × ab . Figure 1a plots two “nowcasts”: the FMI for the current outlook, and the cd Greenbook forecast for the current quarter. Figure 1b plots two forecasts: the FMI £ ¤ for the future outlook, and the Greenbook forecast for one quarter ahead.1 Figure 1c plots the difference between the FMI for the current outlook and the Greenbook forecast for the current quarter; and Figure 1d plots the difference between the FMI for the future outlook and the Greenbook forecast for one quarter ahead. In Figure 1 (and likewise in Figure 3 below), the time  is the date of the FOMC meeting to which a Greenbook is submitted and from which an FMI is constructed. The graphs 1S&S (footnote 12) compare the FMI for the future outlook with two-quarter-ahead Greenbook forecasts. One-quarter-ahead Greenbook forecasts are employed herein, noting the behavior of the root mean squared forecast errors for the Greenbook forecasts as a function of the forecast horizon; see Ericsson, Hood, Joutz, Sinclair, and Stekler (2013), Ericsson, Fiallos, and Seymour (2014), and Chang and Hanson (2015). That said, the forecast-encompassing tests described below permit sorting out which horizon or horizons are relevant. Additional analysis (not reported herein) suggests that combinations of Greenbook forecasts at different horizons may be particularly useful for explaining the FMI for both the current outlook and the future outlook. 5

Table2: DatesofFOMCmeetingsforwhichtheFMImayhavetruncatedtheFOMC’s forecast, and (in percent per annum) the corresponding FMI (), Greenbook fore-  cast (), and their differential ( ) at horizons  = 0 and  = 1.    − Date  0 0 0 0 1 1 1 1         − − March 2006 +4.0 4.7 —0.7 – – – May 2006 +4.0 3.7 +0.3 – – – March 2008 – – – —1.0 —1.0 +0.0 December 2008 —1.0 —4.7 +3.7 —1.0 —5.0 +4.0 January 2009 —1.0 —5.6 +4.6 – – – March 2009 —1.0 —6.5 +5.5 —1.0 —2.0 +1.0 in Figure 1 show that the FMI and the Greenbook forecasts are generally very close numerically, whether for the current outlook or for the future outlook. There are, however, largedeviations betweentheFMI andtheGreenbookforecast for the FOMC meetings in December 2008, January 2009, and March 2009: see the boxed-inareasinFigure1. Forthesemeetings, S&S’sscoreisatitsminimum(= 1), − the most pessimistic outlook. However, the FMI corresponding to that score is only a 10% per annum rate of decline in GDP: see S&S (Table 4), which maps the score to the FMI. The Greenbook forecasts were typically much more negative for those meetings. This discrepancy (or “deviation”) between the FMI and the Greenbook forecast arises because the score (and hence the FMI) truncates both tails of the distribution of the implicit forecast in the FOMC minutes. Truncation. To examine this issue further, Table 2 lists the dates of all FOMC meetings for which the score may have truncated the forecast implicit in the minutes, along with the FMI , the Greenbook forecast , and the differential between them. The forecast horizon  appears as a superscript, where  = 0 and  = 1 respectively denote the current outlook and future outlook for the FMI (and below, for the score  ), or the current quarter and one quarter ahead for the Greenbook forecast. 2 The deviations for the current outlook are large for all three meetings with very pessimistic assessments; and the FMI is much larger than the corresponding Greenbook forecast, by 3%—6% per annum. The deviation for the future outlook is large for December 2008, but not for January 2009 (when the future outlook was less pessimistic) or for March 2009 (when the discrepancy was only 1% per annum). For the current outlook, the score is at its most optimistic value for the FOMC meetings in March 2006 and May 2006, with the FMI being +40% per annum; but the corresponding deviations are smaller than 1% per annum in absolute value. For the future outlook, there are no meetings for which the score (and hence the FMI) is at its most 6

optimistic value. S&S’sscorealgorithmcanthustruncatetheforecastimplicitintheFOMCminutes. That algorithm appears to do so for three FOMC meetings at the onset of the Great Recession. These large deviations also could have arisen if the forecast implicit in the FOMC minutes itself deviated markedly from the Greenbook forecast; but that explanation seems less plausible in light of the close match between the FMI and the Greenbook forecast for other dates. Linearity in calibration. The function  (  ) calibrates S&S’s score to have 2 2 2 units comparable to forecasts of GDP growth rates. That calibration function is very close to linear, as a simple regression of the current-outlook FMI (0) on the  current-outlook score (0 ) demonstrates. 2 0 = 1515 + 2508 0 (2)  2 (0005) (0009)  = 40 [2006F1—2010F8] R2 = 09995 ˆ = 00315% Here and below, estimated standard errors are in parentheses ( ) under coefficient · estimates;  is the total number of observations; the letter “F” indexes the FOMC meeting within the year (F1, ..., F8), paralleling the common notation of “Q” for “quarter”(Q1,...,Q4); R2 isthesquaredmultiplecorrelationcoefficient; andˆ isthe estimated residual standard error. Equation (2) is a regression, and so the left-hand side variable is implicitly the fitted value of 0, not 0 itself.   Toputtheregression(2)incontext, noneofitsresidualsarelargerthan005%per annum, whereas the actual GDP growth rates in S&S (Table 5) and the Greenbook forecasts thereof are reported to a tenth of a percent per annum. Thus, to a very good degree of approximation, the FMI  is 25 times S&S’s score  , plus 15. That  2 is, for the current outlook, the transformation  =  (  ) is essentially linear in  2 2 2 the score  ; and  is approximately zero. Results are numerically very similar for 2 2 the future outlook. Actual GDPgrowthrates. Figure2plotstheFMIsandactualGDPgrowthratesin a2 2panelsimilartotheoneinFigure1,albeitwithtimeinFigure2beingthedate × of the actual GDP growth rate, and with each FMI forecast aligned correspondingly. Figure 2a and 2b graph FMIs for the current outlook and future outlook respectively, each along with the actual GDP growth rate as measured 90 days after quarter end and as measured historically. Figure 2c and 2d plot the corresponding discrepancies, which are interpretable as forecast errors in this context. As S&S note, and as is apparent in Figures 2a—b, the actual values were revised markedly downward for several quarters at the beginning of the financial crisis, including by 22% for 2007Q3, 37%for2008Q1, and20%for2008Q4. Figures2c—dshowthatthelargeandnegative FMI “forecast errors” for late 2008 and early 2009 reflect these data revisions and the effects of truncation discussed above. 7

8 CCuurrrreenntt--oouuttllooookk FFMMII 8 FFuuttuurree--oouuttllooookk FFMMII AAccttuuaall ggrroowwtthh rraattee ((9900--ddaayy mmeeaassuurree)) AAccttuuaall ggrroowwtthh rraattee ((9900--ddaayy mmeeaassuurree)) 6 AAccttuuaall ggrroowwtthh rraattee ((hhiissttoorriiccaall mmeeaassuurree)) 6 AAccttuuaall ggrroowwtthh rraattee ((hhiissttoorriiccaall mmeeaassuurree)) 4 4 2 2 0 0 -2 -2 -4 -4 -6 -6 -8 -8 2006 2007 2008 2009 2010 2011 2006 2007 2008 2009 2010 2011 8 8 FFoorreeccaasstt eerrrroorr ((hh==00,, 9900--ddaayy mmeeaassuurree)) FFoorreeccaasstt eerrrroorr ((hh==11,, 9900--ddaayy mmeeaassuurree)) 6 FFoorreeccaasstt eerrrroorr ((hh==00,, hhiissttoorriiccaall mmeeaassuurree)) 6 FFoorreeccaasstt eerrrroorr ((hh==11,, hhiissttoorriiccaall mmeeaassuurree)) 4 4 2 2 0 0 -2 -2 -4 -4 -6 -6 -8 -8 2006 2007 2008 2009 2010 2011 2006 2007 2008 2009 2010 2011 Figure 2: FMIs for the current outlook and future outlook, and the actual GDP growth rate as measured 90 days after quarter end and as measured historically; and the corresponding FMI forecast errors. The actual GDP growth rates are quarterly, expressed at annual rates and spanning 2006—2010. For the sequential pair of FOMC meetings within a given quarter, the value of an actual GDP growth rate is repeated, thereby providing actual values for comparison with the FMI and Greenbook forecasts. See Ahumada (1992, especially Figure 3) for a similar interpolation of quarterly Argentine total final domestic expenditure to monthly values. Data details. Sources and practical details for the FMIs, Greenbook forecasts, and actual GDP growth rates follow. The units for these series are quarterly growth rates, measured in percent per annum. S&S construct their scores and FMIs for the 40 FOMC meetings during 2006— 2010, a period that spans the financial crisis and leads into the Great Recession. The minutes of the FOMC meetings themselves are available online at: www.federalreserve.gov/monetarypolicy/fomc_historical.htm(for2006—2009)and www.federalreserve.gov/monetarypolicy/fomccalendars.htm (for 2010). In S&S, Table 3 lists the scores, Figures 1A and 1B plot them, Table 4 provides the algorithm for converting the scores to the FMIs, and Figure 2 plots the currentoutlook score in its original units and in the units for the FMI. The Fed staff’s Greenbooks are available on the first website listed above. Currently, the Greenbooks are available through only December 2009, given the approx- 8

imate 5-year delay in their release to the public. There are thus 32 observations of Greenbook forecasts for a given horizon, with those forecasts obtained from Greenbooks spanning 2006—2009. S&S (Figure 2) plot the Greenbook forecasts of the current quarter through 2009, with comparable SPF forecasts for 2010 appended at the end. (The SPF forecasts are not considered in the analysis below.) S&S (Table 5) list the “90-day estimate” and “historical” values (S&S’s terminology) for the quarterly GDP growth rates over 2006—2010. These two measurements correspondtotheU.S. Bureauof EconomicAnalysis’s“thirdestimate”release, which is typically available about 90 days after the end of the relevant quarter; and to the complete set of values for 2006—2010 as reported by the Bureau of Economic Analysis when S&S was initially written; see Stekler and Symington (2014). Remarks. Before proceeding to a detailed assessment of the FMIs, two observations about Stekler and Symington (2016) are germane. First, Stekler andSymington follow a common approach to textual analysis. For instance, Boukus and Rosenberg (2006) use latent semantic analysis to assess the roles of different themes in the FOMC’s minutes; and Meade, Burk, and Josselyn (2015) calculate the changing frequencies of different quantitative words in the FOMC’s minutes to ascertain the diversity of views among the FOMC members and participants. Quantitative textual analysisalsohasnumerousprecedentsoutsideofeconomics,asinCoxandBrandwood (1959). Second, Stekler and Symington’s analysis is novel by quantifying qualitative text from the minutes on the outlook and comparing that quantification with the Greenbook forecasts. By contrast, Meade, Burk, and Josselyn focus on the disparity of viewsintheminutes, ratherthansomecentraltendencyofviews. Banternghansaand McCracken (2009, 2014) likewise focus on the disparity of views, albeit as measured by individual participants’ economic forecasts. Yet other researchers have compared the Greenbook forecasts and FOMC participants’ forecasts with each other and with other forecasts. Stekler and Symington (2016) thus apply an innovative and creative quantitative textual analysis to elicit a measure of the nowcasts and forecasts implicit in the minutesoftheFOMCmeetings. ThatmeasureistheFMIand, becauseitisaforecast elicited fromtheFOMCminutes,itisdenotedan“el-cast”. Eventhoughthetextthat Stekler and Symington examine includes little or no quantitative information, their FMIs reveal much about the thinking of the FOMC participants and (as Section 4 below shows) about the Fed staff’s input to the FOMC meetings. 3 Assessing and Comparing Forecasts This section considers different approaches for assessing and comparing forecasts, starting with the standard test of forecast bias by Mincer and Zarnowitz (1969). The 9

Mincer—Zarnowitz framework is then extended to allow for (a) the suspected truncations noted in Section 2, (b) general time variation in forecast bias, and (c) comparison of multiple forecasts, rather than of a single forecast against the outcome. These extensions include many standard procedures for forecast evaluation as special cases. Mincer—Zarnowitz. Mincer and Zarnowitz (1969, pp. 8—11) suggest testing for forecast bias by regressing the forecast error on an intercept and testing whether the intercept is statistically significant. That is, for a variable  at time  and its  forecast ˆ, estimate the equation:  ( ˆ) =  +    = 1 (3)    − where  is the intercept, and  is the error term at time . In the present context,   is the growth rate of real GDP (either the 90-day estimate or the historical value),  and ˆ is either the FMI or the Greenbook forecast.  A test of  = 0 is interpretable as a test that the forecast ˆ is unbiased for the  variable  . For one-step-ahead forecasts, the error  may be serially uncorrelated,   in which case a standard -statistic or -statistic may be appropriate. For multistep-ahead forecasts,  generally will be serially correlated; hence inference about  the intercept  may require some accounting for that autocorrelation. Suspected systematic effects. The potential truncations documented in Table 2 may distort the FMI as a measure of the forecast implicit in the FOMC minutes. If a measured forecast is distorted in certain time periods, it may be advantageous to account for those distortions in the Mincer—Zarnowitz regression (3), both to correct for any biases that those distortions induce in estimating , and to improve the efficiency of the estimation of . Direct adjustment of regression (3) is feasible for known dates of potential distortions: ( ˆ) =   +  +    = 1 (4)  −      ∈J P where  is a dummy variable that is unity for  =  and zero otherwise,  is the   corresponding coefficient for  , and is the set of dates for which distortions are  J suspected. Equation (4) thus provides the basis for calculating the Mincer—Zarnowitz test while correcting for suspected distortions in the measurement of the forecast. Arbitrarily time-varying forecast bias. Equation (4) is also interpretable as allowing for a specific form of time dependence in the bias of the measured forecast, where that bias depends on the degree of truncation. That time dependence could be completely general, while also explicitly allowing for the date-specific potential distortions: ( ˆ) =   +  +   −       ∈J = P      +  =1     +     = 1 (5) ∈J P P 10

The intercept (now denoted  ) is explicitly time-dependent; the impulse indicator    is a dummy variable that is unity for  =  and zero otherwise; and  is the corre-  sponding coefficient for  . Because the  may have any values whatsoever, the   { } intercept  in (5) may vary arbitrarily over time. In this context, a test that all  coefficients  are equal to zero is a generic test of forecast unbiasedness, corrected  for potential truncation distortions. In determining the form of  , equation (5) thus  retains some variables (the  ) without selection, paralleling the model-selection  { } approach in Hendry and Johansen (2015). Because equation (5) includes at least  coefficients, equation (5) cannot be estimated unrestrictedly. However, the question being asked can be answered with impulse indicator saturation. Impulse indicator saturation. Impulse indicator saturation (IIS) uses the zeroone dummies  to analyze properties of a model. Unrestricted inclusion of all  { }  dummies in the model (thereby “saturating” the sample) is infeasible. However, blocks of dummies can be included, and statistically significant dummies can be retained from those blocks. That insight provides the basis for IIS. See Ericsson and Reisman (2012) for an intuitive non-technical exposition of IIS, and Hendry and Doornik (2014) for extensive analysis in the context of automatic model selection. As background, IIS provides a general procedure for robust estimation and for model evaluation–in particular, for testing parameter constancy. IIS is a generic test for an unknown number of structural breaks, occurring at unknown times, with unknown duration and magnitude, anywhere in the sample. IIS is a powerful empirical tool for both evaluating and improving existing empirical models. Hendry (1999) proposes IIS as a procedure for testing parameter constancy. Further discussion, recent developments, and applications appear in Hendry, Johansen, and Santos (2008), Doornik (2009), Johansen and Nielsen (2009, 2013, 2015), Hendry and Santos (2010), Ericsson (2011a, 2011b, 2012), Ericsson and Reisman (2012), Bergamelli and Urga (2013), Hendry and Pretis (2013), Hendry and Doornik (2014), Pretis, Mann, and Kaufmann (2015), and Castle, Doornik, Hendry, and Pretis (2015). Ericsson (2015) proposes a new application for IIS–as a generic test for time-varying forecast bias. Section 4 applies IIS to test for potential bias in the FMI as a forecast; and IIS addresses measurement errors in the FMI due to suspected truncation biases. Direct comparison of two forecasts. Equation (5) also provides the basis for direct comparison of forecasts. Specifically, consider an alternative forecast of  , denoted  ˜. The corresponding forecast error ( ˜) is equivalent to  ( ˜)  , which   −  =1  −  ·  is a weighted sum of the impulse indicators. Equation (5) may thus be rewritten as: P ( ˆ) =   + ( ˜) +    +    = 1 (6)  −   ∗   −  =1 ∗   ∈J Subtracting ( P ˜  ) from both sides of equPation (6) obtains: − (˜ ˆ) =   +    +   −   ∗  =1 ∗   ∈J = P   ∗   + P ∗ +     = 1 (7) ∈J P 11

The coefficients  ,  , and  in equations (6)—(7) need not equal the coefficients  , ∗ ∗ ∗   , and  in equation (5). Also, tests on   (or, equivalently, on   ) in   { ∗ ∗} { ∗ ∗} equation (7) allow direct comparison of the two forecasts ˆ and ˜.   Remarks. Equations (3)—(7) are directly related to several common forecast tests. First, MincerandZarnowitz(1969,p.11)proposeavariationonequation(3)inwhich the coefficient on ˆ is estimated rather than imposed. See Holden and Peel (1990),  Stekler (2002), and Sinclair, Stekler, andCarnow(2015) inter alia; and note that this modified formulation generates a test of forecast efficiency as well as a test of forecast unbiasedness. Second, if ˜ (=  ˜  ) rather than ( ˜) is included on the  =1  ·   −  right-hand side of equation (6), Chong and Hendry’s (1986) forecast-encompassing P test emerges. Third, as Ericsson (1992b) discusses, a necessary condition for forecast encompassing is having the smallest root mean squared forecast error (RMSFE); and Granger(1989)andDieboldandMariano(1995)proposetestsofwhetheronemodel’s RMSFE is less than another model’s RMSFE. Fourth, the regression for the forecastencompassing test can be extended to compare several forecasts at once, in which case the right-hand side of the regression includes the differential of each alternative forecast relative to the “primary” forecast; see Ericsson and Marquez (1993). Fifth, theregressionfor theforecast-encompassingtestmayincludeforecast revisions across multiple horizons, paralleling the direct extension of forecast encompassing to include a comparison with multiple forecasts. Nordhaus (1987) proposes this revisions-based test as a test of efficiency; see also Dovern, Fritsche, Loungani, and Tamirisa (2015) and Coibion (2015). Tversky and Kahneman (1974) earlier described “anchoring” as a potential phenomenon exhibited by forecast revisions; see Campbell and Sharpe (2009)forempiricalevidenceonanchoring. Inarelatedvein, equation(5)extendsthe framework for regression-based tests of predictive accuracy in West and McCracken (1998). Many common procedures for evaluating forecasts are thus derivable from the generic framework in equation (5), which characterizes general and potentially arbitrarilytime-varyingforecastbias. Equally,thesecommonproceduresareinterpretable as specific forms of potentially time-varying forecast bias. As the simplest case, the original Mincer—Zarnowitz test from (3) is a test of constant (i.e., time-invariant) forecast bias. This observation highlights the strength of the Mincer—Zarnowitz test (that it focuses on detecting a constant nonzero forecast bias) and also its weakness (that it assumes that the forecast bias is constant over time). These characteristics of the Mincer—Zarnowitz test bear directly on the empirical results in Section 4. 4 The FMI and the Greenbook Forecasts This section examines the FMI as an elicited forecast (el-cast) of the U.S. real GDP growthrateover 2006—2010, anditconsiderstheFMI’srelationshiptotheGreenbook 12

forecast. Section 4.1 shows that standard (Mincer—Zarnowitz) tests of forecast bias detect economically small and statistically insignificant biases for the FMI compared against the 90-day estimate of the growth rate; and IIS-type tests find similar results. However, standard tests find economically large and statistically significant biases of the FMI relative to the historical growth rates, whereas IIS-type tests find much smaller and less significant bias. Those differences arise because the standard tests ignore truncation, whereas the IIS-type tests adjust for it. Section 4.2 shows how the FMI can post-cast as-yet-unreleased Greenbook forecasts. Section 4.3 considers some implications. 4.1 Forecast Biases for the FMI The FMI is a relatively unbiased forecast of the U.S. real GDP growth rate, once accounting for truncation. Tables 3 and 4 respectively report analyses of the FMI relative to the 90-day estimate of the growth rate, and of the FMI relative to the historical growth rate. Each table lists regression coefficients, estimated standard errors, and various summary statistics for regressions in which the FMI forecast error ( )isthedependentvariable. Resultsarereportedforthecurrent-outlook( = 0) − and future-outlook ( = 1) FMIs, and for regressions with and without truncation adjustment (“TA”). The sample periods are 2006F1—2009F8 ( = 32) for  = 0 and 2006F3—2009F8 ( = 30) for  = 1, noting that each quarter has two FOMC meetings. The sample periods used in these regressions end in 2009F8 in order to maintain comparability with publicly available Greenbook forecasts. (Section 4.2 considers the FMI in 2010.) Dates on impulse indicator dummies correspond to the dates of actual GDP growth rates. In Table 3, the first column of numbers lists results for the standard Mincer— Zarnowitz regression(3)forthecurrent-outlookFMI, with no truncationadjustment. The estimated bias is 017% and is statistically insignificant with an -statistic of − 015 (-value of 0704); ˆ is 2445%; and (for comparison) the RMSFE of the FMI is 2412%. The first column also lists diagnostic statistics for testing against various alternative hypotheses, thereby helping reveal the equation’s strengths and weaknesses. These statistics test for autocorrelation in the residuals (AR), residual autoregressive conditional heteroscedasticity (ARCH), and residual nonnormality as characterized by skewness and excess kurtosis (Normality); see Godfrey (1978), Engle (1982), and Jarque and Bera (1980) and Doornik and Hansen (2008) respectively. For a given diagnostic statistic, the entries within ablockof numbers are the value of the statistic for testing the null hypothesis against the designated maintained hypothesis, its tail probability(insquare brackets)calculatedfor theapproximatedistributionunderthe null hypothesis, and that distribution (either 2( ) or (  ), with degrees of freedom · · · in parentheses). Superscript asterisks and denote rejections of the null hypothesis ∗ ∗∗ 13

Table 3: Coefficients, estimated standard errors, and summary statistics for regressions with FMI forecast errors ( ) for the U.S. real GDP growth rate, with the − growth rate as measured 90 days after quarter end. Regressor  = 0  = 0  = 1  = 1 or statistic no TA with TA no TA with TA Intercept 017 031 036 023 −(043) (038) −(051) (044)  _ 561 _ 683 20088 −(206) −(230)  _ 481 _ 603 20091 −(206) −(230)  _ 481 _ 473 20092 −(206) −(230) ˆ 2445% 2025% 2803% 2260% RMSFE 2412% 2014% 2779% 2230% Mincer—Zarnowitz 015 068 049 028 (-statistic) [0704] [0416] [0491] [0602] (131) (128) (129) (126) -statistic for _ 573 _ 620 ∗∗ ∗∗ truncation adjustment [0003] [0003] (328) (326) AR(3) LM statistic 718 214 147 677 ∗∗ ∗∗ ∗∗ [0001] [0120] [0000] [0002] (328) (325) (326) (323) ARCH(3) LM statistic 400 027 931 033 ∗ ∗∗ [0018] [0845] [0000] [0804] (326) (326) (324) (324) Normality statistic 575 604 366 718 ∗ ∗ [0056] [0049] [0160] [0028] 2(2) 2(2) 2(2) 2(2) Dates of additional 20087 20087 20087 20087 impulses selected by 20088 20088 IIS (1% target size) 20091 20091 20092 -statistic for the 996 144 956 144 ∗∗ ∗∗ ∗∗ ∗∗ additional impulses [0000] [0001] [0000] [0001] (427) (127) (326) (125) Notes. Regressions are for current-outlook ( = 0) and future-outlook ( = 1) FMIs, without and with truncation adjustment (“TA”). 14

Table 4: Coefficients, estimated standard errors, and summary statistics for regressions with FMI forecast errors ( ) for the U.S. real GDP growth rate, with the − growth rate as measured historically. Regressor  = 0  = 0  = 1  = 1 or statistic no TA with TA no TA with TA Intercept 123 080 146 093 −(043) −(039) −(053) −(048)  _ 650 _ 767 20088 −(213) −(254)  _ 360 _ 477 20091 −(213) −(254)  _ 360 _ 347 20092 −(213) −(254) ˆ 2445% 2095% 2915% 2490% RMSFE 2701% 2209% 3215% 2613% Mincer—Zarnowitz 808 423 749 373 ∗∗ ∗ ∗ (-statistic) [0008] [0049] [0010] [0064] (131) (128) (129) (126) -statistic for _ 473 _ 459 ∗∗ ∗ truncation adjustment [0009] [0010] (328) (326) AR(3) LM statistic 406 142 855 412 ∗ ∗∗ ∗ [0016] [0262] [0000] [0018] (328) (325) (326) (323) ARCH(3) LM statistic 433 004 900 032 ∗ ∗∗ [0013] [0990] [0000] [0812] (326) (326) (324) (324) Normality statistic 620 936 529 709 ∗ ∗∗ ∗ [0045] [0009] [0071] [0029] 2(2) 2(2) 2(2) 2(2) Dates of additional 20087 20087 20087 20087 impulses selected by 20088 20088 IIS (1% target size) 20091 -statistic for the 128 199 101 153 ∗∗ ∗∗ ∗∗ ∗∗ additional impulses [0000] [0000] [0000] [0001] (229) (127) (326) (125) Notes. Regressions are for current-outlook ( = 0) and future-outlook ( = 1) FMIs, without and with truncation adjustment (“TA”). 15

at the 5% and 1% levels respectively. Finally, the column lists dates of impulse dummies selected by IIS at a 1% target size,andthe-statisticforthoseretainedimpulses. Notably,IISselectsimpulsedummies for dates for which truncation is suspected (2008F8, 2009F1, and 2009F2), and for the date immediately preceding (2008F7). Even without focusing the regression on dates with possible truncation, IIS detects those dates. The second numerical column in Table 3 reports the same regression, but where impulse dummies  are included for the three suspected truncation dates. Those  dummies are highly significant (-statistic of 573, with a -value of 0003); and the only additional impulse dummy selected by IIS is for 2008F7. The estimated bias is +031% and is statistically insignificant. The third and fourth numerical columns report results for the future outlook, both without and with truncation adjustment; and those results are similar to those for the current outlook. Table 4 repeats the analysis in Table 3 but, in calculating the forecast errors, uses the historical values of the GDP growth rate rather than the 90-day estimates. As S&S note, the historical values and the 90-day estimates differ markedly for several periods. Those differences are reflected in Table 4’s regressions. For the current outlook and future outlook, the “pure” Mincer—Zarnowitz regression (3) detects large and highly significant negative biases: of 123% and 146% respectively. However, − − with adjustment for possible truncation, estimated biases are only about two thirds of those values ( 080% and 093%); and they are only marginally significant or − − insignificant at the 5% level. Standard tests of time-invariant forecast bias can and do mislead. Also, IIS again detects 2008F7 as an anomaly. 4.2 Post-casting the Greenbook Forecasts The FMI very closely approximates the Greenbook forecast, once accounting for the FMI’s truncation in three observations. Table 5 reports regressions of the form in equation (7), with the dependent variable being the differential between the FMI and the Greenbook forecast ( ). The sample period is 2006F1—2009F8 ( = 32) for − both  = 0 and  = 1. Each date in that sample period corresponds to the date of an FOMC meeting, to which a Greenbook is submitted and from which an FMI is constructed. The first numerical column in Table 5 indicates a statistically significant differential of 066% between the FMI and the Greenbook forecast for the current quarter ( = 0). However, onceadjustedforpossibletruncationeffects(secondnumerical column), that estimated bias is only 026% and is statistically insignificant. For  = 1, no statistically significant or numerically large differential is detectable, whether or not the regression is adjusted for possible truncation effects. For both  = 0 and  = 1, the (truncation-adjusted) FMI and the Greenbook forecast are very close numerically. Deviations between them are small, typically less 16

Table 5: Coefficients, estimated standard errors, and summary statistics for regressions of ( ), the deviation between the FMI and the Greenbook forecast. − Regressor  = 0  = 0  = 1  = 1 or statistic no TA with TA no TA with TA Intercept 066 026 027 009 (028) (017) (019) (016)  _ 344 _ 391 20088 (095) (086)  _ 434 _ 081 20091 (095) (086)  _ 524 _ 091 20092 (095) (086) ˆ 1582% 0938% 1078% 0850% RMSE 1692% 0956% 1094% 0840% Mincer—Zarnowitz 561 215 194 032 ∗ (-statistic) [0024] [0154] [0173] [0574] (131) (128) (131) (128) -statistic for _ 201 _ 728 ∗∗ ∗∗ truncation adjustment [0000] [0001] (328) (328) AR(3) LM statistic 797 048 091 113 ∗∗ [0000] [0699] [0446] [0356] (328) (325) (328) (325) ARCH(3) LM statistic 537 131 012 029 ∗∗ [0005] [0292] [0948] [0830] (326) (326) (326) (326) Normality statistic 183 265 106 060 ∗∗ ∗∗ [0000] [0265] [0005] [0741] 2(2) 2(2) 2(2) 2(2) Dates of additional 20088 None 20088 None impulses selected by 20091 IIS (1% target size) 20092 -statistic for the 201 _ 200 _ ∗∗ ∗∗ additional impulses [0000] [0000] (328) (130) Notes. Thedependentvariableiseitherthecurrent-outlookFMIminustheGreenbookforecast ofthecurrent-quarterU.S.realGDPgrowthrate(=0),orthefuture-outlookFMIminusthe Greenbook forecast of the one-quarter-ahead U.S. real GDP growth rate (=1). Regressions are without and with truncation adjustment (“TA”). 17

than 1% per annum and often less than 05% per annum. The root mean squared errors (RMSEs) and residual standard errors are correspondingly small, under 1% per annum. From IIS, the adjustment for possible truncation in the FMI appears sufficient to correct for that truncation; and only that truncation appears to need correction. Notably, the one-quarter-ahead RMSEs are smaller than the currentquarter RMSEs. That is, S&S’s indexes are more accurate at inferring the onequarter-ahead Greenbook forecast than the current-quarter Greenbook forecast. This empirical analysis establishes a close relationship between the FMI and the Greenbook forecast. This relationship presents a special opportunity for “postcasting” Greenbook forecasts, conditional on the FMI, noting that the minutes of an FOMC meeting are available three weeks after the meeting, whereas the Greenbook forecasts are not released to the public until approximately five years after the GreenbookitselfispresentedtotheFOMC. Specifically, theregressionsinTable5suggesta potentiallyviable mechanismfor post-casting the Greenbook forecasts in 2010, a year for which the Greenbook forecasts have not yet been released publicly. Noting that theregressions’dependentvariableisthedifferential( ), theGreenbookforecast − can be post-cast, conditional on the calculated value of the FMI . While inversion of a statistical relationship–as between  and –generally is inadvisable, inversion of a deterministic or near-deterministic relationship may be useful; see Ericsson (1992a, Section 2C) and Ericsson, Hendry, and Hood (2015, Section 4.6). Figures 3a and 3b plot the FMI-based post-casts of the 2010 Greenbook forecasts, with 1 standard error bands. These post-casts range between 21% and 34% for ± both  = 0 and  = 1, with somewhat different dynamics for each forecast horizon . The forecast standard error is under 1%. Figures 3c and 3d plot the observed insample discrepancies between the FMI and the Greenbook forecasts for 2006—2009 with truncation adjustment. 4.3 Remarks and Implications Several implications follow directly from the results above. First, and foremost, Stekler and Symington’s indexes provide a proximate mechanism for inferring Greenbook forecasts of the current-quarter and one-quarter-ahead U.S. real GDP growth rates, years in advance of the public release of the Greenbook. Second, the current-outlook FMI draws on text about the Fed staff’s views, whereas the future-outlook FMI ostensibly reflects the views of the FOMC participants on both current conditions and future outlook. In principle, these nuances are germanetotheinterpretationof theFMIswhencomparedwiththeFedstaff’sGreenbook forecasts. However, Section 4.2 implies that these distinctions are relatively unimportant over the sample considered, Nunes’s (2013) analysis notwithstanding. Third, it may seem surprising that FOMC participants’ views for the future outlook–as measured by the future-outlook FMI–are well-captured by the one- 18

8 8 CCuurrrreenntt--oouuttllooookk FFMMII,, aanndd 22001100 ppoosstt--ccaassttss FFuuttuurree--oouuttllooookk FFMMII,, aanndd 22001100 ppoosstt--ccaassttss 6 CCuurrrreenntt--qquuaarrtteerr GGrreeeennbbooookk 6 OOnnee--qquuaarrtteerr--aahheeaadd GGrreeeennbbooookk 4 4 2 2 0 0 -2 -2 -4 -4 -6 -6 -8 -8 2006 2007 2008 2009 2010 2011 2006 2007 2008 2009 2010 2011 8 8 FFMMII--GGrreeeennbbooookk ddiiffffeerreennttiiaall ((hh==00,, wwiitthh TTAA)) FFMMII--GGrreeeennbbooookk ddiiffffeerreennttiiaall ((hh==11,, wwiitthh TTAA)) 6 6 4 4 2 2 0 0 -2 -2 -4 -4 -6 -6 -8 -8 2006 2007 2008 2009 2010 2011 2006 2007 2008 2009 2010 2011 Figure 3: The post-casts of the 2010 Greenbook forecasts for current-quarter and one-quarter-ahead U.S. real GDP growth rates, with 1 standard error bands; and ± the observed in-sample discrepancies between the FMI and Greenbook forecasts for 2006—2009 with truncation adjustment. quarter-ahead Greenbook forecast, noting that the policy-relevant horizon may be somewhat longer than one quarter ahead. The future-outlook FMI may thus be an even better proxy for Greenbook forecasts at longer horizons. Or, participants’ views maydown-weightGreenbookforecastsatlongerhorizons,notingthatGreenbookforecasts at two or more quarters ahead appear relatively uninformative when compared with naive forecasts such as a random walk or an unconditional mean growth rate. See Ericsson, Fiallos, and Seymour (2014) and Chang and Hanson (2015), noting that the former includes evaluation of U.S. as well as foreign growth. Also, because the future-outlook FMI quantifies the participants’ views for the future outlook in the minutes, comparison of the future-outlook FMI with the FOMC’s Summary of Economic Projections may be of interest. Fourth, the FMI itself could be improved, extended, and expanded. The measurementerrorsintheFMIduetotruncationcouldbereducedoreliminatedbyexpanding the range of Stekler and Symington’s score. The FMI could be constructed on a finer grid, noting that it currently has intervals of 06% or 07% per annum between adjacent values; see S&S (Table 4). And, the FMI could be extended through 2015, from which post-casts of the Greenbook for 2011 through 2015 could be constructed. The FMI also could be extended to meetings prior to 2006. Indexes over those periods 19

might be more or less accurate than those over 2006—2009; and the Greenbooks from 2010 onward have yet to be released publicly.2 FMIs also could be constructed for other economic variables, such as inflation and unemployment. Fifth, IIS serves as a diagnostic and exploratory tool for detecting potential missinginformationinaregression. IISalsoservestocorrectoradjustforsuchanomalies: here, specifically, by robustifying the results to the truncation effects. Focusing on specific known or suspected dates can enhance the analysis, albeit with supplemental IIS offering some protection from unsuspected events. IIS thus helps detect when the FMI deviated substantially from outcomes, and from the Greenbook forecasts. Clearly, “rejection of the null doesn’t imply the alternative”. However, for time series data, the date-specific nature of IIS-type procedures can aid in identifying important sources of forecast error. Use of these tests in forecast development is consistent with a progressive modeling approach; see White (1990). Sixth, the analysis above implicitly involves factorizing the joint distribution of the FMI, the Greenbook forecast, and the actual GDP growth rate into conditional and marginal distributions. As Engle, Hendry, and Richard (1983) and Ericsson (1992a) discuss, some factorizations may prove more fruitful than others; and this aspect merits further investigation. Seventh, textualanalysisofFeddocumentssuchastheGreenbookmaybefruitful. For instance, while the Greenbook does include numerical forecasts, it also includes extensive discussion of the factors underpinning those forecasts. The Greenbook constituteswhatHendry(2001, p.18)calls“fore-diction”, acombinationofnumerical forecasts with a narrative. Eighth, the largest inaccuracies in the FMI occur at the onset of the financial crisis and Great Recession–in effect, at a turning point in the business cycle. As Fildes and Stekler (2002) and others document, turning points have been difficult to forecast; andtheFMI’s inaccuracies mayreflect that challenge. Froman institutional perspective, it may be useful to isolate the causes of the forecast errors according to the various assumptions made. Such an analysis could lead to improved forecasts, or at least provide a deeper understanding of the sources of the inaccuracies. Equally, robust forecasting procedures could reduce or eliminate systematic forecast errors; see Hendry (2006), Castle, Fawcett, and Hendry (2010), and Castle, Clements, and Hendry (2015) inter alia. 5 Conclusions Stekler and Symington (2016) propose and construct ingenious quantitative indexes that measure the extent of optimism or pessimism expressed in the FOMC’s minutes 2Also, starting in 2010, the Greenbook forecasts appear in a Fed document called the Tealbook, which combines the previous Fed documents called the Greenbook and the Bluebook. 20

on the current and future outlook for the U.S. economy. These indexes help provide insights on the views of the FOMC, especially those held at the onset of the recent financial crisis and Great Recession. The current paper complements Stekler and Symington’s analysis by showing that, aside from a few specific instances, their indexes reveal relatively minor bias in the FOMC’s views when the indexes are reinterpreted as forecasts. These indexes also provide a proximate mechanism for inferring Greenbook forecasts of the U.S. real GDP growth rate, years before the public release of the Greenbook. Impulse indicator saturation proves central to obtaining these results. References Ahumada, H. (1992) “A Dynamic Model of the Demand for Currency: Argentina 1977—1988”, Journal of Policy Modeling, 14, 3, 335—361. Banternghansa, C., and M. W. McCracken (2009) “Forecast Disagreement Among FOMC Members”, Federal Reserve Bank of St. Louis Working Paper No. 2009— 059A, Research Division, Federal Reserve Bank of St. Louis, St. Louis, Missouri, December. Banternghansa, C., andM.W.McCracken(2014)“TheEffectofFOMCForecastDisagreement on U.S. Treasuries”, Presentation, International Symposium on Forecasting, Rotterdam, The Netherlands, July 1. Bergamelli, M., and G. Urga (2013) “Detecting Multiple Structural Breaks: A Monte Carlo Study and an Application to the Fisher Equation for the US”, draft, Cass Business School, London, March. Bernanke, B. S. (2012) “U.S. Monetary Policy and International Implications”, remarks at the seminar “Challenges of the Global Financial System: Risks and Governance under Evolving Globalization”, Bank of Japan, Tokyo, Japan, October 14. Boukus, E., and J. V. Rosenberg (2006) “The Information Content of FOMC Minutes”, mimeo, Federal Reserve Bank of New York, New York, July. Campbell, S. D., and S. A. Sharpe (2009) “Anchoring Bias in Consensus Forecasts and Its Effect on Market Prices”, Journal of Financial and Quantitative Analysis, 44, 2, 369—390. Castle, J. L., M. P. Clements, and D. F. Hendry (2015) “Robust Approaches to Forecasting”, International Journal of Forecasting, 31, 1, 99—112. Castle, J. L., J. A. Doornik, D. F. Hendry, and F. Pretis (2015) “Detecting Location Shifts During Model Selection by Step-indicator Saturation”, Econometrics, 3, 2, 240—264. Castle, J. L., N. W. P. Fawcett, and D. F. Hendry (2010) “Forecasting with Equilibrium-correction Models During Structural Breaks”, Journal of Econometrics, 158, 1, 25—36. 21

Chang, A. C., and T. J. Hanson (2015) “The Accuracy of Forecasts Prepared for the Federal Open Market Committee”, Finance and Economics Discussion Series PaperNo.2015—062,BoardofGovernorsoftheFederalReserveSystem,Washington, D.C., July. Chong, Y. Y., and D. F. Hendry (1986) “Econometric Evaluation of Linear Macroeconomic Models”, Review of Economic Studies, 53, 4, 671—690. Clements, M. P., and D. F. Hendry (2002) “An Overview of Economic Forecasting”, Chapter 1 in M. P. Clements and D. F. Hendry (eds.) A Companion to Economic Forecasting, Blackwell Publishers, Oxford, 1—18. Coibion, O. (2015) “Comments on Dovern, Fritsche, Loungani and Tamirisa”, International Journal of Forecasting, 31, 1, 155—156. Cox, D. R., and L. Brandwood (1959) “On a Discriminatory Problem Connected with the Works of Plato”, Journal of the Royal Statistical Society, Series B, 21, 1, 195—200. Danker, D. J., and M. M. Luecke (2005) “Background on FOMC Meeting Minutes”, Federal Reserve Bulletin, 2005, Spring, 175—179. Diebold, F. X., and R. S. Mariano (1995) “Comparing Predictive Accuracy”, Journal of Business and Economic Statistics, 13, 3, 253—263. Doornik, J. A. (2009) “Autometrics”, Chapter 4 in J. L. Castle and N. Shephard (eds.) The Methodology and Practice of Econometrics: A Festschrift in Honour of David F. Hendry, Oxford University Press, Oxford, 88—121. Doornik, J. A., and H. Hansen (2008) “An Omnibus Test for Univariate and Multivariate Normality”, Oxford Bulletin of Economics and Statistics, 70, supplement, 927—939. Doornik, J. A., and D. F. Hendry (2013) PcGive 14, Timberlake Consultants Press, London (3 volumes). Dovern, J., U. Fritsche, P. Loungani, and N. Tamirisa (2015) “Information Rigidities: Comparing Average and Individual Forecasts for a Large International Panel”, International Journal of Forecasting, 31, 1, 144—154. Engle, R. F. (1982) “Autoregressive Conditional Heteroscedasticity with Estimates of the Variance of United Kingdom Inflation”, Econometrica, 50, 4, 987—1007. Engle, R. F., D. F. Hendry, and J.-F. Richard (1983) “Exogeneity”, Econometrica, 51, 2, 277—304. Ericsson, N. R. (1992a) “Cointegration, Exogeneity, and Policy Analysis: An Overview”, Journal of Policy Modeling, 14, 3, 251—280. Ericsson, N. R. (1992b) “Parameter Constancy, Mean Square Forecast Errors, and Measuring Forecast Performance: An Exposition, Extensions, and Illustration”, Journal of Policy Modeling, 14, 4, 465—495. 22

Ericsson, N. R. (2011a) “Improving Global Vector Autoregressions”, draft, Board of Governors of the Federal Reserve System, Washington, D.C., June. Ericsson, N. R. (2011b) “Justifying Empirical Macro-econometric Evidence in Practice”, invited presentation, online conference Communications with Economists: Current and Future Trends commemorating the 25th anniversary of the Journal of Economic Surveys, November. Ericsson, N. R. (2012) “Detecting Crises, Jumps, and Changes in Regime”, draft, Board of Governors of the Federal Reserve System, Washington, D.C., November. Ericsson, N. R. (2015) “How Biased Are U.S. Government Forecasts of the Federal Debt?”, International Journal of Forecasting, forthcoming. Ericsson, N. R., E. J. Fiallos, and J. E. Seymour (2014) “Assessing Greenbook Forecasts of Foreign GDP Growth”, draft, Board of Governors of the Federal Reserve System, Washington, D.C., August. Ericsson, N. R., D. F. Hendry, and S. B. Hood (2015) “Milton Friedman as an Empirical Modeler”, Chapter 6 in R. Cord and J. D. Hammond (eds.) Milton Friedman: Contributions to Economics and Public Policy, Oxford University Press, Oxford, forthcoming. Ericsson, N. R., S. B. Hood, F. Joutz, T. M. Sinclair, and H. O. Stekler (2013) “Greenbook Forecasts and the Business Cycle”, draft, Board of Governors of the Federal Reserve System, Washington, D.C., December. Ericsson, N. R., and J. Marquez (1993) “Encompassing the Forecasts of U.S. Trade Balance Models”, Review of Economics and Statistics, 75, 1, 19—31. Ericsson,N.R.,andE.L.Reisman(2012)“EvaluatingaGlobalVectorAutoregression for Forecasting”, International Advances in Economic Research, 18, 3, 247—258. Fildes, R., and H. O. Stekler (2002) “The State of Macroeconomic Forecasting”, Journal of Macroeconomics, 24, 4, 435—468. Godfrey, L. G. (1978) “Testing for Higher Order Serial Correlation in Regression Equations When the Regressors Include Lagged Dependent Variables”, Econometrica, 46, 6, 1303—1310. Granger, C. W. J. (1989) Forecasting in Business and Economics, Academic Press, Boston, Massachusetts, Second Edition. Hendry, D. F. (1999) “An Econometric Analysis of US Food Expenditure, 1931— 1989”,Chapter17inJ.R.MagnusandM.S.Morgan(eds.)MethodologyandTacit Knowledge: Two Experiments in Econometrics, JohnWileyandSons, Chichester, 341—361. Hendry, D. F. (2001) “How Economists Forecast”, Chapter 2 in D. F. Hendry and N. R. Ericsson (eds.) Understanding Economic Forecasts, MIT Press, Cambridge, 15—41. Hendry, D. F. (2006) “Robustifying Forecasts from Equilibrium-correction Systems”, Journal of Econometrics, 135, 1—2, 399—426. 23

Hendry, D. F., andJ. A.Doornik(2014) Empirical Model Discovery and Theory Evaluation: Automatic Selection Methods in Econometrics, MIT Press, Cambridge, Massachusetts. Hendry, D. F., and S. Johansen (2015) “Model Discovery and Trygve Haavelmo’s Legacy”, Econometric Theory, 31, 1, 93—114. Hendry, D. F., S. Johansen, and C. Santos (2008) “Automatic Selection of Indicators in a Fully Saturated Regression”, Computational Statistics, 23, 2, 317—335, 337— 339. Hendry, D.F., andF.Pretis(2013)“AnthropogenicInfluencesonAtmosphericCO ”, 2 Chapter12inR.Fouquet(ed.)Handbook on Energy and Climate Change, Edward Elgar, Cheltenham, 287—326. Hendry, D. F., and C. Santos (2010) “An Automatic Test of Super Exogeneity”, Chapter 12 in T. Bollerslev, J. R. Russell, and M. W. Watson (eds.) Volatility and Time Series Econometrics: Essays in Honor of Robert F. Engle, Oxford University Press, Oxford, 164—193. Holden, K., and D. A. Peel (1990) “On Testing for Unbiasedness and Efficiency of Forecasts”, The Manchester School, 58, 2, 120—127. Jarque,C.M.,andA.K.Bera(1980)“EfficientTestsforNormality,Homoscedasticity and Serial Independence of Regression Residuals”, Economics Letters, 6, 3, 255— 259. Johansen, S., and B. Nielsen (2009) “An Analysis of the Indicator Saturation EstimatorasaRobustRegressionEstimator”, Chapter1inJ. L. CastleandN. Shephard (eds.) The Methodology and Practice of Econometrics: A Festschrift in Honour of David F. Hendry, Oxford University Press, Oxford, 1—36. Johansen, S., and B. Nielsen (2013) “Outlier Detection in Regression Using an Iterated One-step Approximation to the Huber-skip Estimator”, Econometrics, 1, 1, 53—70. Johansen, S., and B. Nielsen (2015) “Asymptotic Theory of Outlier Detection Algorithms for Linear Time Series Regression Models”, Scandinavian Journal of Statistics, in press. Meade, E. E., N. A. Burk, andM. Josselyn(2015) “The FOMCMeeting Minutes: An Assessment of Counting Words and the Diversity of Views”, FEDS Note, Board of Governors of the Federal Reserve System, Washington, D.C., May 26. Mincer, J., and V. Zarnowitz (1969) “The Evaluation of Economic Forecasts”, Chapter 1 in J. Mincer (ed.) Economic Forecasts and Expectations: Analyses of Forecasting Behavior and Performance, National Bureau of Economic Research, New York, 3—46. Nordhaus, W.D.(1987)“ForecastingEfficiency: ConceptsandApplications”, Review of Economics and Statistics, 69, 4, 667—674. 24

Nunes,R.(2013)“DoCentralBanks’ForecastsTakeIntoAccountPublicOpinionand Views?”, International Finance Discussion Paper No. 1080, Board of Governors of the Federal Reserve System, Washington, D.C., May. Pretis, F., M. L. Mann, and R. K. Kaufmann (2015) “Testing Competing Models of the Temperature Hiatus: Assessing the Effects of Conditioning Variables and Temporal Uncertainties Through Sample-wide Break Detection”, Climatic Change, 131, 4, 705—718. Romer, C. D., and D. H. Romer (2008) “The FOMC versus the Staff: Where Can MonetaryPolicymakersAddValue?”,AmericanEconomicReview,98,2,230—235. Sheng, X. (2015) “Evaluating the Economic Forecasts of FOMC Members”, International Journal of Forecasting, 31, 1, 165—175. Sinclair, T. M., F. Joutz, and H. O. Stekler (2010) “Can the Fed Predict the State of the Economy?”, Economics Letters, 108, 1, 28—32. Sinclair, T. M., H. O. Stekler, and W. Carnow (2015) “Evaluating a Vector of the Fed’s Forecasts”, International Journal of Forecasting, 31, 1, 157—164. Stekler, H. O. (2002) “The Rationality and Efficiency of Individuals’ Forecasts”, Chapter 10 in M. P. Clements and D. F. Hendry (eds.) A Companion to Economic Forecasting, Blackwell Publishers, Oxford, 222—240. Stekler, H. O., and H. Symington (2014) “How did the FOMC View the Great Recession As It Was Happening?: Evaluating the Minutes from FOMC Meetings, 2006—2010”, RPF Working Paper No. 2014—005, Research Program on Forecasting, Center of Economic Research, Department of Economics, The George Washington University, Washington, D.C., September. Stekler, H. O., and H. Symington (2016) “Evaluating Qualitative Forecasts: The FOMC Minutes, 2006—2010”, International Journal of Forecasting, in press. Tversky, A., and D. Kahneman (1974) “Judgment under Uncertainty: Heuristics and Biases”, Science, 185, 4157, 1124—1131. West, K. D., and M. W. McCracken (1998) “Regression-based Tests of Predictive Ability”, International Economic Review, 39, 4, 817—840. White, H. (1980) “Using Least Squares to Approximate Unknown Regression Functions”, International Economic Review, 21, 1, 149—170. White, H. (1990) “A Consistent Model Selection Procedure Based on -testing”, Chapter 16 in C. W. J. Granger (ed.) Modelling Economic Series: Readings in Econometric Methodology, Oxford University Press, Oxford, 369—383. Yellen, J. L. (2012) “Perspectives on Monetary Policy”, remarks at the Boston Economic Club Dinner, Federal Reserve Bank of Boston, Boston, Massachusetts, June 6. 25