Recent Developments in Bootstrapping Time Series

RecentDevelopmentsin BootstrappingTime Series Jeremy Berkowitz and Lutz Kilian” September 25, 1996 Abstract Inrecentyears,severalnewparametricandnonparametricbootstrapmethods havebeenproposedfor timeseriesdata. Which of thesemethodsshould appliedresearchersuse? We provideevidencethat for many applicationsin timeserieseconometricsparametricmethodsaremoreaccurate,andweidentify directionsfor futureresearchon improvingnonparametricmethods. We explicitlyaddressthe important,but oftenneglectedissueof modelselection inbootstrapping.Inpartictiartweemphasizethe advantagesof theAIC over otherlag orderselectioncriteriaandtheneedto accountfor lagorderuncertaintyinresampfing.We alsoshowthattheblocksizeplaysanimportantrole indeterminingthesuccessoftheblockbootstrap,andweproposeadata-based block sizeselectionprocedure. Our discussionalsohighlightsthe importance of accountingfor small-samplebiasin autoregressionsandsomeshortcomings of thestandardpercentileandpercentile-tintervalsinthetimeseriescontext. Key Words: Bootstrap, ARMA, Frequency Domain, Blocks JEL Classification: C13, C22 * We would liketo thank FrankDie~old, Robert StineandPeter Christoffersen for helpfulsuggestions. Any remainingerrorsareours. Correspondence to: JeremyBerkowitz, FederalReserve Board, Stop 61A, Washington, D,C. 20551, Telephone: (202) 736-5581, Email: mljmb02QFRB.GOV.

i 1 Introduction In recent years, many exciting developmentshave taken place in bootstrapping time series. Advances have proceeded along a number of distinct paths. Some authors have focused on adapting the familiar residual-based resampling approach of Efron (1979) to finite-order ARMA models. More recently, the focus has shifted toward residualbased nonparametric methods such asthe sievebootstrap and the Cholesky factor bootstrap, which treat the underlying population model as unknown. Other researchershave explored resampling blocks of time series data. Still another approach has been to develop algorithms which operate in the frequency domain. The advantage of this approach is that in the frequency domain there are iid variables which can be exploited for bootstrapping even when the original data are non-iid. Given this array of alternative bootstrap methods, which method should applied econometricians use? We observe that these algorithms differ in the extent to which they impose parametric structure on the data. Algorithms which makefew parametric assumptions are relativelylikelyto encompass the true model. However,methods which condition on aparticular parametricmodel affordhigherprecision. We provide evidence that suggests that for many applications in time serieseconometrics parametric methods may be preferable. Rather than rejecting nonparametric methods, we identify directions for future improvements. An important, but oftenneglectedissueinevaluatingtheperformanceof bootstrap algorithms is model selection. For lag order selection in autoregressive models, we stress the advantages of the AIC compared to more parsimonious criteria as well as the need to explicitly account for lag order uncertainty. Similar problems arise in nonparametric resampling. In particular, we show that the block size plays an important role in determining the successof the block bootstrap, and we propose an automatic data-based selection procedure. We also discuss the choice of bandwidth selection criteria for frequency domain bootstraps. Another concern in applied work is the presence of nonstationarities. We argue 2 —

that the asymptotic bootstrap theory for nonstationary data is more fully developed for the parametric case. In particular, parametric resamplingmethods have recently been shown to be valid, even for processeswith some explosive roots (Datta (1995)). Nevertheless, care must be taken in applying parametric bootstrap algorithms. In particular, we emphasize the importance of accounting for small-sample bias in autoregressive models. We also stress that in the time series context percentile-t intervalsmay perform poorly and their use should be supported with Monte Carlo evidence. In addition, standard percentile intervalsmay also require modifications. Incontrastto the discussioninL6ger,PolitisandRomano (1992), Carlstein(1992), Jeong and Maddala (1993), Young (1994) and Horowitz (1995), our main focus is on inferencein the linearstochastic regressormodel. Our discussioncomplements recent work by Li and Maddala (1996). Section 2 reviews parametric and nonparametric residual-basedbootstrap algorithms. Section3 discussesthe block bootstrap. Section 4 presentsbootstrap algorithms for the frequency domain. Section5 focuses on the treatment of nonstationary data. Section 6 contains a Monte Carlo study which compares these three approaches to bootstrapping. Section 7 concludes. 2 Residual Based Resampling Efron’s (1979) original bootstrap algorithm requiredresamplingfrom data which are, in population, independent and identically distributed. In the iid case,one can create artificialrepeated samplesby random resamplingwith replacementfrom the data. If the data display heteroskedasticityor serialcorrelation, a randomly resampled set of datawillnot preservetheseproperties,sothat statisticscalculatedfrom theresampled data (or from transformations of resampled data) will be inconsistent. Thus, the iid bootstrap fails for time-dependent data. One way to reduce time dependent data to an iid structure is to fit a parametric model. 3

2.1 Parametric Methods Early applications of the bootstrap algorithm to time-dependent data assumed that the underlying process follows a stationary finite order autoregressionof the form: A(L)gt = Et (1) whereEt- iid with E(~t) = Oand E(~2) < co. Y = (Y1)...)y~)~ denotes the observed data. A(L) is an invertible polynomial in the lag operator. For example, Efron and Tibshirani (1986) and De Wet and van Wyk (1986) bootstrapped the AR(1) model. Stine (1987) extended the analysis to the AR(p) model and RunNe (1987) to the finite orderVAR(p) model. The AR(p) model may be bootstrapped asfollows: 1. Determine the order of the AR(p) process. 2. Estimate the parametersA(L). 3. Generate bootstrap innovations E; by resamplingwith replacement from the empirical residualsEt= A(L)yt. 4. Generate a random draw for the vector of p initial observations Y: = (y;, ...,y;)’. . 5. Generate pseudo-data: A(L)y~ = S; conditional on Y;. 6. Calculate the bootstrap parameterestimates: A*(L). 7. Repeat steps 3-6 many times and build up the empirical distribution of interest.1 Under some additional regularity conditions, Bose (1988) proves that the bootstrap approximation improves the asymptotic accuracy of the OLS estimates in the AR(p) model from O(T-$) to o(T-~) almost surely. In practice, AR(p) models are almost always estimated by least-squares. If no intercept is included in the regression model, the residuals ;t must be recentered prior to resamplingto ensurethat theirbootstrap population mean is zero. It is also 1For a discussion on choosing the number of bootstrap replications, see Efron and Tibshirani (1993). 4

common to rescalethe empiricalresidualsby afactor of [~ —p/(~ —p —d)]l’2, where d denotes the number of estimated coefficients. The aim is to give the E: the desired variance (see Stine (1987), Peters and Freedman (1984)). The p initial conditions Y. = (gl, ...,~P)’likeall observations for g~are stochastic. While the effect of conditioning on a particular set of initial conditions is asymptotically negligible, it is not appropriate to condition on Y. in order to generate the bootstrap replicates. One way to randomize Y; is to set Y; = ~-1/2(~;l/2e~), where ~ isthe estimateof ~(YtY~) definedby A(L) and Yt= (g~,g~-l, ...,y~-P+l)’ (Lutkepohl (1991), p. 496). This procedure will preserve the second moment structure in the data. The problem with this method is that it requires the estimated process to be stationary. For nonstationary coefficient estimates, the procedure breaks down because r is noninvertible. Even for borderline stationary processes, there is a positive probability that some least-squaresestimateswill be explosive. A method which does not require matrix inversion is to pick arbitrary values for Y; in the recursion A(L) y; = e; and to discard the start-up transientsfor {y;}. Alternatively, one could build up the initial observations from the estimated moving average representation Y; = A-l(L)ej as in Rayner (1990), but this requires the truncation of an infinite sum. A third approach which avoids the truncation of an infinite sum and does not require start-up transientsis to divide the observed data into T – p + 1 overlapping blocks of length p and randomly selectone block with replacementfor Y;. This block initialization has been used for example in Stine (1987). An alternative class of parametric models are stationary MA(q) models: y~= (2) B(L)et whereB(L) denotesalag polynomial andEtand ytaredefinedasabove. MA(q) modelsarerarelybootstrapped in econometric practice, but it isstraightforwardto adapt the bootstrap algorithm for AR(p) models to the presentcontext. Simulation results for the bite order stationary MA(1) model can be found in De Wet and van Wyk 5

i (1986) and Bose (1990). Underfurtherregularityconditions, Bose (1990) provesthat the bootstrap approximation of the parameterestimatesin moving averagemodels is accurate to the order o(T–~ ). In contrast, the asymptotic normal approximation is accurate only to the order O(T–* ). Chatterjee (1986) applies the bootstrap algorithm to generalARMA(p,q) models of the form: A(L)yt = B(L)~t (3) where Etand yt are defined as above. A(L), B(L) are invertible polynomials in the lag operator, satisfying the assumption that together they imply var(yt) < M. The ARMA bootstrap algorithm proceeds as follows: 1. Determine the order of the ARMA(p,q) process. 2. Estimate the parameters: A(L), B(L). 3. Resample from: tt = B-l(L) A(L)yt (after recenteringthe ;t around zero). 4. Choose a large positive integer~, set y; = Ofor t < –~ and generate iid drawsfor e; for t = –T, ..., T. 5. Generate pseudo-data: y; = A-l(L) B(L)Ej for t= –T,...T, and retain the last T valuesof y;. 6. Calculate the bootstrap parameter estimates: A*(L), B*(L). 7. Repeat steps 3-6 many times and build up the empirical distribution of interest. Underregularityconditions, KreissandFranke(1989)prove theasymptotic validityof the bootstrap approximation for ML estimatorsin the finite-order stationary ARMA model.2 zThe parametric bootstrap may be robustified against possibleserialcorrelationinEt bY ‘ensamplingblocks of residuals using the block methods discussed in section 3 (e.g., Li and Maddala (1993)), or by explicitly modeling the error term (e.g., Lamoureux and Lastrapes (1990)).

2.1.1 Generating Bootstrap-Data in VAR Models Superficially, the bootstrap algorithm for VAR models is similar to the familiar algorithm for the regression model with fixed regressors. However, in autoregressive models the OLS estimates of the slope coefficients are systematically biased away from their population values. As a result, the standard bootstrap algorithm used by Runkle (1987) may be misleading in small samples. The size of the bias depends on the sample size, the persistence of the data generating process and whether a deterministic time trend is included in the regression. The tendency of the bootstrap to intensify the deficiencies of the OLS estimator was first observed by Kiviet (1984) in a linear regressionmodel with lagged dependent variables. Pope (1987) and Nichollsand Pope (1988) suggestbias-correcting the slope coefficients“prior to bootstrapping)) in order to improve the bootstrap approximation in the vector autoregressivemodel. They develop closed form expressionsfor the asymptotic first-order bias of the slope coefficients in the VAR model without a deterministic time trend. Recent work by Kilian (1995) implements and extends NichollsandPope)s (1988) proposal. Kilianusesresamplingto estimatethefirst-order coefficient bias in the VAR model with and without deterministic time trends. 2.1.2 Pitfalls in Constructing Confidence Intervals in VAR Models Even if we bias-correct the autoregressive coefficients A(L) prior to resampling the data, small-sample bias may cause additional problems in bootstrap inference. This is because the bootstrap estimates A*(L) themselves will be subject to stochastic regressor bias. This bootstrap bias tends to undermine the coverage accuracy of bootstrap confidence intervalsfor statistics that are functions of A(L), regardlessof the type of confidence intervalused. There is a widespread perception (e.g, Horowitz (1995), Li and Maddala (1996)) that more accurate finite-sample confidence intervalscan be obtained by bootstrap- 7

ping asymptotically pivotal statistics. A statistic is saidto be asymptotically pivotal if its limiting distribution does not depend on any unknowns. Many statistics of interestbased on AR(p) and ARMA(p,q) models areasymptotically normal and can be studentzzed to make them asymptotically pivotal. Consider a statistic ~. Percentile-t bootstrap intervalsarebased on the bootstrap approximation (~”– e)/S~(#*) of the studentized statistic, (e – O)/SD(b) . Unfortunately, in many time series models reliable measures of scale, SD(#*), do not exist. In particular, the distribution of the slope coefficients in the VAR model may undergo drastic changes as the dominant root of the process approaches unity. Bias in A*(L) tends to move the estimate @*(A*(L)) awayfrom its true value, changing its variance. As a result, the percentile-t bootstrap interval is unlikely to perform well without a suitable bias correction prior to estimating the variance. For example, Kilian (1995) reports that the percentile-t interval for VAR impulse responseestimatestendsto failspectacularly in smallsamples. Similarresultsfor the correlationcoefficientarewellknowninthebootstrap literature(Efron (1987)). While the percentile-t interval promises higher order asymptotic accuracy, the asymptotic behavior may be a poor indicator of its accuracy in finite samples. Notethat inthe context of hypothesistesting,thispitfalliseffectivelyovercome by resampling under the null (e.g., Zivot and Andrews (1992)) and/or using restricted estimation techniques (e.g., Nankervis and Savin (1996)). Li and Maddala (1996) provide an extensive discussion. Unfortunately, for interval estimation there is no specific null to refer to. In principle, it is possible to improve the small-sample performance of the percentile-t intervalwith variance stabilizing transformations, but these transformations are not generallyknown and have to be simulated. This adds another layerof bootstrapping and makesthe percentile-t method computationally burdensome (Efron and Tibshirani (1993)). We conclude that the percentile-t method should not be used blindly without supporting Monte Carlo evidence of its small-sampleproperties. 8

Why not directly bootstrap the unstudentized statistic of interest? As with the percentile-t, the percentile interval requires the statistic to be unbiased and scale invariant. This assumption is asymptotically valid for many statistics based onVAR slope coefficients, but it is not reasonable for their small-sample distribution. As a result, the percentile intervalcan be expected to perform poorly. The bias-corrected (BC) percentileintervaldiscussedin Efron and Tibshirani (1993) does not necessarily remedy this problem either, because it ignores scale effects. Intuitively, shifting the intervalendpoints to account for median bias in a scalardistribution failsto account for the changesin the shape of the distribution acrossthe parameterspace. The more generalBCQpercentile intervalof Efron and Tibshirani (1993) is designedto account forbiasand changesinthevarianceofthe statisticof interest. However,itsadaptation to time-dependent data has not been investigated. To date empirical evidence on the coverage accuracy of the percentile and BC intervals is scant. Kilian (1995) finds that these intervals perform erratically for VAR impulse response estimates. He also explores an alternative approach to removing bias and scale effects prior to bootstrapping. 2.1.3 Lag Order Uncertainty in Parametric Models The bootstrap can only be expected to perform well when the parametric model provides a good approximation to the true model. Determining the correct orders of an ARMA or AR model is thus a crucial issue. Chatterjee (1986), for example, reports simulation resultsfor ARMA(l ,1), ARMA(2,0) and ARMA(0,2) models. He compares bootstrap and asymptotic estimatesof standard errors. Chatterjee regards the bootstrap results as quite satisfactory, but observes that much of the attraction of this method depends on selecting the right order. He notes that the bootstrap performs poorly if the selected order is not correct. Recent work by Kilian (1996a) offers some guidance on selecting the lag order. For the AR(p) model, the lag order selection criterion need not be consistent for 9

the lag order for the bootstrap algorithm to be asymptotically valid. However, it is necessarythat the probability of underestimatingthe true lag order is aspptotically zero. Provided that the rangeof lagordersconsideredincludesthe true lagorder, this suggeststhat a wide range of information-based lag order selection criteria including the Akaike Information Criterion (AIC) are potentially valid criteria.3 Kilian (1996a) also points out that the consequences of bootstrapping an overparameterized VAR model may be very different from those of bootstrapping an under-parameterizedmodel. This suggeststhat lagorderselection criteriasuchasthe SchwarzInformation Criterion (SIC), whichareknownto be biaseddownwardinsmall samples, will result in poor bootstrap estimates. Kilian)s simulation results confirm that in small and moderate samples the coverage accuracy of bootstrap confidence intervalsfor VAR impulse response estimatesis much closer to nominal coverage for the AIC than for more parsimonious criteria such as the SIC or the Hannan-Quinn Criterion. Once it isexplicitly recognized that the lag order must be estimated, anothermajor difference between bootstrapping a fixed design model and a stochastic regressor model becomes apparent. The standard bootstrap algorithm for AR(p) models conditions on the lag order estimate asthough it werethe true lag order. Even if the lag order isestimated correctly, the standard algorithm ignoresthe sampling uncertainty about the lag orderestimateandmay leadto misleadinginferences. Masarotto (1990) and Kilian (1996b) therefore propose a generalizationof the bootstrap algorithm for VAR(p) models whichreflectsthe truesamplinguncertaintyof the lagorderestimate. This “endogenous lagorder” bootstrap algorithm doesnot condition on the initiallag orderestimate,but re-estimatesthe lag order in eachbootstrap iteration. Extensions of this idea to ARMA(p,q) models are straightforward. sThe ~~ymptotic validity of the AIC for bootstrapping followsfromresultsin paulsenand Tj@theim (1985) and Quinn (1988). Also see Potscher (1991, p. 179). 10

2.1.4 Conditional Bootstrap Prediction Applied researchersareoften interestedin the distribution of forecasts conditional on the lastp observationsof the samplepath. Bootstrapping the conditional distribution requiresthe last p observations in each bootstrap sample to be identical to the last p observations in the original data. The standardbootstrap algorithm for the AR(p) model is not appropriate for this purpose, because it does not constrain the values of the last p bootstrap observations. To solve this problem, Thombs and Schucany (1990) propose initializing the AR(p) with the last p observations and backcasting the time seriesusing the ‘backward representation’: A(L-l)yt = (4) W~ where the backward noise wt is the sequence defined by: A(L-l) (5) ‘t= A(L) ‘t If etisnormally distributed, one can useiid resamplingof the backward residuals, tit, based on (4) to generate conditional bootstrap sample paths. However, this algorithm is not valid for non-Gaussian innovations St. Findley (1986) shows that in the non-Gaussian AR(1) model, wt (though uncorrelated) is not iid. Breidt and Davis (1991) prove this result for the AR(p) model. Breidt, Davis, and Dunsmuir (1992, 1995) observe that the distribution and the dependence structure of wt are complicated, but that the sequence can be simulated if we rewrite 5 as: A(L)wt = A(L-l)st, (6) where wt is an ARMA (p,p) process driven by the iid sequence et. They propose the following algorithm: 11

1. Determine the lag order of the AR(p) process. 2. Estimate the parametersA(L) for the observed data {gt}, t = 1,..., T. 3. Compute ;t = A(L)yt for t = p+ 1,...T,. 4. Generate a bootstrap realizationw: of the backward noise Wtvia: A(L-l)E;, A(L)W; = using standard bootstrap techniquesfor ARMA(p,q) models. 5. Generate a bootstrap realization {g;} of {yt} passing through the last p observations of the sample path via: Y; = Yt t = T,T – 1,...T,– p + 1 (7) Y; = A(L-l)y; + W; t = T – p,T – p – 1,...,1 using the sequence of observations for w; from step 4. 6. Calculate the bootstrap estimates A*(L). 7. Repeat steps4-6manytimesandbuildup the conditional empiricaldistribution of the h-step ahead forecasts ~~+~. McCullough (1994) findsthat the conditional forecast distributions implied by the Thombs and Schucany (1990) procedure are very differentfrom those implied by the Breidt et al. procedure. Kabaila (1993) showsthat the conditional sample paths generated by the Breidt et al. procedure are not entirely correct. He proposes an exact bootstrap procedure based on an estimate of the pdf of Et. However, as Breidt, Davis, and Dunsmuir (1995) point out, estimating the density in small samples may be problematic, and the gainsfrom using exact ratherthan approximate conditional samplepaths appear small. 12

2.1.5 Bootstrapping State Space Models ARMA processes can also be cast in state-space form and consistently estimated using Kalman filtertechniques (e.g., Harvey (1989)). Stofferand Wall (1991) propose a bootstrap algorithmfor linearstatespacemodels. They showthat theiralgorithm deliversconsistent bootstrap standard errorsunder some regularity conditions . Define the state-space model as: St+l = Fst + Gxt + wt (8) Yt = ~st + Dxt + vt, whereytis a q x 1 vector of observed data, st is a p x 1 unobserved state vector and xt is an r x 1 vector of exogenous variables. F,G, H, and D are coefficient matrices. The innovations wtand vtareiid with zero mean andnonnegative definitecovariance matrices. E(vtvt) = R, E(wtwt) = Q, and E(wtvt) = O.The model coefficients and correlation structure are assumedto be uniquely parameterizedby the vector 0. The model (8) may, alternatively,be representedin innovations form. Let st+llt denote thebestlinearpredictor of st+l.Then theforecasterrorsareet = yt–~ stlt-l– Dxt with covariance matrix Zt = H Ptlt_lH’ + R . The innovations form is: St+llt = Fstlt_l + Gxt + FKt~t Yt = ~stlt-l + ~xt + Et, where Kt = Ptlt_lH’Z~l (the Kalman gain) and where Ptlt–listhe covariance matrix of St—st[t_l. Let e denote the Gaussian ML estimate, and define the (p+q) x 1 vector &t= [s~+llt,g;]’. Stacking the equations in (9) and evaluating at ~resultsin: 13

(10) F07 where A = 1 Ho ‘=[:1) c’=[F:$Y~:)l Stoffer and Wall 1991) devise the following bootstrap procedure: 1. Calculate 0 = argmax~ [- ~1 {logl~,(o)l +~t(~)~~;l(~)~t(o)}]. This A implies a set of forecast errors, &t(0)and the forecast error covariance matrix St(e). 2. Generate e; by sampling with replacement from the normalized residuals 2,= fi;l/2(e);,(e). 3. Generate pseudo-data, y~,by substituting e: for ;t in equation (10), holding fixed the exogenous variablesZt and the initial conditions. 4. Calculate bootstrap parameter estimates,d“. 5. Repeat steps 2-4 many times and build up the empirical distribution of interest. Nonparametric Methods 2.2 The bootstrap algorithmsdiscussedabove assumethat thetrue model isafiniteorder ARMA process with iid innovations. However, these models are at best viewed as approximations. A broader classof models arelinearautoregressionsof infiniteorder. If the true model is not finite-ordered, the asymptotic justification of the bootstrap approximation proposed by Bose (1988) and Kreiss and I?ranke(1992) is no longer valid. We will discuss two bootstrap algorithms designed for this class of processes: the sieve bootstrap and the Cholesky factor bootstrap. ,

2.2.1 The Sieve Bootstrap Buhlmann (1995, 1996a)considersaclassof linear,infinitedimensionalprocess which can be approximated by a sequence of finite-dimensional autoregressiveapproximations of order p (T) where p (T) ~ 00 and p (T) = o(T) as T + m. He argues that the standard OLS bootstrap for the AR(p) model may be given a nonparametric interpretation. In particular, he proposes estimating an AR(p(T)) model using the AIC and generating a bootstrap sample by resamplingthe residualsof the fitted model. This so-called sievebootstrap ismodel free within the classof linearMA(oo) processeswith polynomial decay. Buhlmann (1996a) proves that the sievebootstrap givescorrect approximations to thedistribution of smooth functions of linearstatistics of the data. Under the more restrictive assumption of exponential decay, Paparoditis (1996) proved that this bootstrap procedure delivers an asymptotically valid bootstrap approximation for the autoregressive coefficients and for the moving average representation of the VAR(p) model. Similar procedures have also been proposed in Swanepoel and Van Wyk (1986), Paparoditis and Streitberg (1992), and Kreiss (1992). Inrelatedwork, Bickeland Buhlmann (1996) propose a smoothed sievebootstrap for nonlinear, nonregular statistics. Their proposal involves resampling from the smoothed distribution of the empirical residualsof the approximating autoregressive model. Buhlmann (1996b) studies the sievebootstrap for autoregressivemodels including a deterministic time trend. 2.2.2 The Cholesky Factor Bootstrap While the sieve bootstrap effectively reinterpretsthe familiar parametric AR model as a device for nonparametric estimation, Diebold, Ohanian and Berkowitz (1995) formulate a bootstrap algorithm which does not require conditioning on any particular parametric model of the VARMA type. The context is the vector covariance stationary MA(oo). Any finite realization of length T thus has representation: 15

Y = PE, (11) whereP isrT x rT and e isrT x 1. The nonparametric bootstrap proceeds asfollows: 1. Consistently estimate COV(Y)=E, by applying a suitable truncation lag rule. 2. Take the Cholesky decomposition: PP’= ~. 3. Resample from the normal distribution: e“ w N(o,5). 4. Generate pseudo-data: Y*= ~~”. 5. Calculate bootstrap statistics: 8( y“). 6. Repeat steps 3-5 many times and build up the empirical distribution of interest. Alternatively, in step 3 the residualsmay be resampled without imposing Gaussianity by drawing from the empirical distribution of ; = P–lY, after resealingthe 2 so that they have a variance of 1. This “Cholesky factor” algorithm is a model-free method for generating pseudodata focusing on the second moment properties of the observed data. Note that the ARMA(p,q) parametric bootstrap generatespseudo-data from: Y; = A-l(L) B(L) E;. (12) The Cholesky factor bootstrap replacesthe parametric estimatesA-l(L)~(L) with a non-parametric estimate of the dynamics. Specifically, P = fili2 is lower triangular so that, T (13) j=l Inorderto consistentlyestimateP, thenumberof autocovariancesbeing estimated must grow with (but slower than) the sample size. This can be achieved by dOwnweighting the off-diagonal elementsof ~. Selecting a particular sequence of weights 16

amountsto choosing a bandwidth. This choice will, of course, affect the performance of the bootstrap. Severedownweightinginduces bias, while too little downweighting reduces efficiency. Thus, in place of the lag order selection problem in parametric models, the nonparametric Cholesky factor bootstrap requires a bandwidth choice. Data-based bandwidth selection procedures for consistent covariance matrix estimation may be found, for example, in Andrews (1991), Andrews and Monahan (1992) or Neweyand West (1994). 3 Resampling Blocks of Data The bootstrap algorithms discussed in section 2 all transform stationary time series data in a way that gives rise to iid residuals. These residuals may then be resampled with replacement. A different strategy has focused on resampling ‘blocks’ of contiguous time seriesobservations. Throughout this paper we focus on methods for resamplingoverlappingblocksof data (moving blocks). Resamplingoverlappingblocks may provide somewhat higher bootstrap estimation efficiency than non-overlapping blocks, although the availableevidence indicates that the efficiencygain issmall (e.g., Hall, Horowitz and Jing (1995)). Given a set of observations, yt, t = 1,..., T, define b = T – k + 1 blocks of data Xt = (yt,...,Yt+~-1) of length k. Kunsch (1989) and Liu and Singh (1992) independently propose resampling with replacement from the blocks (zl, Z2,...,x~) to form pseudo-data, (z~,z~,...,z;) of length T = Zk. The statisticof interestisthen calculated for each of many sets of pseudo-data. The distribution of the bootstrap statistic approximates the asymptotic distribution, as long as the size of the blocks increaseswith sample size. Under some conditions on the mixing coefficients of the data process, Kunsch (1989)provesthat theblock bootstrap providesavalidapproximation to the unknown distribution of the normalized univariate sample mean. Liu and Singh (1992) prove 17

the optimal block size tends to increase with the persistence of the time series as measuredbythe dominant root. Third, figure 1suggeststhat the performance of the moving blocks bootstrap tends to be fairly stable in the neighborhood ofthe optimal block size. This suggests that even afairly coarse grid for k will provide valuable information. Otherworkon block selectionincludesHall,HorowitzandJing (1995) andBuhlmann and Kunsch (1996) . Hall et al. propose an iterative empirical procedure for deter- . miningthe optimal block size. Startingwith aninitialguessfor k, they firstdetermine the optimal block size for a subseriesof the original data of length m < T. Using asymptotic expressionsfor the optimal block length, the resultfor the subseriesis recalibrated, so that it appliesto the originalfull sample size. This procedure may then be iterated until convergence is achieved. However,an important issuenot discussed in Hall et al. is the selection of the tuning parameter, m. Buhlmann andKunsch (1996) propose adata-drivenprocedure basedontheequivalence of the block size to the inverseof the bandwitdh of a lag weight estimator of the spectral density at frequency zero. This allows them to apply an iterative plugin method to select the optimal block size. In practice, their method requires an estimate of the influence function of the statistic of interest. 4 Resampling in the Frequency Domain A differentapproach to bootstrapping time seriesdata isto resampleinthe frequency domain. This researchis motivated by noting that, even for non-iid data, there are iid relationships in the frequency domain which can be exploited for bootstrapping. The algorithmsconsideredin this section requirethe data to be covariance stationary or appropriately detrended prior to bootstrapping. Ramos (1984) makesuse of the fact that taking the discrete Fouriertransform of 22

the covariance stationary data gives rise to iid normal variables. Specifically, 1 T–1 E Yt exp(~wjt) = a(wj) + b(wj)i (14) d’T ,=, where a(wj) and b(wj) are known as the Fourier coefficients. For any finite number of frequencies, CL(Wj) iid N(O,1/2) (15) = m Wj ~ mb(wj) asy iid N(O,~fz), Wj where~(w) isthespectral densityfunction (e.g., Brillinger(1981)). The independence holds across frequenciesand between the two coefficientsat each frequency. Ramos makesuse of the asymptotic independence in the following algorithm: 1. Consistently estimate the spectral density function (s.d.f). 2. Generate pseudo-Fourier coefficients: b*(Wj) = ~f(wj)~~(uj) with z~(w) iid N(O,l/2), z~(w)iid N(O,l/2). 3. Calculate pseudo-data by taking the inverseFouriertransform: T–1 y: =x [a*(wj) Cos(wjt) + b“(wj)Sin(Wjt)] j=() 4. Calculate the statistic of interestfrom the pseudo-data. 5. Repeat many times and build up the empirical distribution of 23

1 thevalidityof theblock bootstrap form-dependent data. Buhlmann (1994) provesthe asymptotic validity of the block bootstrap for statistics given by smooth functional of sample meansof vector valued observations. Related resultscan be found in Naik- Nimbalkarand Rajarshi (1994). In related work, Lahiri (1995) proposes a modified block bootstrap procedure for normalized sums of heavy-tail dependent variables. While these results establish asymptotic validity, Lahiri (1991, 1992), Gotze and Kunsch (1993), and Davison and Hall (1993) prove that the block bootstrap is second order correct for a wide class of studentized statistics based on sample means in the multivariate setting. That is, this bootstrap algorithm corrects for the second order term in the Edgeworth expansion which the asymptotic approximation cannot. However,the asymptotic refinementscannot in generalbe obtained by applying the usual formula for the test statistic to the block-bootstrap data. In fact, without suitable modifications, the rate of convergence by Kunsch)s method may be worse than the rate of normal approximation (e.g. Hall and Horowitz (1996)). Politis and Romano (1992a) extend the idea of the block bootstrap to estimates of parametersof the infinite-dimensionaljoint distribution of a stationary time series. Their procedure is as follows: 1. Resample blocks of data (x~,x~,...,x;) from the original blocks (ZI,Z2,...,Zb). 2. Define a function T(z~) on each block, such that the statistic of interest~ can be written, ~=~ ~ T(x~). i=l 3. Define blocks of statistics, Bj = (T(xj), T(xj+l), ...,T(x~+~)) . 4. Resample with replacement from the Bj which gives a sequence of statistics (TAT, ...,T(x~)) . 5. Calculate the bootstrap statistic T“=+ 5 T(x;). i=l Politis and Romano (1992a) motivate this ‘blocks of blocks’ bootstrap in part by showing that Kunsch’s (1989) original block bootstrap fails in the case of spectral densityestimators. The ‘blocks of blocks’ bootstrap, in contrast, deliversa consistent bootstrap distribution. Related work can be found in Politis and Romano (1992b), 18

Politis, Romano and Lai (1992), and Buhlmann and Kunsch (1995). Kunsch and Carlstein (1990) and Carlsteinet al. (1995) observe that pseudo-data generatedby concatenating resampledblocks of datawillnot preservethe dependence structure of the original data nearblock ‘endpoints’. They propose linkingthe blocks in a way designed to deliver a more natural transition from one block to the next. Even if the true data process is stationary,a particular draw of pseudo-data may not be. Politis and Romano (1994) propose the ‘stationary bootstrap’ which guaranstationary pseudo-data However,theirmethod requiresthe choice of additional tees tuning parameters. Usually,theoretical work on the moving block bootstrap assumesshort-range dependence; that is, the observations areassumedto satisfysome form of mixing conditionswith arapidly decayingmixing coefficient. Lahiri (1993) relaxesthisassumption and investigatesthe behavior of the moving block booststrap when the data exhibit long-range dependence. 3.1 How to Select the Block Size Moving blocks bootstrap algorithms requirethe researcherto choose a block size. Li and Maddala (1996) discuss several rules for block size selection based on specific models or on asymptotic mean-squared error (MSE) considerations. In this section, we propose a data-based procedure for choosing the block size in finite samples. We observe that choosing a block size involves a tradeoff. As the block size becomes too small, the moving blocks bootstrap destroys the time dependency of the data and its averageaccuracy will decline. As the block sizebecomes too large, there are few blocks and pseudo-data will tend to look alike. As a result, the average accuracy of the moving blocks bootstrap also will decline. This suggests that there . exists an optimal block size k which maximizes accuracy. The proposed procedure automatically selectsthisblock sizefor agivenseriesand statisticof interest,regard- 19

lessof the sample size,persistenceor lag structureof the underlying process. We will effectively use the bootstrap as an aid in block size selection. Consider a stationary series {Yt o } f length T: 1. Approximate the underlying MA(m) process by a parametric ARMA(p,q) or AR(p) model. 2. Generate many Monte Carlo trialsof length T from this fitted model. 3. Foreach Monte Carlo trial generatemoving blocks bootstrap data {y;} for alternative block sizesk. 4. Calculate the statistic of interestfor {g:(k)}. 5. Select the block size k which on averageproduces the most accurate test statistic, point estimate, or confidence intervalacross Monte Carlo trials. 6. Use that block size ~ to apply the fully nonparametric moving blocks algorithm to the original data {gt}. We illustrate this procedure for the quarterly time series on the S&P common stock earnings-price ratio (CitiBase code: FSEXP) for 1947.2-1994.3. Based on the AIC, we fit an AR(2) model with intercept to this series. After bias-correcting the autoregressivecoefficients using the closed-form expressionof Pope (1990) we obtain the estimate: y~= 0.3814+ 1.2514y~-~– 0.2902y~-~+ ~t, o: = 0.6105 Our statistic of interest is the response of the earnings-price ratio to a one-standard deviation shock. We will trace out this impulse response function for 16 quarters after the initial shock. We generate 1,000Monte Carlo trialsfrom this model to evaluatethe probability contentof the 90percentmoving blocks bootstrap confidenceintervalsfor thestatistic of interest. For each Monte Carlo trial we generate 1,000 moving blocks bootstrap replications of length T = 190for alternativeblock sizesk c {4, 12,24,36,48,60,72, 20

I 84,96,108,120,136, 144} using Kunsch’s (1989) algorithm. For each bootstrap replication, we resamplethe data conditional on k, fit an AR(@*) model to {g;} based on the AIC lag order estimate, and construct the implied impulse response function for up to 16 quarters after the initial shock. Then we build up the empirical distributionsof the impulse response coefficientsand construct nominal 90percent bootstrap confidence intervalsfor each time horizon. The probability content of these intervals is evaluated across the 1,000 Monte Carlo trials. Rather than compare the coverage accuracy of the intervals for each block size by visual inspection of the coverage plots across the time horizon z = O,...,16, we construct a simple statistical summary measure. Assuming a quadratic loss function and equal weights for all time horizons of the impulse response function, we average the squared deviations from nominal coverage across the time horizon and tabulate them as a function of the block size. Then the optimal block size is: fi~ 1 1 1 = argmin (coverage(k, i) – 0.9)2 . k [ 2—0 Figure 1 plots the mean squared deviations from nominal coverage as a function of k. As expected the curve follows a U-shape. The global minimum is at ~ = 48 (quarters). This resultis of interestfor severalreasons. First, it shows that the performance of the moving blocks bootstrap can be highly sensitive to the choice of the block size. Second, figure 1 indicates that appropriate block sizes are much larger than some illustrative examples in the literature would suggest. For example, Efron and Tibshirani (1993) considerblock sizesof 3and 5forbootstrapping theslope coefficient in an AR(1) model with T = 48. Forthe samemodel, Kunsch (1989) considersk = 4 for a data setof length T = 120. Our procedure indicates that the optimal block size formacroeconomic time seriesmaybe up to 12timeshigherthanthevaluessometimes considered for similar models in the literature. Additional examples suggest that 21

I interest. Despite the frequency domain setting, this algorithm is a generic method for generatingpseudo-data. Ramos (1984) and Stine (1985) also discussa versionof this method which does not impose normality. Ramos (1988) provides a general and thorough treatment of frequency domain resamplingplans for linearfunctional of the spectral density, o~=Jo(w)f(w)d(w)> (16) where O(w) is an even periodic function satisfying some regularity conditions and ~(~) is the s.d.f. of the process. Examples of statistics of this form include the variance and autocovariances. The corresponding estimates must be a linear functional of the periodogram: ~y= f O(w)IY(w)d(w). Ramos therefore proposes the following procedure: 1. Obtain a consistent estimate ~(w). 2. Generate pseudo-data y: from the Gaussian distribution described by ~(w). Since a Gaussian distribution is completely characterizedby its s.d.f. (ignoring the mean of the process for convenience), N(O,fi) may be equivalently written as N(O,~(w)). How can we draw from data from N(O,~(w))? Ramos suggests: Takethe first, say,m terms in the Fourierseriesexpansion of f(w): ‘y,,...,~~. These are the firstm covariancesof the process, with which the covariance matrix may then be constructed. In Ramos) notation, ~ = Top(yl, ...,y~). Pseudo-data is then drawn from N(O,~). 3. Calculate ey. = ~O(w)IV.(w)d(w). 4. Repeat many times and build up the empirical distribution of interest. 24

Ramos’ main resultis that under some regularityconditions, including normality, (17) that is, the bootstrap principle holds. Note that in both the Ramos (1988) and Gaussian Cholesky factor procedures, the pseudo-data are drawn from y* ~ N(O,~). Ramos estimates S by inverting a consistently estimated spectral density function. In contrast, the Cholesky factor procedure generates consistent estimates E in the time domain by setting ~ = Top(Aoyo,...,~~-1~~-1), whereTop(”) is the operator which createsa toeplitz matrix from a singlerow of covariances,and where Atare decreasingweights. One can show that, if the Atform a Bartlett window, the Cholesky factor procedure is equivalentto inverting the s.d.f estimated with a Fejer spectral window in Ramos) procedure. An algorithm explicitly designed to bootstrap the spectral density function itself is given in Franke and Hardle (1992). They make use of the same as~ptotic relationships asRamos (1984), but in a differentway. Equations (15) immediately imply that, ( 2 & m a(w) b(u) 2 ~ +& x; (18) w u ( m )d and so 2 /. (19) But from the definition of Fouriercoefficients, equation (19) implies that 2 —I(w) L x;, (20) w f( ) where I(w) is the periodogram. Equation 20 holds approximately in finite samples. Thus, Frankeand Hardle (1992) suggestthe following bootstrap algorithm: 25

1. Compute the periodogram, 1(w), and consistently estimate the s.d.f., ~(u). 2a. Resample from E*= x~/2 or 2b. Resample from the empirical distribution: ~“ = ~, where the number of estimated residualsequals the number of frequencies. 3. Calculate the bootstrap periodogram ordinates: 1(w)* = &*~(w). 4. Calculate bootstrap spectral density estimates: ~“(w) =x k(w)l”(u), by smoothing the periodogram ordinates. 5. Repeat steps 2-4 many times and build up empirical distribution of interest. Frankeand Hardle (1992) prove the consistency of the bootstrap distribution of the pivoted s.d.f. They also report the results of a simulation study for an AR(5) model at five discrete frequencies. They find that the bootstrap performs favorably relativeto the asymptotic in capturing the finitesample skewnessof the distribution of the s.d.f. estimates. Berkowitzand Diebold (1996) describe amultivariategeneralizationof the Franke andHardle(1992)procedure. Equation 20is,infact, aspecialcaseof themoregeneral result: (see, for example, Brillinger (1981)). IVV(U)is the r x r periodogram matrix and FYV(W)isthe rx rspectral densitymatrix of anr-dimensionalvector random variable, Yt. The asymptotic distribution is complex Wishart. A multivariate version of the bootstrap is implemented by noting that Thus, step 2 above is replaced with 26

2a. Resample horn the empirical distribution: ~“w F&l/2(w)Iyy(u)Fy;’/2(w) or 2b. from the parametric distribution: E*~ W~(l, I.). The frequency domain methods discussedin this section maybe distinguishedby the statistics they are designed to bootstrap. Whereas some algorithms are specifically designed for bootstrapping the s.d.f. or linear functional of the s.d.f., others are omnibus procedures for generating pseudo-data. A common feature is that they generally require a consistent estimate of the s.d.f. and thus a bandwidth choice. However, the effect of the bandwidth choice on the performance of the bootstrap remains an open question. For frequency domain bootstraps which require the calculation of the entire spectral density function (rather than the spectral density at frequency zero), the data-based bandwidth selection procedures of section 2 are not appropriate. Automatic bandwidth selection procedures for the entire s.d.f. have been suggested, for example, by Beltra6 and Bloomfield (1987). 5 Bootstrapping Nonstationary Data If the true process is nonstationary, many of the standard resultsof the asymptotic validity of the bootstrap approximation no longer apply. For example, the Cholesky factor and the frequency domain bootstrap assume stationarity, as does nearly all the work on moving blocks bootstraps. Lahiri (1992) showsthat, under appropriate conditions, the Kunsch procedure is second-order correct even for the sample mean of nonstationary data. It is not known to what extent his resultsgeneralizeto other statistics. Forthe parametric AR(I) model, Basawa,Mallik, McCormick, Reeves,and Taylor (1991) and Datta (1992) prove that the standard bootstrap algorithm for the level autoregressionis invalid if the true model is a random walk model. Intuitively, this result arisesbecause of the discontinuity of the asymptotic distribution at the unit circle and generalizesto all exact unit root VAR models estimated in levels. If the model is known to contain an exact unit root, resampling remains valid if 27

weimpose that unit root in estimation. Inparticular, in the absenceof cointegration, autoregressiveI(1) processes can be estimated in firstdifferencesand the coefficients converted to the level representation. Similarly,nonparametric methods continue to be valid if resampling is based on first-differenceddata. While nonparametric bootstrap methods can easily deal with I(1) processes, there are no theoretical results to show that nonparametric resampling preserves cointegration relationships in the data. In fact, cointegration itselfmay be viewed asa parametric notion. Thus, if the data are known to be cointegrated, parametric methods are preferable. For example, if the true process were a cointegrated VAR, one would resample from the ML estimate of a vector error correction model. Since the innovations in that model are identical to the innovations in the levelautoregression,the usualbootstrap algorithm is valid. Kilian (1995) reports that in many cases the bootstrap algorithm is quite accurate for vector errorcorrection models, provided the cointegration rankisknown. However,the estimation of a lineartime trend or the presence of additional roots of large magnitude in the system may lead to similar bias problems as in stationary autoregressions. Li and Maddala (1996) provide a detailed discussion of the special problems involved in bootstrapping the cointegration relationship itself. In applied work the existence of a unit root or cointegration is rarelyknown with certainty. For some econometric questions this is not a problem. For example, if we are interested in approximating the finite sample distribution of a test statistic under the null hypothesis of a unit root, it is correct to simply impose the unit root in estimation. However, not all inference problems involve a unit root null hypothesis. For example, bootstrap confidence intervals do not involve a specific null. Rather, the user faces the choice between ignoring the possible presence of unit roots or relying on the result of pre-tests with low power. one would like to think that the level bootstrap still provides a satisfactory approximation for roots arbitrarily close to unity. Howevertthere is little evidence to support that view. For example, the accuracy of the bootstrap approximation for the level autoregression 28

can be expected to deteriorate, as the persistence of the process rises (Bose (1988)). Simulationevidence in De Wet andvanWyk (1986) for the AR(1) coefficient seemsto confirm that conjecture. We conjecture that this resultislargely due to small-sample bias, and that, in light of the argumentsin sections 2.1.1. and 2.1.2, the accuracy of the bootstrap could be substantially improved by appropriate bias corrections. What if the roots of the process are explosive? There are no known results for nonparametric bootstrap methods involving explosive roots, but, interestingly, the parametric bootstrap can be shown to be theoretically valid for such models. Basawa, Mallik, McCormick, and Taylor (1989) prove the validity of the bootstrap in the explosive AR(1) model with finite error variance. Datta (1995) shows that the limit of the bootstrap distribution in the AR(p) model convergesto that of the OLS slope coefficient estimate in probability without any additional moment restrictions. In particular, Datta proves that the bootstrap approximation as measured by the Kolmogoroff distance goes to zero almost surely provided E Ietl < m. Thus for the explosive case the bootstrap principle works, even if the error distribution is heavy tailed. These resultsseemof limited practical relevance,but Datta (1995) takesthe analysis one step further. He shows that for E le~l < m, the standard OLS bootstrap offers an asymptotically valid approximation even in the partially explosive AR(p) model (when the characteristic polynomial admits roots both inside and outside the unit circle). Datta assumesthat for some k, 1 < k ~ p, exactly k roots of the characteristic polynomial lie inside and the remaining s = p—k roots lie outside the unit circle. Note that for k = p the model is stationary and for k < p it is partially explosive. Exact unit roots are ruled out by assumption. For the OLS estimator, Datta proves that the error in bootstrap approximation (measuredin sup norm) converges to zero almost surely. His proof subsumes the stationary case, strengthening Kreiss and Franke’s (1992) result about convergence in probability. Ignoring the possibility of purely explosive AR(p) models, Datta’s result suggests that, at least asymptoti- 29

Cally,the standardbootstrap isvalidforprocesseswith roots arbitrarilycloseto unity, both insideand outside the unit circle. It thus strengthensthe casefor bootstrapping level autoregressions. 6 A Monte Carlo Comparison of Bootstrap Methods The various bootstrap algorithms of the previous sections differ in how much parametric structure they impose in estimation. Forexample, Cholesky factor, frequency domain and block bootstrap procedures are completely nonparametric. They do not use any particular model to generate pseudo-data. on the other hand, parametric AR(p) bootstrap procedures make strong assumptions about the form of the time dependency of the data. Which procedures should applied time serieseconometricians use? We conjecture that for the sample sizes macroeconomists tend to work with, parsimony is essentialand thus parametric models may be the only reasonable choice. To verify this conjecture, we compare the accuracy of the parametric AR(p) bootstrap, the Cholesky factor bootstrap, the Ramos (1984) frequency domain bootstrap, and the moving blocks bootstrap algorithm of Kunsch (1989) by Monte Carlo simulation. We consider confidence intervals for the responses of the T-bill rate to a onestandard deviation shock over the subsequent 16 quarters. We believe that this example isof broad interestto applied users. Impulse responsesplay an important role inmacroeconometrics. Further,they sharemanystatisticalpropertieswithmulti-step ahead forecasts. In the Monte Carlo experiment, we consider three sample sizes, corresponding to 20 and 40 years worth of quarterly data and 40 years worth of monthly data: T = 80,160,480. Our data generating process is based on quarterly U.S. T-bill data 30

(CitiBase code: FYGM3) for 1971.4-1993.4. The AIC suggests an ARMA(2,4) data generatingprocess: yt = 0.8871+ 0.3499 ~~-1+ 0.5231y~-z+ 1.0004~t-l – 0.1103 ~t_z + 0.0021et-s + 0.3492 Et-4+ Et, with ~z = ().7124.The innovations Etarenormally distributed. & An important concern in this study is model selection. The AR(p) estimate underlying the parametric bootstrap isbased on the minimum of the AIC for 1< p <8. For the Cholesky factor bootstrap, we use the automatic bandwidth selection procedure of Andrews (1991). For computational reasons,the bandwidth is not permitted to exceed three quarters of the sample size. We use the algorithm of Beltra6 and Bloomfield (1987) for the Ramos (1984) bootstrap. For the moving blocks bootstrap we use the procedure outlined in the previous section to select the block size.4 Foreach bootstrap replication {y;} weusethe AIC to estimatethe lagorder {p*}, fit an AR({p*}), and calculate the statistic of interest. The confidence intervalsare based on the 0.05 and 0.95 percentile intervalendpoints of the empirical distribution of the impulse response coefficient estimates. Figures2-4 plot the effective coverageratesof the nominal 90percent intervalsfor each method and sample size. Forthe sample sizesconsidered the parametric AR(p) bootstrap (labeled AR) intervalclearly dominatestheother threebootstrap intervals. The Cholesky factor (CHOL), the Kunsch (1989) block bootstrap (BLOCK) and the Ramos (1984) frequency domain bootstrap (RAMOS) intervals perform erratically AFO~rOmPUtatiO~neaal~onw~edeterminetheoptimalblocksizea prioribasedontheAR(P) approximatioonftheactualdata.TheAICsuggestsanAR(6)modelfortheT-Billratedata.We bias-correcttheautoregressivceoefficienetstimateussingtheexpressionisnPope(1990).Conditional on this DGP, we find k = 36 for T = 80and k c {12, 24,36,48,60, 72}, k = 60 for T = 160 and k G{12,24, 36,48,60,72,84, 96}, and k = 120for T = 480and k l {80,120,160,200,2402,80}. 31

i and are generally unreliable. However, the Cholesky bootstrap performs somewhat better than the moving blocks bootstrap. The extremely poor performance of the Ramos bootstrap may in part reflect the bandwidth selection criterion employed. It also suggests that the Ramos procedure may not be consistent for our statistic of interest. With the exception of the Ramos bootstrap, coverage accuracy tends to improve with sample size. However,in absolute terms none of the methods can be considered adequate. Even the AR(p) bootstrap interval consistently falls short of nominal coverage. This coverage deficiency is consistent with the arguments we presentedin section 2. Further improvements in coverage accuracy are likely to require interval calibration (Breidt et al. (1995)) or some form of bias correction in the statistic of interest (e.g., Rudebusch (1993), Andrews and Chen (1994)). To illustratethispoint, we added resultsfor the sameAR(p) bootstrap with additional bias corrections (AR- BC) for the impulse responseestimatesbased on Kilian (1995). The improvement in coverage accuracy is sizable. Similarimprovements are likelyfor the other methods. 7 Conclusion We reviewed a range of alternative parametric and nonparametric bootstrap algorithms for time-dependent data. These methods differ in the extent to which they impose parametric structure on the data. In highly parsimonious models, parameters are estimated with many degreesof freedom. Bootstrap estimates, in turn, are comparatively accurate. However,parsimony is also likely to increase the ‘)distance” between the fitted and the true model. Conditioning on a m.isspecifiedmodel may cause the misspecification to be propagated (and possibly magnified) through resampling. We provided some preliminary Monte Carlo evidence that for typical sample sizesfaced by macroeconomists, parsimony is essentialand thus parametric models may be the only reasonablechoice. However,care must be exercised to overcome the 32

i drawbacks of bootstrapping parametric models in small samples. In particular, we stressedthe choice of lag order selection criteria, the treatment of lag order uncertainty,and the need for bias corrections in small samples. We also showed that the accuracy of the moving block bootstrap can be highly sensitiveto the choice of block size. We proposed an automatic data-based procedure forselectingthe block size. We successfullyapplied thisprocedure to severaleconomic timeseriesand determinedthat formacroeconomic time seriesthe optimal block sizes tend to be much larger than those sometimesused in the literature. While our preliminary Monte Carlo evidence suggests that nonparametric bootstrap methods may perform poorly in small samples, resultsfor other statistics and data generating processes would be useful. In particular, the use of an ARMA data generating process may have biased the results in favor of parametric bootstraps. Additional researchis needed to determine whether nonparametric algorithms enjoy special advantages for data generating processes that are not encompassed by the ARMA framework. Moreover, it would be important to obtain a senseof the sample sizesrequired for reliable inference based on nonparametric methods. That information would be useful for the analysis of higher frequency financial data. However, thesecaveatscannot obscure that nonparametric methods need to be improved to be of much use in small and moderately large samples. In particular, the comparatively poor performance of the Cholesky factor, the block bootstrap and the frequency domain bootstrap for the sample sizesconsidered is surprising. A partial explanation could be the slow rate of convergence of nonparametric methods, but it would be premature to discard these methods. Indeed, Diebold, Ohanian and Berkowitz (1995) presentMonte Carlo evidence which suggests that the Choleskybootstrap deliversfarbetter coveragefor spectral densityestimates than the asymptotic approximation. The Ramos bootstrap does not appear suitable for impulse response estimates. More theoretical work is needed to establish a class of statistics for which this algorithm may be used. 33

More research is needed to clarify how the choice of the bandwidth affects the performance of these algorithms. We conjecture that their performance could be improved by refining the bandwidth selection process. In this paper, we used the Andrews (1991) and the Beltra6 and Bloofield (1987) automatic bandwidth selection criteria. It would be of interest to systematically compare the performance of frequency domain methods to other data-based bandwidth selection criteria (e.g., Andrews and Monahan (1992), Newey and West (1994)) or parametric devices for estimating the spectral density. For example, den Haan and Levin (1996) report that autoregressiveestimatesof the spectral density at frequency zero outperform kernelbased estimates. In addition, bootstrapping the bandwith selection process is likely to improve small-sample performance. Similarly,theperformance of the Kunsch (1989) moving block bootstrap leftmuch to be desired. In the future, it would be valuable to study more sophisticated block bootstraps such as the linked-block bootstrap of Kunsch and Carlstein (1990), the stationary block bootstrap of Politis and Romano (1994) or the asymptotic refinements proposed by Lahiri (1991, 1995) and Gotze and Kunsch (1993). Moreover! a systematic comparison of the block sizeselectionmethods proposed in this paper and elsewherewould be useful. 34

References [I] Andrews, D.W.K. (1991), “Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation,” Econometrzca, 59, 817-858. [2]Andrews, D.W.K, and H.Y. Chen (1994), “Approximately Median-Unbiased Estimation of Autoregressive Models,” Journal of Business and Economic Statistics, 12, 187-204. [3] Andrews, D.W.K, and J.C. Monahan (1992), “An Improved Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimator,)’ Econometrzca, 60, 953-966. [4] Basawa, L.V., A.K.Mallik, W.P. McCormick, and R.L. Taylor (1989), “Bootstrapping Explosive Autoregressive Processes,” Annals of Statistics, 17, 1479- 1486. [5] Basawa, L.V., A.K.Mallik, W.P. McCormick, J.H. Reeves, and R.L. Taylor (1991), “Bootstrapping Unstable First Order Autoregressive Processes,” Annals of Statistics, 19, 1098-1101. [6] Beltra6, K.I. and P.Blootield (1987), “Determining the Bandwidth of aKernel Spectrum Estimate,” Journal of Time Seties Analysis, 8, 21-38. [7] Berkowitz, J. and F.X. Diebold (1996), “Bootstrapping Multivariate Spectra,” manuscript, Department of Economics, Universityof Pennsylvania. [8] Bickel, P., and P. Buhlmann (1996), “Mixing Property and Functional Central Limit Theorems for a SieveBootstrap in Time Series,” manuscript, Department of Statistics, Universityof California, Berkeley. [9] Bose, A. (1988), ‘)Edgeworth Correction by Bootstrap in Autoregressions,))Annals of Statistics, 16, 1709-1722. 35

[10]Bose, A. (1990), “Bootstrap in Moving Average Models,” Annals. Institute of StatisticalMathematics, 42, 753-768. Breidt, F.J., and R.A. Davis (1991), “Time-reversibility, Identifiability, and In- [11] dependence of Innovations in Stationary Time Series,” Journal of Time Series Analysis, 13, 377-39o. [12]Breidt, F.J., R.A. Davis, and W.T.M. Dunsmuir (1992), “On Backcasting in Linear Time SeriesModels,” in New Directions in Time Seties Analysis. Part I, D. Brillinger et al., eds., Springer-Verlag,New York. [13]Breidt, F.J., R.A. Davis, and W.T.M. Dunsmuir (1995), “Improved Bootstrap Prediction Intervalsfor Autoregressions,” Journal of Time Series Analysis, 16, 177-200. [14]Brillinger, D.R. (1981), Time Series: Data Analysis and Theory. San Francisco: Holden-Day. [15]Buhlmann, P. (1994), “Blockwise Bootstrapped Empirical Processesfor Stationary Sequences,” Annals of Statistics, 22, 995-1012. [16]Buhlmann, P. (1996a), “Sieve Bootstrap forTime Series,”to appearin BernouZZz. [17]Buhlmann, P. (1996b), “Confidence Regions for Trendsin Time Series: A Simultaneous Approach with a Sieve Bootstrap,” Technical Report 447, Department of Statistics, Universityof California, Berkeley. [18]Buhlmann, P., and H.R. Kunsch (1996), “Block Length Selection in the Bootstrap for Time Series,)’manuscript, Seminarfur Statistik, ETH Zurich. [19]Buhlmann, P., and H.R. Kunsch (1995), ‘The Blockwise Bootstrap for General Parametersof a Stationary Time Series,))ScandinavianJournalof Statistics, 22, 35-54. 36

[20]Carlstein, E. (1992), “Resampling Techniquesfor Stationary Time Series: Some Recent Developments,” in New Directions in Time Series Analysis. Part I, D. Brillinger et al., eds., Springer-Verlag,NewYork. [21]Carlstein, E.l K.-A. Do, P. Hall, T. Hesterberg, and H.R. Kunsch (1995), “Matched Block Bootstrap for Dependent Data,” Technical Report 74, Seminar fur Statistik, ETH Zurich. [22]Chatterjee, S. (1986), ’’Bootstrapping ARMAModels: Some Simulations,” IEEE Transactions Systems, Man and Cybernetics, SMC-16, 294-299. [23] Datta, S. (1992), “On Asymptotic Properties of Bootstrap for Unstable AR(1) Processes,” Technical Report, Department of Statistics, Universityof Georgia. [24] Datta, S. (1995), “Limit Theory and Bootstrap for Explosive and Partially ExplosiveAutoregression,” StochasticProcesses andtheirApplications, 57,285-304. [25] Davison, A.C., and P. Hall (1993), “On Studentizing and Blocking Methods for Implementing the Bootstrap with Dependent Data,” Australian Journal of Statistics, 35, 215-224. [26] De Wet, T., and J.W.J. van Wyk (1986), “Bootstrap Confidence Intervalsfor RegressionCoefficientswhenthe ResidualsareDependent,” JournalofStatistical Computation and Simulation,23, 317-327. [27] Den Haan, W.J. and A. Levin (1996), “Inferences from Parametric and Nonparametric Covariance Matrix Estimation Procedures,” manuscript, Board of Governors of the FederalReserve System. [28] Diebold, F.X., L.E. Ohanian and J. Berkowitz (1995), “Dynamic Equilibrium Economies: A Frameworkfor Comparing Models and Data,” National Bureau of Economic Research Technical Working Paper No. 174. 37

[29] Efron, B. (1979), “Bootstrap Methods: Another Look at the Jackknife,” AnnaZs of Statistics, 9, 1218-1228. [30] Efron, B. (1987), “Better Bootstrap Confidence Intervals,” JournaZof the American StatisticalAssociation, 82, 171-200. [31] Efron, B. and R. Tibshirani (1986), “Bootstrap Methods for Standard Errors, Confidence Intervals,and other Measuresof StatisticalAccuracy,” StatisticalScience, 1754-77. [32] Efron, B. and R. Tibshirani (1993), An Introduction to the Bootstrap. Chapman and Hall, New York. [33] Findley, D.F. (1986), “On Bootstrap Estimates of Forecast Mean Square Errors for Autoregressive Processes,)) in: Computer Science and Statistics: The Interface. D.M. Allen, cd., Elsevier Science Publishers, Amsterdam. [34] Franke,J.andW. Hardle(1992), “On Bootstrapping KernelSpectralEstimates,” Annals of Statistics, 20, 121-145. [35] Gotze, F., and H.R. Kunsch (1993), ‘)Clockwise Bootstrap for Dependent Observation: Higher-Order Approximations for Studentized Statistics,’) Technical Report, Universitat Bielefeld, Germany. [36] Hall, P., and J.L. Horowitz (1996), “ Bootstrap Critical Values for Tests Based on Generalized-Method-of-Moments Estimators,” Econometric, 64, 891-916. [37] Hall, P., J.L. Horowitz and B.-Y. Jing (1995), “On Blocking Rules for the Bootstrap with Dependent Data,)’ Biomettika, 82, 561-574. [38] Harvey,A.C. (1989), Forecasting,Structural Time Series Models andtheKalman Filter. Cambridge: Cambridge UniversityPress. 38

[39] Horowitz, J.L. (1995), “Bootstrap Methods inEconometrics: Theory andNumerical Performance,’) manuscript, Department of Economics, Universityof Iowa. [40]Jeong, J. and G.S. Maddala (1993), “A Perspective on Application of Bootstrap Methods in Econometrics,” Handbookof Statistics, vol 11. [41] Kabaila, P. (1993), “On Bootstrap Predictive Inference for Autoregressive Processes,” Journal of Time Series Analysis, 14, 473-484. [42] Kilian, L. (1995), “Small-Sample Confidence Intervals for Impulse Response Functions,” manuscript, Department of Economics, Universityof Pennsylvania. [43] Kilian, L. (1996a), “Impulse Response Analysis in Vector Autoregressions with Unknown Lag Order,” manuscript, Department of Economics, University of Pennsylvania. [44] Kilian, L. (1996b), “AnEndogenous Lag-Order Bootstrap Algorithm withan Application ImpulseResponse Analysis,” manuscript, Departmentof Economics, to Universityof Pennsylvania. [45] Kiviet, J.F. (1984), “Bootstrap Inferencein Lagged-dependent Variable models,” University of Amsterdam Working Paper. [46] Kreiss, J.P. (1992), “Bootstrap Procedures for AR(m)-Processes,” in Bootstrapping and Related Techniques, Proceedings, Lecture Notes in Economics and Mathematical Systems, eds. K.H. Jockel, G. Rothe, and W. Sendler. New York: Springer-Verlag. [47] Kreiss, J.P., and J. Franke (1992), “Bootstrapping Stationary Autoregressive Moving-Average Models,” Journal of Time Series Analysis, 13, 287-317. [48] Kunsch, H.R. (1989), “The Jackknife and the Bootstrap for General Stationary Observations”, Annals of Statistics, 17, 1217-1241. 39

[49] Kunsch, H.R., and E. Carlstein (1990), “The Linked Blockwise Bootstrap for SeriallyDependent Observations,” TechnicalReport in preparation, Department of Statistics, University of North Carolina, Chapel Hill. [50] Lahiri, S.N. (1991), “Second-Order Optimality of Stationary Bootstrap,” Statistics and ProbabilityLetters, 11,335-341 [51] Lahiri, S.N. (1992), “Edgeworth Correction by ‘Moving Block’ Bootstrap for Stationary and Nonstationary Data”, in Exploring the Limits of Bootstrap, R. LePage and L. Billard, eds., John Wiley k Sons, New York. [52] Lahiri, S.N. (1993), “On the Moving Block Bootstrap under Long Range Dependence,” Statistics and ProbabilityLetters, 18, 405-413. [53] Lahiri,S.N. (1995), “On the Asymptotic Behaviorofthe Moving Block Bootstrap for Normalized Sumsof Heavy-Tail Random Variables,” Annals of Statistics, 23, 1331-1349. [54] Lamoureux, C.G., and W.D. Lastrapes (1990), “Persistence in Variance, Structural Change, and the GARCH Model)”Journal of Business and Economic Statistics, 8, 225-234. [55] L6ger,c.,D.N. Politis, and J.P. Romano (1992), “Bootstrap Technology and Applications,” Technomettics, 34, 378-398. [56] Li, H., and G.S. Maddala (1993), “Bootstrapping Cointegrating Regressions,” paper presented at the Fourth Meeting of the European Conference Series in Quantitative Economics and Econometrics, December 16-18, 1993. [57] Li, H., and G.S. Maddala (1996), “Bootstrapping Time SeriesModels,” Econometric Reviews, 15, 115-158. [58] Liu, R.Y. (1988), “Bootstrap Procedures under Some Non-i.i.d Models,” Annals of Statistics, 16, 1696-1708. . / 40

[59] Liu, R.Y. andK. Singh (1992), “Moving Blocks JackknifeandBootstrap Capture Weak Dependence”, in Exploting the Limits of Bootstrap, R. LePage and L. Billard, eds. New York: John Wiley & Sons [60] Masarotto, G. (1990), “Bootstrapping Prediction Intervalsfor Autoregressions,” International Journal of Forecasting,6, 229-239. [61] McCullough, B.D. (1994), ‘)Bootstrapping ForecastIntervals: An Application to AR(p) Models,” Journal of Forecasting,13, 51-66. [62] Naik-Nimbalkar,U.V., and M.B. Rajarshi (1994), “Validity of Blockwise Bootstrap for Empirical Processes with Stationary Observations,’) Annals of Statzstzcs,22, 980-994. [63] Nankervis,J.C., and N.E. Savin (1996), “The Level and Power of the Bootstrap t Test in the AR(1) Model with Trend,” Journal of Business and Economic Statistics, 14, 161-168. [64] Newey, W.K. and K.D. West (1994)~‘)Automatic Lag Selection in Covariance Matrix Estimation,” Review of Economic Studies,61, 631-653. [65] Nicholls, D.F. and A.L. Pope (1988), “Bias in the Estimation of Multivariate Autoregressions, “ Australian Journal of Statistics, 30A, 296-309. [66] Paparoditis, E. (1996), “Bootstrapping Autoregressive and Moving Average Parameter Estimates of Infinite Order Vector Autoregressive Processes, “ Journal of Multivatiate Analysis, 57, 277-296. [67] Paparoditis, E., and B. Streitberg (1992), ‘)Order Identification in Stationary Autoregressive Moving Average Models: Vector Autocorrelations and the Bootstrap,” Journal of Time Series Analysis, 13, 415-435. 41

[68] Paulsen, J., and D. Tj@stheim (1985), “On the Estimation of Residual Variance and Orderin AutoregressiveTime Series,))Journalofthe RoyalStatisticalSociety B,47,216-228. [69] Peters, S.C., and D.A. Freedman (1984), “Some Notes on the Bootstrap in Regression Problems,’) Journal of Business andEconomic Statistics, 2, 406-409. [70] Potscher, B.M. (1991), ‘)Effects of Model Selection on Inference,” Econometric Theo~, 7, 163-185. [71] Politis, D.N. and J.P. Romano (1992a), “A General Resampling Schemefor Triangular Arrays of a-Mixing Random Variableswith Application the Problem to of Spectral Density Estimation,” Annals of Statistics, 20, 1985-2007. [72] Politis, D.N., and J.P. Romano (1992b), “A Nonparametric Resampling Procedurefor Multivariate Confidence Regions in Time Series Analysis,” in Computing Science andStatistics, Proceedings of the 22nd Symposiumon the Interface, eds. C. Page and R. LePage, Springer-Verlag,New York [73] Politis, D.N. and J.P. Romano (1994), “The Stationary Bootstrap,” Journal of the American StatisticalAssociation, 89, 1303-1313. [74] Politis, D.N., J.P. Romano and T. Lai (1992), “Bootstrap Confidence Intervals for Spectra and Cross-Spectra,” IEEE Transactions on Signal Processing, 40, 1206-1215. [75] Pope, A.L. (1987), “Small-Sample Bias Problems in Time Series,” unpublished MSCThesis, Australian National University. [76] Pope, A.L. (1990), ‘)Biasesof Estimators in Multivariate Non-Gaussian Autoregression,” Journal of Time Series Analysis, 11, 249-258. [77] Priestley,M.B. (1981), SpectralAnalysis and Time Series. NewYork: Academic Press. 42

[78] Quinn, B.G. (1988), “A Note on AIC Order Determination for Multivariate Autoregression,” Journal of Time Series Analysis, 9, 241-245. [79] Ramos, E. (1984), “A Bootstrap for Time Series,” Qualifying paper, Department of Statistics, Harvard University. [80] Ramos, E. (1988), “Resampling Methods for Time Series,” Technical Report No. ONR- C-2, Department of Statistics, Harvard University. [81]Rayner (1990), “Bootstrapping p Values and Power in the First-Order Autoregression: A Monte Garlo Investigation,” Journal of Business and Economic Statistics, 8, 251-263. [82] Rudebusch, G.D. (1993), “The Uncertain Unit Root in Real GNP,” American Economic Review, 83, 264-272. [83] Stine, R.A. (1985), “A Frequency Domain Bootstrap for Time Series,” manuscript, Department of Statistics,Wharton School, Universityof Pennsylvania. [84] Stine ,R.A. (1987), “Estimating Properties ofAutoregressive Forecasts,” Journal of the Ametican StatisticalAssociation, 82, 1072-1078. [85] Stoffer, D.S. and K.D. Wall (1991), “Bootstrapping State-Space Models: Gaussian Maximum Likelihood Estimation and the Kalman Filter))) Journal of the American StatisticalAssociation, 86, 1024-1033. [86] Swanepoel, J.W.H and J.W.Y. van Wyk (1986), “The Bootstrap Applied to Power Spectral Density Function Estimation,” Bzometrzka,73, 135-141. [87] Thombs, L.A., and W.R. Schucany (1990), “Bootstrap Prediction Intervalsfor Autoregressions,” Journal of the Ametican StatisticalAssociation, 85, 486-492. 43

[88] Young, G.A. (1994), “Bootstrap: More than Stab in the Dark? “ Statistical a Science, 9, 382-415. [89] Zivot, E., and D. Andrews (1992), FurtherEvidence on the Great Crash,the Oil Price Shock, and The Unit Root Hypothesis,” Journalof Business andEconomic Statistics, 10, 237-250. 44

Figure 1 Selectingthe Optimal BlockSizeby Minimizing Mean Squared Deviations from NominalCoverage 0.8 0.7 0.6 0.5 0.4 0.: o.: o.’ ( 20 40 60 80 100 120 140 160 Block Size inQuarters Notes: Quarterly S&Pcommon stock earnings-priceratio, 1947.2to 1994.3.For eachblock size,~ 16 mean squareddeviationfrom nominalcoverage isdefinedas ~ ~ [coverage(k,i)-O.9] 2, i=O where i =O,..., 16denotesthe time horizon ofthe impulseresponse.

Figure 2 Effective Coverage of Nominal 90 0/0 Bootstrap Confidence Intervals for ImpulseResponses T =80 1 i I I I I 1 I 0.9 - AR-BC j 0.7 - \ ‘, ~‘ /-\ \... ‘ .. \ \ . .- .-. - “\ -. —.—. -. -.-, i. -. _ , ---- -. - .- 0.6 - ; AR - :I 0.5 - ;; :/ 0.4 - ;; “,.... ... \ ... :/ \ \ ...... “ ......, ........... C .. H .. O . L ........ - 0.3 -; \ \ :/ \ \ 0.2 J \ \ \\ BLOCK \ \ . \ \- 0 1 I I I 1 -—— L 0 2 4 6 8 10 12 14 – – ‘;6 Quarters Source: 1,000Monte Carlo trials for ARMA(2,4)-DGP based on U.S. T-Billrate for 1971.4-1993.4.

Figure 3 Effective Coverage of Nominal 90 0/0 BootstrapConfidence Intervals for ImpuiseResponses T = 160 1 0.9- : “’-”””’” AR-BC 0.8 - \ &#45;&#45;&#45; -. —.—. -,- .. CI-IOL ... \ 0.4- : ,/ \ 0.3-~•,ò’ì(cid:129) •ò(cid:129) \ BLOCK :1 \ \ : I 0.2-,,’ \ \ :I \ \ 0.1-; \ RAMOS \ :/ \ \ o t I I I I \_ I -—_ + 1 0 2 4 6 8 10 12 14 16 Quarters Source: 1,000Monte Carlo trialsfor ARMA(2,4)-DGP based on U.S. T-Billrate for 1971.4-1993.4.

Figure 4 Effective Coverage of Nominal 90 ‘!/0 Bootstrap Confidence Intervals for Impuise Responses T = 480 1 I I I I [ i 1 \ - -. - .-. - - -“ - - 0.8 - 0.7 - 0.6 - 1 \ \ \ / \ 002 -“ I \ / ~ RAMOS \ \ o.l; )’ / \ \ / \ o- // 1 I I I \ I I I 0 2 4 6 8 10 12 14 16 Quarters Source: 1,000Monte Carlo trials for ARMA(2,4)-DGP based on U.S. T-Billrate for 1971.4-1993.4.