Robust inference under time‐varying volatility: A real‐time evaluation of professional forecasters

Abstract

In many forecast evaluation applications, standard tests as well as tests allowing for time‐variation in relative forecast ability build on heteroskedasticity‐and‐autocorrelation consistent (HAC) covariance estimators. Yet, the finite‐sample performance of these asymptotics is often poor. “Fixed‐ b$$ b $$” asymptotics, used to account for long‐run variance estimation, improve finite‐sample performance under homoskedasticity, but lose asymptotic pivotality under time‐varying volatility. Moreover, loss of pivotality due to time‐varying volatility is found in the standard HAC framework in certain cases as well. We prove a wild bootstrap implementation to restore asymptotically pivotal inference for the above and new CUSUM‐ and Cramér‐von Mises‐based tests in a fairly general setup, allowing for estimation uncertainty from either a rolling window or a recursive approach when fixed‐ b$$ b $$ asymptotics are adopted to achieve good finite‐sample performance. We then investigate the (time‐varying) performance of professional forecasters relative to naive no‐change and model‐based predictions in real‐time. We exploit the Survey of Professional Forecasters (SPF) database and analyze nowcasts and forecasts at different horizons for output and inflation. We find that not accounting for time‐varying volatility seriously affects outcomes of tests for equal forecast ability: wild bootstrap inference typically yields convincing evidence for advantages of the SPF, while tests using non‐robust critical values provide remarkably less. Moreover, we find significant evidence for time‐variation of relative forecast ability, the advantages of the SPF weakening considerably after the “Great Moderation.”