Please use this identifier to cite or link to this item: http://bura.brunel.ac.uk/handle/2438/9833
Title: On the usefulness of cross-validation for directional forecast evaluation
Authors: Bergmeir, C
Costantini, M
Benítez, JM
Keywords: Blocked cross-validation;Forecast directional accuracy;Linear models;Monte Carlo analysis;Out-of-sample evaluation
Issue Date: 2014
Publisher: Elsevier
Citation: Computational Statistics and Data Analysis, 76: 132–143, (August 2014)
Abstract: The usefulness of a predictor evaluation framework which combines a blocked cross-validation scheme with directional accuracy measures is investigated. The advantage of using a blocked cross-validation scheme with respect to the standard out-of-sample procedure is that cross-validation yields more precise error estimates of the prediction error since it makes full use of the data. In order to quantify the gain in precision when directional accuracy measures are considered, a Monte Carlo analysis using univariate and multivariate models is provided. The experiments indicate that more precise estimates are obtained with the blocked cross-validation procedure. An application is carried out on forecasting UK interest rate for illustration purposes. The results show that in such a situation with small samples the cross-validation scheme may have considerable advantages over the standard out-of-sample evaluation procedure as it may help to overcome problems induced by the limited information the directional accuracy measures contain due to their binary nature.
URI: http://www.sciencedirect.com/science/article/pii/S0167947314000310#
http://bura.brunel.ac.uk/handle/2438/9833
DOI: http://dx.doi.org/10.1016/j.csda.2014.02.001
ISSN: 0167-9473
Appears in Collections:Dept of Economics and Finance Research Papers

Files in This Item:
File Description SizeFormat 
Fulltext.pdf270.63 kBAdobe PDFView/Open


Items in BURA are protected by copyright, with all rights reserved, unless otherwise indicated.