The full text on this page is automatically extracted from the file linked above and may contain errors and inconsistencies.
Working Paper 8711 ESTIMATING MULTIVARIATE ARIMA MODELS: WHEN IS CLOSE NOT GOOD ENOUGH? by Michael L. Bagshaw Michael L. Bagshaw is a statistical analysis administrator at the Federal Reserve Bank of Cleveland. Working papers of the Federal Reserve Bank of Cleveland are preliminary materials circulated to stimulate discussion and critical comment. The views expressed herein are those of the author and not necessarily those . of the Federal Reserve Bank of Cleveland or of the Board of Governors of the Federal Reserve System. December 1987 ABSTRACT Key words: multivariate ARIMA, forecasting performance The purpose of this study is to examine the forecasting abilities of the same multivariate autoregressive model estimated using two methods. The first method is the "exact method" used by the SCA System from Scientific Computing Associates. The second method is an approximation method as implemented in the MTS system by Automatic Forecasting Systems, Inc. The two methods were used to estimate a five-series multivariate autoregressive model for the Quenouille series on hog numbers, hog prices, corn prices, corn supply, and farm wage rates. The 82 observations were arbitrarily divided into two periods: the first 60 observations were used to estimate the models; then forecasts for one through eight years ahead were calculated for each possible point in the remaining 22 observations. The root mean square error (RMSE) using the SCA-estimated parameters was smaller than the RMSE using the MTS-estimated parameters for 38 of the 40 possible values (five variables by eight forecast horizons) and tied for one point. The average increase in the RMSE when using the MTS parameters was approximately 9 percent. Using the SCA parameters for forecasting provided smaller mean absolute error (MAE) for 35 of the 40 values, with the average increase from using the MTS parameters being approximately 5 . 6 percent. Using the SCA parameters provided smaller mean errors(ME) for 39 of the 40 values, with the average increase from using the MTS parameters being approximately . 0 2 3 . Thus, the SCA estimation method is shown to provide better forecasts than the MTS method for this one example. ESTIMATING MULTIVARIATE ARIMA MODELS: WHEN IS CLOSE NOT GOOD ENOUGH? I. Introduction Little study appears to have been done on the effects of different methods of estimation of the parameters in a multivariate autoregressive integrated moving average (MARIMA) model on forecasting. It is extremely difficult to estimate a multivariate model with more than a few series. Consequently, if approximate methods can provide estimations that are close enough to provide "adequate" forecasts, the savings in computer cost can be substantial. In this study, we examine the forecasting performance of the same model estimated using two methods. The first method is the "exact method" used by the SCA System from Scientific Computing Associates. The second method is an approximation method due to Spliid (1983) as implemented in the MTS system by Automatic Forecasting Systems, Inc. The two methods were used to estimate a five-series multivariate autoregressive model for the data on hog numbers, hog prices, corn prices, corn supply, and farm wage rates as given in Quenouille (1957). The data consisted of 82 yearly observations from 1867 to 1948. The 82 observations were arbitrarily divided into two periods: the first 60 observations were used to estimate the models; then forecasts for one through eight years ahead were calculated for each possible point in the remaining 22 observations. The models were actually estimated in the natural logarithm of the original data and the results given in this paper are in terms of forecasting the logged data. 11. Time Series Models The following is a very brief description of the general MARIMA model. Tiao and Box (1981) provide a more detailed description of the multivariate ARIMA models. These models are particular versions of the general time series model of order (p,q) given by: where @(B) -F' = z - LIB - . . . - e (B) = I - llB - ... - -q ) BP, 7' B Bq, -q and B = backshift operator(e .g., Bszi, = zi, t-s) , - k x k identity matrix, z - - vector of k variables in the model, I - - ) 's and ljts k x k matrices of unknown parameters, -j B -0 = k x 1 vector of unknown parameters, and a = k x 1 vector of random errors that are identically and independently distributed as N(0,Z). Thus, it is assumed that the -aj.t 's at different points in time are independent, but not necessarily that the elements of 2, are independent at a given point in time. The n-period-ahead forecasts from these models at time t (z,(n)) are given by: ] implies the conditional where, for any value of t,n,m, [gtin-m expected values of the random variables ztin-m at time t. If n-m is less than or equal to zero, then the conditional expected values are the actual values of the random variables and the error terms. If n-m is greater than zero, then the expected values are the best forecasts available for these random variables and error terms at time t. Because the error terms are uncorrelated with present and past information, the best forecasts of the error terms for n-m greater than zero are their conditional means, which are zero. The forecasts can be generated iteratively with the one-period-ahead forecasts that depend only on known values of the variables and error terms. The longer-length forecasts, in turn, depend on the shorter-length forecasts. 111. Development of Models for Forecasting Because we wish to test which method provides better forecasts, we divided the data into two periods. The data from 1867 through 1926 were used to estimate the model for each method with adjustments in the starting period for the lags involved in the model. The last 22 observations(from 1927 through 1948) were used to test the forecast accuracy of these models in terms of root mean square error (RMSE) of the forecasts for one to eight years ahead. For the MARIMA model, we developed a model by using the method of Tiao and Box (1981). This method is similar to the Box and Jenkins (1976) method for developing univariate models, except that cross-correlations between the series are added and modeled for. This is an iterative method that involves: 1) tentatively identifying a model by examining autocorrelations of the series, 2) estimating the parameters of this model, and 3) applying diagnostic checks to the residuals. If the residuals do not pass the diagnostic checks, then the tentative model is modified, and steps two and three are repeated. This process continues until a satisfactory model is obtained. The resulting model was an MARIMA (1,0,1) model. That is, it was first order in both the autoregressive and the moving-average terms. It thus can be represented as: where 4 -1 and B1 are 5 by 5 matrices of unknown parameters that must be estimated. These matrices were estimated by the two different methods discussed in the next section. IV. Estimation Methods and Resulting Parameter Estimates The MARIMA (1,0,1)model was estimated using two different methods. The first method is the "exact method" used in the SCA Statistical System, Version I11 from Scientific Computing Associates. This method is an implementation of the estimation method using the "exact" likelihood function given by Hillmer and Tiao (1979). This method actually approximates the likelihood function based on the stochastic structure of n-1 observations with izl considered fixed for models with an autoregressive part of order 1. Because this method is extremely technical, the details are not presented here. The second method is an approximation implemented in the MTS system from Automatic Forecasting Systems, Inc. This method is based on the results given in Spliid (1983). Spliid believes that this approximation method is an economical alternative to maximum-likelihood methods, which can be expensive, that this method can provide good starting values for maximum-likelihood estimation, and that this method can be used in initial studies to help determine an appropriate model by the estimation of different forms and orders of models. The results of estimating the model using the two methods are given in table 1. The results are fairly close for most parameters, but in one case, the difference is substantial. This is the moving-average term corresponding to the effect of the lagged error in forecasting hog numbers on the farm wage rates. The next step, determining how these differences affect the forecast performance of the model, is addressed in the next section. V. Forecastine Results The models developed for this study were used to forecast the five variables for a forecast horizon of up to eight years from 1927 through 1948. These were actual forecasts and did not use any information within the forecast horizon. Thus, the number of forecasts we have for each forecast length varies. For one-quarter-ahead forecasts, we have 22 observations; for two quarters ahead, we have 21 obsenrations, etc. For the purposes of this study, we calculated the root mean square error (RMSE) , the mean absolute error (MAE), and the mean error (ME) as measures of forecast accuracy. The results are presented in tables 2 through 4. The RMSE using the SCA-estimated parameters was smaller than the RMSE using the MTS-estimated parameters for 38 of the 40 possible values (five variables by eight forecast horizons) and tied for one point. The average increase in the RMSE when using the MTS parameters was approximately 9 percent. For individual variables, the increases in RMSE from using MTS were: Hog numbers Hog prices Corn prices Corn supply Farm wage rates 5.9 7.6 1.5 5.3 24.6 percent percent percent percent percent Thus, in terms of RMSE, the forecasts produced from using the SCA parameters dominate the results using the MTS parameters. A major difference in the results for farm wages parallels the difference in the estimated parameter that indicates the effect of hog numbers on farm wages, as shown in table 1. Using the SCA parameters for forecasting provided smaller MAE for 35 of the 40 values, with the average increase from using the MTS parameters being approximately 5.6 percent. For individual variables, the increases in MAE were: Hog numbers Hog prices Corn prices Corn supply Farm wage rates 1.2 percent 3.3 percent 5.8 percent 7.0 percent 15.9 percent The results are again consistent with the difference in the estimated parameters. The farm wage forecast is substantially different, with not as much difference for the other variables. Using the SCA parameters provided smaller ME for 39 of the 40 values, with the average increase from using the MTS parameters being approximately .023. The MEs were always of the same sign for both sets of estimated parameters. For the individual variables, the increases in ME were: Hog nmbers Hog prices Corn prices Corn supply Farm wage rates .0060 .0157 .0431 .0104 .0396 VI. Summary In this study, we have compared the forecasting performance of the same multivariate autoregressive moving average model estimated by two different methods. The "exact method" used in SCA dominates the approximate method used in MTS for all variables and time lengths used in this study. The results indicate that for at least one of the five variables studied here (farm wage rates), there is a substantial difference in the forecasting ability. For the other four variables, there is not as substantial a difference, but the difference could be very meaningful, depending on the application. The results of this study indicate the importance of using as accurate an estimation method as possible and indicate that for at least one variable in this study, the forecasting performance can be substantially improved by using the better methods. This result may have implications for studies that have shown that Box-Jenkins methods do not perform as well in forecasting as other methods. Most of these studies use univariate models in which the result may not be as dramatic. However, there has been no study of this effect in univariate models. This is an area for further research. These results are, of course, based only on one set of data and may not carry over to other cases. However, the results do indicate that whenever a study compares forecasting abilities of different methods, the method of estimating should be clearly identified. Further work is needed to determine whether these results are general or are specific to this data set. References Box, George E. P., and Gwilym M. Jenkins. Time Series Analysis: Forecasting and Control. San Francisco: Holden-Day Inc., 1976. Hillmer, S.C., and G. C. Tiao. "Likelihood Function of Stationary Multiple Autoregressive Moving Average Models," Journal of the American Statistical Association, vol. 74, no. 367 (1979), 652-660. Quenouille, M.H. The Analysis of Multiple Time Series. London: Griffing, 1957. Spliid, H. "A Fast Estimation Method for the Vector Autoregressive Moving Average Model with Exogenous Variables," Journal of the American Statistical ~ssociation,vol. 78, no. 384,(1983) 843-849. Tiao, George C., and George E. P. Box. "Modeling Multiple Time Series with Applications," Journal of the American Statistical Association, vol. 76, no. 376 (December 1981), 802-16. Table 1 Estimated Models Phi Matrix Using MTS Using SCA Theta Matrix Using MTS Using SCA Table 2 Comparison of Root Mean Square Error Forecast horizons (years) HOE numbers MTS model SCA Ratio MTS/SCA .0062 .0060 .0104 .0098 1.0038 1.0437 1.0529 1.0557 1.0867 1.0457 1.0461 1.0843 1.1286 1.1129 1.0791 1.0582 1.0463 1.0486 .0836 .0830 .0941 .0937 .lo02 .0993 .0977 .0956 1.0070 1.0038 1.0092 1.0217 1.0237 1.0151 1.1086 1.1352 1.1219 1.0547 1.0270 1.3578 1.2802 1.2182 1.1756 1.1509 .0026 .0026 .0123 .0117 .0139 .0128 .0172 .0164 .0187 -0233 . O M 1 -0203 1.0371 1.1486 Hog prices MTS SCA Ratio MTS/SCA Corn prices MTS SCA Ratio MTS/SCA .0309 .0302 .0632 .0622 1.0212 1.0151 .0979 .lo80 .0956 .I064 Corn supply MTS SCA Ratio MCTS/SCA .9572 1.0000 1.0190 Farm wage rates MTS SCA Ratio 1.9600 1.5873 1.4586 Table 3 Comparison of Mean Absolute Error Forecast horizons (years) Hog numbers MTS model SCA MTS-SCA (MTS-SCA)/SCA .0392 .0404 .0568 .0598 .0741 .0740 .0883 .0845 .lo12 .0966 .I105 .lo70 -1161 -1123 -.OO12 -.0030 -.0297 -.O502 .OOOl .0014 .0038 .0450 .0046 .0476 .0035 .0327 .0038 .0338 Hog prices MTS model SCA .I102 .lo86 .I725 .I649 .2335 .2189 .2872 .2735 .3148 .3051 -3351 -3301 -3555 -3498 MTS - SCA (MTS-SCA)/SCA .0016 .0147 .0076 .0461 .0146 .0667 .0137 .0501 .0097 .0318 .0050 .0151 .0057 .0163 MTS model SCA .I469 .I447 .2209 .2181 .2526 .2533 .2562 .2569 .2654 .2646 .2575 .2554 -2529 -2.509 MTS- SCA (MTS-SCA)/SCA .0022 .0152 .0028 -.0007 -.0007 .0128 -.0028 -.0027 .0008 .0030 .0021 .0082 .0020 .0080 .0579 .0590 .0556 .0516 .0690 .0655 .0745 .0684 .0823 .0749 .0897 .0797 .0959 .0890 - .OO11 -.0186 .0040 .0775 .0035 .0534 .0061 .0892 .0074 .0988 .0100 .I255 .0069 .0775 MTS model SCA .0724 .0541 .I219 .0995 .I768 .I485 .2153 .I862 .2441 .2199 .2643 .2482 .2884 .2688 MTS- SCA (MTS-SCA)/SCA .0183 .3383 .0224 .2251 .0283 .I906 .0291 .I563 .0242 .I101 .0161 .0649 .0196 .0729 Corn prices Corn supply MTS model SCA MTS- SCA (M.TS-SCA)/SCA Farm wage rates Table 4 Comparison of Mean Error Forecast horizons (years Hog numbers MTS model SCA -.0056 -.0094 .0066 .0008 .0215 .0151 .0342 .0271 .0474 -0394 -0585 .0502 -0643 -0653 -0562 -0575 ABS(MTS) -ABS(SCA) -.0038 .0058 .0064 .0071 .0080 .0083 .0081 .0078 MTS model SCA .0627 .0571 .I332 .I217 .I765 .I595 .2053 .I858 .2243 .2044 .2413 .2224 .2718 .3158 .2547 .3001 ABS(MTS ) -ABS(SCA) .0056 .0115 .0170 .0195 .0199 .0189 .0171 Hog prices Corn prices MTS model SCA .0601 .0545 ABS(MTS) -ABS(SCA) .0056 Corn supply MTS model SCA .0083 .0042 ABS(MTS) -ABS(SCA) .0041 Farm wage rates MTS model SCA .0679 .0444 ABS(MTS) -ABS(SCA) .0235
@FedFRASER