Univariate Time Series Forecasting Using k-Nearest Neighbors Algorithm: A Case for GDP

Georgios Rigopoulos

doi:10.18535/ijsrm/v10i9.em01

Abstract

k-Nearest Neighbors (k-NN) is a well know algorithm used for classification and regression. Its usage in time series forecasting is limited though, despite its simplicity and competitive accuracy, as it has been demonstrated in relevant research. This work presents a method for time series forecasting based on k Nearest Neighbors regression, which can be utilized for macroeconomic variable forecasting, like Gross Domestic Product. The approach focuses on one-step ahead forecast, and uses R package libraries for the implementation. The method is applied to forecast Greek Gross Domestic Product and the accuracy results are high and comparable to ARIMA approach. The work offers a competent approach for time series and GDP forecasting, which is comparable in accuracy to traditional statistical approaches, and can be further developed experimentation on diverse data sets can improve parameter tuning and aggregation approach.

References

Hyndman, R. J., & Athanasopoulos, G. (2018). Forecasting: principles and practice. OTexts.Google Scholar ↗
Box, G. E., Jenkins, G. M., Reinsel, G. C., & Ljung, G. M. (2015). Time series analysis: forecasting and control. John Wiley & Sons.Google Scholar ↗
Bontempi, G., Ben Taieb, S., & Borgne, Y. A. L. (2012, July). Machine learning strategies for time series forecasting. In European business intelligence summer school (pp. 62-77). Springer, Berlin, Heidelberg.Google Scholar ↗
De Gooijer, J.G., Hyndman, R.J.: 25 years of time series forecasting. International Journal of Forecasting 22(3), 443–473 (2006)Google Scholar ↗
Poskitt, D.S., Tremayne, A.R.: The selection and use of linear and bilinear time series models. International Journal of Forecasting 2(1), 101–114 (1986)Google Scholar ↗
Engle, R.F.: Autoregressive conditional heteroscedasticity with estimates of the variance of united kingdom inflation. Econometrica 50(4), 987–1007 (1982)Google Scholar ↗
Ahmed, N.K., Atiya, A.F., El Gayar, N., El-Shishiny, H.: An empirical comparisonof machine learning models for time series forecasting. Econometric Reviews 29(5- 6) (2010)Google Scholar ↗
Crone SF,Hibon M, NikolopoulosK(2011) Advances in forecastingwith neural networks? Empirical evidence from the NN3 competition on time series prediction. Int J Forecast 27(3):635–660Google Scholar ↗
Martínez, F., Frías, M. P., Pérez, M. D., & Rivera, A. J. (2019). A methodology for applying k-nearest neighbor to time series forecasting. Artificial Intelligence Review, 52(3).Google Scholar ↗
Palit, A.K., Popovic, D.: Computational Intelligence in Time Series Forecasting: Theory and Engineering Applications. Advances in Industrial Control. Springer- Verlag New York, Inc., Secaucus (2005)Google Scholar ↗
Werbos, P.J.: Generalization of backpropagation with application to a recurrent gas market model. Neural Networks 1(4), 339–356 (1988)Google Scholar ↗
Lapedes, A., Farber, R.: Nonlinear signal processing using neural networks: prediction and system modelling. Technical Report LA-UR-87-2662, Los Alamos National Laboratory, Los Alamos, NM (1987)Google Scholar ↗
Hastie, T., Tibshirani, R., Friedman, J.: The elements of statistical learning: data mining, inference and prediction, 2nd edn. Springer (2009)Google Scholar ↗
Masini, R. P., Medeiros, M. C., & Mendes, E. F. (2021). Machine learning advances for time series forecasting. Journal of Economic Surveys. 2021;1–36.Google Scholar ↗
Pavlyshenko, B. M. (2019). Machine-learning models for sales time series forecasting. Data, 4(1), 15.Google Scholar ↗
Lendasse, A. (ed.): ESTSP 2008: Proceedings. Multiprint Oy/Otamedia (2008) ISBN: 978-951-22-9544-9Google Scholar ↗
Crone, S.F.: Mining the past to determine the future: Comments. International Journal of Forecasting 5(3), 456–460 (2009); Special Section: Time Series MonitoringGoogle Scholar ↗
Sorjamaa, A., Hao, J., Reyhani, N., Ji, Y., Lendasse, A.: Methodology for long-term prediction of time series. Neurocomputing 70(16-18), 2861–2869 (2007)Google Scholar ↗
Anderson, T.W.: The statistical analysis of time series. J. Wiley and Sons (1971)Google Scholar ↗
Farmer, J.D., Sidorowich, J.J.: Predicting chaotic time series. Physical Review Letters 8(59), 845–848 (1987)Google Scholar ↗
Takens, F.: Detecting strange attractors in fluid turbulence. In: Dynamical Systems and Turbulence. Springer, Berlin (1981)Google Scholar ↗
Casdagli, M., Eubank, S., Farmer, J.D., Gibson, J.: State space reconstruction in the presence of noise. PHYD 51, 52–98 (1991)Google Scholar ↗
Atkeson, C.G., Moore, A.W., Schaal, S.: Locally weighted learning. AIR 11(1-5), 11–73 (1997)Google Scholar ↗
Lorenz, E.N.: Atmospheric predictability as revealed by naturally occurring analogues. Journal of the Atmospheric Sciences 26, 636–646 (1969)Google Scholar ↗
Ikeguchi, T., Aihara, K.: Prediction of chaotic time series with noise. IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences E78-A(10) (1995)Google Scholar ↗
Priestley, M.B.: Non-linear and Non-stationary time series analysis. Academic Press (1988)Google Scholar ↗
Birattari, M., Bontempi, G., Bersini, H.: Lazy learning meets the recursive least-squares algorithm. In: Kearns, M.S., Solla, S.A., Cohn, D.A. (eds.) NIPS 11, pp.375–381. MIT Press, Cambridge (1999)Google Scholar ↗
Weigend, A.S., Gershenfeld, N.A.: Time Series Prediction: forecasting the future and understanding the past. Addison Wesley, Harlow (1994)Google Scholar ↗
Sorjamaa, A., Lendasse, A.: Time series prediction using dirrec strategy. In: Verleysen, M. (ed.) European Symposium on Artificial Neural Networks, ESANN 2006, Bruges, Belgium, April 26-28, pp. 143–148 (2006)Google Scholar ↗
Rigopoulos, G., GDP Modeling Using Autoregressive Integrated Moving Average (ARIMA): A Case for Greek GDP, International Journal of Business Marketing and Management (IJBMM), Volume 7 Issue 4, 2022, P.P. 66-75 ISSN: 2456-4559Google Scholar ↗
R Core Team, R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria, 2017, URL https://www.R-project.orgGoogle Scholar ↗

[refR-1] Hyndman, R. J., & Athanasopoulos, G. (2018). Forecasting: principles and practice. OTexts.Google Scholar ↗

[refR-2] Box, G. E., Jenkins, G. M., Reinsel, G. C., & Ljung, G. M. (2015). Time series analysis: forecasting and control. John Wiley & Sons.Google Scholar ↗

[refR-3] Bontempi, G., Ben Taieb, S., & Borgne, Y. A. L. (2012, July). Machine learning strategies for time series forecasting. In European business intelligence summer school (pp. 62-77). Springer, Berlin, Heidelberg.Google Scholar ↗

[refR-4] De Gooijer, J.G., Hyndman, R.J.: 25 years of time series forecasting. International Journal of Forecasting 22(3), 443–473 (2006)Google Scholar ↗

[refR-5] Poskitt, D.S., Tremayne, A.R.: The selection and use of linear and bilinear time series models. International Journal of Forecasting 2(1), 101–114 (1986)Google Scholar ↗

[refR-6] Engle, R.F.: Autoregressive conditional heteroscedasticity with estimates of the variance of united kingdom inflation. Econometrica 50(4), 987–1007 (1982)Google Scholar ↗

[refR-7] Ahmed, N.K., Atiya, A.F., El Gayar, N., El-Shishiny, H.: An empirical comparisonof machine learning models for time series forecasting. Econometric Reviews 29(5- 6) (2010)Google Scholar ↗

[refR-8] Crone SF,Hibon M, NikolopoulosK(2011) Advances in forecastingwith neural networks? Empirical evidence from the NN3 competition on time series prediction. Int J Forecast 27(3):635–660Google Scholar ↗

[refR-9] Martínez, F., Frías, M. P., Pérez, M. D., & Rivera, A. J. (2019). A methodology for applying k-nearest neighbor to time series forecasting. Artificial Intelligence Review, 52(3).Google Scholar ↗

[refR-10] Palit, A.K., Popovic, D.: Computational Intelligence in Time Series Forecasting: Theory and Engineering Applications. Advances in Industrial Control. Springer- Verlag New York, Inc., Secaucus (2005)Google Scholar ↗

[refR-11] Werbos, P.J.: Generalization of backpropagation with application to a recurrent gas market model. Neural Networks 1(4), 339–356 (1988)Google Scholar ↗

[refR-12] Lapedes, A., Farber, R.: Nonlinear signal processing using neural networks: prediction and system modelling. Technical Report LA-UR-87-2662, Los Alamos National Laboratory, Los Alamos, NM (1987)Google Scholar ↗

[refR-13] Hastie, T., Tibshirani, R., Friedman, J.: The elements of statistical learning: data mining, inference and prediction, 2nd edn. Springer (2009)Google Scholar ↗

[refR-14] Masini, R. P., Medeiros, M. C., & Mendes, E. F. (2021). Machine learning advances for time series forecasting. Journal of Economic Surveys. 2021;1–36.Google Scholar ↗

[refR-15] Pavlyshenko, B. M. (2019). Machine-learning models for sales time series forecasting. Data, 4(1), 15.Google Scholar ↗

[refR-16] Lendasse, A. (ed.): ESTSP 2008: Proceedings. Multiprint Oy/Otamedia (2008) ISBN: 978-951-22-9544-9Google Scholar ↗

[refR-17] Crone, S.F.: Mining the past to determine the future: Comments. International Journal of Forecasting 5(3), 456–460 (2009); Special Section: Time Series MonitoringGoogle Scholar ↗

[refR-18] Sorjamaa, A., Hao, J., Reyhani, N., Ji, Y., Lendasse, A.: Methodology for long-term prediction of time series. Neurocomputing 70(16-18), 2861–2869 (2007)Google Scholar ↗

[refR-19] Anderson, T.W.: The statistical analysis of time series. J. Wiley and Sons (1971)Google Scholar ↗

[refR-20] Farmer, J.D., Sidorowich, J.J.: Predicting chaotic time series. Physical Review Letters 8(59), 845–848 (1987)Google Scholar ↗

[refR-21] Takens, F.: Detecting strange attractors in fluid turbulence. In: Dynamical Systems and Turbulence. Springer, Berlin (1981)Google Scholar ↗

[refR-22] Casdagli, M., Eubank, S., Farmer, J.D., Gibson, J.: State space reconstruction in the presence of noise. PHYD 51, 52–98 (1991)Google Scholar ↗

[refR-23] Atkeson, C.G., Moore, A.W., Schaal, S.: Locally weighted learning. AIR 11(1-5), 11–73 (1997)Google Scholar ↗

[refR-24] Lorenz, E.N.: Atmospheric predictability as revealed by naturally occurring analogues. Journal of the Atmospheric Sciences 26, 636–646 (1969)Google Scholar ↗

[refR-25] Ikeguchi, T., Aihara, K.: Prediction of chaotic time series with noise. IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences E78-A(10) (1995)Google Scholar ↗

[refR-26] Priestley, M.B.: Non-linear and Non-stationary time series analysis. Academic Press (1988)Google Scholar ↗

[refR-27] Birattari, M., Bontempi, G., Bersini, H.: Lazy learning meets the recursive least-squares algorithm. In: Kearns, M.S., Solla, S.A., Cohn, D.A. (eds.) NIPS 11, pp.375–381. MIT Press, Cambridge (1999)Google Scholar ↗

[refR-28] Weigend, A.S., Gershenfeld, N.A.: Time Series Prediction: forecasting the future and understanding the past. Addison Wesley, Harlow (1994)Google Scholar ↗

[refR-29] Sorjamaa, A., Lendasse, A.: Time series prediction using dirrec strategy. In: Verleysen, M. (ed.) European Symposium on Artificial Neural Networks, ESANN 2006, Bruges, Belgium, April 26-28, pp. 143–148 (2006)Google Scholar ↗

[refR-30] Rigopoulos, G., GDP Modeling Using Autoregressive Integrated Moving Average (ARIMA): A Case for Greek GDP, International Journal of Business Marketing and Management (IJBMM), Volume 7 Issue 4, 2022, P.P. 66-75 ISSN: 2456-4559Google Scholar ↗

[refR-31] R Core Team, R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria, 2017, URL https://www.R-project.orgGoogle Scholar ↗