A Review of Current Machine Learning Approaches for Anomaly Detection in Network Traffic
Main Article Content
Keywords
Anomaly Detection, Intrusion, Networks, Supervised, Unsupervised
Abstract
Due to the advance in network technologies, the number of network users is growing rapidly, which leads to the generation of large network traffic data. This large network traffic data is prone to attacks and intrusions. Therefore, the network needs to be secured and protected by detecting anomalies as well as to prevent intrusions into networks. Network security has gained attention from researchers and network laboratories. In this paper, a comprehensive survey was completed to give a broad perspective of what recently has been done in the area of anomaly detection. Newly published studies in the last five years have been investigated to explore modern techniques with future opportunities. In this regard, the related literature on anomaly detection systems in network traffic has been discussed, with a variety of typical applications such as WSNs, IoT, high-performance computing, industrial control systems (ICS), and software-defined network (SDN) environments. Finally, we underlined diverse open issues to improve the detection of anomaly systems.
Downloads
Download data is not yet available.
References
Aburomman, A. A., & Reaz, M. B. I. (2017). A survey of intrusion detection systems based on ensemble and hybrid classifiers. Computers & Security, 65, 135-152.
Agrawal, S., & Agrawal, J. (2015). Survey on anomaly detection using data mining techniques. Procedia Computer Science, 60, 708-713.
Ahmad, S., Lavin, A., Purdy, S., & Agha, Z. (2017). Unsupervised real-time anomaly detection for streaming data. Neurocomputing, 262, 134-147.
Aissa, N. B., & Guerroumi, M. (2016). Semi-supervised statistical approach for network anomaly detection. Procedia Computer Science, 83, 1090-1095.
Akhi, A. B., Kanon, E. J., Kabir, A., & Banu, A. (2019). Network Intrusion Classification Employing Machine Learning: A Survey. (Doctoral dissertation) Department of Computer Science and Engineering, United International University, Bangladesh.
Alizadeh, H., Khoshrou, A., & Zuquete, A. (2015). Traffic classification and verification using unsupervised learning of Gaussian Mixture Models. In 2015 IEEE international workshop on measurements & networking (M&N). 1-6. IEEE.
Amangele, P., Reed, M. J., Al-Naday, M., Thomos, N., & Nowak, M. (2019). Hierarchical Machine Learning for IoT Anomaly Detection in SDN. In 2019 International Conference on Information Technologies (InfoTech). 1-4. IEEE.
Anderson, J. P. (1980). Computer security threat monitoring and surveillance. Technical Report, Fort Washington, PA, James P. Anderson Co.
Ashfaq, R. A. R., Wang, X. Z., Huang, J. Z., Abbas, H., & He, Y. L. (2017). Fuzziness based semi-supervised learning approach for intrusion detection system. Information Sciences, 378, 484-497.
Aung, Y. Y., & Min, M. M (2018). An analysis of K-means algorithm based network intrusion detection system. Advances in Science, Technology and Engineering Systems Journal, 3(1), 496-501.
Bauer, F. C., Muir, D. R., & Indiveri, G (2019). Real-Time Ultra-Low Power ECG Anomaly Detection Using an Event-Driven Neuromorphic Processor. IEEE Transactions on Biomedical Circuits and Systems, 13, 1575–82. https://doi.org/10.1109/TBCAS.2019.2953001
Bhati, B. S., Rai, C. S., Balamurugan, B., & Al-Turjman, F. (2020). An intrusion detection scheme based on the ensemble of discriminant classifiers. Computers & Electrical Engineering, 86, 106742.
Bhattacharyya, D. K., & Kalita, J. K. (2013). Network anomaly detection: A machine learning perspective. CRC Press.
Blanco, R., Malagón, P., Briongos, S., & Moya, J. M. (2019). Anomaly Detection Using Gaussian Mixture Probability Model to Implement Intrusion Detection System. In International Conference on Hybrid Artificial Intelligence Systems, 648-659. Springer, Cham.
Bock, T., Displayr blog, https://www.displayr.com/what-is-hierarchical-clustering/
Borghesi, A., Bartolini, A., Lombardi, M., Milano, M., & Benini, L. (2019). A semisupervised autoencoder-based approach for anomaly detection in high performance computing systems. Engineering Applications of Artificial Intelligence, 85, 634-644.
Chakir, E. M., Moughit, M., & Khamlichi, Y. I. (2018). An effective intrusion detection model based on SVM with feature selection and parameters optimization. Journal of Theoretical and Applied Information Technology, 96(12), 3873–85. https://www.researchgate.net/publication/326391656
Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM Computing Surveys (CSUR), 41(3), 1-58.
Chauhan, P., & Shukla, M. (2015). A review on outlier detection techniques on data stream by using different approaches of K-Means algorithm. In 2015 International Conference on Advances in Computer Engineering and Applications. 580-585. IEEE.
Chen, C. M., Guan, D. J., Huang, Y. Z., & Ou, Y. H. (2016). Anomaly network intrusion detection using hidden Markov model. International Journal of Innovative Computing, Information and Control, 12, 569-580.
Chew, Y. J., Ooi, S. Y., Wong, K. S., & Pang, Y. H. (2020). Decision Tree with Sensitive Pruning in Network-based Intrusion Detection System. In Computational Science and Technology, 1-10. Springer, Singapore.
DataRobot AI Wiki. https://www.datarobot.com/wiki/semi-supervised-machine-learning/
Ding, M., & Tian, H. (2016). PCA-based network traffic anomaly detection. Tsinghua Science and Technology, 21(5), 500-509.
Dua, S., & Du, X. (2016). Data mining and machine learning in cybersecurity. CRC press.
Duong, N. H., & Hai, H. D. (2015). A semi-supervised model for network traffic anomaly detection. In 2015 17th International Conference on Advanced Communication Technology (ICACT), 70-75. IEEE.
Fernandes G., Rodrigues, J. J., Carvalho, L. F., Al-Muhtadi, J. F., & Proença, M. L. (2019). A comprehensive survey on network anomaly detection. Telecommunication Systems, 70(3), 447-489.
Gu, J., Wang, L., Wang, H., & Wang, S. (2019). A novel approach to intrusion detection using SVM ensemble with feature augmentation. Computers & Security, 86, 53-62.
Han, X., Xu, L., Ren, M., &Gu, W. (2015). A Naive Bayesian network intrusion detection algorithm based on Principal Component Analysis. In 2015 7th International Conference on Information Technology in Medicine and Education (ITME), 325-328. IEEE.
Haripriya, L.A., Jabbar, M., & Seetharamulu, B. (2018). A Novel Intrusion Detection System Using Artificial Neural Networks and Feature Subset Selection. International Journal of Engineering and Technology, 7(4), 181. http://doi.org/10.14419/ijet.v7i4.6.20458
Hoang, D. H., & Nguyen, H. D. (2018). A PCA-based method for IoT network traffic anomaly detection. In 2018 20th International Conference on Advanced Communication Technology (ICACT), 381-386. IEEE.
Hodo, E., Bellekens, X., Hamilton, A., Dubouilh, P. L., Iorkyase, E., Tachtatzis, C., & Atkinson, R. (2016). Threat analysis of IoT networks using artificial neural network intrusion detection system. In 2016 International Symposium on Networks, Computers and Communications (ISNCC), 1-6. IEEE.
Hu, J., Ma, D., Liu, C., Shi, Z., Yan, H., & Hu, C. (2019). Network Security Situation Prediction Based on MR-SVM. IEEE Access, 7, 130937-130945.
Idhammad, M., Afdel, K., & Belouch, M. (2018). Semi-supervised machine learning approach for DDoS detection. Applied Intelligence, 48(10), 3193-3208.
Karim, S., Rousanuzzaman, P. A. Y., Khan, P. H., & Asif, M. (2019). Implementation of K-Means Clustering for Intrusion Detection. International Journal of Scientific Research in Computer Science, Engineering and Information Technology, 5, 1232–41.
Kevric, J., Jukic, S., & Subasi, A. (2017). An effective combining classifier approach using tree algorithms for network intrusion detection. Neural Computing and Applications, 28(1), 1051-1058.
Khraisat, A., Gondal, I., & Vamplew, P. (2018). An anomaly intrusion detection system using C5 decision tree classifier. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, 149-155. Springer, Cham.
Kim, E., & Kim, S. (2015). A novel hierarchical detection method for enhancing anomaly detection efficiency. In 2015 International Conference on Computational Intelligence and Communication Networks (CICN), 1018-1022. IEEE.
Knapp, E. D., & Langill, J. T. (2014). Industrial Network Security: Securing Critical Infrastructure Networks for Smart Grid, SCADA, and Other Industrial Control Systems. Syngress.
Kotu, V., & Deshpande, B. (2018). Data Science: Concepts and Practice. Morgan Kaufmann.
Kumar, D. A., & Venugopalan, S. R. (2018). A novel algorithm for network anomaly detection using adaptive machine learning. In Progress in Advanced Computing and Intelligent Engineering, 59-69. Springer, Singapore.
Kusyk, J., Uyar, M.U., & Sahin, C. S. (2018). Survey on evolutionary computation methods for cybersecurity of mobile ad hoc networks. Evolutionary Intelligence, 10, 95–117. https://doi.org/10.1007/s12065-018-0154-4
Lakhina, A., Crovella, M., & Diot, C. (2004). Diagnosing network-wide traffic anomalies. ACM SIGCOMM computer communication review, 34(4), 219-230.
Lalitha, K. V., & Josna, V. R. (2016). Traffic verification for network anomaly detection in sensor networks. Procedia Technology, 24, 1400-1405.
Larriva-Novo, X. A., Vega-Barbas, M., Villagra, V. A., & Sanz Rodrigo, M. (2020). Evaluation of Cybersecurity Data Set Characteristics for Their Applicability to Neural Networks Algorithms Detecting Cybersecurity Anomalies. IEEE Access, 8, 9005–14.
Lee, W., & Stolfo, S. (1998). Data mining approaches for intrusion detection In the 7th USENIX Security Symposium, San Antonio, Texas, USA.
Liang, W., Long, J., Chen, Z., Yan, X., Li, Y., Zhang, Q., & Li, K. C. (2018). A security situation prediction algorithm based on HMM in mobile network. Wireless Communications and Mobile Computing, 2018.
Liu, Y., Xu, H., Yi, H., Lin, Z., Kang, J., Xia, W., Shi, Q., Liao, Y., & Ying, Y (2017). Network anomaly detection based on dynamic hierarchical clustering of cross domain data. In 2017 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C), 200-204. IEEE.
Manasa, K. N., & Padma, M. C. (2019). A Study on Sentiment Analysis on Social Media Data. In Emerging Research in Electronics, Computer Science and Technology, 661-667. Springer, Singapore.
Mehmood, A., Mukherjee, M., Ahmed, S. H., Song, H., & Malik, K. M. (2018). NBC-MAIDS: Naïve Bayesian classification technique in multi-agent system-enriched IDS for securing IoT against DDoS attacks. The Journal of Supercomputing, 74(10), 5156-5170.
Meng, X., Mo, H., Zhao, S., & Li, J. (2017). Application of anomaly detection for detecting anomalous records of terrorist attacks. In 2017 IEEE 2nd International Conference on Cloud Computing and Big Data Analysis (ICCCBDA), 70-75. IEEE.
Mohammadi, S., Mirvaziri, H., Ghazizadeh-Ahsaee, M., & Karimipour, H. (2019). Cyber intrusion detection by combined feature selection algorithm. Journal of information security and applications, 44, 80-88.
Mohd Ali, A. (2018). Anomalous behaviour detection using heterogeneous data. (Doctoral dissertation) Lancaster University.
Münz, G., Li, S., & Carle, G. (2007). Traffic anomaly detection using k-means clustering. In GI/ITG Workshop MMBnet, 13-14.
Omar, S., Ngadi, A., & Jebur, H. H. (2013). Machine learning techniques for anomaly detection: an overview. International Journal of Computer Applications, 79(2).
Paffenroth, R., Kay, K., & Servi, L. (2018). Robust pca for anomaly detection in cyber networks. arXiv preprint arXiv:1801.01571.
Peng, H., Sun, Z., Zhao, X., Tan, S., & Sun, Z. (2018). A detection method for anomaly flow in software defined network. IEEE Access, 6, 27809-27817.
Pham, N. T., Foo, E., Suriadi, S., Jeffrey, H., & Lahza, H. F. M. (2018). Improving performance of intrusion detection system using ensemble methods and feature selection. In Proceedings of the Australasian Computer Science Week Multiconference, 1-6.
Rai, A. (2020). Optimizing a New Intrusion Detection System Using Ensemble Methods and Deep Neural Network. In 2020 4th International Conference on Trends in Electronics and Informatics (ICOEI) (48184), 527-532. IEEE.
Rai, K., Devi, M. S., & Guleria, A. (2016). Decision tree based algorithm for intrusion detection. International Journal of Advanced Networking and Applications, 7(4), 2828.
Reddy A., Ordway-West, M., Lee, M., Dugan, M., Whitney, J., Kahana, R., & Rao, M. (2017). Using Gaussian mixture models to detect outliers in seasonal univariate network traffic. In 2017 IEEE Security and Privacy Workshops (SPW), 229-234. IEEE.
Rettig, L., Khayati, M., Cudré-Mauroux, P., & Piórkowski, M. (2019). Online anomaly detection over big data streams. In Applied Data Science, 289-312. Springer, Cham.
Shukur, H. A., & Kurnaz, S. (2019). Credit Card Fraud Detection using Machine Learning Methodology. International Journal of Computer Science and Mobile Computing, 8, 257-260.
Stefanidis, K., & Voyiatzis, A. G. (2016). An HMM-based anomaly detection approach for SCADA systems. In IFIP International Conference on Information Security Theory and Practice, 85-99. Springer, Cham.
Swarnkar, M., & Hubballi, N. (2016). OCPAD: One class Naive Bayes classifier for payload based anomaly detection. Expert Systems with Applications, 64, 330-339.
Tang, C., Xiang, Y., Wang, Y., Qian, J., & Qiang, B. (2016). Detection and classification of anomaly intrusion using hierarchy clustering and SVM. Security and Communication Networks, 9(16), 3401-3411.
Techopedia - IT Education Site. https://www.techopedia.com/definition/30331/gaussian-mixture-model-gmm
Thakare, Y. S., & Bagal, S. B. (2015). Performance evaluation of K-means clustering algorithm with various distance metrics. International Journal of Computer Applications, 110(11), 12-16.
Vasan, K. K., & Surendiran, B. (2016). Dimensionality reduction using principal component analysis for network intrusion detection. Perspectives in Science, 8, 510-512.
Veselý, A., & Brechlerova, D. (2009). Neural networks in intrusion detection systems. Agricultural Economics (Zem?d?lská ekonomika), 156-165.
Vinayakumar, R., Soman, K. P., & Poornachandran, P. (2017). Evaluation of recurrent neural network and its variants for intrusion detection system (IDS). International Journal of Information System Modeling and Design (IJISMD), 8(3), 43-63.
Wang, L., Li, J., Bhatti, U. A., & Liu, Y. (2019). Anomaly Detection in Wireless Sensor Networks Based on KNN. In International Conference on Artificial Intelligence and Security, 632-643. Springer, Cham.
Weerasinghe, S., Erfani, S. M., Alpcan, T., & Leckie, C. (2019). Support vector machines resilient against training data integrity attacks. Pattern Recognition, 96, 106985.
Wu, K., Chen, Z., & Li, W. (2018). A novel intrusion detection model for a massive network using convolutional neural networks. IEEE Access, 6, 50850-50859.
Xiao, P., Qu, W., Qi, H., & Li, Z. (2015). Detecting DDoS attacks against data center with correlation analysis. Computer Communications, 67, 66-74.
Yuan, Y., Kaklamanos, G., & Hogrefe, D. (2016). A novel semi-supervised Adaboost technique for network anomaly detection. In Proceedings of the 19th ACM International Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems, 111-114.
Yun, J. H., Hwang, Y., Lee, W., Ahn, H. K., & Kim, S. K. (2018). Statistical similarity of critical infrastructure network traffic based on nearest neighbor distances. In International Symposium on Research in Attacks, Intrusions, and Defenses, 577-599. Springer, Cham.
Zegeye, W. K., Dean, R. A., & Moazzami, F. (2019). Multi-layer hidden Markov model based intrusion detection system. Machine Learning and Knowledge Extraction, 1(1), 265-286.
Zhang, T., Wang, X., Li, Z., Guo, F., Ma, Y., & Chen, W. (2017). A survey of network anomaly visualization. Science China Information Sciences, 60(12), 121101.
Zhao, Q., Zhang, Y., Shi, Y., & Li, J. (2019). Analyzing and Visualizing Anomalies and Events in Time Series of Network Traffic. In International Conference on Computing and Information Technology, 15-25. Springer, Cham.
Article Sidebar
Published
Dec 2, 2020
Article Details
How to Cite
Ali, W. A., N, M. K., Aljunid, M., Bendechache, M., & Sandhya, P. (2020). A Review of Current Machine Learning Approaches for Anomaly Detection in Network Traffic. Journal of Telecommunications and the Digital Economy, 8(4), 64–95. https://doi.org/10.18080/jtde.v8n4.307
Issue
Section
Review/Tutorial
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Copyright Telecommunications Association Inc.
Author Biographies
Wasim A. Ali, Department of Computer Science and Engineering, PG Centre, Visvesvaraya Technological University, Mysore, Karnataka, India
Wasim A. Ali received his B.E. Information technology from the University of science and technology, Taiz, Yemen (2014). He did his master's degree in Computer Science from St. Philomena’s College, University of Mysore, India(2017). Currently, he is pursuing a Ph.D. in computer science and engineering from Visvesvaraya Technological University (VTU), Mysore, India. He is working with the Public Corporation of Telecommunication (PTC), Yemen, since 2003.
Manasa K. N, Research scholar, PET Research Center (affiliated to University of Mysore), PES college of engineering, Mandya, Karnataka, India
I received B.E. and M. Tech. in Computer Science and Engineering from Visvesvaraya Technological University, Belgavi. Currently pursuing Ph.D., under the guideship of Dr. Padma M C, PET Research Centre (affiliated to the University of Mysore), Mandya.
Mohammed Fadhel Aljunid, Mangalore University, India
Mohammed Fadhel Aljunid,
(B.sc, M.Sc, Research Scholar),
Department of Post-Graduate Studies and Research in Computer Science,
Mangalore University, India,
Lecturer, Thamar University, Thamar, Yemen
Research interest: Artificial Intelligent, Information Filtering, Machine learning, Deep learning, Data mining, & Expert system
Dr Malika Bendechache, Dublin City University, Ireland.
Dr Malika Bendechache is an Assistant Professor in the School of Computing at Dublin City University. She lectures in Blockchain and Distributed Ledger Technology. Malika's research interests span the areas of Big data Analytics, Machine Learning, Data Governance, Parallel and Distributed Systems, Cloud/Edge/Fog Computing, Blockchain, Security and Privacy.
Dr P. Sandhya, Department of Computer Science and Engineering, PG centre, Visvesvaraya Technological University, Mysore, Karnataka, India
Dr. P. Sandhya is an associate Professor, Department of Computer Applications, Centre for Postgraduate Studies, Visvesvaraya technological University, Mysore, Research areas Biomedical Data mining, CDSS, Soft Computing. Professional Memberships in CSI (Life member), IEEE, and ARDSI.
