Data Warehouse, Detection and Transfer of Anomalies in Retail Data
Main Article Content
Abstract
In this article, we offer some suggestions for anomaly detection on the data received from the source to the Data warehouse. As a result, it is aimed to prevent the entry of dirty and noisy data into the data warehouse. We think that knowing that there is clean and healthy data in the data warehouse will be resistant to anomalies in the processed data used for data science. In order to reach our goal, studies were carried out on the data in the retail sector. We aimed to determine our theoretical thoughts from some topics such as user erroneous login data in the retail and energy industry, abnormal sales over employees during the campaign period, product stock abnormality, and incorrect pricing. When we examined many studies, we saw that they made anomaly detection after estimation. Before taking the data from the source to the data warehouse, we thought that anomaly detection would be more efficient and healthier. Analysis and results were evaluated on the data obtained in the wiseboard retail project of Gtech company.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
References
Nguyen, H. D., et al. "Forecasting and Anomaly Detection approaches using LSTM and LSTM Autoencoder techniques with the applications in supply chain management." International Journal of Information Management 57 (2021): 102282. DOI: https://doi.org/10.1016/j.ijinfomgt.2020.102282
Jansen, Maarten, Laurens Swinkels, and Weili Zhou. "Anomalies in the China A-share market." Pacific-Basin Finance Journal 68 (2021): 101607. DOI: https://doi.org/10.1016/j.pacfin.2021.101607
Hampton, Harrison, and Aoife Foley. "A review of current analytical methods, modelling tools and development frameworks applicable for future retail electricity market design." Energy (2022): 124861. DOI: https://doi.org/10.1016/j.energy.2022.124861
Oliveira, João Pedro, and Rui Dinis Sousa. "Unsupervised Anomaly Detection of Retail Stores Using Predictive Analysis Library on SAP HANA XS Advanced." Procedia Computer Science 181 (2021): 882-889. DOI: https://doi.org/10.1016/j.procs.2021.01.243
Ramakrishnan, Jagdish, et al. "Anomaly detection for an e-commerce pricing system." U.S. Patent Application No. 17/721,594.
Chen, Xu, et al. "GraphAD: A Graph Neural Network for Entity-Wise Multivariate Time-Series Anomaly Detection." arXiv preprint arXiv:2205.11139 (2022). DOI: https://doi.org/10.1145/3477495.3531848
Vincent, Vercruyssen, Meert Wannes, and Davis Jesse. "Transfer learning for anomaly detection through localized and unsupervised instance selection." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 34. No. 04. 2020. DOI: https://doi.org/10.1609/aaai.v34i04.6068
Putra, Hafid Yoza. "Fraud detection at self-checkout retail using data mining." 2020 International Conference on Information Technology Systems and Innovation (ICITSI). IEEE, 2020. DOI: https://doi.org/10.1109/ICITSI50517.2020.9264919
Pourhabibi, Tahereh, et al. "Fraud detection: A systematic literature review of graph-based anomaly detection approaches." Decision Support Systems 133 (2020): 113303. DOI: https://doi.org/10.1016/j.dss.2020.113303
Leite, Roger A., et al. "Visual analytics for event detection: Focusing on fraud." Visual Informatics 2.4 (2018): 198-212. DOI: https://doi.org/10.1016/j.visinf.2018.11.001
Laptev, Nikolay, Saeed Amizadeh, and Ian Flint. "Generic and scalable framework for automated time-series anomaly detection." Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining. 2015. DOI: https://doi.org/10.1145/2783258.2788611
Haldar, Malay, et al. "Applying deep learning to airbnb search." Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2019. DOI: https://doi.org/10.1145/3292500.3330658
Liu, Xiufeng, and Per Sieverts Nielsen. "Regression-based online anomaly detection for smart grid data." arXiv preprint arXiv:1606.05781 (2016).
Shipmon, Dominique T., et al. "Time series anomaly detection; detection of anomalous drops with limited features and sparse examples in noisy highly periodic data." arXiv preprint arXiv:1708.03665 (2017).
Thomassey, Sébastien. "Sales forecasting in apparel and fashion industry: A review." Intelligent fashion forecasting systems: Models and applications (2014): 9-27. DOI: https://doi.org/10.1007/978-3-642-39869-8_2
Greff, Klaus, et al. "LSTM: A search space odyssey." IEEE transactions on neural networks and learning systems 28.10 (2016): 2222-2232. DOI: https://doi.org/10.1109/TNNLS.2016.2582924