A Multimodal Deep Learning Framework for Predicting Machine Anomalies Using IoT-Enabled Vibration and Sound Data

Abstract

Unplanned machine downtimes caused by component failures, overheating, or mechanical stress significantly impact manufacturing efficiency and profitability. Predicting such failures before they occur is a core objective of smart manufacturing and Industry 4.0. Leveraging recent advances in sensor technology and machine learning, this study proposes an anomaly detection architecture that predicts the operational state of manufacturing machines one step ahead, enabling early detection of potential downtime.

The system integrates two primary data sources: vibration signals collected by an IoT-enabled device and sound recordings obtained from a microphone positioned close to the manufacturing equipment. These complementary signals capture the machine’s dynamic behaviour under varying operational conditions. While vibration and line status data are directly utilized, sound recordings undergo pre-processing using a low-pass filter to remove irrelevant background noise. The filtered recordings are segmented into one-minute intervals, and statistical features are extracted in both time and frequency domains, including mean, standard deviation, skewness, and kurtosis. Since the available dataset covers only one day, a moving block bootstrap technique is employed to improve robustness and generalization.

Two deep learning architecture, Long Short-Term Memory (LSTM) and Multi-Layer Perceptron (MLP), are implemented to forecast the machine state at time t + 1. The dataset, consisting of nine features and approximately 13,200 samples, is divided into training, validation, and test sets in a 70/15/15 ratio. Both models are trained using the Adam optimizer and binary cross-entropy loss. Performance is evaluated using precision, recall, and F1 score metrics.

Overall, the proposed approach demonstrates that combining vibration and acoustic data with deep learning can effectively predict machine anomalies in real time, contributing to proactive maintenance and reduced production downtime in smart manufacturing environments.