A Deep Learning Approach for Fault Diagnosis of Induction Motors in Manufacturing

Failures often occur in manufacturing machines, which may cause disastrous accidents, such as economic losses, environmental pollution, and even casualties. Effective diagnosis of these failures is essential in order to enhance reliability and reduce costs for operation and maintenance of the manufacturing equipment. As a result, research on fault diagnosis of manufacturing machines that utilizes data acquired by advanced sensors and makes decisions using processed sensor data has been seen success in various applications [1–3]. Induction motors, as the source of actuation, have been widely used in many manufacturing machines, and their working states directly influence system performance, thus affecting the production quality. Therefore, proper grasping of data reflecting the working states of induction motors can obtain early identification of potential failures [4]. During recent years, various approaches for induction motor fault diagnosis have been developed and innovated continuously [5–8].

Artificial intelligence (AI)-based fault diagnosis techniques have been widely studied, and have succeeded in many applications of electrical machines and drives [9, 10]. For example, a two-stage learning method including sparse filtering and neural network was proposed to form an intelligent fault diagnosis method to learn features from raw signals [11]. The feed-forward neural network using Levenberg-Marquardt algorithm showed a new way to detect and diagnose induction machine faults [12], where the results were not affected by the load condition and the fault types. In another study, a special structure of support vector machine (SVM) was proposed, which combined Directed Acyclic Graph-Support Vector Machine (DAG-SVM) with recursive undecimated wavelet packet transform, for inspection of broken rotor bar fault in induction motors [13]. Fuzzy system and Bayesian theory were utilized in machine health monitoring in Ref. [14]. Although these studies have shown the advantages of AI-based approaches for induction motor fault diagnosis, most of these approaches are based on supervised learning, in which high quality training data with good coverage of true failure conditions are required to perform model training [15]. However, it is not easy to obtain sufficient labelled fault data to train the model in practice.

Furthermore, many fault diagnosis tasks in induction motors depend on feature extraction from the measured signals. The feature characteristics directly affect effectiveness of fault recognition. In the existing literature, many feature extraction methods are suitable for fault diagnosis tasks, such as time-domain statistical analysis, frequency-domain spectral analysis [16], and time-scale/frequency analysis [17], among which wavelet analysis [18], which belongs to time-scale analysis, is a powerful tool for feature extraction and has been well applied to processing non-stationary signals. Whereas, the problem is that different features extracted from these methods may affect the classification accuracy. Therefore, an automatic and unsupervised feature learning from the measured signals for fault diagnosis is needed.

Limitations above can be overcome by deep learning algorithms which follow an effective way of learning multiple layers of representations [19]. Essentially, a deep learning algorithm uses deep neural networks which contain multiple hidden layers to learn information from the input, but was not put into practice because of its training difficulty until Geoffrey Hinton proposed layer-wise pre-training algorithm to effectively train deep networks in 2006 [20]. Since then, deep learning techniques have been advanced significantly and their successful applications have been seen in various fields [21], including hand written digit recognition [22], computer vision [23–26], Google Map [27], and speech recognition [28–30]. In addition, For natural language processing (NLP), deep learning has achieved several successful applications and made significant contributions to its progress [31–33]. In the area of fault diagnosis, deep learning theory also has many applications. For example, deep neural network built for fault signature extraction was utilized for bearings and gearboxes [34], while a classification model based on deep network architecture was proposed in the task of characterizing health states of the aircraft engine and electric power transformer [35]. The deep belief network (DBN) was also used for identifying faults in reciprocating compressor valves [36]. Sparse coding was used to built deep architecture for structural health monitoring [37], and a unique automated fault detection method named “Tilear” using deep learning concepts was proposed for the quality inspection of electromotor [38]. Furthermore, auto-encoder based DBN model was successfully applied to quality inspection [39], while a sparse model based on auto-encoder was shown to form a deep architecture, which realized induction motor fault diagnosis [40].

Inspired by the prior research, this paper presents a deep learning model based on DBN for induction motor fault diagnosis. The deep model is built on restricted Boltzmann machine (RBM) which is the building unit of a DBN and by stacking multiple RBMs one by one, the whole deep network architecture can be constructed. It can learn high-level features from frequency distribution of measured signals for diagnosis tasks. Including this section, this paper is organized with 5 sections. Section 2 provides theoretical background of the deep learning algorithm. Section 3 presents the proposed fault diagnosis approach, where the deep architecture based on DBN is described in detail. Experiments are carried out in Section 4 to verify the effectiveness of the proposed deep model, where classification performance is discussed. Section 5 summarizes the whole study and gives future directions.

The DBN is a deep architecture with multiple hidden layers that has the capability of learning hierarchical representations automatically in an unsupervised way and performing classification at the same time. In order to accurately structure the model, it contains both unsupervised pre-training procedure and supervised fine-tuning strategy. Generally, it is difficult to learn a large number of parameters in a deep architecture which has multiple hidden layers due to the vanishing gradient problem. To address this issue, an effective training algorithm, which learns one layer at a time and each pair of layers is seen as one RBM model, is proposed and introduced in Refs. [41, 42]. As DBN is formed by units of RBM, the basic unit of DBN, i.e., RBM, is introduced first.

2.1 Architecture of RBM

The RBM is a common used mathematical model in probability statistics theory and follows the theory of log-linear Markov Random Field (MRF) [36] which has several special forms and RBM is one of them. A RBM model contains two layers: One layer is the input layer which is also called visible layer, and the other layer is the output layer which also called hidden layer. RBM can be represented as a bipartite undirected graphical model. All the visible units of the RBM are fully connected to hidden units, while units within one layer do not have any connetion between each other. That is to say, there are no connection between visible units or between hidden units. The architecture of a RBM is shown in Figure 1.

In Figure 1, v represents the visible layer, i is the ith visible unit, h is the hidden layer, and j is the jth hidden unit. Connections between these two layers are undirected. An energy function is proposed to describe the joint configuration (v, h) between them, which is expressed as

$$E\left( {v,h} \right) = - \sum\limits_{i \in visible} {a_{i} v_{i} } - \sum\limits_{j \in hidden} {b_{j} h_{j} } - \sum\limits_{i,j} {v_{i} h_{j} w_{ij} } .$$

(1)

Here, v _i , and h _j represent the visible unit i and hidden unit j respectively; a _i , and b _j are their biases. w _ij denotes the weight between these two units. Therefore, the joint distribution of this pair can be obtained using the energy function where θ is the model parameter set containing a, b, and w:

$$p\left( {v,h} \right) = \frac{1}{Z\left( \theta \right)}\exp \left( { - E\left( {v,h} \right)} \right),$$

(2)

$$Z\left( \theta \right) = \sum\limits_{v} {\sum\limits_{h} {\exp \left( { - E\left( {v,h} \right)} \right)} } .$$

(3)

Due to the particular connections in RBM model, it satisfies conditional independent. Therefore, conditional probability of this pair of layers can be written as:

$$p\left( {h\left| v \right.} \right) = \prod\limits_{i} {p\left( {h_{i} \left| v \right.} \right)} ,$$

(4)

$$p\left( {v\left| h \right.} \right) = \prod\limits_{j} {p\left( {v_{i} \left| h \right.} \right)} .$$

(5)

Mathematically,

$$p\left( {h_{j} = 1\left| v \right.} \right) = \sigma \left( {b_{j} + \sum\limits_{i} {v_{i} w_{ij} } } \right),$$

(6)

$$p\left( {v_{i} = 1\left| h \right.} \right) = \sigma \left( {a_{i} + \sum\limits_{j} {h_{j} w_{ij} } } \right),$$

(7)

where σ(x) is the activation function. Generally, σ(x)=1/(1+exp(-x)) is adopted.

2.2 Training RBM

In order to set the model parameters, the RBM needs to be trained using training dataset. In the procedure of training a RBM model, the learning rule of stochastic gradient descent is adopted. The log-likelihood probability of the training data is calculated, and its derivative with respect to the weights is seen as the gradient, shown in Eq. (8). The goal of this training procedure is to update network parameters in order to obtain a convergence model.

$$\frac{\partial \log p(v)}{{\partial w_{ij} }} = < v_{i} h_{j} >_{data} - < v_{i} h_{j} >_{\text{model}} .$$

(8)

Parameter update rules are originally derived by Hinton and Sejnowki, which can be written as:

$$\Delta_{{w_{ij} }} = \varepsilon \left( { < v_{i} h_{j} >_{data} - < v_{i} h_{j} >_{\text{model}} } \right),$$

(9)

where ε is the learning rate, the symbol <·> _data represents an expectation from the data distribution while the symbol <·> _model is an expectation from the distribution defined by the model. The former term is easy to compute exactly, while the latter one is intractable to compute [43].

An approximation to the gradient is used to obtain the latter one which is realized by performing alternating Gibbs sampling, as illustrated in Figure 2(a).

Later, a fast learning procedure is proposed, which starts with the visible units, then all the hidden units are computed at the same time using Eq. (6). After that, visible units are updated in parallel to get a “reconstruction” by Eq. (7), as illustrated in Figure 2(b), and the hidden units are updated again [44]. Model parameters are updated as:

$$\Delta w_{ij} = \varepsilon \left( {\left. {\left\langle {v_{i} h_{j} } \right\rangle_{data} - \left\langle {v_{i} h_{j} } \right\rangle_{recon} } \right)} \right..$$

(10)

In addition, for practical problems that come down to real-valued data, Gaussian-Bernoulli RBM is introduced to deal with this issue. Input units of this model are linear while hidden units are still binary. Learning procedure for Gaussian-Bernoulli RBM is very similar to binary RBM introduced above.

2.3 DBN Architecture

DBN model is a deep network architecture with multiple hidden layers which contain many nonlinear representation. It is a probabilistic generative model and can be formed by RBMs as shown in Figure 3. It illustrates the way of stacking one RBM on top of another. DBN architecture can be built by stacking multiple RBMs one by one to form a deep network architecture.

As DBN has multiple hidden layers, it can learn from the input data and extract hierarchical representation corresponding to each hidden layer. Joint distribution between visible layer v and the l hidden layers h _k can be calculated mathematically from conditional distribution P(h ^k−1 |h ^k ) for the (k–1)th layer conditioned on the kth layer and visible-hidden joint distribution P(h ⁿ⁻¹, h ⁿ ):

$$P\left( {v,h^{1} , \ldots ,h^{n} } \right) = \left( {\prod\limits_{k = 1}^{n - 1} {P\left( {h^{k - 1} |h^{k} } \right)} } \right)P\left( {h^{n - 1} ,h^{n} } \right).$$

(11)

For deep neural networks, learning such amount of parameters using traditional supervised training strategy is impractical because errors transferred to low level layers will be faint through several hidden layers and the ability to adjust the parameters is weak for traditional back propagation method. It is difficult for the network to generate globally optimal parameters. Here the greedy layer-by-layer unsupervised pre-training method is used for training DBNs. This procedure can be illustrated as follows: The first step is to train the input units (v) and the first hidden layer (h ₁) using RBM rule(denoted as RBM1). Next, the first hidden layer (h ₁) and the second hidden layer (h ₂) are trained as a RBM (denoted as RBM2) where the output of RBM1 is used as the input for the RBM2. Similarly, the following hidden layers can be trained as RBM3, RBM4,…, RBMn until the set number of layers are met. It is an unsupervised pre-training procedure, which gives the network an initialization that contributes to convergence on the global optimum.

For classification tasks, fine-tuning all the parameters of this deep architecture together is needed after the layer-wise pre-training, as shown in Figure 4. It is a supervised learning process using labels to eliminate the training error and improve the classification accuracy [45, 46].

Based on the DBN, a fault diagnosis approach for induction motor has been developed, as illustrated in Figure 5, where the DBN model is built to extract multiple levels of representation from the training dataset.

Vibration signals are selected as the input of the whole system for fault diagnosis as they usually contain useful information that can reflect the working state of induction motors. However, there exists correlation between sampled data points. This is difficult for DBN architecture to model as it does not have the ability to function the correlation between the input units which may influence the following classification task. Therefore, in this study the vibration signals are transformed from time domain to frequency domain using Fast Fourier Transform (FFT), and then frequency distribution of each signal is used as the input of the DBN architecture. This is beneficial to classification task during the training procedure. Specifically, DBN learns a model that generates input data, which can obtain more intrinsic characteristics of the input, thus improving classification accuracy eventually. In this module, DBN stacked by a number of RBMs is built and then trained by training dataset from data preparation module to obtain the model parameters. The DBN training process is shown in Figure 6. Input parameters of the architecture will be first initialized including a set of neuron numbers and hidden layer numbers, together with training epochs. Each layer of the architecture is then trained as a RBM unit, and the output of lower-layer RBM is used as the training input for the next layer RBM.

After layer-by-layer learning, synaptic weights and biases are settled and the basic structure is determined. Classification process is then followed to predict the fault category. It is a supervised fine-tuning procedure and the proposed method adopts the back-propagation training algorithm to realize fine-tuning which uses labeled data for training, so that it can improve the discriminative ability for classification task. The unsupervised training process trains one RBM at a time and afterwards supervised fine-tuning process using labels adjusts weights of the whole model. The difference between DBN outputs and the target label is regarded as training error. In order to obtain the minimum error, the deep network parameters will be updated based on learning rules.

After training the DBN model, all the DBN parameters are fixed, and the next procedure is to test the classification capability of the trained DBN model and classification rate is calculated as an index for evaluation. The vibration signal is the input of the constructed fault diagnosis system, and its output indicates working states of the induction motor.

This paper presents a deep learning model based on DBN, where frequency distribution of the measured data is used as input, for fault diagnosis of induction motors in manufacturing. The construction of this deep architecture uses restricted Boltzmann machine as a building unit, and uses greedy layer-wise training for model construction. The presented approach makes use of strong capabilities of DBN, which can model high-dimensional data and learn multiple layers of representation, thus can reduce training error and improve classification accuracy. Experimental studies are carried out using vibration signals to verify the effectiveness of the DBN model for feature learning, providing a new way of feature extraction for automatic fault diagnosis in manufacturing.

In future work, methods to improve the performance of the DBN model in fault diagnosis will be explored. Generalization ability of the model will also be investigated to overcome the problem of overfitting. Using both labeled and unlabeled datasets to train the DBN model is also of interest. In addition, the performances corresponding to different model parameters need to be further researched.