Rail Internal Defect Detection Method Based on Enhanced Network Structure and Module Design Using Ultrasonic Images

Abstract

Improving the detection accuracy of rail internal defects and the generalization ability of detection models are not only the main problems in the field of defect detection but also the key to ensuring the safe operation of high-speed trains. For this reason, a rail internal defect detection method based on an enhanced network structure and module design using ultrasonic images is proposed in this paper. First, a data augmentation method was used to extend the existing image dataset to obtain appropriate image samples. Second, an enhanced network structure was designed to make full use of the high-level and low-level feature information in the image, which improved the accuracy of defect detection. Subsequently, to optimize the detection performance of the proposed model, the Mish activation function was used to design the block module of the feature extraction network. Finally, the proposed rail defect detection model was trained. The experimental results showed that the precision rate and ${F}_{1}$ score of the proposed method were as high as 98%, while the model’s recall rate reached 99%. Specifically, good detection results were achieved for different types of defects, which provides a reference for the engineering application of internal defect detection. Experimental results verified the effectiveness of the proposed method.

1 Introdution

Strong friction, extrusion, and load impact are produced on the rail during high-speed train operation, which results in surface and internal defects in the rail [1,2,3]. If these defects are not identified in a timely and accurately manner, and the corresponding treatment is not implemented rapidly, they will further propagate and may cause major accidents, such as train derailment or overturning [4, 5], resulting in serious casualties and economic losses [6]. In addition, under the influence of long-term loads and the external environment, various internal defects such as rail head flaws and screw hole cracks are likely to develop on the rail. Such defects are more difficult to detect than surface defects, and they also result in hidden dangers to train operation. Therefore, it is of great significance to study an accurate and fast rail-internal-defect detection method to realize efficient maintenance and management of railways.

Manual methods are typically used to detect internal rail defects; however, because of their low detection efficiency and limitations in the detection experience of existing workers, manual methods are gradually being replaced by other detection methods [7]. In recent years, various nondestructive testing methods have been developed to detect rail internal defects. Common rail defect detection technologies include ultrasonic testing, eddy current testing, visual inspection, and magnetic flux leakage testing [8,9,10]. Ultrasonic testing is usually applied to rail internal defect detection because of its good performance in directivity, strong penetration, and high sensitivity [11].

Recently, Kim et al. [12] designed a phased array ultrasonic system for detecting rail cracks based on the phased array ultrasonic transducer. The system can effectively detect star cracks in the bolt holes, as well as longitudinal and welding cracks in the rail. For better detection of rail bottom defects, Pathak et al. [13] developed a laser-induced ultrasonic guided wave detection method based on finite element simulation. Their experimental results show that this method can identify the effective ultrasonic detection frequency and sensor position, which is conducive to reliable defect detection. To quickly detect rail bottom defects on site, a fast ultrasonic B-scan image detection method based on a shear horizontal guided wave electromagnetic acoustic transducer was proposed in Ref. [14]. This method can eliminate the strong noise in the echo signal and improve the quality of B-scan images to realize effective detection of rail bottom defects. By combining laser ultrasonic detection and variational mode decomposition, Jiang et al. [15] proposed a quantitative method to detect rail surface defects. The quantitative error of the method was less than 5%, which implies that it could effectively detect surface defects of different lengths. This method laid the foundation for the visual inspection of laser ultrasonic rail surface defects. In addition, a defect detection system [16] based on laser ultrasonic technology was proposed to detect and locate rail surface defects. The experimental results show that the proposed system can be used to detect defects with a depth greater than 0.5 mm on the rail surface and is a good solution for evaluating the service life of rail.

Evidently, the rail defect detection methods mentioned above, which are all based on ultrasonic testing, are more efficient than manual detection. However, the poor robustness and detection effects remain challenging problems in this process. Fortunately, because deep learning can automatically extract rich and deep-level features from image samples to identify different types of rail defects, the problems mentioned above are expected to be solved in future work. Presently, studies on have been conducted on using deep learning for rail defect detection. For example, Aydin et al. [17] proposed a rail defect detection and classification method based on image processing and deep learning to detect rail surface defects quickly and in real-time. The experimental results showed that the classification accuracy of this method was as high as 97.10% and that rail surface defects could be better recognized under low contrast. To realize automatic detection of external substances at the bottom of a high-speed train, Yao et al. [18] developed a fast object detection method based on an improved YOLO v3 model. The experimental results also showed that the mAP rate, accuracy, and recall rate of the method were 14.82%, 4.3%, and 9.54%, respectively, higher than those of the original YOLO v3, respectively. Considering the problem of fewer fastener defect samples in ballastless rail, Ref. [19] proposed a fastener defect identification method based on semi-supervised deep learning. The experimental results showed that this method could quickly locate and identify defects such as missing fasteners and broken elastic strips at an effective detection rate of 95%. In addition, to solve the problems of poor robustness and missed detection of small-area defects in existing rail surface defect detection methods, Han et al. [20] proposed a rail surface defect detection method with multilevel feature fusion. Compared with existing methods, the proposed method performed better and could be applied in the engineering field owing to its high recall rate and ${F}_{1}$ score.

In summary, exiting methods can detect rail defects in a certain degree, while improving the detection accuracy of rail internal defects and generalization ability of detection models remains a challenge. To this end, we propose a method based on an enhanced network structure and module design using ultrasonic images to accurately detect rail defects. First, data augmentation technology is used to expand the existing image data set, which avoids overfitting in the proposed model. Second, an enhanced network structure is designed to fully integrate the high-level and low-level feature information of the rail image to improve the accuracy of defect detection. Finally, the block module in the ResNet-50 feature extraction network is designed to optimize the detection performance of the proposed model.

The remainder of this paper is organized as follows. In Section 2, the rail defect types and their features are analyzed based on the B-scan images. The rail internal defect detection method designed to detect objects is described in Section 3. Subsequently, relevant experiments conducted to evaluate the effectiveness of the proposed method are presented in Section 4. Finally, conclusions are presented in Section 5.

2 Rail Internal Defect Classification and Feature Analysis

2.1 B-scan Image Analysis

Currently, B-scan image flaws are mostly detected manually, resulting in low detection efficiency and strong subjectivity. Although it is difficult to extract all types of features from defects in B-scan images and accurately model them using traditional image processing techniques. However, B-scan flaw detection has received increasing attention compared to A-type flaw detection for the following three main reasons: (1) B-scan images (as shown in Figure 1) can directly reflect the location of defects, which is convenient for identifying and judging defects for workers. (2) B-scan flaw detection have display delay, which means that the acquired B-scan image does not disappear immediately. (3) B-scan images are easy to store, and their data volume is one-thousandth that of A-type images [21]. In addition, compared with the other images, the pixels in the B-scan images do not change with light and noise. Therefore, B-scan images can be widely used to detect and identify internal rail defects.

Figure 1

B-scan image with rail internal defects

2.2 Defect Classification and Feature Analysis of B-scan Image

According to the defect categories in the existing data analysis software for rail flaw detectors and the classification standards for rail defects published by the Ministry of Railways [22], rail defects are divided into 11 categories, as shown in Table 1.

Table 1 Rail internal defects

The various defects in Table 1 are shown in the B-scan image in Figure 2. From top to bottom, the B-scan image can be divided into four areas: the rail head, rail jaw, rail waist, and rail bottom. The positions, shapes, colors, and imaging modes of different types of defects in B-scan images are usually different. For example, rail-head flaws can be detected using a 70° probe. Because the defects are in different specific positions, they also show different shapes, positions, and colors in the rail head area of the B-scan image. Different defect types, including inside, middle, and outside flaws of the rail head may arise. Then, the screw-hole crack can be detected by a 37° probe because its imaging position is usually near the screw-hole and its imaging color is related to its position. For example, it is imaged in green when inside a rail but imaged in blue on the outside of the rail. The horizontal crack of the screw hole is usually detected by a 0° probe because its imaging position is located below the screw-hole and its imaging color is red. In contrast, rail-bottom transverse cracks can be detected by a 37° probe and imaged in pairs at the rail bottom, and their imaging thickness and length are also different.

Figure 2

Rail defects in B-scan images: (a) Inside flaw of rail head, (b) Middle flaw of rail head, (c) Outside flaw of rail head, (d) Upper crack on the inside of the screw hole, (e) Lower crack on the inside of the screw hole, (f) Inverted lower crack on the inside of the screw hole, (g) Upper crack on the outside of the screw hole, (h) Lower crack on the outside of the screw hole, (i) Inverted lower crack on the outside of the screw hole, (j) Horizontal crack of screw hole, (k) Rail bottom transverse crack

3 Defect Detection Method Based on Enhanced Network Structure and Module Design Under Ultrasonic Image

As mentioned above, there are many types of defects in the B-scan images, and different defects have similarities and clutter interference to some extent. Although traditional image-processing technology can recognize and classify these defects, it is still difficult to detect them accurately. Fortunately, deep learning methods have developed rapidly in the field of image recognition and have shown strong detection performance [23]. Convolutional neural networks usually include input, convolution, pooling, fully connected, and output layers. In this study, the image feature is extracted accurately by inputting the image pixel information and using a multilayer convolution operation, which retains the information of the input image to the greatest extent

Therefore, to detect internal defects quickly and effectively, a method based on an enhanced network structure and module design using ultrasonic images is proposed in this paper. As shown in Figure 3, ResNet-50 [24] was used as the feature extraction network to analyze the characteristics of rail defects in B-scan image, an enhanced network structure was designed to extract the feature information of rail defects, and the Mish activation function was reconstructed to improve the generalization ability of the proposed detection network.

Figure 3

Network structure of the proposed method

3.1 Design of Enhanced Network Structure

Detailed information is usually contained in low-level features, whereas semantic information can be found in high-level features [25]. In particular, existing feature pyramid networks (FPNs) usually focus on the application of high-level features and enhance the detection accuracy of network structures. In such a network structure, the semantic information of high-level features is transmitted to the low-level feature map via a top-down path, which improves the classification accuracy of low-level features. However, the detailed information in the low-level features is not fully utilized; consequently, the positioning accuracy of high-level feature maps still has room for improvement.

Therefore, to accurately detect rail defects and reduce the loss of feature information, an enhanced network structure is proposed based on PANet [26]. Specifically, a location information feature channel was designed based on the FPN structure, thereby transferring the low-level feature into a high-level feature map for feature fusion. This makes full use of high-level and low-level feature information, which further improves the positioning accuracy of small objects.

A defect detection model based on the enhanced network structure is designed, as shown in Figure 4. A bottom-up path is built after the top-down path of the FPN in the proposed structure. Hierarchical feature maps, {C2, C3, C4, C5}, are obtained after the input image is processed using the ResNet-50 backbone network. Subsequently, their corresponding hierarchical feature maps, {P2, P3, P4, P5}, are obtained using the FPN. Finally, the corresponding hierarchical feature maps, {T2, T3, T4, T5}, are outputted using the designed bottom-up path for post-processing.

Figure 4

The proposed detection model of rail defects based on enhanced network structure

Taking T2 and T3 as examples, the fusion process of the bottom-up path is analyzed as follows. First, P2 is down-sampled by a 3×3 convolution with a step size of 2. Second, P3 and P2, which are down-sampled, are fused by pixel-by-pixel addition. Finally, the fused feature is processed by a 3×3 convolution and then outputted as a T3 feature map to enhance its characterization ability. The T4 and T5 feature maps were obtained in the same manner. In this model, T2 is equal to the P2 outputted by the original FPN, and T6 is obtained by maximum pooling based on T5.

3.2 Block Module Design

Previously, ReLU [27] was used as the activation function in the ResNet-50 feature-extraction network. In this study, the Mish function was adopted to replace the ReLU function in ResNet-50 and improve the detection performance of the proposed network. The curves of the ReLU and Mish activation functions are shown in Figure 5. Their corresponding expressions are shown as Eqs. (1) and (2), respectively.

Figure 5

The activation function of the ReLU and Mish: (a) ReLU activation function, (b) Mish activation function

$$h\left( x \right) = \left\{ {\begin{array}{*{20}c} {x,\quad \left( {x > 0} \right),} \\ {0,\quad \left( {x \le 0} \right),} \\ \end{array} } \right.$$

(1)

$$f\left(x\right)=x\cdot \mathrm{tanh}\left(\mathrm{ln}\left(1+{\it{e}}^{x}\right)\right).$$

(2)

As shown in Figure 5, the Mish activation function is continuous and differentiable. Therefore, gradient optimization can easily cope with back propagation, and the optimization and network generalization performance of the proposed model will be better. In addition, the Mish function diverges in the positive infinite direction and converges to 0 in the negative infinite direction, and it has a minimum value and non-monotonicity below the x-axis, which can produce a strong regularization effect. A relatively large negative output can also be obtained using the Mish function, but the absolute value of this negative input is relatively small. Therefore, this is conducive to improving the expression ability of the proposed model. Therefore, to detect rail internal defects accurately and effectively, a block module is designed by replacing the ReLU function with the Mish function in the ResNet-50 feature extraction network. Taking the conv2_x layer as an example, a diagram of the block module structure is shown in Figure 6.

Figure 6

Block module structure diagram

The module structure can be expressed as:

$$H\left(x\right)=F\left(x\right)+x,$$

(3)

where $x$ and $H\left(x\right)$ are the input and output values, respectively, of the module. $F\left(x\right)$ represents the internal operation of the block module. $H\left(x\right)=F\left(x\right)$ if $F\left(x\right)=0$, indicating identity mapping. Therefore, ResNet-50 learns the difference between $H\left(x\right)$ and $x$ rather than focusing on a complete output. That is, it learns the residual value, $F\left(x\right)=H\left(x\right)-x$. The residual result can be approximated to zero when the network is trained. With the deepening of the network layer, the accuracy does not decline, which solves the problem of gradient disappearance after the deepening of the network layer. Furthermore, it would be better to extract semantic information from hundreds or thousands of layers for a neural network using the proposed structure.

3.3 Loss Function and Evaluation Index

3.3.1 Loss Function

During the training process, the multitask loss function was used to train the proposed model to predict the defect type and position correctly. These equations are expressed as follows:

$$L\left(\left\{{p}_{i}\right\},\left\{{t}_{i}\right\}\right)=\frac{1}{{N}_{\mathrm{cls}}}{\sum }_{i}{L}_{\mathrm{cls}}\left({p}_{i},{p}_{i}^{*}\right)+\lambda \frac{1}{{N}_{\mathrm{reg}}}{\sum }_{i}{p}_{i}^{*}{L}_{\mathrm{reg}}\left({t}_{i},{t}_{i}^{*}\right),$$

(4)

$${L}_{\mathrm{cls}}\left({p}_{i},{p}_{i}^{*}\right)=-\mathrm{log}\left[{p}_{i}{p}_{i}^{*}+\left(1-{p}_{i}^{*}\right)\left(1-{p}_{i}\right)\right],$$

(5)

$${L}_{\mathrm{reg}}\left({t}_{i},{t}_{i}^{*}\right)=smoot{h}_{{L}_{1}}\left({t}_{i}-{t}_{i}^{*}\right),$$

(6)

$$smoot{h}_{{L}_{1}}\left(x\right)=\left\{\begin{array}{c}0.5{x}^{2},\,\,\, \left|x\right|<1,\\ \left|x\right|-0.5, {\text{otherwise}}.\end{array}\right.$$

(7)

where $L\left(\left\{{p}_{i}\right\}, \left\{{t}_{i}\right\}\right)$ is the loss function of the RPN, ${L}_{\mathrm{cls}}$ is the classification loss, ${L}_{\mathrm{reg}}$ is the regression loss, and subscript $i$ is the anchor index. ${p}_{i}$ is the probability that the ith anchor is a real tag. ${p}_{i}^{*}$ can be 0 or 1. Specifically, ${p}_{i}^{*}$ equals 1 if the anchor is a positive sample. Otherwise, ${p}_{i}^{*}$ is 0. ${t}_{i}$ is the bounding box regression parameter for predicting the ith anchor and ${t}_{i}^{*}$ is the bounding box regression parameter of the ground truth corresponding to the ith anchor. ${N}_{\mathrm{cls}}$ is the number of samples in a mini-batch, and its value was 256. ${N}_{\mathrm{reg}}$ is the number of anchor locations. $\lambda$ is the equilibrium parameter for the classification and regression.

3.3.2 Evaluation Indices

Three indices, including recall rate, precision rate and F1-Measure (${F}_{1}$ score), were adopted to accurately and effectively evaluate the defect detection performance of the proposed model. The equations are as follows:

$$R=\frac{TP}{TP+FN},$$

(8)

$$P=\frac{TP}{TP+FP},$$

(9)

$${F}_{1}=\frac{2PR}{P+R},$$

(10)

where $R$ is the recall rate, $P$ is the precision rate, and ${F}_{1}$ is the harmonic mean of $R$ and $P$. $TP$ (Ture-positive) indicates the number of defects correctly identified, $FP$ (false positive) indicates the number of incorrectly identified defects, and $FN$ (false negative) indicates the number of missed defects. ${F}_{1}$ combines precision and recall rates. The larger the value, the better the detection performance of the proposed model.

4 Experimental Results and Analysis

4.1 Experimental Environment and Parameter Settings

The hardware environment mainly includes an Intel i7-10700k eight-core processor and a GeForce RTX 2080ti graphics card, while the software environment mainly includes a 64-bit Windows 10 operating system, a deep learning framework with PyTorch1.6, and integrated development based on CUDA10.1 and PyCharm. Python 3.7 was used as the running environment.

After repeatedly testing the parameters, the final hyperparameters were as follows: Stochastic gradient descent (SGD) was used as the optimizer, and its initial learning rate was set to 0.005, the momentum was 0.9, and the weight attenuation was 0.0005. In addition, the batch size was set to 4, while the number of epochs was 30. Every time three epochs were trained; the learning rate was attenuated and its attenuation factor was 0.33.

4.2 Sample Labeling and Model Training

As shown in Figure 7, sample images were acquired using the ultrasonic flaw detector, EGT-60, which was developed by the authors and a cooperative company. The rail internal defect database was established based on the above experimental environment and parameter setting conditions. First, the number of database images was expanded to 1875 using data augmentation. The image resolution was 1000×400 pixels, and at least one type of defect was included in each image. Second, the database was divided into training, validation, and test sets at a ratio of 6:2:2. Finally, defects in the images of the training and validation sets were labeled using the annotation tool. The annotation format was the PASCAL VOC dataset format, while the type of generated annotation file was XML. As listed in Table 1, various types of defects are marked by their corresponding abbreviations to clearly display the prediction results of the proposed model.

Figure 7

Field operation scenario

Furthermore, a suitable anchor size for detecting rail defects was designed using the K-means clustering algorithm [28]; its area and aspect ratio were {82, 102, 152, 172, 232} and {0.5:1, 1.8:1, 2.5:1}, respectively. To accelerate the training, parameter-based transfer learning was used to pre-train the proposed model. Specifically, the parameter weights of the pre-training were obtained by training the COCO dataset. An appropriate classifier was also designed to correctly classify different defects. Finally, the proposed model was trained using the experimental environment described above.

4.3 Experimental Results and Analysis

To validate the performance of the proposed model, the images in the test set were detected. The test results are shown in Figure 8. Good detection effect of the proposed model can be achieved using this testing process for different types of rail defects. The proposed model can also identify the positions of defects accurately and has the ability to predict categories. Subsequently, the results in Tables 2 and 3 were obtained after statistical testing of various defects and analysis of their relevant evaluation indices.

Figure 8

Defect-detection results: (a) Overall defect-detection results, (b) Rail head defect-detection results, (c) Defect-detection results for the rail waist, (d) Defect-detection results for the rail bottom

Table 2 Test results for various defects

Table 3 Evaluation index analysis for various defects

It can be seen from Table 2 that there were 3431 defects in the test set. Specifically, 3393 defects were correctly detected by the proposed model. However, 73 non-defect objects were misidentified as defects, whereas 38 defects were not detected. Therefore, the overall average prediction probability of the model was as high as 97%. The precision rate, recall rate, and ${F}_{1}$ score were 98%, 99%, and 98%, respectively. However, the model performed well when detecting a single type of defect. Specifically, the lowest precision rate was 96% and the highest was 100%. The lowest and highest recall rates were 95% and 100%, respectively. The minimum and maximum ${F}_{1}$ scores were 95% and 100%, respectively. These results show that the proposed model can effectively detect various types of defects, providing an effective method for detecting internal rail defects.

However, the proposed model can still be improved in certain cases. For example, the frequency of misidentifying non-defect objects as defects is relatively high when detecting certain types of defects (such as Ir and Ui in Table 2). A possible reason for this phenomenon is that non-defect objects are very similar to defects in location, color, shape, and other characteristics, which causes the model to identify them as probable defects. However, the problem of missed detection of defects was found in the model. This may be because the number of related defects in the training samples and the defect features obtained by model learning were insufficient, which resulted in an unsatisfactory generalization ability and caused the model to fail to detect all defects.

Tables 4 and 5 were obtained by comparing the proposed method with the defect detection method proposed in a previous study [29]. As can be observed from Table 4, fewer types of defects were detected by the method proposed in Ref. [29], whereas 11 types of common defects were detected by the method proposed in this paper. Furthermore, the precision rate, recall rate, and ${F}_{1}$ score of the proposed method both exceed 98%, which implies that the proposed method can effectively detect rail internal defects with a higher detection ability.

Table 4 Data analysis of the two methods

Table 5 Performance comparison of the two methods

5 Conclusions

An accurate detection method for rail internal defects based on an enhanced network structure and module design using ultrasonic images is proposed in this paper. The main conclusions are summarized as follows.

• (1) An enhanced network structure was designed to transfer the detailed information in the low-level features into the high-level feature map, which improved the positioning accuracy of rail defects by effectively using low-level and high-level feature information.

• (2) In the ResNet-50 feature extraction network, the block module was designed using the Mish activation function to optimize the detection performance of the proposed model. It detected rail internal defects effectively, as evidenced by the experimental results.

• (3) Experimental results showed that the precision rate, recall rate, and ${F}_{1}$ score of the proposed method were as high as 98%, which indicates that the proposed method is effective for detecting rail internal defects. Moreover, for the detection of a single type of defect, the precision rate, recall rate, and ${F}_{1}$ score of the method were as high as 100%. Therefore, the proposed method has high detection ability and provides an effective method for detecting internal rail defects.

However, this method still suffers from the problem of missed detection of defects. A possible reason is that the defect features learned by the model for small samples do not meet all defect detection requirements, which results in inadequate engineering application. Further work will avoid the missed detection of defects and improve the defect-detection ability of the proposed method.