Novel Hybrid Physics-Informed Deep Neural Network for Dynamic Load Prediction of Electric Cable Shovel

Abstract

Electric cable shovel (ECS) is a complex production equipment, which is widely utilized in open-pit mines. Rational valuations of load is the foundation for the development of intelligent or unmanned ECS, since it directly influences the planning of digging trajectories and energy consumption. Load prediction of ECS mainly consists of two types of methods: physics-based modeling and data-driven methods. The former approach is based on known physical laws, usually, it is necessarily approximations of reality due to incomplete knowledge of certain processes, which introduces bias. The latter captures features/patterns from data in an end-to-end manner without dwelling on domain expertise but requires a large amount of accurately labeled data to achieve generalization, which introduces variance. In addition, some parts of load are non-observable and latent, which cannot be measured from actual system sensing, so they can’t be predicted by data-driven methods. Herein, an innovative hybrid physics-informed deep neural network (HPINN) architecture, which combines physics-based models and data-driven methods to predict dynamic load of ECS, is presented. In the proposed framework, some parts of the theoretical model are incorporated, while capturing the difficult-to-model part by training a highly expressive approximator with data. Prior physics knowledge, such as Lagrangian mechanics and the conservation of energy, is considered extra constraints, and embedded in the overall loss function to enforce model training in a feasible solution space. The satisfactory performance of the proposed framework is verified through both synthetic and actual measurement dataset.

1 Introduction

Electric cable shovel (ECS) is one of the most essential production equipment, which is widely applied in surface-mining operations to peel off the surface cover and load ore materials [1]. Currently, the increasing requirements of high production efficiency and low operating costs have boosted the intelligent or unmanned operation of ECS [2, 3]. Complex construction machinery automation involves a major challenge, that is, the accurate measurement and estimation of the load, which is a crucial precondition for the development of intelligent or unmanned ECS, since it directly influences the planning of digging trajectories and energy consumption.

The load acting on the ECS generally refers to the resistive force between the dipper and surrounding media and digging forces, which includes the crowd force and hoist force provided by the crowd and hoist motor, respectively. For modeling of the resistive force due to media-dipper interactions, there is a considerable number of methods have been developed [4]. Reece [5] proposed a universal earthmoving equation for the cutting model. Based on Reece’s model, McKyes et al. [6] added the inertia. The aforementioned methods are mainly based on the analysis of mining principles and theoretically deduced, have the advantage of universality in calculation, and provide insights for resistive force prediction. However, the resistive force due to media-dipper interactions not only depends on the physical mechanical properties of the excavated medium and the geometrical characteristics of the dipper but also is related to other effects, such as digging strategies. Therefore, some parts of the abovementioned resistive force model are difficult to model with a constant analytical expression. For example, the normal cutting force, which is perpendicular to the speed of dipper teeth, is difficult to express analytically due to compression because it depends on both the digging operation and the medium’s hardness [7].

Digging forces as driving forces are mainly employed to overcome two components of external effects, namely, the dynamic part of ECS and resistive force between the dipper and surrounding media. In recent decades, researchers have focused on using the estimated resistive force to predict digging forces by establishing kinetics or dynamic models of the digging process [8, 9]. For example, Stavropoulou et al. [10] calculated digging forces in excavation based on mechanical equilibrium. Wang et al. [11] adopted an analytic model to calculate digging forces in the mining process based on an empirical resistance equation and the Lagrange method. To realize the accurate prediction of digging forces, it is necessary to accurately establish the kinetics or dynamic models of the digging process under the premise of accurately simulating the resistive force due to media-dipper interaction. In terms of establishing the kinetics and dynamic models of the digging process, Li et al. [12], Rasuli et al. [13], and Frimpong et al. [14] developed dynamic models of the ECS to describe the evolution of the excavator motion with time using the Newton-Euler formulation or iterative Newton-Euler method. The formulation of the dynamic models presented above depends on precise knowledge of the various kinematic and dynamic parameters. When modeling the ECS system’s dynamic parts, such methods make many idealized assumptions about the equipment, such as its kinematic structure, inertia properties, and assumptions regarding the forces acting on the system, which may not capture hard-to-model effects, leading to inaccuracies via model bias.

Based on this analysis, the classical mechanistic method of predicting load of ECS can be used to determine only certain parts and aspects, such as the average value and approximate range, which is insufficiently detailed for the optimal digging trajectory planning of an unmanned ECS. Recently, with the development of sensing and measurement technology, operation data generated in practical digging processes can be effectively recorded. Furthermore, artificial neural networks (ANNs) have been proven to be a powerful tool and approximator to capture potential nonlinear coupling between input and states of complex systems [15]. These results provide many opportunities for the practical application of data-driven methods to improve the accuracy of dynamic load prediction and identification [16]. Data-driven methods typically reduce the problem of predicting load to that of optimizing the parameters of an expressive function class by minimize some form of a prediction loss in an end-to-end way [17]. However, there are many circumstances where data-driven approaches can reach their limits or lead to unsatisfactory results, due to model variance [18]. Additionally, some parts of load, such as the resistive force due to media-dipper interactions, are non-observable and latent which cannot be measured from actual system sensing, so they can’t be directly predicted by data-driven methods. A large number of studies have shown that neural networks that are endowed with good physical priors, namely, physics-informed neural networks (PINNs), can constrain and boost learning within a feasible solution space [19, 20]. PINNs require substantially less training data than other models and can produce simpler neural network structures while achieving high accuracy. Lutter et al. [21]incorporated the physical prior originating from Lagrange mechanics into the model architecture of simulated and real robot systems. Jia et al. [22] and Muralidhar et al. [23] incorporated monotonic physical relationships as additional constraints in loss functions for a neural network that models lake temperature. Zhang et al. [24] embedded the physics constraints in the loss function to capture latent nonlinear state variables, where measurement is unavailable.

Herein, we follow this line of research and develop a hybrid physics-informed deep neural network (HPINN) framework that combines first-principles models and data-driven methods, to improve the performance of load predictor. More concretely, for modeling of the resistive force due to media-dipper interactions, it delicately incorporates some parts of the theoretical model of the resistive force, while capturing the difficult-to-model parts by training a highly expressive approximator, such as an LSTM neural network from data. In terms of modeling the digging forces, it establishes the kinetics and dynamic models of the digging process, and a novel extended deep Lagrangian networks (DeLaN) is applied to model the system’s dynamic part, including the inertial effect, quality matrix, and gravity. The proposed framework presents salient features that include ① the combination of the flexibility of deep learning with theoretical insights, ② distinct interpretability with physical meaning, and ③ some variables with specific physical meaning that cannot be directly measured can be accurately modeled.

The remainder of this paper is organized as follows: Section 2 describes the mathematical definition of load modeling, theoretical background for the resistant force model, foundations of Lagrangian dynamics, and some preliminaries on LSTM. Section 3 introduces the proposed HPINN architectures for dynamic load prediction. In Section 4, a synthetic dataset is used to test the performance of the proposed framework. In Section 5, the effectiveness of the proposed framework is validated with an actual measurement dataset from an ECS prototype and compared with the classical mechanistic method and pure data-driven method. Section 6 summarizes the conclusions.

2 Background

The framework for dynamic load prediction proposed in this paper draws inspiration from various fields, including the resistive force model, classical mechanics, and advanced machine learning methods, such as PINNs.

2.1 Problem Definition

As a complex multidisciplinary system, an ECS consists of three major assemblies, namely, the upper assembly, lower assembly, and front-end attachments, as illustrated in Figure 1. Among them, the attachments, consisting of the boom, hoist ropes, crowd machinery, dipper handle, and dipper, etc., are the main operating mechanism that directly contacts the media to complete the digging task.

Figure 1

For a WK-55-type electric shovel, the capacity of the dipper is 55 m³

While digging, two types of motions are simultaneously performed by the dipper handle: extension/retraction motion in the direction parallel to the major axes of dipper handle and circular motion around the axis of crowd gear. Therefore, the digging mechanism of ECS can be considered as a two degree of freedom (DOF) mechanism. Based on the motion characteristics, a polar coordinate system is established to describe the dynamic system, as illustrated in Figure 2, where the axis of the crowd gear, O, is set to be the origin; the stretching length of dipper handle is set to be the polar diameter (r), and the angle between vertical direction and axes of dipper handle is set to be the independent variable (ψ).

Figure 2

Polar coordinate system built in the dynamic analysis

Generally, three main factors affect the dynamic load of ECS in practice: trajectory parameters (e.g., velocity and acceleration), ore pile parameters (e.g., material mechanical properties and terrain of the ore pile), and structural parameters (e.g., dipper width and boom length). Since the structural parameters are static and remain relatively unchanged during the digging process, the mapping relationship among the digging trajectory, ore pile parameters, and corresponding dynamic load is considered in this paper.

Based on sensor measurements, the dipper motion information (consisting of the angle between the vertical direction and the axis of dipper handle $\psi$, stretching length of dipper handle $r$, the angular velocity of dipper handle $\dot{\psi }$, stretching velocity of dipper handle $\dot{r}$, the angular acceleration of dipper handle $\ddot{\psi }$, the stretching acceleration of dipper handle $\ddot{r}$), and the corresponding digging forces (consisting of the hoist force $F_{r}$ and crowd force $F_{h}$) can be synchronously acquired at every moment. A 3D scanner is used to measure the profile of the ore pile being excavated, and the obtained laser scanning data can be applied to establish an accurate geometric model, which is capable of describing the dynamic shape of the ore pile, as shown in Figure 3. The digging is a dynamic process, and based on the obtained 3D model, the digging depth d corresponding to the digging trajectory can be obtained. In the digging process, the medium loaded into the dipper continues to accumulate. Therefore, the mass, moment of inertia, and centroid of the dipper-material system are constantly changing, and the required digging force depends not only on the current working conditions but also on the historical digging trajectory. Essentially, dynamic load prediction can be summarized as a time series modeling task. For simplicity, let ${\varvec{x}} = [\psi ,r,\dot{\psi },\dot{r},\ddot{\psi },\ddot{r},d]$ and ${\varvec{y}}{\mathbf{ = }}[F_{r} ,F_{h} ,F_{t} ,F_{n} ]$. The goal of load prediction is to find a suitable function $f$ that can efficiently and accurately map from input ${\varvec{x}}$ to digging forces ${\varvec{y}}$, e.g.,

$$ ({\varvec{x}}_{0} ,{\varvec{x}}_{{1}} , \cdots ,{\varvec{x}}_{t - 1} ,{\varvec{x}}_{t} )\to ^{{\varvec{f}}} {\varvec{y}}_{t} . $$

(1)

Figure 3

Constructing the ore pile surface

In addition, the non-observable variable (e.g., the resistive force due to media-dipper interactions) which is not measured in data, and cannot be directly modeled and solved. We can introduce available prior physics knowledge (e.g., Lagrangian mechanics, the conservation of energy) as extra constraints into the function $f$ for indirectly solving the latent variable.

2.2 Digging Resistance Model

During the digging process, the dipper interacts with the excavated material, and the excavated material produces a great resistance to the dipper originating from the presence of surcharge, cohesion, etc. As mentioned, empirical equations of modeling the resistance of complex digging process may not be reliable, but give insights on digging resistant force prediction. Figure 4 shows the various forces components due to the interaction of media tools that the dipper needs to overcome during the digging process. $F_{t}$ is the tangential resistance parallel to the direction of tip motion; $F_{n}$ is the normal resistance normal to the direction of tip motion, and $G$ is the gravity of the medium loaded in the dipper.

Figure 4

Resistive force analysis of the digging process

Herein, the dynamic prediction model of the digging resistant force based on the method of trial wedges proposed by McKyes et al. [6] is selected to predict the tangential resistance. With this method, the tangential resistance can be divided into three parts, including the cutting resistance tangential components $F_{c}$, the velocity effect resistance $F_{v}$, and the resistance caused by the extrusion from the two sides of the dipper $F_{s}$.

$$ F_{t} = F_{c} + F_{v} + F_{s} , $$

(2)

where $F_{c}$ can be obtained as Eq. [6].

$$ \left\{ \begin{gathered} F_{c} = \omega \left( {\gamma gd^{2} N_{\gamma } + cdN_{c} + \gamma v^{2} dN_{a} } \right), \hfill \\ N_{\gamma } = 0.5\left( {\cot \beta + \cot \rho } \right)/E_{N} , \hfill \\ N_{c} = \left[ {1 + \cot \rho \cot \left( {\rho + \varphi } \right)} \right]/E_{N} , \hfill \\ N_{a} = \left[ {\tan \rho + \cot \left( {\rho + \varphi } \right)} \right]/\left[ {1 + \tan \rho \cot \beta /E_{N} } \right], \hfill \\ E_{N} = \cos \left( {\beta + \delta } \right) + \sin \left( {\beta + \delta } \right)\cot \left( {\rho + \varphi } \right), \hfill \\ \rho = \left( {\uppi - \varphi } \right), \hfill \\ \end{gathered} \right. $$

(3)

where ω is the dipper width, γ is medium specific mass, d is the digging depth, c is the medium cohesion, v is the speed of dipper teeth, β is the digging angle, ρ is the failure plane angle, ψ denotes the internal friction angle of the medium, and δ denotes the external friction angle.$F_{v}$ and $F_{s}$ can be obtained through Eqs. (4) and (5).

$$ F_{v} = \frac{{\omega d\nu^{2} \gamma [\tan \rho \sin (\rho + \varphi ) + \cos (\rho + \varphi )]}}{\sin (\beta + \delta + \rho + \varphi )(1 + \tan \rho \cot \beta )}, $$

(4)

$$ F_{s} = \frac{{2d^{3} \gamma (\cot \beta + \cot \rho )\sin (\beta + \delta )\sqrt {\cot^{2} \rho + \cot \beta \cot \rho } }}{3\omega \sin (\beta + \rho + \varphi + \delta )}. $$

(5)

When the dipper cuts through the medium, the bottom of the tip compresses the medium, thus, the normal resistance due to the extrusion reaction arises whose orientation is perpendicular to the speed of dipper teeth. And it’s difficult to obtain an analytical expression of the normal resistance $F_{n}$ because it depends on both digging operations and the medium’s hardness. Usually, the value is obtained by multiplying the tangential cutting resistance by a factor obtained from experience [7]. However, in this article, we use a neural network to represent the normal resistance:

$$ F_{n} = f_{\theta } \left( {{\varvec{x}}_{{\varvec{0}}} ,{\varvec{x}}_{{\varvec{1}}} , \cdots ,{\varvec{x}}_{{\user2{t - 1}}} ,{\varvec{x}}_{{\varvec{t}}} } \right). $$

(6)

2.3 Deep Lagrangian Networks (DeLaN)

The purpose of establishing dynamic models of the digging process is to predict how the state of the system evolves over time by a vector of generalized variables ${\varvec{q}} \in {\mathbb{R}}^{N}$ and velocities $\dot{\user2{q}} \in {\mathbb{R}}^{N}$, where N is the number of coordinates. DeLaN uses knowledge originating from the Euler-Lagrange equation and encodes this prior within a flexible deep learning architecture [21]. Based on this architecture, all learned models adhere to Lagrangian mechanics. Specifically, the Lagrangian of a rigid body is generally defined as

$$ L = T - V, $$

(7)

where $T = {1 \mathord{\left/ {\vphantom {1 2}} \right. \kern-\nulldelimiterspace} 2}\dot{\user2{q}}\user2{H}({\varvec{q}})\dot{\user2{q}}$ is the kinetic energy, V is the potential energy, which can be defined as a scalar function V(q), and H is the positive definite mass matrix. Substituting L into the Euler-Lagrange differential equation yields the second order ordinary differential equation (ODE) described by:

$$ {\varvec{H}}{(}{\varvec{q}}{)}\user2{\ddot{q}} + \underbrace {{\dot{\user2{H}}{(}{\varvec{q}}{)}\dot{\user2{q}} - \frac{1}{2}\left( {\frac{\partial }{{\partial {\varvec{q}}}}\left( {\dot{\user2{q}}^{{\text{T}}} {\varvec{H}}{(}{\varvec{q}}{)}\dot{\user2{q}}} \right)} \right)^{{\text{T}}} }}_{{: = {\varvec{c}}({\varvec{q}}{,}\dot{\user2{q}})}} - \frac{\partial V}{{\partial {\varvec{q}}}} = {\varvec{\tau}}, $$

(8)

where τ represents the nonconservative generalized forces, for ECS system, mainly refers to motors force effects and the resistive force due to media-dipper interaction, c describes Coriolis and centripetal effects, and ${{ - \partial V} \mathord{\left/ {\vphantom {{ - \partial V} {\partial {\varvec{q}}}}} \right. \kern-\nulldelimiterspace} {\partial {\varvec{q}}}}$ is gravity [25]. In DeLaN, the unknown functions ${\varvec{H}}({\varvec{q}})$ and $V({\varvec{q}})$ are represented as a feed-forward network, i.e.,

$$\left\{ \begin{gathered} \user2{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{H} }{ = }\user2{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{L} }{(}{\varvec{q}}{;}\theta_{1} {)}\user2{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{L} }^{{\text{T}}} {(}{\varvec{q}}{;}\theta_{1} {), } \hfill \\ \overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{V} = {{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{V} (}}{\varvec{q}}{;}\theta_{2} {),} \hfill \\ \end{gathered} \right.$$

(9)

where $\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{.}$ refers to an approximation, $\user2{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{L} }$ is a lower triangular matrix with a nonnegative diagonal, and θ₁ and θ₂ are the network parameters, and one can encode the ODE by exploiting the full differentiability of the neural networks. The parameters θ₁ and θ₂ can be learned online and end-to-end by minimizing the violation of the physical law described by the ODE. The basic architecture of the DeLaN [26] can be found in Figure 5.

Figure 5

Structure of DeLaN

2.4 Long-Short-Term Memory Networks (LSTM)

LSTMs have achieved state-of-the-art performance in a range of different domains comprising sequential data, such as natural language processing (NLP) [27], load prediction [28], and remaining useful life (RUL) estimation [29]. In Figure 6, we show a typical structure of an LSTM's hidden nodes incorporating four interacting units, including an internal cell, an input gate, a forget gate, and an output gate [30]. The internal cell memorizes the cell state at the previous time step via a self-recurrent connection. The input gate controls the flow of input activation into the internal cell state. The output gate regulates the flow of output activation into the LSTM cell output. The forget gate scales the internal cell state, enabling the LSTM cell to adaptively forget or reset the cell’s memory [24]. Specifically, given the previous hidden output $h_{{t{ - 1}}}$, cell state memory $C_{t - 1}$, and current input $x_{t}$, the current hidden output $h_{t}$ can be computed in the following way:

$$ \left\{ \begin{gathered} {\varvec{i}}_{t} = \sigma \left( {{\varvec{U}}^{[i]} {\varvec{x}}_{t} + {\varvec{W}}^{[i]} {\varvec{h}}_{t - 1} } \right), \hfill \\ {\varvec{f}}_{t} = \sigma \left( {{\varvec{U}}^{[f]} {\varvec{x}}_{t} + {\varvec{W}}^{[f]} {\varvec{h}}_{t - 1} } \right), \hfill \\ {\varvec{o}}_{t} = \sigma \left( {{\varvec{U}}^{[o]} {\varvec{x}}_{t} + {\varvec{W}}^{[o]} {\varvec{h}}_{t - 1} } \right), \hfill \\ \tilde{\user2{c}}_{t} = \tanh \left( {{\varvec{U}}^{[g]} {\varvec{x}}_{t} + {\varvec{W}}^{[g]} {\varvec{h}}_{t - 1} } \right), \hfill \\ {\varvec{c}}_{t} = {\varvec{c}}_{t - 1} \odot {\varvec{f}}_{t} + \tilde{\user2{c}}_{t} \odot {\varvec{i}}_{t} , \hfill \\ {\varvec{h}}_{t} = \tanh ({\varvec{c}}_{t} ) \odot {\varvec{o}}_{t} , \hfill \\ \end{gathered} \right. $$

(10)

where $\sigma$ is the logistic sigmoid function, $\odot$ represents for the Hadamard product, and ${\varvec{U}}^{[\zeta ]}$ (${\varvec{W}}^{[\zeta ]}$) denotes the weight matrix between the current input ${\varvec{x}}_{t}$(${\varvec{h}}_{t - 1}$) and the operations $\zeta (\zeta \in i,f,o,g)$.

Figure 6

Structure of LSTMs

3 Methodology

In the proposed framework, we incorporate available physics information (e.g., theoretical model of digging resistant force, Lagrangian mechanics, and energy conservation) into the PINN architecture to predict the dynamic load under different positions and postures of the ECS during digging process.

3.1 Overall Framework

In its most general form, we model the continuous-time dynamic digging process of an ECS by modeling ① the digging resistant force between the dipper and the surrounding medium, and ② the dynamic part, including the inertial effect, Coriolis and centripetal effects, and gravity. The overall framework of the HPINN mainly consists of four components, namely, an empirical resistance equation for modeling the tangential digging resistance, a deep LSTM network for modeling the normal digging resistance, and a central finite difference filter-based numerical differentiator for calculating the derivatives of the kinetic energy with respect to time, as described in Figure 7.

Figure 7

The proposed HPINN architecture

For modelling of the digging resistant force, we use an empirical resistance equation proposed by McKyes et al. [6] to model the tangential resistance, which is mainly related to the physical mechanical properties of the excavated medium, and has an analytical expression. Different from the classical mechanistic model, which calculates the normal digging resistance based on the tangential digging resistance multiplied by a proportionality coefficient, in this paper, we present a methodology for modeling the difficult-to-model normal digging resistance $F_{n}$ using an LSTM network with the trainable weights and biases contained in $\theta$. The normal digging resistance is a function of the digging trajectory, digging speed and digging depth. Therefore, the input of the LSTM network includes digging trajectory, digging speed and the digging depth corresponding to the digging profile. The digging trajectory and digging speed can be obtained directly through sensor acquisition, and the digging depth can be calculated according to the digging trajectory and the shape of ore pile. Since the digging resistance is generated by the contact between the dipper and the surrounding medium, which cannot be directly measured, in order to establish end-to-end supervised training, the output of the LSTM network needs to be projected into the generalized coordinates to participate in the calculation as part of the mechanical equation.

Akin to Lutter et al. [21], we incorporate the physical prior of Lagrangian mechanics into the framework to calculate the dynamic part of the ECS. Meanwhile, there are key distinctions between the approaches. As previously mentioned, during the digging process, the mass, rotational inertia, and centroid of the dipper-medium system are related not only to the current position but also to the digging profile that the dipper has traveled, which can be summarized as a time series modeling task. It is known that LSTM is uniquely capable of modeling long-term dependence in time series data. In order to capture the sequence-to-sequence input-output relationship, we extended the DeLaN using an LSTM neural network instead of a feed-forward network as the baseline neural network model. All parameters of the extended DeLaN are contained in $\vartheta$. The input of DeLaN is the sensor data sequence collected during the digging process, including digging trajectory, digging speed, digging acceleration and digging depth information calculated according to the digging profile and the shape of ore pile, and the output is the dynamic part of the system.

3.2 Loss Function

Having established the parameterization for the model of the complex digging process, we now discuss how to optimize these parameters from data using standard end-to-end optimization techniques. With the available training data, the “Lagrangian mechanics loss function” of the HPINN can be formulated as Eq. (11).

$$ {\varvec{J}}_{d} (\theta ,\vartheta ) = \sum\limits_{i = 1}^{N} {\sum\limits_{t = 0}^{T - 1} {\left\| {\tau_{{M_{t}^{(i)} }} - \left. {f^{ - 1} \left( \vartheta \right)_{t}^{(i)} } \right\|} \right.} + \lambda{\varvec{\varOmega}}(\theta ,\vartheta )} , $$

(11)

where N represents the number of measurement (data) samples, T is the number of sampling steps in the time series, $f^{ - 1} (\vartheta )$ denotes the inverse model of Eq. (9), which is the dynamic part of ECS, and ${\varvec{\tau}}_{{\varvec{M}}}$ denotes the external forces, including motor forces $\{ F_{r} ,F_{h} \}$ projected to the generalized coordinate and the interaction force between the dipper and the surrounding medium $\{ F_{t} ,F_{n} \}$. $\theta$ and $\vartheta$ are the trainable weights and biases of the extended deep Lagrangian network and LSTM, respectively. $\lambda$ is a suitable hyperparameter that controls the regularization strength. ${\varvec{\varOmega}}$ denotes the regularization term, and ${\varvec{\varOmega}}= \sum\nolimits_{{\omega \in \{ \theta ,\vartheta \} }} {{\varvec{\omega}}^{2} }$ , where ω is the parameter to be trained.

In addition to the Lagrangian mechanics objective, ECS needs to observe the conservation of energy that is the total energy of the system must be equal at every moment during its operation. This objective can also be expressed using the change in energy, i.e.,

$$ \dot{E} = {\varvec{\tau}}_{{\varvec{M}}} \dot{\user2{q}} = \dot{T} + \dot{V}. $$

(12)

In particular, the derivatives of kinetic energy with respect to time, namely, ${{{\text{d}}T} \mathord{\left/ {\vphantom {{{\text{d}}T} {{\text{d}}t}}} \right. \kern-\nulldelimiterspace} {{\text{d}}t}}$, respectively, cannot be computed using automatic differentiation, as t is not an input of the network. Thus, we developed a central finite difference filter-based numerical differentiator for the calculation. Adding energy conservation to the optimization problem yields “energy loss” for the HPINN.

$$ {\varvec{J}}_{e} \left( {\theta ,\vartheta } \right) = \sum\limits_{i = 1}^{N} {\sum\limits_{t = 0}^{T - 1} {\left\| {\dot{T}_{t}^{(i)} + \dot{V}_{t}^{(i)} - \left. {\tau_{{M_{t}^{(i)} }} \dot{q}_{t}^{(i)} } \right\|} \right.} } . $$

(13)

As a result, the proposed HPINN architecture can be trained by solving the following optimization problem through a standard training algorithm (e.g., gradient descent):

$$ \left\{ {\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{\theta } ,\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{\vartheta } } \right\} = \mathop {\arg }\limits_{{\left\{ {\theta ,\vartheta } \right\}}} \min {\varvec{J}}\left( {\theta ,\vartheta } \right), $$

(14)

where ${\varvec{J}}\left( {\theta ,\vartheta } \right)$ is the generic total hybrid loss function composed of both data loss and domain loss, as follows:

$$ {\varvec{J}}\left( {\theta ,\vartheta } \right) = \alpha {\varvec{J}}_{d} \left( {\theta ,\vartheta } \right) + \eta {\varvec{J}}_{e} \left( {\theta ,\vartheta } \right), $$

(15)

where, α and η are user-defined hyper-parameter determining the weight of each term in the objective function for convergence control (e.g., inversely proportional to the magnitude of each term, or for simplicity α = η= 1). The purpose here is to optimize the network parameters $\left\{ {\theta ,\vartheta } \right\}$ for both the deep LSTM networks and extended DeLaN, such that the HPINN is able to accurately fit the measurement samples while meeting the domain constraints. This process improves the capabilities of the HPINN to model potential nonlinear, sequence-to-sequence, input-output relationships within a physically feasible solution space.

4 Experimental Investigation on Synthetic Dataset

To comprehensively demonstrate the applicability and superiority of proposed framework, as well as the ability of accurately model latent variables that cannot be directly measured but have specific physical meaning, a synthetic dataset, which from the ideal dynamic equation of WK-55-type ECS, is used to test. Latent variable values cannot be obtained from field measurements, but they have explicit values when synthesizing a dataset.

4.1 Dataset Preparation

To generate a synthetic dataset for model training, we first need to describe the digging trajectory of the ECS. As described in Section 2, the hoist motor and crowd motor cooperate to drive the dipper teeth along a predetermined trajectory and complete a digging task. Here, two sixth-order polynomials are applied to describe the digging trajectory, as given in Ref. [11]. The digging trajectory is formulated as

$$ \left\{ \begin{gathered} s_{x} (t) = a_{6} t^{6} + a_{5} t^{5} + a_{{4}} t^{{4}} + a_{3} t^{3} + a_{2} t^{2} + a_{1} t + a_{0} , \hfill \\ s_{y} (t) = b_{6} t^{6} + b_{5} t^{5} + b_{{4}} t^{{4}} + b_{3} t^{3} + b_{2} t^{2} + b_{1} t + b_{0} , \hfill \\ \end{gathered} \right. $$

(16)

where $a_{0\sim 6}$ and $b_{0\sim 6}$ denote polynomial coefficients, $s_{x}$ and $s_{y}$ are the positions of the dipper tip in the horizontal and vertical directions based on the reference Cartesian coordinates.

Figure 8 shows the trajectory of a dipper tip of the excavator during the digging stage using two sixth-order polynomials. The origin of the digging cycle is at the bottom of media point B, and the end of the cycle is at point C. The ECS is located on the A-B plane.

Figure 8

The Cartesian coordinate system built to describe the digging trajectory

By solving inverse kinematics of the shovel, the motion information of the joint space in the digging process can be obtained, including the angle between the vertical direction and the axis of the dipper handle $\psi$, the stretching length of the dipper handle r, the angular velocity of the dipper handle $\dot{\psi }$, the stretching velocity of the dipper handle $\dot{r}$, the angular acceleration of the dipper handle $\ddot{\psi }$, and the stretching acceleration of the dipper handle $\ddot{r}$. The digging resistive forces were calculated based on the method of trial wedges proposed by McKyes et al. [6]. Then based on the dynamic Eq. (17), the corresponding digging forces can be calculated.

$$ \left\{ \begin{gathered} \left[ {m_{b} \left( {r^{2} - L_{b} r + \tfrac{1}{3}L_{b}^{2} } \right) + m_{d} \left( {r^{2} + L_{d} r + \tfrac{1}{3}L_{d}^{2} } \right)} \right]\ddot{\psi } + \cdots \hfill \\ \, \left[ {2\left( {m_{b} + m_{d} } \right) \cdot r - \left( {m_{b} L_{d} - m_{d} L_{d} } \right)} \right]\dot{\psi } \cdot \dot{r} + \cdots \hfill \\ \, \left[ {m_{b} g\sin \psi \left( {r - \frac{{L_{b} }}{2}} \right) + m_{d} g\sin \psi \left( {r + \frac{{L_{d} }}{2}} \right)} \right] \hfill \\ \, = F_{r} \cdot r \cdot \sin \vartheta - F_{t} \cdot \left( {L_{d} + r} \right), \hfill \\ \left( {m_{b} + m_{d} } \right)\ddot{r} - \left[ {\left( {m_{b} + m_{d} } \right) \cdot r - \tfrac{1}{2}\left( {m_{b} L_{b} - m_{d} L_{d} } \right)} \right]\dot{\psi }^{2} - \cdots \hfill \\ \, \left( {m_{b} + m_{d} } \right)g\cos \psi = F_{h} - F_{n} - F_{r} \cdot \cos \vartheta , \hfill \\ \end{gathered} \right. $$

(17)

where $L_{b}$ is the length of the dipper handle, $L_{d}$ is the length of the dipper, $F_{r}$ is the hoist force, $F_{h}$ is the crowd force, $F_{n}$ is the normal excavating resistance, $F_{t}$ is the tangential excavating resistance, ϑ is the angle between the hoist rope and dipper handle. $m_{b}$ is the mass of dipper handle, $m_{d}$ is the total mass of the dipper including the mass of the dipper itself m₀ and the mass of the loaded material $m_{t}$ ($m_{d} = m_{0} + m_{t}$). And the related parameter values of the shovel in simulation can be found in Ref. [11].

To generate the input-output pairs provided for model training and testing, the different coefficients of the polynomial in Eq. (16) are chosen to obtain different digging trajectories. The total time in digging is set to 12 s and sampling time interval is set to 0.1 s. Based on the inverse kinematics and dynamics of the shovel, 1000 time series samples can be obtained. The entire dataset is randomly divided into two parts, including the training dataset with 800 samples, and test dataset with 200 samples. In practical, due to the unevenness of trajectories in real digging, vibration and shock occur during digging, and noise interference inevitably exists in measurement data, which may have a substantial impact on performance. To further examine the noise immunity of the proposed framework, we also added random errors of ±5%, ±10%, ±15%, ±20% and ±25% in positions (ψ, r), velocity ($\dot{\psi }$, $\dot{r}$), acceleration ($\ddot{\psi }$, $\ddot{r}$) and forces ($F_{r}$, $F_{h}$) components, respectively.

4.2 Measurement Indexes

The prediction results were evaluated based on three performance metrics, i.e., R-squared (R²), mean absolute error (MAE) and root mean square error (RMSE).

$$ R^{2} = 1 - \frac{1}{N}\sum\limits_{i = 1}^{N} {\frac{{\sum\limits_{t = 0}^{T - 1} {\left( {\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{y}_{t}^{(i)} - y_{t}^{(i)} } \right)^{2} } }}{{\sum\limits_{t = 0}^{T - 1} {\left( {y_{t}^{(i)} - \overline{y}^{(i)} } \right)^{2} } }}} , $$

(18)

$$ MAE = \frac{1}{N}\sum\limits_{i = 1}^{N} {\frac{1}{T}\sum\limits_{t = 0}^{T - 1} {|\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{y}_{t}^{(i)} - y_{t}^{(i)} |} } , $$

(19)

$$ RMSE = \sqrt {\frac{1}{N}\sum\limits_{i = 1}^{N} {\frac{1}{T}\sum\limits_{t = 0}^{T - 1} {\left( {\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{y}_{t}^{(i)} - y_{t}^{(i)} } \right)^{2} } } } , $$

(20)

where N represents the number of samples, T represents the number of sampling steps in the time series, and $\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\frown}$}}{y}_{t}^{(i)}$, $y_{t}^{(i)}$ represent the predicted value and actual value, respectively, at time step t for the n-th sample, respectively. $\overline{y}^{(n)}$ is the average value of the n-th sample. The R² is closer to 1, the better the performance is. MAE is used to estimate the difference between the ground-truth value and predicted value. This function has strong robustness to large errors, and RMSE also reflects the discreteness of the model. However, compared with the MAE, RMSE is more sensitive to large errors because the error is squared, and large errors are further amplified. These three indices can be utilized to evaluate the performance of this framework from different perspectives.

4.3 Experimental Results and Discussion

The experimental data modeling was performed on a computer with an Intel Xeon Silver 4114 CPU at 2.2 GHz, 64 GB RAM and an NVIDIA Tesla P100 graphics card. The framework is coded by the authors in PyTorch and set as follows: the LSTM network in the proposed HPINN architecture has two LSTM layers and one fully connected layer. The hyperbolic tangent function (tanh) is employed as the nonlinearity, and the standard automatic differentiation toolkit is used to compute the gradients of ${\varvec{H}}({\varvec{q}},d;\theta )$ with respect to ${\varvec{q}}$. The learning rate for the proposed hybrid deep neural network is set to 0.001, and the Adam optimizer is selected as the optimizer.

The prediction results of digging forces under different noise levels are shown in Tables 1 and 2. It can be seen that the performance of HPINN decreases as the noise level increases, but still has high accuracy. For intuitive comparisons, the hoist force and crowd force predicted by the HPINN and real values are also provided in Figures 9 and 10. It can be seen from Figures 9 and 10 that digging forces can be accurately predicted, which indicates that the proposed framework is still able to provide competitive prediction results even if the level of noise is relatively high at 25%.

Table 1 Prediction performance of hoist forces at different noise levels

Table 2 Prediction performance of crowd forces at different noise levels

Figure 9

Prediction results of hoist force under different noise levels

Figure 10

Prediction results of crowd force under different noise levels

The HPINN embedded available physics information in model, which has the advantage of clear interpretability with physics meaning. Therefore, it can not only accurately fit the visible variables, but also accurately predict hidden variables. The prediction results of the normal digging resistance under different noise levels are shown in Table 3 and Figure 11. It can be observed that the HPINN is able to provide reasonable predictions under different noise levels, which implies that the proposed framework can successful capture the latent nonlinear expressions of normal digging resistance.

Table 3 Prediction performance of the normal digging resistance at different noise levels

Figure 11

Prediction results of normal digging resistance under different noise levels

5 Experimental Investigation on Real-world Dataset

For the aim of testing and evaluation, we conducted digging experiments and used sensors to record operating data. The performance of the HPINN is compared to that of the classical mechanistic model and a purely data-driven method, that are most common dynamic load prediction methods, to verify its effectiveness and superiority.

5.1 Experimental Setup

As shown in Figure 12, the experimental equipment used here is an intelligent ECS prototype, that is a 1:7 scale model of the WK-55-type, and the capacity of the dipper is about 0.16 m³. The main geometry and physical data of the digging device are listed in Table 4, and the corresponding explanatory diagram is illustrated in Figure 13. The hardware for synchronous data acquisitions mainly consists of an industrial computer, a LiDAR for obtaining information about the material to be excavated, two crowd/hoist motor encoders for obtaining position information. And digging forces is uploaded through the drive inverter. Experiments were carried out on a test site constructed from real mine materials, and the whole digging operations are shown in Figure 14.

Figure 12

Experimental equipment: ECS prototype of the 1/7 scale model

Table 4 Structural parameters of the ECS prototype working device

Figure 13

Explanatory diagram of the ECS prototype working device

Figure 14

The whole digging operations

To obtain sufficient training data, 500 complex pile digging experiments are carried out, and the number of training samples and test samples are set to 400 and 100, respectively. To verify the effective of the HPINN, a pure data-driven method without including physics is constructed, which have two LSTM layers and one fully connected layer, and the Adam optimizer and mean-square error (MSE) are selected as the optimizer and data loss function, respectively.

5.2 Experimental Results and Discussion

This section presents experimental results of the proposed HPINN for dynamic load prediction on a real-world dataset. The prediction results were evaluated with performance metrics used in Section 3, and the digging forces prediction results are shown in Table 5. The hoist and crowd forces predicted by the HPINN, classical analytical method and pure data driven method are also provided in Figure 15 for intuitive comparisons. According to Table 5, the HPINN outperforms the classical analytical method and pure data-driven method on the real-world datasets, i.e., the RMSE and MAE are smaller and the coefficient of determination, R², is higher. More concretely, the RMSE of the proposed framework is 31.8 %, 21.7% less than that of the classical analytical method and pure data-driven method, respectively. The MAE of the proposed framework is 27.9%, 13.2% less than that of the classical analytical method and pure data-driven method, respectively. Moreover, the R² of the proposed method is 8.6%, 2.3% higher than that of the classical analytical method and pure data-driven method, respectively. These results verify the applicability and superiority of the proposed framework in dynamic digging force prediction.

Table 5 Performance comparison of digging forces on real-world measurement data

Figure 15

The digging forces predicted by three models on real-world data

As shown in Figure 15, compared with the classical analytical method, the predicted curves of HPINN and pure data driven method are more consistent with the ground-truth curve. And the classical analytical method can predict the changing trend of crowd force and hoist force to some extent but cannot make accurate predictions. The main reason lies in that classical analytical approaches are mainly derived from physics and make many idealized assumptions about the system and excavated material, such as its kinematic structure, inertia properties, assumptions regarding the forces acting on the physical system, and the assumption that the excavated material is uniform, continuous and isotropic, which may fail to capture hard-to-model effects, causing inaccuracies via model bias. Moreover, the classical method assumes or simplifies the dynamic digging process as a steady process and does not consider the influence of time-varying dipper penetration and digging speed. Therefore, this approach can predict only a range of crowd force and hoist force and cannot sensitively capture small changes.

Both pure data-driven method and HPINN are able to produce reasonable predictions of digging forces, and the HPINN slightly yields better prediction on whole tendency, which implies that the proposed framework provides reasonable nonlinear expressions and can capture latent correlations for the complicated time-dependent data. In many cases, purely data-based methods, in particular deep learning models, which ground in the nature of data-based and captures rules present in the training data, may reach their limits and produce inaccurate prediction results, due to model variance. Additional, pure data-driven method works in an end-to-end manner and cannot model non-observable variables. The HPINN combines the flexibility of deep learning with the theoretical insights of physics and retains certain physical interpretability, which explores a deeper coupling of ML methods with scientific knowledge. HPINN only restricts the system to comply with general laws of physics, such as energy conservation and force balance, and does not impose too many constraints on the system, so that the model maintains greater flexibility. Moreover, the change of some underlying variables with specific physical meanings can be obtained by using the framework proposed in this paper, such as resistive force, potential, and kinetic energies, which are not observable and hence, cannot be learned supervised. In this paper, the resistive force between the dipper and surrounding media is analyzed, and compared with traditional classical analytical method. The tangential component and normal component of the resistance are illustrated in Figure 16.

Figure 16

The resistive forces predicted by analytical method and HPINN

During real digging, it is difficult for the bucket to enter the material at the ideal cutting angle, and the material will inevitably be squeezed, so the normal resistance is related to both the digging operation and the medium’s hardness, and has a relatively complex expression. Therefore, the normal resistance cannot be obtained simply by multiplying the tangential resistance by the coefficient.

6 Conclusions

• (1) This paper presents a hybrid physics-informed deep neural networks framework, named the HPINN, which combines first-principles method and data-driven modeling, to predict dynamic load of the ECS.

• (2) In the proposed framework, some parts of the theoretical model of the resistive force are incorporated, while capturing the difficult-to-model part by training a highly expressive LSTM neural network with data.

• (3) The HPINN can not only accurately fit the visible variables, but also accurately predict hidden variables.

• (4) The HPINN combines the flexibility of deep learning with the theoretical insights of physics and retains certain physical interpretability, achieving a deep coupling of ML methods with scientific knowledge.

• (5) Both synthetic data and actual measurement dataset are used to validate the proposed framework, which can also provide a competitive prediction performance with different noise levels.