Skip to main content

Table 4 Optimal activation functions for different algorithms

From: Practical Options for Adopting Recurrent Neural Network and Its Variants on Remaining Useful Life Prediction

Algorithms Optimal activation functions
RNN_1LAYER Leaky_ReLU
RNN_2LAYERS Leaky_ReLU, ELU
RNN_3LAYERS ELU
LSTM_1LAYER ReLU
LSTM_2LAYERS PReLU
LSTM_3LAYERS ELU
Bi_LSTM_1LAYER PReLU
Bi_LSTM_2LAYERS ReLU, Leaky_ReLU
Bi_LSTM_3LAYERS PReLU
GRU_1LAYER ReLU
GRU_2LAYERS ReLU
GRU_3LAYERS Swish