Algorithms | Optimal activation functions |
---|---|
RNN_1LAYER | Leaky_ReLU |
RNN_2LAYERS | Leaky_ReLU, ELU |
RNN_3LAYERS | ELU |
LSTM_1LAYER | ReLU |
LSTM_2LAYERS | PReLU |
LSTM_3LAYERS | ELU |
Bi_LSTM_1LAYER | PReLU |
Bi_LSTM_2LAYERS | ReLU, Leaky_ReLU |
Bi_LSTM_3LAYERS | PReLU |
GRU_1LAYER | ReLU |
GRU_2LAYERS | ReLU |
GRU_3LAYERS | Swish |