Mish
Mish()
Content copied to clipboard
Mish activation function.
Transforms input 'x' according formula:
mish(x) = x * tanh(softplus(x))
It is a smooth, non-monotonic function that consistently matches or outperforms ReLU and Swish on deep networks, it is unbounded above and bounded below. It also smoothens the loss landscape of the network.
Calls MishActivation under the hood.