ELU

class ELU(alpha: Float, name: String) : AbstractActivationLayer

Exponential Unit activation function.

It follows:

f(x) = x,                    if x 0
f(x) = alpha * (exp(x) - 1), if x <= 0

In contrast to ReLU it has negative values which push the mean of the activation closer to zero which enable faster learning as they bring the gradient to the natural gradient.

Since

0.3

Constructors

ELU

fun ELU(alpha: Float = 1.0f, name: String = "")

Creates ELU object.

Functions

build

open override fun build(tf: Ops, input: Operand<Float>, isTraining: Operand<Boolean>, numberOfLosses: Operand<Float>?): Operand<Float>

open fun build(tf: Ops, input: List<Operand<Float>>, isTraining: Operand<Boolean>, numberOfLosses: Operand<Float>?): Operand<Float>

Extend this function to define variables in the layer and compute layer output.

forward

open override fun forward(tf: Ops, input: Operand<Float>): Operand<Float>

Applies the activation functions to the input to produce the output.

invoke

operator fun invoke(vararg layers: Layer): Layer

Important part of functional API. It takes layers as input and saves them to the inboundLayers of the given layer.

toString

open override fun toString(): String

Properties

alpha

val alpha: Float = 1.0f

Hyperparameter that controls the value to which an ELU saturates for negative net inputs. Should be 0.

hasActivation

open override val hasActivation: Boolean

Returns True, if layer has internal activation function.

inboundLayers

var inboundLayers: MutableList<Layer>

Returns inbound layers.

name

var name: String

Layer name. A new name is generated during model compilation when provided name is empty.

outboundLayers

var outboundLayers: MutableList<Layer>

Returns outbound layers.

outputShape

lateinit var outputShape: TensorShape

Output data tensor shape.

parentModel

var parentModel: GraphTrainableModel? = null

Model where this layer is used.