ELU

class ELU(alpha: Float, name: String) : AbstractActivationLayer

Exponential Unit activation function.

It follows:

f(x) = x,                    if x 0
f(x) = alpha * (exp(x) - 1), if x <= 0

In contrast to ReLU it has negative values which push the mean of the activation closer to zero which enable faster learning as they bring the gradient to the natural gradient.

Since

0.3

Constructors

ELU

fun ELU(alpha: Float = 1.0f, name: String = "")

Creates ELU object.

Functions

build

open override fun build(tf: Ops, kGraph: KGraph, inputShape: Shape)

Extend this function to define variables in layer.

buildFromInboundLayers

fun buildFromInboundLayers(tf: Ops, kGraph: KGraph)

Extend this function to define variables in layer.

computeOutputShape

open override fun computeOutputShape(inputShape: Shape): Shape

Computes output shape, based on inputShape and Layer type.

computeOutputShapeFromInboundLayers

open fun computeOutputShapeFromInboundLayers(): TensorShape

Computes output shape, based on input shapes of inbound layers.

forward

open override fun forward(tf: Ops, input: Operand<Float>): Operand<Float>

Applies the activation functions to the input to produce the output.

open override fun forward(tf: Ops, input: Operand<Float>, isTraining: Operand<Boolean>, numberOfLosses: Operand<Float>?): Operand<Float>

Builds main layer input transformation with tf. Depends on Layer type.

open fun forward(tf: Ops, input: List<Operand<Float>>, isTraining: Operand<Boolean>, numberOfLosses: Operand<Float>?): Operand<Float>

Builds main layer input transformation with tf. Depends on Layer type.

invoke

operator fun invoke(vararg layers: Layer): Layer

Important part of functional API. It takes layers as input and saves them to the inboundLayers of the given layer.

toString

open override fun toString(): String

Properties

alpha

val alpha: Float = 1.0f

Hyperparameter that controls the value to which an ELU saturates for negative net inputs. Should be 0.

hasActivation

open override val hasActivation: Boolean

Returns True, if layer has internal activation function.

inboundLayers

var inboundLayers: MutableList<Layer>

Returns inbound layers.

isTrainable

var isTrainable: Boolean = true

True, if layer's weights could be changed during training. If false, layer's weights are frozen and could be changed during the training.

name

var name: String

outboundLayers

var outboundLayers: MutableList<Layer>

Returns outbound layers.

outputShape

lateinit var outputShape: TensorShape

Output data tensor shape.

paramCount

open override val paramCount: Int

Returns amount of neurons.

parentModel

var parentModel: TrainableModel? = null

Model where this layer is used.

weights

open override var weights: Map<String, Array<*>>

Layer's weights.