ReLU

class ReLU(maxValue: Float?, negativeSlope: Float, threshold: Float, name: String) : AbstractActivationLayer

Rectified Linear Unit activation function.

With default values, it returns element-wise max(x, 0).

Otherwise, it follows:

f(x) = maxValue,                        if x >= maxValue
f(x) = x,                               if threshold <= x < maxValue
f(x) = negativeSlope * (x - threshold), if x < threshold

Since

0.2

Constructors

ReLU

fun ReLU(maxValue: Float? = null, negativeSlope: Float = 0.0f, threshold: Float = 0.0f, name: String = "")

Creates ReLU object.

Functions

build

open override fun build(tf: Ops, kGraph: KGraph, inputShape: Shape)

Extend this function to define variables in layer.

buildFromInboundLayers

fun buildFromInboundLayers(tf: Ops, kGraph: KGraph)

Extend this function to define variables in layer.

computeOutputShape

open override fun computeOutputShape(inputShape: Shape): Shape

Computes output shape, based on inputShape and Layer type.

computeOutputShapeFromInboundLayers

open fun computeOutputShapeFromInboundLayers(): TensorShape

Computes output shape, based on input shapes of inbound layers.

forward

open override fun forward(tf: Ops, input: Operand<Float>): Operand<Float>

Applies the activation functions to the input to produce the output.

open override fun forward(tf: Ops, input: Operand<Float>, isTraining: Operand<Boolean>, numberOfLosses: Operand<Float>?): Operand<Float>

Builds main layer input transformation with tf. Depends on Layer type.

open fun forward(tf: Ops, input: List<Operand<Float>>, isTraining: Operand<Boolean>, numberOfLosses: Operand<Float>?): Operand<Float>

Builds main layer input transformation with tf. Depends on Layer type.

invoke

operator fun invoke(vararg layers: Layer): Layer

Important part of functional API. It takes layers as input and saves them to the inboundLayers of the given layer.

toString

open override fun toString(): String

Properties

hasActivation

open override val hasActivation: Boolean

Returns True, if layer has internal activation function.

inboundLayers

var inboundLayers: MutableList<Layer>

Returns inbound layers.

isTrainable

var isTrainable: Boolean = true

True, if layer's weights could be changed during training. If false, layer's weights are frozen and could be changed during the training.

maxValue

val maxValue: Float? = null

Maximum activation value. Should be >= 0.

name

var name: String

negativeSlope

val negativeSlope: Float = 0.0f

Negative slope coefficient. Should be >= 0.

outboundLayers

var outboundLayers: MutableList<Layer>

Returns outbound layers.

outputShape

lateinit var outputShape: TensorShape

Output data tensor shape.

paramCount

open override val paramCount: Int

Returns amount of neurons.

parentModel

var parentModel: TrainableModel? = null

Model where this layer is used.

threshold

val threshold: Float = 0.0f

Threshold value for threshold activation.

weights

open override var weights: Map<String, Array<*>>

Layer's weights.