kotlin-spark-api_3.3.2_2.13/org.jetbrains.kotlinx.spark.api/udaf

udaf

inline fun <IN, OUT, AGG : Aggregator<IN, *, OUT>> udaf(agg: AGG, nondeterministic: Boolean = false): NamedUserDefinedFunction1<IN, OUT>
inline fun <IN, OUT, AGG : Aggregator<IN, *, OUT>> udaf(    name: String,     agg: AGG,     nondeterministic: Boolean = false): NamedUserDefinedFunction1<IN, OUT>

Obtains a NamedUserDefinedFunction1 that wraps the given agg so that it may be used with Data Frames.

Return

a NamedUserDefinedFunction1 that can be used as an aggregating expression

Parameters

name

Optional. Tries to obtain name from the class of agg if not supplied. Use udafUnnamed if no name is wanted.

agg

the given Aggregator to convert into a UDAF. Can also be created using aggregatorOf.

nondeterministic

Optional. If true, sets the UserDefinedFunction as nondeterministic.

See also

functions.udaf

for a named variant.

inline fun <IN, BUF, OUT> udaf(    noinline zero: () -> BUF,     noinline reduce: (b: BUF, a: IN) -> BUF,     noinline merge: (b1: BUF, b2: BUF) -> BUF,     noinline finish: (reduction: BUF) -> OUT,     bufferEncoder: Encoder<BUF> = encoder(),     outputEncoder: Encoder<OUT> = encoder(),     nondeterministic: Boolean = false): UserDefinedFunction1<IN, OUT>

Obtains a UserDefinedFunction1 created from an Aggregator created by the given arguments so that it may be used with Data Frames.

Return

a UserDefinedFunction1 that can be used as an aggregating expression

Parameters

zero

A zero value for this aggregation. Should satisfy the property that any b + zero = b.

reduce

Combine two values to produce a new value. For performance, the function may modify b and return it instead of constructing new object for b.

merge

Merge two intermediate values.

finish

Transform the output of the reduction.

bufferEncoder

Optional. Specifies the Encoder for the intermediate value type.

outputEncoder

Optional. Specifies the Encoder for the final output value type.

nondeterministic

Optional. If true, sets the UserDefinedFunction as nondeterministic.

See also

functions.udaf

for a named variant.

inline fun <IN, BUF, OUT> udaf(    name: String,     noinline zero: () -> BUF,     noinline reduce: (b: BUF, a: IN) -> BUF,     noinline merge: (b1: BUF, b2: BUF) -> BUF,     noinline finish: (reduction: BUF) -> OUT,     bufferEncoder: Encoder<BUF> = encoder(),     outputEncoder: Encoder<OUT> = encoder(),     nondeterministic: Boolean = false): NamedUserDefinedFunction1<IN, OUT>

Obtains a NamedUserDefinedFunction1 that wraps the given agg so that it may be used with Data Frames. so that it may be used with Data Frames.

Return

a UserDefinedFunction1 that can be used as an aggregating expression

Parameters

name

Optional. Name for the UDAF.

zero

A zero value for this aggregation. Should satisfy the property that any b + zero = b.

reduce

Combine two values to produce a new value. For performance, the function may modify b and return it instead of constructing new object for b.

merge

Merge two intermediate values.

finish

Transform the output of the reduction.

bufferEncoder

Optional. Specifies the Encoder for the intermediate value type.

outputEncoder

Optional. Specifies the Encoder for the final output value type.

nondeterministic

Optional. If true, sets the UserDefinedFunction as nondeterministic.

See also

functions.udaf

for an unnamed variant.