kotlin-spark-api_3.3.2_2.13/org.jetbrains.kotlinx.spark.api/col

col

inline fun <T, R> Dataset<T>.col(colName: String): TypedColumn<T, R>

Selects column based on the column name and returns it as a TypedColumn.

For example:

dataset.col<_, Int>("a")
Content copied to clipboard
inline fun <T, U> Dataset<T>.col(column: KProperty1<T, U>): TypedColumn<T, U>

Helper function to quickly get a TypedColumn (or Column) from a dataset in a refactor-safe manner.

    val dataset: Dataset<YourClass> = ...
    val columnA: TypedColumn<YourClass, TypeOfA> = dataset.col(YourClass::a)
Content copied to clipboard

See also

inline fun <DsType, U> col(colName: String): TypedColumn<DsType, U>

Returns a TypedColumn based on the given column name and type DsType.

This is just a shortcut to the function from org.apache.spark.sql.functions combined with an as call. For all the functions, simply add import org.apache.spark.sql.functions.* to your file.

See also

fun col(colName: String): Column

Returns a Column based on the given column name.

inline fun <DsType, U> col(column: KProperty1<DsType, U>): TypedColumn<DsType, U>

Returns a Column based on the given class attribute, not connected to a dataset.

    val dataset: Dataset<YourClass> = ...
    val new: Dataset<Tuple2<TypeOfA, TypeOfB>> = dataset.select( col(YourClass::a), col(YourClass::b) )
Content copied to clipboard

See also