cast

Changes the type argument of the DataFrame instance without changing its contents.

cast<T>(verify = false)

Parameters:

verify: Boolean = false — when true, the function throws an exception if the DataFrame instance doesn't match the given schema. Otherwise, it just changes the format type without actual data checks.

Use this operation to change the formal type of a DataFrame instance to match the expected schema and enable generated extension properties for it.

@DataSchema
interface Person {
    val age: Int
    val name: String
}

df.cast<Person>()

To convert DataFrame columns to match given schema, use convertTo operation.

Reusing implicitly generated schema

castTo<T>(df: DataFrame<T>)

In notebooks, dataframe types are implicitly generated.

This type can be referred to, but its name will change whenever you re-execute cells. Here how you can do it in a more robust way:

val sample = DataFrame.readJson("sample.json")

for (file in files) {
    // df here is expected to have the same structure as sample
    val df = DataFrame.readJson(file).castTo(sample)
    val count = df.count { perf > 10.0 }
    println("$file: $count")
}

Last modified: 27 September 2024