Data Schemas and Extension Properties Troubleshooting

Sometimes you can get an exception with a message containing

..exception in generated DataFrame extension property..

This means there is a runtime error while accessing a DataFrame extension property, generated by the Compiler Plugin or in Kotlin Notebook.

Such errors are caused by generating extension properties for data schemas that are not compatible with the DataFrame, DataRow, etc. In most cases, the schema contains columns with an incorrect name or type.

For example:

@DataSchema
interface Schema {
    val age: String
}

Read a simple csv with a column of integers:

age
17
32
26

val df = DataFrame.readCsv(simpleCsvFile).cast<Schema>()

// Compiles correctly but fails on runtime
df.filter { age > "20" }

Possible reasons

Incompatible manually defined data schema

If you define the initial data schema manually, make sure your data schema is compatible with the dataframe.

Use .cast<Schema>() with verify=true for verifying the Schema compatibility.
Use special methods for generating a data schema code instead of defining data schema manually. However, you may still need to edit a generated schema — adjust a type nullability or the whole type, since sometimes type can be inferred incorrect if the data sample used for generation is not representative or if there's a bug in this DataFrame.

Bug in the DataFrame reader

Sometimes a column is created internally with an incorrect KType that doesn't represent actual runtime values. As a result, schema().print(), generateInterfaces, print(columnTypes = true) show misleading type.

This can happen when reading a dataframe from files or databases.

Possible workarounds (for ValueColumns):

Update runtime column type:
- Specify the correct type using .replace {} and ValueColumn.changeType():
df.replace { wrongTypeCol }.with { it.asValueColumn().changeType(typeOf<ActualType>()) }
- Use .inferType { columns } to infer the correct types for the selected columns from the actual values. Doesn't work for generic types like Map.
inferType() can take a long time and use up a lot of resources for large dataframes, so we recommend using it once to figure out the actual type and change it to changeType() for production code.
Change the compiler-time schema of a dataframe

You need to edit (you can view the correct one with .schema().print()) or regenerate the correct data schema (with generate..() methods) manually and apply it (with cast() or convertTo()).

Problems with SQLite type affinity

Because of SQLite type affinity, the column type reported by the JDBC driver may not match the actual types of the values stored in that column.

This commonly occurs when reading SQLite columns declared with custom or non-standard SQL types.

You can explicitly specify the resulting Kotlin type using the Sqlite.withCustomConverters { ... } DSL:

forColumn<T>(columnName) sets the T Kotlin type for a specific column.
forType<T>(typeName) sets the T Kotlin type for all columns declared with the specified SQL type.
forColumn(...) { ... } and forType(...) { ... } additionally let you transform the raw stored value.

For example:

You have a LONGVARCHAR column containing text (i.e., String or String? values). However, SQLite type affinity assigns it the NUMERIC basic type, so the JDBC driver expects Int or Long values. Use forType<String?>("LONGVARCHAR") to explicitly specify the expected column type.
You have several columns of the DATETIME type. They are stored as TEXT, and Kotlin DataFrame can automatically parse their values into date-time types. However, suppose the time_stamp column of the DATETIME type contains date-time values in an unusual format. Automatic parsing fails and causes an exception. Use forColumn("time_stamp") { ... } to provide a custom converter specifically for this column.

val sqliteCustom = Sqlite.withCustomConverters {
    // SQLite assigns `NUMERIC` affinity to the custom `LONGVARCHAR` type,
    // so the JDBC driver reports the column type as Int.
    // However, the actual stored values are strings, so we explicitly
    // set the resulting Kotlin type to String?.
    forType<String?>("LONGVARCHAR")

    // Convert values from the "time_stamp" column regardless of its SQL type.
    // The raw values are stored as strings and parsed into LocalDateTime values;
    //  the resulting column has LocalDateTime type as well.
    forColumn("time_stamp") { raw: String ->
        LocalDateTime.parse(raw, customFormat)
    }
}

val df = DataFrame.readSqlTable(
    connectionConfig,
    "table_name",
    dbType = sqliteCustom,
)

17 September 2026