DataSchema
This annotation marks an interface or data class as a data schema. An annotated declaration should be a non-local and non-private interface or class with only constructor properties.
This annotation generates an extension properties API for a declaration according to its properties. The aim is to provide a convenient class-like syntax for working with a dataframe instance after reading it from CSV, JSON, databases, Arrow, or other sources.
After a val df = DataFrame.read* operation, df is the source of truth for the data schema. One way to look at it is that a data schema tells the compiler what is already there; it does not affect reading.
Given the initial schema of the data you read, the compiler plugin provides a typed result for most operations.
Example:
@DataSchema
data class Group(
val id: String, // DataColumn<String>
val participants: List<Person>, // FrameColumn<Person>
)
@DataSchema
data class Person(
val name: Name, // ColumnGroup<Name>
val age: Int, // DataColumn<Int>
val city: String?, // DataColumn<String?>
)
@DataSchema
data class Name(
val firstName: String, // DataColumn<String>
val lastName: String, // DataColumn<String>
)
fun main() {
val url = "https://raw.githubusercontent.com/Kotlin/dataframe/refs/heads/master/data/participants.json"
val df = DataFrame.readJson(url).cast<Group>()
val groupId: String = df.id[0] // Properties-style access to columns and values.
val df1 = df.asGroupBy { participants }.aggregate {
count() into "groupSize"
city.distinct().toList() into "cities"
}
// The compiler plugin uses prior knowledge of `Group` and the aggregate operation to infer new columns.
val cities: List<String?> = df1.cities[0]
}Schema properties describe the corresponding dataframe columns:
A property of a regular type
Vcorresponds to aDataColumn<V>.A property of type
T, whereTis annotated with@DataSchema, corresponds to aColumnGroup<T>.A property of type
List<T>, whereTis annotated with@DataSchema, corresponds to aFrameColumn<T>.
After casting a dataframe to DataFrame<Group>, it can be converted to data-class instances:
val url = "https://raw.githubusercontent.com/Kotlin/dataframe/refs/heads/master/data/participants.json"
val df = DataFrame.readJson(url).cast<Group>()
val groups: List<Group> = df.toList()
val participants: List<Person> = groups.first().participants