# Data Schemas

> For the complete documentation index, see [llms.txt](https://kotlin.github.io/dataframe/llms.txt).

The Kotlin DataFrame library provides typed data access via [generation of extension properties](extensionpropertiesapi.html) for the type [DataFrame&lt;T&gt;](dataframe.html) (as well as for [DataRow&lt;T&gt;](datarow.html) and [ColumnGroup&lt;T&gt;](datacolumn.html#columngroup)), where `T` is a marker class representing the `DataSchema` of the [DataFrame](dataframe.html).

API reference: [DataSchema](https://kotlin.github.io/dataframe/api-docs/core/org.jetbrains.kotlinx.dataframe.annotations/-data-schema/index.html), [ImportDataSchema](https://kotlin.github.io/dataframe/api-docs/core/org.jetbrains.kotlinx.dataframe.annotations/-import-data-schema/index.html)

A schema of a [DataFrame](dataframe.html) is a mapping from column names to column types.

This data schema can be expressed as a Kotlin interface or data class by annotating it with `@DataSchema`.

If the dataframe is hierarchical — contains a [column group](datacolumn.html#columngroup) or a [column of dataframes](datacolumn.html#framecolumn) — the data schema reflects this structure, with a separate class representing the schema of each column group or nested `DataFrame`.

For example, consider a simple hierarchical dataframe from [](resources/example.csv).

> **Note:**
> Note that this is not a regular CSV file — it contains a column with embedded JSON values.
>
>
>
> To read such files correctly, both the [dataframe-csv](modules.html#dataframe-csv) and [dataframe-json](modules.html#dataframe-json) modules must be included.

This dataframe consists of two columns:

* `name`, which is a `String` column

* `info`, which is a [column group](datacolumn.html#columngroup) containing two nested [value columns](datacolumn.html#valuecolumn): * `age` of type `Int` * `height` of type `Double`

| name | info |
| --- | --- |
|  | age | height |
| Alice | 23 | 175.5 |
| Bob | 27 | 160.2 |

The data schema corresponding to this `DataFrame` can be represented as:

```KOTLIN
// Data schema of the "info" column group
@DataSchema
interface Info {
    val age: Int
    val height: Float
}

// Data schema of the entire DataFrame
@DataSchema
interface Person {
    val info: Info
    val name: String
}
```

[Extension properties](extensionpropertiesapi.html) for `DataFrame<Person>`

are generated based on this schema and allow accessing columns or using them in operations:

```KOTLIN
// Assuming `df` has type `DataFrame<Person>`

// Get "age" column from "info" group
df.info.age

// Select "name" and "height" columns
df.select { name and info.height }

// Filter rows by "age"
df.filter { age >= 18 }
```

See [Extension Properties API](extensionpropertiesapi.html) for more information.

##  `@DataSchema` annotation 

`@DataSchema` is a Kotlin annotation that marks a data class or interface as a data schema. [The compiler plugin](compiler-plugin.html) generates [extension properties](extensionpropertiesapi.html) for the `DataFrame` (or [DataRow](datarow.html), [ColumnGroup](datacolumn.html#columngroup), etc.) with a type parameter annotated with `@DataSchema`.

> **Warning:**
> While you can annotate any Kotlin class or object with a `@DataSchema`, we highly recommend using it only on interfaces and data classes specially made for representing the data schema of a `DataFrame`.
>
>
>
> Use only trivial properties, avoiding computed, `lateinit`, or delegated properties. In data classes, provide only constructor properties.
>
>
>
> In all other cases, the behavior may be undefined. If you do need to use `@DataSchema` on a “complex” class, please let us know via the [issues](https://github.com/Kotlin/dataframe/issues).

Each property of an annotated class or interface corresponds to a column in the `DataFrame` (or [DataRow](datarow.html), [ColumnGroup](datacolumn.html#columngroup), etc.). The property name is the column name, and the property type is the column type.

> **Warning:**
> Data schema is considered compatible if it contains any subset of actual dataframe columns with correct types. If the data schema contains columns that are not present in the dataframe, it is considered incompatible.
>
>
>
> This is checked in [.cast()](cast.html) with `verify=true` and [.convertTo()](convertto.html) methods.

## Data Schema Retrieving

Defining a data schema manually can be difficult, especially for dataframes with many columns or deeply nested structures, and may lead to mistakes in column names or types. Kotlin DataFrame provides several methods for generating data schemas.

* [generate..() methods](dataschemagenerationmethods.html) are extensions for [DataFrame](dataframe.html) (or for its [schema](schema.html)) that generate a code string representing its `DataSchema`.

* [Kotlin DataFrame Compiler Plugin](compiler-plugin.html) cannot automatically infer a data schema from external sources such as files or URLs. However, it can infer the schema if you construct the [DataFrame](dataframe.html) manually — that is, by explicitly declaring the columns using the API. It will also automatically update the schema during operations that modify the structure of the DataFrame.

> **Tip:**
> For best results when working with the Compiler Plugin, it's recommended to generate the initial schema using one of the [generate..() methods](dataschemagenerationmethods.html). Once generated, the Compiler Plugin will automatically keep the schema up to date after any operations that change the structure of the DataFrame.

### Plugins

> **Warning:**
> The current Gradle and KSP plugins are deprecated. Their latest release is 1.0.0-Beta4; there will be no future releases.
>
>
>
> At the moment, [data schema generation is handled via dedicated methods](dataschemagenerationmethods.html) instead of relying on the plugins.

* The [Gradle plugin](gradle-plugin.html) allows generating a data schema automatically by specifying a source file path in the Gradle build script.

* The KSP plugin allows generating a data schema automatically using [Kotlin Symbol Processing](https://kotlinlang.org/docs/ksp-overview.html) by specifying a source file path in your code file.

## Specifying Data Schema

To bring the `DataFrame` into the desired schema, you can use one of two operations:

* Specify the schema using [cast](cast.html).

* Convert the `DataFrame` to the target schema using [convertTo](convertto.html).

## Extension Properties Generation

Once you have a data schema, you can generate [extension properties](extensionpropertiesapi.html).

The easiest and most convenient way is to use the [Kotlin DataFrame Compiler Plugin](compiler-plugin.html), which generates extension properties on the fly for declared data schemas and automatically keeps them up to date after operations that modify the structure of the [DataFrame](dataframe.html).

> **Warning:**
> Extension properties generation was deprecated from the Gradle plugin in favor of the Compiler Plugin.

* When using Kotlin DataFrame inside [Kotlin Notebook](setupkotlinnotebook.html), the schema and extension properties are generated automatically after each cell execution for all `DataFrame` variables declared in that cell. See [extension properties example in Kotlin Notebook](extensionpropertiesapi.html#example).

> **Tip:**
> Compiler Plugin is coming to Kotlin Notebook soon.

* If you're not using the Compiler Plugin, you can still generate [extension properties](extensionpropertiesapi.html) for a [DataFrame](dataframe.html) manually by calling one of the [generate..() methods](dataschemagenerationmethods.html) with the `extensionProperties = true` argument.

### Custom extension properties

Sometimes it is also useful to define your own extension properties based on a [data schema](schema.html).

```KOTLIN
@DataSchema
interface BranchData {
    val expenses: Long
    val revenue: Long
}

val DataRow<BranchData>.profit get() = revenue - expenses
```

```KOTLIN
val dfProfitable = df.filter { it.profit > 0 }
```

See [Custom extension properties](extensionpropertiesapi.html#custom-extension-properties) for more information.

