What is Data Frame
Data frame is an abstraction for working with structured data. Essentially it’s a 2-dimensional table with labeled columns of potentially different types. You can think of it like a spreadsheet or SQL table, or a dictionary of series objects.
The handiness of this abstraction is not in the table itself but in a set of operations defined on it. The Kotlin Dataframe library is an idiomatic Kotlin DSL defining such operations. The process of working with data frame is often called data wrangling which is the process of transforming and mapping data from one "raw" data form into another format that is more appropriate for analytics and visualization. The goal of data wrangling is to assure quality and useful data.
Main Features and Concepts
Hierarchical — the Kotlin DataFrame library provides an ability to read and present data from different sources including not only plain CSV but also JSON. That’s why it has been designed hierarchical and allows nesting of columns and cells.
Interoperable — hierarchical data layout also opens a possibility of converting any objects structure in application memory to a data frame and vice versa.
Safe — the Kotlin DataFrame library provides a mechanism of on-the-fly generation of extension properties that correspond to the columns of a data frame. In interactive notebooks like Jupyter or Datalore, the generation runs after each cell execution. In IntelliJ IDEA there's a Gradle plugin for generation properties based on CSV file or JSON file. Also, we’re working on a compiler plugin that infers and transforms
DataFrameschema while typing. The generated properties ensures you’ll never misspell column name and don’t mess up with its type, and of course nullability is also preserved.
Generic — columns can store objects of any type, not only numbers or strings.
Polymorphic — if all columns of
DataFrameare presented in some other
DataFrames, then the first one could be a superclass for latter. Thus, one can define a function on an interface with some set of columns and then execute it in a safe way on any
DataFrameswhich contains this set of columns.
Immutable — all operations on
DataFrameproduce new instance, while underlying data is reused wherever it's possible
Contribute and give feedback
If you find a bug, or have an idea for a new feature, file an issue in our DataFrame GitHub repository.
Additionally, we welcome contributions. To get stared, choose an issue and let us know that you're working on it. When you're ready, create a pull request.
You can also contact us in the #datascience channel of Kotlin Slack.
For more information on how to contribute, see our Contributing guidelines.