parse

parse tries to parse every String column into one of supported types in the following order:

• Int

• Long

• Instant (kotlinx.datetime and java.time)

• LocalDateTime (kotlinx.datetime and java.time)

• LocalDate (kotlinx.datetime and java.time)

• Duration (kotlin.time and java.time)

• LocalTime (java.time)

• URL (java.net)

• Double (with optional locale settings)

• Boolean

• BigDecimal

• JSON (arrays and objects) (requires the org.jetbrains.kotlinx:dataframe-json dependency)

DataFrame supports multiple parser options that can be used to customize the parsing behavior. These can be supplied to the parse function (or any other function that can implicitly parse Strings) as an argument.

For each option you don't supply (or supply null) DataFrame will take the value from the Global Parser Options.

Available parser options:

• locale: Locale is used to parse doubles

  • Global default locale is Locale.getDefault()

• dateTimePattern: String is used to parse date and time

  • Global default supports ISO (local) date-time

• dateTimeFormatter: DateTimeFormatter is used to parse date and time

  • Is derived from dateTimePattern and/or locale if null

• nullStrings: List<String> is used to treat particular strings as null value

  • Global default null strings are "null" and "NULL"

  • When reading from CSV, we include even more defaults, like "", and "NA". See the KDocs there for the exact details

• skipTypes: Set<KType> types that should be skipped during parsing

  • Empty set by global default; parsing can result in any supported type

• useFastDoubleParser: Boolean is used to enable or disable the new fast double parser

  • Enabled by global default

df.parse(options = ParserOptions(locale = Locale.CHINA, dateTimeFormatter = DateTimeFormatter.ISO_WEEK_DATE))

As mentioned before, you can change the default global parser options that will be used by read, convert, and other parse operations. Whenever you don't explicitly provide parser options to a function call, DataFrame will use these global options instead.

For example, to change the locale to French and add a custom date-time pattern for all following DataFrame calls, do:

DataFrame.parser.locale = Locale.FRANCE
DataFrame.parser.addDateTimePattern("dd.MM.uuuu HH:mm:ss")

For locale, this means that the one being used by the parser is defined as:

↪ The locale given as function argument directly, or in parserOptions, if it is not null, else

↪ The locale set by DataFrame.parser.locale = ..., if it is not null, else

↪ Locale.getDefault(), which is the system's default locale that can be changed with Locale.setDefault().

DataFrame has a new fast and powerful double parser enabled by default. It is based on the FastDoubleParser library for its high performance and configurability (in the future, we might expand this support to Float, BigDecimal, and BigInteger as well).

The parser is locale-aware; it will use the locale set by the (global) parser options to parse the doubles. It also has a fallback mechanism built in, meaning it can recognize characters from all other locales (and some from Wikipedia) and parse them correctly as long as they don't conflict with the current locale.

For example, if your locale uses ',' as decimal separator, it will not recognize ',' as thousands separator, but it will recognize ''', ' ', '٬', '_', ' ', etc. as such. The same holds for characters like "e", "inf", "×10^", "NaN", etc. (ignoring case).

This means you can safely parse "123'456 789,012.345×10^6" with a US locale but not "1.234,5".

Aside from this, DataFrame also explicitly recognizes "∞", "inf", "infinity", and "infty" as Double.POSITIVE_INFINITY (as well as their negative counterparts), "nan", "na", and "n/a" as Double.NaN, and all forms of whitespace are treated equally.

If FastDoubleParser fails to parse a String as Double, DataFrame will try to parse it using the standard NumberFormat.parse() function as a last resort.

If you experience any issues with the new parser, you can turn it off by setting useFastDoubleParser = false, which will use the old NumberFormat.parse() function instead.

Please report any issues you encounter.