readDelimImpl

internal fun readDelimImpl(inputStream: InputStream, delimiter: Char, header: List<String>, charset: Charset?, hasFixedWidthColumns: Boolean, fixedColumnWidths: List<Int>, colTypes: Map<String, ColType>, skipLines: Long, readLines: Long?, parserOptions: ParserOptions?, ignoreEmptyLines: Boolean, allowMissingColumns: Boolean, ignoreExcessColumns: Boolean, quote: Char, ignoreSurroundingSpaces: Boolean, trimInsideQuoted: Boolean, parseParallel: Boolean, compression: Compression<*>, adjustCsvSpecs: AdjustCsvSpecs): DataFrame<*>

Implementation to read delimiter-separated data from an InputStream based on the Deephaven CSV library.

Parameters

inputStream

Represents the file to read. Use charset to specify the encoding.

charset

The character set the input is encoded in. Default: null

If null, the Charset will be read from the BOM of the provided input, defaulting to UTF-8 if no BOM is found.

delimiter

The field delimiter character. The default is ',' for CSV, 't' for TSV.

header

Optional column titles. Default: empty list.

If non-empty, the data will be read with header as the column titles (use skipLines if there's a header in the data). If empty (default), the header will be read from the data.

colTypes

The expected ColType per column name. Default: empty map, a.k.a. infer every column type.

If supplied for a certain column name (inferred from data or given by header), the parser will parse the column with the specified name as the specified type, else it will infer the type.

e.g. colTypes = mapOf("colName" toColType.Int). You can also set ColType.DEFAULTtoColType.X to set a default column type, like ColType.String.

skipLines

The number of lines to skip before reading the header and data. Default: 0.

Useful for files with metadata, or comments at the beginning, or to give a custom header.

readLines

The maximum number of lines to read from the data. Default: null.

If null, all lines will be read.

hasFixedWidthColumns

Whether the data has fixed-width columns instead of a single delimiter. Default: false.

Fixed-width columns can occur, for instance, in multi-space delimited data, where the columns are separated by multiple spaces instead of a single delimiter, so columns are visually aligned. Column widths are determined by the header in the data (if present), or manually by setting fixedColumnWidths.

fixedColumnWidths

The fixed column widths. Default: empty list.

Requires hasFixedWidthColumns. If empty, the column widths will be determined by the header in the data (if present), else, this manually sets the column widths. The number of widths should match the number of columns.

parserOptions

Optional parsing options for columns initially read as String. Default: null.

Can configure locale, date format, double parsing, skipping types, etc.

If parserOptions or any of the arguments are null, the global parser configuration (DataFrame.parser) will be queried.

The only exceptions are:

ignoreEmptyLines

Whether to skip intermediate empty lines. Default: false.

If false, empty lines will be interpreted as having empty values if allowMissingColumns.

allowMissingColumns

Whether to allow rows with fewer columns than the header. Default: true.

If true, rows that are too short will be interpreted as empty values.

ignoreExcessColumns

Whether to ignore rows with more columns than the header. Default: true.

If true, rows that are too long will have those columns dropped.

quote

The quote character. Default: ".

Used when field- or line delimiters should be interpreted as literal text.

For example: 123,"hello, there",456, would correspond to: 123; hello, there; 456.

ignoreSurroundingSpaces

Whether to ignore leading and trailing blanks around non-quoted fields. Default: true.

trimInsideQuoted

Whether to ignore leading and trailing blanks inside quoted fields. Default: false.

parseParallel

Whether to parse the data in parallel. Default: true.

If true, the data will be read and parsed in parallel by the Deephaven parser. This is usually faster but can be turned off for debugging.

compression

The compression of the data. Default: Compression.None, unless detected otherwise from the input file or url.

adjustCsvSpecs

Optional extra CsvSpecs configuration. Default: { it }.

Before instantiating the CsvSpecs, the CsvSpecs.Builder will be passed to this lambda. This will allow you to configure/overwrite any CSV / TSV parsing options.