The table type(s) (TABLE_TYPE) of ordinary tables in the SQL database, used by the readAllSqlTables functions as a filter when querying the database for all the tables it has using DatabaseMetaData.getTables.

Functions

buildDataColumn

override fun <D, P> buildDataColumn(name: String, values: List<D>, tableColumnMetadata: TableColumnMetadata, targetColumnSchema: ColumnSchema?, inferNullability: Boolean): DataColumn<P>

Builds a DataColumn from the given (preprocessed) values, adhering to targetColumnSchema.

buildSelectTableQueryWithLimit

open fun buildSelectTableQueryWithLimit(tableName: String, limit: Int?): String

Builds a SELECT query for reading from a table.

buildSqlQueryWithLimit

open fun buildSqlQueryWithLimit(sqlQuery: String, limit: Int = 1): String

Constructs a SQL query with a limit clause.

buildTableMetadata

open override fun buildTableMetadata(tables: ResultSet): TableMetadata

How to retrieve the correct table metadata when using DataFrame.readAllSqlTables and DataFrameSchema.readAllSqlTables. The names of these can be found in the DatabaseMetaData implementation of the DuckDB JDBC integration.

configureReadStatement

open fun configureReadStatement(statement: PreparedStatement)

Configures the provided PreparedStatement for optimized read operations.

createConnection

open override fun createConnection(dbConfig: DbConnectionConfig): Connection

Creates a database connection using the provided configuration.

generateConverter

protected open override fun generateConverter(tableColumnMetadata: TableColumnMetadata): AnyJdbcToDataFrameConverter

getConverter

protected open fun getConverter(tableColumnMetadata: TableColumnMetadata): AnyJdbcToDataFrameConverter

getExpectedJdbcType

override fun getExpectedJdbcType(tableColumnMetadata: TableColumnMetadata): KType

Returns the type of the objects returned by getValueFromResultSet for the given column. Also called type J.

getPreprocessedValueType

override fun getPreprocessedValueType(tableColumnMetadata: TableColumnMetadata, expectedJdbcType: KType): KType

Returns the type of the objects returned by preprocessValue for the given column. Also called type D.

getTableColumnsMetadata

open fun getTableColumnsMetadata(resultSet: ResultSet): List<TableColumnMetadata>

Retrieves column metadata from a JDBC ResultSet.

getTargetColumnSchema

override fun getTargetColumnSchema(tableColumnMetadata: TableColumnMetadata, expectedValueType: KType): ColumnSchema?

Returns the target schema of the given column which buildDataColumn will adhere to. This schema corresponds to type P, in the sense that it will describe the schema of DataColumn<P>.

getValueFromResultSet

override fun <J> getValueFromResultSet(rs: ResultSet, columnIndex: Int, tableColumnMetadata: TableColumnMetadata, expectedJdbcType: KType): J

Extracts a value from the result set for the given column.

isSystemTable

open override fun isSystemTable(tableMetadata: TableMetadata): Boolean

How to filter out system tables from user-created ones when using DataFrame.readAllSqlTables and DataFrameSchema.readAllSqlTables.

parseDuckDbType

internal fun parseDuckDbType(sqlTypeName: String, isNullable: Boolean): AnyJdbcToDataFrameConverter

How a column type from JDBC, sqlTypeName, is read in Java/Kotlin. The returned type must exactly follow ResultSet.getObject of your specific database's JDBC driver. Returning null defer the implementation to the default one (which may not always be correct).

parseListType

internal fun parseListType(typeString: String): String

Parses "X" and "X123" into "X", and "X" into "X"

parseMapTypes

internal fun parseMapTypes(typeString: String): Pair<String, String>

Parses "MAP(X, Y)" into "X" and "Y", taking parentheses into account

parseStructType

internal fun parseStructType(typeString: String): Map<String, String>

Parses

preprocessValue

override fun <J, D> preprocessValue(value: J, tableColumnMetadata: TableColumnMetadata, expectedJdbcType: KType, expectedPreprocessedValueType: KType): D

(Potentially) preprocesses the value for the given column before collecting it in a DataFrame DataColumn.

quoteIdentifier

open fun quoteIdentifier(name: String): String

Quotes an identifier (table or column name) according to database-specific rules.

toDataColumn

protected fun <D, P> List<D>.toDataColumn(name: String, targetColumnSchema: ColumnSchema?, inferNullability: Boolean): DataColumn<P>

Helper function to convert this list of values to a DataColumn<P>.