gather
Converts several columns into two columns key
and value
. key
column will contain names of original columns, value
column will contain values from original columns.
This operation is reverse to pivot
See column selectors for how to select the columns for this operation.
Configuration options:
explodeLists
— gathered values of typeList
will be exploded into their elements, sowhere
,cast
,notNull
andmapValues
will be applied to list elements instead of lists themselvescast
— inform compiler about the expected type of gathered elements. This type will be passed towhere
andmapKeys
lambdasnotNull
— skip gatherednull
valueswhere
— filter gathered valuesmapKeys
— transform gathered column names (keys)mapValues
— transform gathered column values
Storage options:
into(keyColumn, valueColumn)
— store gathered key-value pairs in two new columns with nameskeyColumn
andvalueColumn
keysInto(keyColumn)
— store only gathered keys (column names) in a new columnkeyColumn
valuesInto(valueColumn)
— store only gathered values in a new columnvalueColumn