map
Computes a new value for every value, row, or key–group pair of the receiver, and collects the results into a List, a DataFrame, a DataColumn, or a FrameColumn.
Related operations: Add / map / remove columns
All map operations share the name but differ in what they go over and what they give back:
Operation | Goes over | Returns |
|---|---|---|
rows of a |
| |
rows of a | a single | |
rows of a | a new | |
values of a | a | |
key–group pairs of a |
| |
key–group pairs of a | ||
key–group pairs of a |
Each result keeps the order of the values, rows, or key–group pairs it was computed from.
Every example on this page uses the same DataFrame:
map
Maps the rows of a DataFrame into a List with one element per row.
See row expressions
A ColumnGroup is also a DataFrame, so map on a column group is this operation: it goes over the rows of the group and returns a List. To get a DataColumn of the same size instead — a column of the rows of the group — call asDataColumn() on the group first, and then map.
mapToColumn
Maps the rows of a DataFrame into a single new DataColumn with one value per row.
See row expressions
The new column is standalone: the original DataFrame is not changed and does not contain it. Use add to get a DataFrame with the new column in it.
Inside the row expression, prev()?.newValue() gives the value already computed for the preceding row — null for the first row, where there is no preceding one. This is how running totals and other recurrences are expressed; see add.
mapToFrame
Maps the rows of a DataFrame into a new DataFrame made of the described columns.
The result holds only the described columns, in the order in which they are described. This is what makes mapToFrame different from add, where the columns of the original DataFrame are part of the result as well. In the example above, city is in the result only because of the +city line, and name and age are not in it at all.
map on DataColumn
Maps the values of a DataColumn into a new DataColumn of the same size. mapIndexed gives the position of the value as well, starting at 0.
The new column has the same name as the original one, so it is usually renamed on the spot or given a name by the operation it is passed to.
mapIndexed also gives the position of the value:
Which kind of column you get follows the type of the new column — the type argument, or the type given explicitly — and not the computed values: a DataFrame type gives a FrameColumn, a DataRow type gives a ColumnGroup, and any other type gives a ValueColumn. A nullable DataFrame type belongs to the last group, because a FrameColumn cannot hold null.
infer only concerns a ValueColumn: it decides whether the type of that column is the given one as it is, or the type of the computed values. For a ColumnGroup and a FrameColumn it changes nothing.
The overloads with an explicit type are for the cases where the type of the new column is only known at runtime. The computed values are put into the column as they are, without any conversion, so the type has to fit them.
A ValueColumn can never have a non-nullable DataFrame type, so a call that would give it one fails with an IllegalArgumentException. That happens when the computed values are dataframes and Infer.Type derives a DataFrame type for them, and also under a nullable DataFrame type when none of the computed values is null: the default Infer.Nulls then drops the nullability and leaves exactly that forbidden type. With at least one null among them the same call succeeds and gives a ValueColumn of the nullable type.
map on GroupBy
Maps the key–group pairs of a GroupBy: every row of a GroupBy is one key–group pair — the key values, and the group of rows that belongs to them (see groupBy).
The lambda receives the pair as a GroupWithKey, both as the receiver and as the argument, so the key values are available as key (a DataRow) and the rows of the group as group (a DataFrame).
mapToFrames names the new column after GroupBy.groups ("group" by default); call concat() on it to get all of those dataframes back as one DataFrame. concatWithKeys is built exactly this way.