withCached
inline fun <T, R> Dataset<T>.withCached(blockingUnpersist: Boolean = false, executeOnCached: Dataset<T>.() -> R): R
This function creates block, where one can call any further computations on already cached dataset Data will be unpersisted automatically at the end of computation
it may be useful in many situations, for example, when one needs to write data to several targets
ds.withCached {
write()
.also { it.orc("First destination") }
.also { it.avro("Second destination") }
}Content copied to clipboard
Return
result of block execution for further usage. It may be anything including source or new dataset
Parameters
blockingUnpersist
if execution should be blocked until everything persisted will be deleted
executeOnCached
Block which should be executed on cached dataset.