pyspark.RDD.unpersist#

RDD.unpersist(blocking=False)[source]#

Mark the RDD as non-persistent, and remove all blocks for it from memory and disk.

Added in version 0.9.1.

Parameters:

blockingbool, optional, default False: whether to block until all blocks are deleted

Added in version 3.0.0.

Returns:

RDD: The same RDD

See also

RDD.cache()
RDD.persist()
RDD.getStorageLevel()

Examples

>>> rdd = sc.range(5)
>>> rdd.is_cached
False
>>> _ = rdd.unpersist()
>>> rdd.is_cached
False
>>> _ = rdd.cache()
>>> rdd.is_cached
True
>>> _ = rdd.unpersist()
>>> rdd.is_cached
False
>>> _ = rdd.unpersist()