pyspark.sql.DataFrameWriter.orc¶
-
DataFrameWriter.
orc
(path, mode=None, partitionBy=None, compression=None)[source]¶ Saves the content of the
DataFrame
in ORC format at the specified path.New in version 1.5.0.
- Parameters
- pathstr
the path in any Hadoop supported file system
- modestr, optional
specifies the behavior of the save operation when data already exists.
append
: Append contents of thisDataFrame
to existing data.overwrite
: Overwrite existing data.ignore
: Silently ignore this operation if data already exists.error
orerrorifexists
(default case): Throw an exception if data already exists.
- partitionBystr or list, optional
names of partitioning columns
- compressionstr, optional
compression codec to use when saving to file. This can be one of the known case-insensitive shorten names (none, snappy, zlib, and lzo). This will override
orc.compress
andspark.sql.orc.compression.codec
. If None is set, it uses the value specified inspark.sql.orc.compression.codec
.
Examples
>>> orc_df = spark.read.orc('python/test_support/sql/orc_partitioned') >>> orc_df.write.orc(os.path.join(tempfile.mkdtemp(), 'data'))