Skip to content

lit_deltalake.readers.spark

source module lit_deltalake.readers.spark

Classes

  • SparkDeltaReader — Read Delta tables from a caller-owned Spark or Spark Connect session.

source class SparkDeltaReader(session: SparkSession | None = None)

Bases : DeltaReader

Read Delta tables from a caller-owned Spark or Spark Connect session.

Create a Spark-backed Delta reader.

Parameters

  • session : SparkSession | None — Optional caller-owned Spark or Spark Connect session.

Methods

  • scan — Yield Arrow record batches for a Spark table scan.

source method SparkDeltaReader.scan(request: DeltaScan) → Iterator['pa.RecordBatch']

Yield Arrow record batches for a Spark table scan.

Parameters

  • request : DeltaScan — Configuration for the table snapshot and selected data.

Raises

  • InvalidDatasetConfigurationError

  • UnsupportedReaderOperationError