Skip to content

lit_deltalake.datasets.key_dataset

source module lit_deltalake.datasets.key_dataset

Classes

  • DeltaKeyDataset — Map explicit stable keys to individual Delta table rows.

source class DeltaKeyDataset(reader: DeltaReader, scan: DeltaScan, id_column: str, keys: Sequence[str | int], column_specs: tuple[ColumnSpec, ...] | None = None, row_transform: Transform | None = None, row_type: RowType | None = None, transform: Transform | None = None, cache_size: int = 1024)

Bases : BaseDeltaDataset[SampleT], Dataset[SampleT]

Map explicit stable keys to individual Delta table rows.

Create a map-style dataset for stable keyed Delta table rows.

Parameters

  • reader : DeltaReader — Reader used to scan the table.

  • scan : DeltaScan — Read configuration for the versioned table snapshot.

  • id_column : str — Source column containing each row's stable key.

  • keys : Sequence[str | int] — Stable keys identifying rows to retrieve.

  • column_specs : tuple[ColumnSpec, ...] | None — Optional mappings from source columns to sample fields.

  • row_transform : Transform | None — Optional transform applied to each selected row.

  • row_type : RowType | None — Optional dataclass type used to construct samples.

  • transform : Transform | None — Optional transform applied to each selected row.

  • cache_size : int — Maximum number of version-pinned records retained for repeated lookups.

Raises

  • InvalidDatasetConfigurationError — If dataset configuration is invalid.

  • UnsupportedReaderOperationError — If the reader does not support keyed lookup.