loom.etl.backends.polars¶

Polars backend for the Loom ETL framework.

class loom.etl.backends.polars.PolarsFileWriter[source]¶

Bases: object

Write FILE targets with Polars DataFrame writers.

write(frame, spec, *, streaming=False)[source]¶

Write frame to the path declared in spec.

Parameters:

frame (polars.LazyFrame)
spec (FileSpec)
streaming (bool)

Return type:

None

class loom.etl.backends.polars.PolarsPhysicalSchema(schema, partition_columns=())[source]¶

Bases: object

Physical schema snapshot for a resolved Polars target.

Parameters:

schema (polars.Schema) – Native Polars schema for write-time alignment.
partition_columns (tuple[str, ...]) – Ordered partition columns.

class loom.etl.backends.polars.PolarsSourceReader(locator, *, route_resolver=None, file_locator=None)[source]¶

Bases: SourceReader

Read table and file sources using Polars.

Parameters:

locator (str | TableLocator)
route_resolver (TableRouteResolver | None)
file_locator (FileLocator | None)

read(spec, params_instance)[source]¶

Read source spec and return lazy frame.

Parameters:

spec (TableSourceSpec | FileSourceSpec | TempSourceSpec)
params_instance (Any)

Return type:

polars.LazyFrame

execute_sql(frames, query)[source]¶

Execute SQL query against backend frames.

Parameters:

frames (dict[str, Any])
query (str)

Return type:

polars.LazyFrame

class loom.etl.backends.polars.PolarsTargetWriter(locator, *, route_resolver=None, missing_table_policy=MissingTablePolicy.SCHEMA_MODE, file_locator=None)[source]¶

Bases: _WritePolicy[LazyFrame, DataFrame, PolarsPhysicalSchema]

Polars target writer using delta-rs for Delta tables.

Parameters:

locator (str | TableLocator)
route_resolver (TableRouteResolver | None)
missing_table_policy (MissingTablePolicy)
file_locator (FileLocator | None)

append(frame, table_ref, params_instance, *, streaming=False)[source]¶

Append frame to table (legacy API, creates table on first write).

Parameters:

frame (polars.LazyFrame)
table_ref (Any)
params_instance (Any)
streaming (bool)

Return type:

None

to_frame(records, /)[source]¶

Convert execution records into a Polars LazyFrame.

Parameters:: records (Sequence[PipelineRunRecord | ProcessRunRecord | StepRunRecord])
Return type:: polars.LazyFrame

loom.etl.backends.polars.read_delta_physical_schema(uri, storage_options=None)[source]¶

Read physical schema directly from a Delta table URI.

Parameters:

uri (str) – Delta table URI/path (including uc://catalog.schema.table).
storage_options (dict[str, str] | None) – Optional delta-rs storage options.

Returns:

Physical schema when the table exists, else None.

Return type:

PolarsPhysicalSchema | None