loom.etl.backends._historify

Backend-agnostic SCD Type 2 transform and HistorifyBackend Protocol.

loom.etl.backends._historify.eval_param_expr(expr, params_instance)[source]

Walk the ParamExpr attribute path and return the resolved value.

Parameters:
Return type:

Any

class loom.etl.backends._historify.HistorifyBackend(*args, **kwargs)[source]

Bases: Protocol[F]

Backend-specific frame operations used by SCD2Transform.

history_dtype(spec)[source]

Return the native dtype for valid_from / valid_to columns.

Parameters:

spec (HistorifySpec)

Return type:

Any

stamp_col(frame, name, value, dtype)[source]

Add or overwrite column with literal value.

Parameters:

  • frame (F)

  • name (str)

  • value (Any)

  • dtype (Any)

Return type:

F

null_col(frame, name, dtype)[source]

Add null column with given dtype.

Parameters:

  • frame (F)

  • name (str)

  • dtype (Any)

Return type:

F

rename(frame, rename_map)[source]

Rename columns according to mapping.

Parameters:
Return type:

F

drop(frame, cols)[source]

Drop columns.

Parameters:
Return type:

F

dedup_priority(frame, subset, priority_col)[source]

Deduplicate keeping, per subset group, rows with the max priority_col.

Parameters:
Return type:

F

apply_overwrite_cols(unchanged, incoming, join_key, overwrite)[source]

Replace overwrite columns in unchanged rows with values from incoming.

Drops the overwrite columns from unchanged then joins the fresh values from incoming on join_key. The open row is refreshed in-place; no new history row is created.

Parameters:

  • unchanged (F)

  • incoming (F)

  • join_key (list[str])

  • overwrite (tuple[str, ...])

Return type:

F

rewind_to(frame, spec, eff_date)[source]

Rewind history to its state before any run at or after eff_date.

Parameters:
Return type:

F

collect_future_rows(existing, spec, eff_date)[source]

Materialize keys + valid_from tuples of rows with valid_from > eff_date.

Parameters:
Return type:

list[tuple[object, …]]

build_log_boundaries(frame, spec)[source]

Compute valid_from / valid_to from sorted event frame (LOG mode).

Parameters:
Return type:

F

temporal_conflict_min_date(existing, spec, eff_date)[source]

Return min conflicting valid_from or None if no conflict.

Parameters:
Return type:

Any | None

loom.etl.backends._historify.scd2_transform(ops, frame, existing, spec, params_instance)[source]

Apply SCD Type 2 logic and return the transformed frame plus repair report.

Parameters:

  • ops (HistorifyBackend[F]) – Backend-specific frame operations.

  • frame (F) – Incoming data frame.

  • existing (F | None) – Current target frame, or None for first run.

  • spec (HistorifySpec) – Compiled HistorifySpec.

  • params_instance (object) – Runtime params for ParamExpr resolution.

Returns:

Tuple of the transformed frame ready to be written to Delta and a repair report when a temporal rerun rewound strictly-future history (SNAPSHOT mode with allow_temporal_rerun), otherwise None.

Raises:
Return type:

tuple[F, HistorifyRepairReport | None]

loom.etl.backends._historify.prev_period_value(eff_date, spec)[source]

Return one unit before eff_date (one day or one microsecond).

Parameters:
Return type:

Any

loom.etl.backends._historify.resolve_effective_date(spec, params_instance)[source]

Resolve effective date to a scalar (SNAPSHOT) or column name (LOG).

Parameters:
Return type:

Any

loom.etl.backends._historify.resolve_track_cols(spec, frame_cols)[source]

Return tracked columns — explicit or all non-key, non-history columns.

Parameters:
Return type:

tuple[str, …]