curryer.correction.io

Unified path resolution for the correction pipeline.

All file loading in the correction package should go through resolve_path() to transparently handle local paths and S3 URIs.

S3 access follows the same pattern as curryer.correction.dataio: callers may provide an explicit s3_client (useful for testing) or rely on the default boto3 client.

Attributes

logger

Functions

resolve_path(→ pathlib.Path)

Resolve a file path, downloading from S3 if necessary.

Module Contents

curryer.correction.io.logger
curryer.correction.io.resolve_path(path: str | pathlib.Path, *, s3_client=None) → pathlib.Path

Resolve a file path, downloading from S3 if necessary.

Parameters:

  • path (str or Path) – Local file path or S3 URI (s3://bucket/key).

  • s3_client (boto3 S3 client, optional) – Injected client for testing. If omitted and path is an S3 URI, a default client is created via boto3.

Returns:

Local file path. For S3 URIs, a temporary local file that is cleaned up at process exit via atexit().

Return type:

Path

Raises:

  • ImportError – If path is an S3 URI and boto3 is not installed.

  • FileNotFoundError – If path is a local path that does not exist.

  • ValueError – If path is an S3 URI with no object key (e.g. s3://bucket).