curryer.correction.io

Unified path resolution for the correction pipeline.

All file loading in the correction package should go through resolve_path() to transparently handle local paths and S3 URIs.

S3 access follows the same pattern as curryer.correction.dataio: callers may provide an explicit s3_client (useful for testing) or rely on the default boto3 client.

Attributes

logger

_temp_files

Functions

_cleanup_temp_files(→ None)

Remove temporary files created by S3 downloads.

_require_client(→ object)

Return client if given, otherwise create a default boto3 S3 client.

resolve_path(→ pathlib.Path)

Resolve a file path, downloading from S3 if necessary.

_download_from_s3(→ pathlib.Path)

Download an S3 object to a local temporary file.

Module Contents

curryer.correction.io.logger
curryer.correction.io._temp_files: list[pathlib.Path] = []
curryer.correction.io._cleanup_temp_files() → None

Remove temporary files created by S3 downloads.

curryer.correction.io._require_client(client: object | None) → object

Return client if given, otherwise create a default boto3 S3 client.

This is intentionally identical to curryer.correction.dataio._require_client().

Parameters:

client (object or None) – An injected S3 client, or None to use the default.

Returns:

A boto3 S3 client.

Return type:

object

Raises:

ImportError – If client is None and boto3 is not installed.

curryer.correction.io.resolve_path(path: str | pathlib.Path, *, s3_client=None) → pathlib.Path

Resolve a file path, downloading from S3 if necessary.

Parameters:

  • path (str or Path) – Local file path or S3 URI (s3://bucket/key).

  • s3_client (boto3 S3 client, optional) – Injected client for testing. If omitted and path is an S3 URI, a default client is created via boto3.

Returns:

Local file path. For S3 URIs, a temporary local file that is cleaned up at process exit via atexit().

Return type:

Path

Raises:

  • ImportError – If path is an S3 URI and boto3 is not installed.

  • FileNotFoundError – If path is a local path that does not exist.

  • ValueError – If path is an S3 URI with no object key (e.g. s3://bucket).

curryer.correction.io._download_from_s3(s3_uri: str, *, s3_client=None) → pathlib.Path

Download an S3 object to a local temporary file.

The temp file is registered for cleanup at process exit via _cleanup_temp_files().

Parameters:

  • s3_uri (str) – S3 URI in the form s3://bucket/key.

  • s3_client (boto3 S3 client, optional) – Injected client for testing.

Returns:

Path to the local temporary file.

Return type:

Path

Raises:

  • ImportError – If boto3 is not installed and no client was injected.

  • ValueError – If the URI has no object key.