curryer.correction.io ===================== .. py:module:: curryer.correction.io .. autoapi-nested-parse:: Unified path resolution for the correction pipeline. All file loading in the correction package should go through resolve_path() to transparently handle local paths and S3 URIs. S3 access follows the same pattern as :mod:`curryer.correction.dataio`: callers may provide an explicit ``s3_client`` (useful for testing) or rely on the default boto3 client. Attributes ---------- .. autoapisummary:: curryer.correction.io.logger curryer.correction.io._temp_files Functions --------- .. autoapisummary:: curryer.correction.io._cleanup_temp_files curryer.correction.io._require_client curryer.correction.io.resolve_path curryer.correction.io._download_from_s3 Module Contents --------------- .. py:data:: logger .. py:data:: _temp_files :type: list[pathlib.Path] :value: [] .. py:function:: _cleanup_temp_files() -> None Remove temporary files created by S3 downloads. .. py:function:: _require_client(client: object | None) -> object Return *client* if given, otherwise create a default boto3 S3 client. This is intentionally identical to :func:`curryer.correction.dataio._require_client`. :param client: An injected S3 client, or None to use the default. :type client: object or None :returns: A boto3 S3 client. :rtype: object :raises ImportError: If *client* is None and boto3 is not installed. .. py:function:: resolve_path(path: str | pathlib.Path, *, s3_client=None) -> pathlib.Path Resolve a file path, downloading from S3 if necessary. :param path: Local file path or S3 URI (``s3://bucket/key``). :type path: str or Path :param s3_client: Injected client for testing. If omitted and *path* is an S3 URI, a default client is created via boto3. :type s3_client: boto3 S3 client, optional :returns: Local file path. For S3 URIs, a temporary local file that is cleaned up at process exit via :func:`atexit`. :rtype: Path :raises ImportError: If *path* is an S3 URI and boto3 is not installed. :raises FileNotFoundError: If *path* is a local path that does not exist. :raises ValueError: If *path* is an S3 URI with no object key (e.g. ``s3://bucket``). .. py:function:: _download_from_s3(s3_uri: str, *, s3_client=None) -> pathlib.Path Download an S3 object to a local temporary file. The temp file is registered for cleanup at process exit via :func:`_cleanup_temp_files`. :param s3_uri: S3 URI in the form ``s3://bucket/key``. :type s3_uri: str :param s3_client: Injected client for testing. :type s3_client: boto3 S3 client, optional :returns: Path to the local temporary file. :rtype: Path :raises ImportError: If boto3 is not installed and no client was injected. :raises ValueError: If the URI has no object key.