curryer.correction.image_io =========================== .. py:module:: curryer.correction.image_io .. autoapi-nested-parse:: Image I/O utilities for correction pipeline. Provides format-agnostic loading and saving of image data used in the correction pipeline. All format-specific logic is in private helpers; the public API dispatches on file extension so callers never need to know the underlying format. Supported formats ----------------- ``.mat`` — MATLAB struct files (calibration and legacy test data). ``.nc`` / ``.netcdf`` / ``.nc4`` — NetCDF (regridded GCPs, outputs). ``.hdf`` / ``.h5`` — HDF4/5 raw GCP chips; use :func:`load_gcp_chip_from_hdf` then ``curryer.correction.regrid`` to convert ECEF to an ImageGrid. Public API (9 functions) ------------------------ :func:`load_image_grid` — any image file → :class:`ImageGrid` :func:`load_named_image_grid` — any image file → :class:`NamedImageGrid` :func:`load_observation_file` — observation + spacecraft position :func:`load_los_vectors` — LOS unit vectors from calibration file :func:`load_optical_psf` — PSF entries from calibration file :func:`load_gcp_chip_from_hdf` — raw HDF chip (band + ECEF arrays) :func:`save_image_grid` — write ImageGrid; format from extension :func:`infer_spacecraft_state` — derive boresight/t_matrix from position :func:`geolocated_to_image_grid` — convert geolocated xr.Dataset → :class:`ImageGrid` Attributes ---------- .. autoapisummary:: curryer.correction.image_io.logger curryer.correction.image_io._MAT_EXTS curryer.correction.image_io._NC_EXTS curryer.correction.image_io._TIFF_EXTS Functions --------- .. autoapisummary:: curryer.correction.image_io._load_mat_image_grid curryer.correction.image_io.load_image_grid curryer.correction.image_io.load_optical_psf curryer.correction.image_io.load_los_vectors curryer.correction.image_io._load_netcdf_image_grid curryer.correction.image_io._save_to_netcdf curryer.correction.image_io.load_gcp_chip_from_hdf curryer.correction.image_io.save_image_grid curryer.correction.image_io._save_to_mat curryer.correction.image_io._save_to_geotiff curryer.correction.image_io.load_named_image_grid curryer.correction.image_io.load_observation_file curryer.correction.image_io.infer_spacecraft_state curryer.correction.image_io.geolocated_to_image_grid Module Contents --------------- .. py:data:: logger .. py:data:: _MAT_EXTS .. py:data:: _NC_EXTS .. py:data:: _TIFF_EXTS .. py:function:: _load_mat_image_grid(filepath: pathlib.Path, key: str) -> curryer.correction.grid_types.ImageGrid Load an :class:`ImageGrid` from a MATLAB ``.mat`` struct. :param filepath: Resolved local path to the ``.mat`` file. :type filepath: Path :param key: MATLAB struct key (e.g. ``"subimage"``, ``"GCP"``). :type key: str :rtype: ImageGrid :raises KeyError: If *key* is not present in the file. .. py:function:: load_image_grid(filepath: pathlib.Path | str, mat_key: str = 'subimage') -> curryer.correction.grid_types.ImageGrid Load any supported image file as an :class:`ImageGrid`. Dispatches on file extension — callers do not need to know the underlying format. :param filepath: Path to the image file. Local paths and ``s3://`` URIs supported (S3 requires ``boto3``). :type filepath: path-like :param mat_key: MATLAB struct key. Defaults to ``"subimage"``. Ignored for NetCDF. :type mat_key: str, optional :rtype: ImageGrid :raises ValueError: If the file extension is not recognised. :raises FileNotFoundError: If *filepath* does not exist. .. rubric:: Examples >>> obs = load_image_grid(Path("subimage.mat"), mat_key="subimage") >>> gcp = load_image_grid(Path("GCP12055_regridded.nc")) .. py:function:: load_optical_psf(mat_file: str | pathlib.Path, key: str = 'PSF_struct_675nm') -> list[curryer.correction.grid_types.OpticalPSFEntry] Load optical PSF entries from a ``.mat`` calibration file. :param mat_file: Path to MATLAB file with PSF data (local path or ``s3://`` URI). :type mat_file: str or Path :param key: Primary key to try for PSF data. :type key: str, default="PSF_struct_675nm" :returns: Optical PSF samples with data, x, and field_angle arrays. :rtype: list[OpticalPSFEntry] :raises FileNotFoundError: If mat_file is a local path and doesn't exist. :raises ImportError: If mat_file is an S3 URI and boto3 is not installed. :raises KeyError: If no PSF data found with common key names. :raises ValueError: If PSF entries missing field angle attribute. .. py:function:: load_los_vectors(mat_file: str | pathlib.Path, key: str = 'b_HS') -> numpy.ndarray Load line-of-sight unit vectors from a ``.mat`` calibration file. :param mat_file: Path to MATLAB file with LOS vectors (local path or ``s3://`` URI). :type mat_file: str or Path :param key: Primary key to try for LOS data. :type key: str, default="b_HS" :returns: LOS unit vectors in instrument frame, shape (n_pixels, 3). :rtype: np.ndarray :raises FileNotFoundError: If mat_file is a local path and doesn't exist. :raises ImportError: If mat_file is an S3 URI and boto3 is not installed. :raises KeyError: If no LOS vectors found with common key names. .. py:function:: _load_netcdf_image_grid(filepath: pathlib.Path, band_var: str = 'band_data', lat_var: str = 'lat', lon_var: str = 'lon', height_var: str = 'h') -> curryer.correction.grid_types.ImageGrid Load an :class:`ImageGrid` from a NetCDF file (private helper). Called by :func:`load_image_grid` and :func:`load_named_image_grid`. Handles both regular (1-D lat/lon) and irregular (2-D lat/lon) grids; 1-D coordinate arrays are broadcast to full 2-D. :param filepath: Path to a NetCDF file. :type filepath: Path :param band_var: Band/radiance variable name. Falls back to ``"data"`` for legacy files. :type band_var: str, optional :param lat_var: Latitude variable name. Falls back to ``"latitude"``. :type lat_var: str, optional :param lon_var: Longitude variable name. Falls back to ``"longitude"``. :type lon_var: str, optional :param height_var: Height variable name. Optional — ``h`` is ``None`` when absent. :type height_var: str, optional :rtype: ImageGrid :raises FileNotFoundError: If *filepath* does not exist. :raises KeyError: If a required variable (band, lat, or lon) is not found. .. py:function:: _save_to_netcdf(filepath: pathlib.Path, image_grid: curryer.correction.grid_types.ImageGrid, metadata: dict[str, str] | None = None, compression: bool = True, band_units: str = 'DN', band_long_name: str = 'regridded_radiance') -> None Save *image_grid* to a CF-1.8 NetCDF file (private — called from :func:`save_image_grid`). The coordinate (lat/lon/h) and CRS attributes are fixed CF-1.8 / WGS84 standards and are not configurable. The band variable describes the data itself, so its ``units`` and ``long_name`` are overridable; file-level attributes are set via *metadata*. :param filepath: Output NetCDF file path. :type filepath: Path :param image_grid: :type image_grid: ImageGrid :param metadata: Additional global attributes (override the defaults, e.g. ``"title"``). :type metadata: dict, optional :param compression: Enable zlib compression. :type compression: bool, default=True :param band_units: ``units`` attribute for the band data variable (e.g. ``"W m-2 sr-1 um-1"`` for calibrated radiance). :type band_units: str, default="DN" :param band_long_name: ``long_name`` attribute for the band data variable. :type band_long_name: str, default="regridded_radiance" .. py:function:: load_gcp_chip_from_hdf(filepath: pathlib.Path, band_name: str = 'Band_1', coord_names: tuple[str, str, str] = ('ECR_x_coordinate_array', 'ECR_y_coordinate_array', 'ECR_z_coordinate_array')) -> tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray, numpy.ndarray] Load raw GCP chip data from HDF file (Landsat format). Supports both HDF4 and HDF5 formats. Tries HDF4 first (Landsat standard), then falls back to HDF5 if needed. :param filepath: Path to HDF file containing GCP chip data. :type filepath: Path :param band_name: Name of the dataset containing band/radiometric data. :type band_name: str, default="Band_1" :param coord_names: Names of X, Y, Z coordinate datasets (ECEF coordinates in meters). :type coord_names: tuple[str, str, str], default=("ECR_x_coordinate_array", ...) :returns: * **band_data** (*np.ndarray*) -- 2D array of radiometric values, shape (nrows, ncols). * **ecef_x, ecef_y, ecef_z** (*np.ndarray*) -- 2D arrays of ECEF coordinates in meters, each shape (nrows, ncols). :raises FileNotFoundError: If filepath doesn't exist. :raises KeyError: If required datasets not found in HDF file. :raises ValueError: If array shapes are inconsistent. .. rubric:: Examples >>> band, x, y, z = load_gcp_chip_from_hdf("LT08CHP.20140803.hdf") >>> band.shape (1400, 1400) .. py:function:: save_image_grid(filepath: pathlib.Path, image_grid: curryer.correction.grid_types.ImageGrid, metadata: dict | None = None, compression: bool = True, band_units: str = 'DN', band_long_name: str = 'regridded_radiance') -> None Save an :class:`ImageGrid` to file. The output format is determined by the file extension: * ``.nc`` / ``.netcdf`` / ``.nc4`` → CF-1.8 NetCDF (via ``netCDF4``) * ``.mat`` → MATLAB struct (key ``"GCP"``) * ``.tif`` / ``.tiff`` → GeoTIFF (requires ``rasterio``) :param filepath: Output file path. Extension controls the format. :type filepath: Path :param image_grid: Image data with lat/lon coordinates. :type image_grid: ImageGrid :param metadata: Additional metadata written as file-level attributes or tags. :type metadata: dict, optional :param compression: Enable compression for NetCDF output (default ``True``). Ignored for other formats. :type compression: bool, optional :param band_units: ``units`` / ``long_name`` for the band data variable in NetCDF output (defaults describe a regridded Landsat GCP chip in digital numbers). Ignored for other formats. :type band_units: str, optional :param band_long_name: ``units`` / ``long_name`` for the band data variable in NetCDF output (defaults describe a regridded Landsat GCP chip in digital numbers). Ignored for other formats. :type band_long_name: str, optional :raises ValueError: If the file extension is not supported. .. rubric:: Examples >>> save_image_grid(Path("chip.nc"), regridded, metadata={"band": "red"}) >>> save_image_grid(Path("chip.mat"), regridded) .. py:function:: _save_to_mat(filepath: pathlib.Path, image_grid: curryer.correction.grid_types.ImageGrid, metadata: dict | None) -> None Save ImageGrid to MATLAB .mat file (private helper). .. py:function:: _save_to_geotiff(filepath: pathlib.Path, image_grid: curryer.correction.grid_types.ImageGrid, metadata: dict | None) -> None Save ImageGrid to GeoTIFF file (private helper). .. py:function:: load_named_image_grid(filepath: pathlib.Path | str, mat_key: str = 'subimage') -> curryer.correction.grid_types.NamedImageGrid Load any supported image file as a :class:`NamedImageGrid`. Dispatches on file extension so callers do not need to know the underlying format. The returned grid always carries the file path as its ``name``. Supported formats ----------------- ``.mat`` MATLAB struct file. The struct accessed via *mat_key* must have ``data``, ``lat``, and ``lon`` attributes (and optionally ``h``). ``.nc`` / ``.netcdf`` / ``.nc4`` NetCDF file written by :func:`save_image_grid` or the regridding pipeline (expects ``band_data``, ``lat``, ``lon``). :param filepath: Path to the image file. :type filepath: path-like :param mat_key: MATLAB struct key to read. Defaults to ``"subimage"``. Ignored for NetCDF files. :type mat_key: str, optional :returns: Loaded image grid with ``name`` set to the string representation of *filepath*. :rtype: NamedImageGrid :raises ValueError: If the file extension is not recognised. :raises FileNotFoundError: If *filepath* does not exist. .. rubric:: Examples >>> obs = load_named_image_grid(Path("TestCase1a_subimage.mat"), mat_key="subimage") >>> gcp = load_named_image_grid(Path("GCP12055Dili_regridded.nc")) .. py:function:: load_observation_file(filepath: str | pathlib.Path, mat_key: str = 'subimage') -> tuple[curryer.correction.grid_types.ImageGrid, numpy.ndarray | None] Load one observation file and return ``(ImageGrid, spacecraft_position_m)``. Supports ``.mat`` and NetCDF (``.nc``, ``.nc4``, ``.netcdf``) formats. The spacecraft ECEF position is extracted when available in the file; when absent, ``None`` is returned and callers should fall back to :func:`infer_spacecraft_state`. :param filepath: Local path or ``s3://`` URI to a ``.mat`` or ``.nc`` observation file. :type filepath: str or Path :param mat_key: MATLAB struct key containing the image data. Defaults to ``"subimage"``. Ignored for NetCDF files. :type mat_key: str, optional :returns: * **grid** (*ImageGrid*) -- Radiance data on a lat/lon grid. * **r_spacecraft_m** (*ndarray of shape (3,) or None*) -- Spacecraft ECEF position in meters at the mid-frame, extracted from the file when available. ``None`` when not present — caller should use :func:`infer_spacecraft_state` to approximate. :raises ValueError: If the file extension is not recognised. :raises FileNotFoundError: If *filepath* is a local path that does not exist. :raises ImportError: If *filepath* is an S3 URI and ``boto3`` is not installed. .. py:function:: infer_spacecraft_state(grid: curryer.correction.grid_types.ImageGrid, r_spacecraft_m: numpy.ndarray | None, default_altitude_m: float = 400000.0) -> tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray] Deprecated — use ``resolve_spacecraft_ecef`` from ``curryer.correction.psf`` instead. .. py:function:: geolocated_to_image_grid(geo_dataset: xarray.Dataset) -> curryer.correction.grid_types.ImageGrid Convert a geolocated ``xr.Dataset`` to an :class:`ImageGrid` for image matching. This is the canonical adapter between SPICE geolocation output and the image-matching subsystem. It should be used wherever an :class:`ImageGrid` is needed from an already-geolocated dataset, ensuring a single consistent implementation across the correction pipeline and the verification module. :param geo_dataset: Geolocated dataset with ``latitude`` and ``longitude`` variables (2-D arrays) and optionally ``altitude``/``height``, ``radiance``, or ``reflectance``. :type geo_dataset: xr.Dataset :returns: Image grid with ``lat``, ``lon``, ``h`` from the dataset and ``data`` set to the first available of ``radiance``, ``reflectance``, or an all-ones placeholder. :rtype: ImageGrid .. rubric:: Notes Altitude/height defaults to an all-zeros array when absent. The placeholder ``data`` array (all ones) is used only for geometric operations where actual radiance values are not yet required.