curryer.correction.verification =============================== .. py:module:: curryer.correction.verification .. autoapi-nested-parse:: Verification module for geolocation requirements compliance. Provides :func:`verify`, a standalone entry point that evaluates the current set of SPICE kernels and alignment parameters against mission geolocation requirements — without running the iterative correction loop. Typical use-cases ----------------- Weekly automated check (CLARREO) Pass pre-computed ``image_matching_results`` (the most common path): >>> result = verify(setup, image_matching_results=weekly_datasets, work_dir=work_dir) >>> if not result.passed: ... send_alert(result.summary_table) Post-correction validation After a full GCS run, verify the optimised parameter set: >>> result = verify(setup, image_matching_results=post_correction_datasets, work_dir=work_dir) One-off compliance check with in-memory geolocated data Supply an already-geolocated dataset together with a GCP chip directory and calibration files; verification auto-pairs and image-matches without any additional setup: >>> result = verify( ... setup, ... geolocated_data=raw_dataset, ... gcp_directory="data/gcps/", ... los_file="cal/b_HS.mat", ... psf_file="cal/optical_PSF_675nm.mat", ... ) Models ------ :class:`RequirementsConfig` Verification thresholds (performance limit and pass-rate). :class:`GCPError` Per-measurement/GCP error detail. :class:`VerificationResult` Structured pass/fail result; serialisable via Pydantic JSON methods. Attributes ---------- .. autoapisummary:: curryer.correction.verification.logger Classes ------- .. autoapisummary:: curryer.correction.verification.GCPError curryer.correction.verification.VerificationResult Functions --------- .. autoapisummary:: curryer.correction.verification.image_matching curryer.correction.verification.match_geolocated_to_gcp_files curryer.correction.verification.verify curryer.correction.verification.compare_results Module Contents --------------- .. py:data:: logger .. py:class:: GCPError(/, **data: Any) Bases: :py:obj:`pydantic.BaseModel` Per-measurement/GCP error detail. Each instance corresponds to one row in the aggregated image-matching output — typically one measurement from a single GCP pair. .. attribute:: gcp_index Zero-based measurement index in the aggregated dataset. :type: int .. attribute:: science_key Identifier for the science data segment (dataset label or index). :type: str .. attribute:: gcp_key Identifier for the ground-control-point source. :type: str .. attribute:: lat_error_deg Latitude error in degrees (positive = northward shift). :type: float .. attribute:: lon_error_deg Longitude error in degrees (positive = eastward shift). :type: float .. attribute:: nadir_equiv_error_m Nadir-equivalent total geolocation error in meters, or ``None`` when error-stats processing was not performed. :type: float or None .. attribute:: correlation Image-matching correlation score, or ``None`` when not available. :type: float or None .. attribute:: passed Whether this measurement satisfies the per-measurement threshold. :type: bool .. py:attribute:: model_config Configuration for the model, should be a dictionary conforming to [`ConfigDict`][pydantic.config.ConfigDict]. .. py:attribute:: gcp_index :type: int .. py:attribute:: science_key :type: str .. py:attribute:: gcp_key :type: str .. py:attribute:: lat_error_deg :type: float .. py:attribute:: lon_error_deg :type: float .. py:attribute:: nadir_equiv_error_m :type: float | None :value: None .. py:attribute:: correlation :type: float | None :value: None .. py:attribute:: passed :type: bool .. py:class:: VerificationResult(/, **data: Any) Bases: :py:obj:`pydantic.BaseModel` Structured result from a :func:`verify` call. Most fields are JSON-serialisable via Pydantic's ``model_dump()`` / ``model_dump_json()``. The :attr:`aggregate_stats` field (an ``xr.Dataset``) must be excluded when serialising to JSON — persist it separately (e.g. via ``aggregate_stats.to_netcdf(path)``):: json_str = result.model_dump_json(exclude={"aggregate_stats"}) result.aggregate_stats.to_netcdf("verification_stats.nc") .. attribute:: passed ``True`` when :attr:`percent_within_threshold` ≥ :attr:`requirements.performance_spec_percent`. :type: bool .. attribute:: per_gcp_errors One entry per measurement in the aggregated dataset. :type: list[GCPError] .. attribute:: aggregate_stats Full output from :meth:`~curryer.correction.error_stats.ErrorStatsProcessor.process_geolocation_errors`. :type: xr.Dataset .. attribute:: requirements The thresholds used for this verification run. :type: RequirementsConfig .. attribute:: summary_table Human-readable ASCII table suitable for logging or reports. :type: str .. attribute:: percent_within_threshold Percentage of measurements with nadir-equivalent error below :attr:`requirements.performance_threshold_m`. :type: float .. attribute:: warnings Non-empty when :attr:`passed` is ``False``. :type: list[str] .. attribute:: timestamp UTC wall-clock time when :func:`verify` was called. :type: datetime .. attribute:: files_processed Science/GCP key pairs that were processed, as ``"+"`` strings. Empty when the source mapping is unavailable. :type: list[str] .. attribute:: elapsed_time_s Wall-clock time for the verify call in seconds, or ``None`` when not measured. :type: float or None .. attribute:: config_snapshot Key config fields used for this run (threshold, spec percent, instrument name), for reproducibility records. :type: dict or None .. py:attribute:: model_config Configuration for the model, should be a dictionary conforming to [`ConfigDict`][pydantic.config.ConfigDict]. .. py:attribute:: passed :type: bool .. py:attribute:: per_gcp_errors :type: list[GCPError] .. py:attribute:: aggregate_stats :type: xarray.Dataset .. py:attribute:: requirements :type: curryer.correction.config.RequirementsConfig .. py:attribute:: summary_table :type: str .. py:attribute:: percent_within_threshold :type: float .. py:attribute:: warnings :type: list[str] .. py:attribute:: timestamp :type: datetime.datetime .. py:attribute:: files_processed :type: list[str] :value: None .. py:attribute:: elapsed_time_s :type: float | None :value: None .. py:attribute:: config_snapshot :type: dict | None :value: None .. py:function:: image_matching(geolocated_data: xarray.Dataset, gcp_reference_file: pathlib.Path, telemetry: pandas.DataFrame | None = None, params_info: list | None = None, setup: curryer.correction.config.GeolocationSetup | None = None, los_vectors_cached: numpy.ndarray | None = None, optical_psfs_cached: list | None = None, r_iss_midframe: numpy.ndarray | None = None) -> xarray.Dataset Image matching using :func:`~curryer.correction.image_match.integrated_image_match`. Performs image correlation between geolocated pixels and a Landsat GCP reference image to measure geolocation error. This function is the single implementation used by both the correction loop (:func:`~curryer.correction.pipeline.loop`) and standalone verification (:func:`verify`). ``pipeline.py`` imports it from here. :param geolocated_data: Geolocation output with ``latitude``, ``longitude``, and a ``frame`` coordinate (GPS seconds = ``ugps_times / 1e6``). :type geolocated_data: xr.Dataset :param gcp_reference_file: Path to GCP reference image (``.mat`` or ``.nc``). :type gcp_reference_file: Path :param telemetry: Telemetry DataFrame with spacecraft state. Required when *r_iss_midframe* is not supplied. :type telemetry: pd.DataFrame or None, optional :param params_info: Current parameter values for error tracking. Defaults to ``[]``. :type params_info: list or None, optional :param setup: Setup for coordinate names, calibration paths, and instrument metadata. :type setup: GeolocationSetup or None, optional :param los_vectors_cached: Pre-loaded LOS vectors. :type los_vectors_cached: np.ndarray or None, optional :param optical_psfs_cached: Pre-loaded optical PSF entries. :type optical_psfs_cached: list or None, optional :param r_iss_midframe: Spacecraft ECEF position in meters at mid-frame. When provided, *telemetry* is not consulted for position. :type r_iss_midframe: np.ndarray of shape (3,) or None, optional :returns: Error measurements: ``lat_error_deg``, ``lon_error_deg``, and metadata. :rtype: xr.Dataset :raises ValueError: If neither *telemetry* nor *r_iss_midframe* is supplied, or if calibration files are missing. .. py:function:: match_geolocated_to_gcp_files(geolocated_data: xarray.Dataset, gcp_files: list[pathlib.Path], setup: curryer.correction.config.GeolocationSetup, los_vectors_cached: numpy.ndarray | None = None, optical_psfs_cached: list | None = None) -> list[xarray.Dataset] Run image matching between already-geolocated data and GCP reference files. This is the reusable *pipeline tail*: both the correction loop (after kernel tweaking and geolocation) and standalone :func:`verify` call this function. Given geolocated data and a list of GCP reference files it performs image matching against each file and returns the per-GCP error datasets ready for aggregation and error-stats processing. :param geolocated_data: Geolocated observation dataset with ``latitude``, ``longitude``, and a ``frame`` coordinate (GPS seconds). :type geolocated_data: xr.Dataset :param gcp_files: GCP reference files to match against. :type gcp_files: list of Path :param setup: Mission setup (calibration paths, variable names, instrument name). :type setup: GeolocationSetup :param los_vectors_cached: Pre-loaded LOS vectors. :type los_vectors_cached: np.ndarray or None, optional :param optical_psfs_cached: Pre-loaded optical PSF entries. :type optical_psfs_cached: list or None, optional :returns: One error dataset per successfully matched GCP file. Failures are logged as warnings and skipped. :rtype: list of xr.Dataset .. py:function:: verify(setup: curryer.correction.config.GeolocationSetup, gcp_pairs: list[tuple[str | pathlib.Path, str | pathlib.Path]] | None = None, observation_paths: list[str | pathlib.Path] | None = None, gcp_directory: str | pathlib.Path | None = None, los_file: str | pathlib.Path | None = None, psf_file: str | pathlib.Path | None = None, max_distance_m: float = 0.0, gcp_pattern: str = '*_regridded.nc', default_altitude_m: float = 400000.0, image_matching_results: list[xarray.Dataset] | None = None, geolocated_data: xarray.Dataset | None = None, work_dir: pathlib.Path | None = None) -> VerificationResult Evaluate current alignment against mission requirements. No parameter variation, no kernel creation, no iteration loop. This function checks whether a **given** set of alignment parameters meets geolocation requirements. Input priority (first match wins) ---------------------------------- 1. *image_matching_results* — pre-computed outputs from image matching; the most common entry point for weekly automated checks. 2. *geolocated_data* — raw geolocated data. Either set ``setup.image_matching_func`` for a custom matcher, or supply *gcp_directory*, *los_file*, and *psf_file* to run built-in spatial pairing + image matching. 3. *gcp_pairs* — explicit ``(observation_path, gcp_path)`` file-path pairs. Requires *los_file* and *psf_file*. 4. *observation_paths* + *gcp_directory* — auto-paired via spatial overlap. Requires *los_file* and *psf_file*. 5. None of the above provided — raises :class:`ValueError`. :param setup: Mission setup with all geolocation/calibration settings. :type setup: GeolocationSetup :param gcp_pairs: Explicit ``(observation_path, gcp_path)`` pairs. Each path may be a local path or an ``s3://`` URI (requires ``boto3``). :type gcp_pairs: list of (path, path) or None :param observation_paths: Observation file paths for automatic GCP pairing. Requires *gcp_directory*, *los_file*, and *psf_file*. :type observation_paths: list of path or None :param gcp_directory: Directory of GCP reference images for automatic pairing with *observation_paths*. :type gcp_directory: path or None :param los_file: Instrument line-of-sight vectors (``.mat`` file). Required when *gcp_pairs* or *observation_paths* is provided. :type los_file: path or None :param psf_file: Optical PSF ``.mat`` file. Required when *gcp_pairs* or *observation_paths* is provided. :type psf_file: path or None :param max_distance_m: Spatial pairing margin for the auto-pair mode (default ``0.0`` — GCP center must be inside the observation footprint). :type max_distance_m: float, optional :param gcp_pattern: Glob pattern used to discover GCP chips when *gcp_directory* is provided. Defaults to ``"*_regridded.nc"``. :type gcp_pattern: str, optional :param default_altitude_m: Fallback spacecraft altitude (meters) used when observation files do not contain position data. Default ``400_000.0`` (ISS nominal orbit). Override for other platforms (e.g. ``505_000.0`` for CTIM). :type default_altitude_m: float, optional :param image_matching_results: Pre-computed image-matching datasets, one per GCP pair. :type image_matching_results: list[xr.Dataset] or None :param geolocated_data: Already-geolocated data. Matched either via ``setup.image_matching_func`` (custom override) or, when *gcp_directory*, *los_file*, and *psf_file* are supplied, via built-in spatial pairing + image matching. :type geolocated_data: xr.Dataset or None :param work_dir: Working directory for outputs. Created if absent. :type work_dir: Path or None, optional :returns: Structured pass/fail result with per-GCP detail and a human-readable :attr:`~VerificationResult.summary_table`. :rtype: VerificationResult :raises ValueError: When none of the input modes is provided; when *geolocated_data* is supplied without *gcp_directory* / *los_file* / *psf_file* and ``setup.image_matching_func`` is not set; when *los_file* or *psf_file* is ``None`` for a file-path mode (*gcp_pairs* or *observation_paths* + *gcp_directory*); when *observation_paths* and *gcp_directory* are not both supplied; or when image matching produces no results. :raises FileNotFoundError: If any of the supplied file paths do not exist. .. py:function:: compare_results(before: VerificationResult, after: VerificationResult) -> str Generate a side-by-side comparison of two verification results. Useful for evaluating whether a correction run improved geolocation accuracy relative to a baseline. :param before: Baseline verification result (e.g., pre-correction). :type before: VerificationResult :param after: Updated verification result (e.g., post-correction). :type after: VerificationResult :returns: Human-readable side-by-side comparison table. :rtype: str