MPhil Research • Ongoing Project

Evaluating Error Structures & ML Bias-Correction for Satellite Precipitation Estimates

Azalea Kamellia Abdullah — Universiti Teknologi Malaysia

Executive Summary

Gridded Satellite Precipitation Estimates (SPEs) offer continuous spatial coverage crucial for hydrometeorological monitoring, flood forecasting, and drought assessment. However, in data-scarce tropical regions with complex topography, SPEs exhibit significant systematic biases due to cloud physics, mountain effects, and gauge density limitations.

This research evaluates multi-source gridded datasets—including CHIRPS, PERSIANN-CDR, and ERA5-Land—across a 40-year historical baseline (1983–2023). By combining Triple Collocation Analysis (TCA) with machine learning pipelines, this study quantifies error variance without relying solely on dense rain-gauge networks.

Problem Statement & Context

Traditional bias correction methods heavily depend on high-density, continuously operating ground rain gauge networks. In regions where gauge distribution is sparse or unevenly distributed, conventional spatial interpolation methods (e.g., IDW or Kriging) propagate spatial errors.

The objective of this project is twofold:

  1. Characterize the independent error variances of competing satellite gridded products without treating any single dataset as absolute ground truth.
  2. Develop a robust, machine-learning-based bias-correction framework to improve input reliability for downstream standardized drought index computation (e.g., SPI/SPEI).

Methodological Framework

1. Data Acquisition & Harmonization

Data extraction and spatio-temporal resampling are executed via the Google Earth Engine (GEE) Python API and custom local Python pipelines. Datasets with differing native spatial resolutions (e.g., \(0.05^\circ\) to \(0.25^\circ\)) are resampled and aligned to a standardized spatial grid before error modeling.

2. Triple Collocation Analysis (TCA)

To evaluate dataset uncertainties without assuming a perfect ground reference, Triple Collocation estimates the error variances \( \sigma_{\epsilon}^2 \) among three mutually independent triplet signals \( (X_1, X_2, X_3) \):

\[ \sigma_{\epsilon_1}^2 = \sigma_1^2 - \frac{\sigma_{12} \sigma_{13}}{\sigma_{23}} \]

Where \( \sigma_i^2 \) denotes variance and \( \sigma_{ij} \) denotes covariance between dataset pairs. This allows objective assessment of sensor performance across varied elevation zones and seasonal monsoon regimes.

3. Machine Learning Calibration Pipeline

To correct systematic rainfall distribution shifts (such as wet-day frequency oversaturation and extreme event underestimation), stratified cross-validation is used to train regression frameworks (including ensemble tree models and neural network approaches) against historical observation nodes.

Technical Stack & Libraries

Python • Google Earth Engine (GEE) API • Scikit-Learn • PyTorch • GDAL • PostGIS • Triple Collocation • MathJax

Project Status & Future Scope

Note: Detailed model benchmarks, regional error maps, and specific quantitative findings are intentionally withheld as manuscript preparation and thesis evaluations are currently ongoing.

Calibrated output arrays from this workflow will directly feed into STEMRAINCAL, a MyIPO-registered Python library developed for automated drought index derivation.