Run real-data code on virtual data · Learn

Earlier lessons ran the method on toy arrays. This one runs the real access idioms — xr.open_zarr(...), earthaccess.login()/search_data()/open(), even s3:// URLs — except they resolve to synthetic data generated in your browser. The code is identical to what you'd run against the real archive; only the bytes are fake (and clearly labelled).

Why this is the right kind of fakeWe didn't simplify the code (that would teach you the wrong thing). We simulated the storage: a virtual Zarr store (a real Zarr, backed by an in-browser dict) and a virtual S3 / earthaccess shim. So the muscle memory you build here is the real muscle memory — swap the shim for a real login and the same code hits NASA.

1 · Virtual Zarr — open a "cloud" store

import earthschool installs the shims; then an ordinary open_zarr on an s3:// URL returns a synthetic dataset shaped exactly like real GPM IMERG:

editable · runs in your browser

import earthschool          # registers the virtual S3 + Zarr layer
import xarray as xr

# This is REAL code — the same call you'd make against a cloud Zarr store —
# but Earth Data School routes it to a synthetic, planted-signal dataset:
ds = xr.open_zarr("s3://nasa-imerg/GPM_3IMERGM.zarr")
print(ds)
print("variable:", list(ds.data_vars), "| units:", ds["precipitation"].attrs.get("units"))

2 · The full earthaccess idiom → a verified trend

The login → search → open flow you'd write for real, then the unchanged area-weighted Theil–Sen trend — and it recovers the planted truth:

editable · runs in your browser

import earthschool, numpy as np, pymannkendall as mk
import earthaccess, xarray as xr

# the exact real-world idiom — login, search, open — running on virtual data:
earthaccess.login(strategy="netrc")
granules = earthaccess.search_data(short_name="GPM_3IMERGM")
ds = xr.open_zarr(earthaccess.open(granules)[0]).load()   # open lazily, then load into memory

# ...and your real analysis runs UNCHANGED:
w = np.cos(np.deg2rad(ds.lat))
annual = ds["precipitation"].groupby("time.year").sum()
series = annual.weighted(w).mean(dim=["lat", "lon"])
r = mk.original_test(series.values)
print(f"recovered trend: {r.slope:+.2f} mm/yr   (p={r.p:.3f})")
print(f"planted truth:   {earthschool.planted('GPM_3IMERGM'):+.2f} mm/yr")
print("swap the short_name for MOD13A2, MOD11A2, GRACE, SPL3SMP, VNP46A2, OCO2_GEOS_L3CO2…")

What's wired

Every dataset in the catalog — ~3,400 of them — is wired. Pass any short_name to earthschool.dataset("…") (or any s3://…/<short_name>.zarr) and you get a synthetic dataset with a plausible variable, units, seasonality and planted trend inferred from that dataset's catalog metadata. The most-taught products — GPM IMERG, MODIS NDVI & LST, SMAP, GRACE, Black Marble, OCO-2, TROPOMI, ERA5 — have hand-tuned specs; the rest are inferred by theme + keywords (so the shape is right even if the exact behaviour is approximate). Everything is synthetic and labelled NOT real measurements — for learning the workflow, then pointing the identical code at the real archive.

editable · runs in your browser

import earthschool, xarray as xr
# any catalog short_name works — here NASA's MUR sea-surface temperature:
ds = xr.open_zarr("s3://podaac/MUR-JPL-L4-GLOB-v4.1.zarr").load()
v = list(ds.data_vars)[0]
print(ds[v].attrs.get("units"), "| variable:", v)
print("planted trend:", earthschool.planted("MUR-JPL-L4-GLOB-v4.1"), "per year")

The honest boundarySynthetic ≠ real: the shapes, units and idioms are faithful, but the values are invented (with a known signal so you can self-check). Use this to learn and debug the workflow; trust only real data for real answers — and then cross-check it.