Skip to content

Add nb error multiple ds #208

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cehbrecht merged 2 commits into from
Sep 2, 2026
Merged

Add nb error multiple ds #208

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a62aee5
added error nb for multiple datasets error
cehbrecht Aug 24, 2026
d24751b
added error nb for cmip6 with mixed freqs
cehbrecht Aug 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
239 changes: 239 additions & 0 deletions notebooks/errors/cds-error-2026-08-24-cmip6-pr-multi-model-subset.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,239 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# CMIP6 daily precipitation non-monotonic time error\n",
"\n",
"When: 2026-08-24\n",
"\n",
"This notebook reproduces a reported CDS WPS error while subsetting daily precipitation (`pr`) from 42 CMIP6 historical datasets for 1970–1989 over the area `88.5,20,93,24`. The request explicitly selects every day, month, and year in the time range.\n",
"\n",
"The service returns: `Process error: Resulting object does not have monotonic global indexes along dimension time`.\n",
"\n",
"The notebook preserves the decoded JSON workflow from the WPS `ComplexData` payload and does not download or open any result data."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Original WPS workflow\n",
"\n",
"The `ComplexData` CDATA contains the following JSON request. The complete `time_components` value is assembled programmatically to keep it readable while retaining the exact request semantics."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"collection = [\n",
" \"c3s-cmip6.CMIP.AS-RCEC.TaiESM1.historical.r1i1p1f1.day.pr.gn.v20200626\",\n",
" \"c3s-cmip6.CMIP.AWI.AWI-ESM-1-1-LR.historical.r1i1p1f1.day.pr.gn.v20200212\",\n",
" \"c3s-cmip6.CMIP.BCC.BCC-CSM2-MR.historical.r2i1p1f1.day.pr.gn.v20181116\",\n",
" \"c3s-cmip6.CMIP.BCC.BCC-ESM1.historical.r1i1p1f1.day.pr.gn.v20181220\",\n",
" \"c3s-cmip6.CMIP.CAS.FGOALS-f3-L.historical.r1i1p1f1.day.pr.gr.v20191019\",\n",
" \"c3s-cmip6.CMIP.CAS.FGOALS-g3.historical.r1i1p1f1.day.pr.gn.v20190826\",\n",
" \"c3s-cmip6.CMIP.CCCR-IITM.IITM-ESM.historical.r1i1p1f1.day.pr.gn.v20191226\",\n",
" \"c3s-cmip6.CMIP.CCCma.CanESM5.historical.r1i1p1f1.day.pr.gn.v20190429\",\n",
" \"c3s-cmip6.CMIP.CMCC.CMCC-CM2-HR4.historical.r1i1p1f1.day.pr.gn.v20200904\",\n",
" \"c3s-cmip6.CMIP.CMCC.CMCC-CM2-SR5.historical.r1i1p1f1.day.pr.gn.v20200616\",\n",
" \"c3s-cmip6.CMIP.CMCC.CMCC-ESM2.historical.r1i1p1f1.day.pr.gn.v20210114\",\n",
" \"c3s-cmip6.CMIP.CNRM-CERFACS.CNRM-CM6-1-HR.historical.r1i1p1f2.day.pr.gr.v20191021\",\n",
" \"c3s-cmip6.CMIP.CNRM-CERFACS.CNRM-CM6-1.historical.r1i1p1f2.day.pr.gr.v20180917\",\n",
" \"c3s-cmip6.CMIP.CNRM-CERFACS.CNRM-ESM2-1.historical.r1i1p1f2.day.pr.gr.v20181206\",\n",
" \"c3s-cmip6.CMIP.CSIRO-ARCCSS.ACCESS-CM2.historical.r1i1p1f1.day.pr.gn.v20191108\",\n",
" \"c3s-cmip6.CMIP.CSIRO.ACCESS-ESM1-5.historical.r1i1p1f1.day.pr.gn.v20191115\",\n",
" \"c3s-cmip6.CMIP.EC-Earth-Consortium.EC-Earth3-AerChem.historical.r1i1p1f1.day.pr.gr.v20200624\",\n",
" \"c3s-cmip6.CMIP.EC-Earth-Consortium.EC-Earth3-CC.historical.r1i1p1f1.day.pr.gr.v20210113\",\n",
" \"c3s-cmip6.CMIP.EC-Earth-Consortium.EC-Earth3-Veg-LR.historical.r1i1p1f1.day.pr.gr.v20200217\",\n",
" \"c3s-cmip6.CMIP.INM.INM-CM4-8.historical.r1i1p1f1.day.pr.gr1.v20190530\",\n",
" \"c3s-cmip6.CMIP.INM.INM-CM5-0.historical.r1i1p1f1.day.pr.gr1.v20190610\",\n",
" \"c3s-cmip6.CMIP.IPSL.IPSL-CM5A2-INCA.historical.r1i1p1f1.day.pr.gr.v20200729\",\n",
" \"c3s-cmip6.CMIP.IPSL.IPSL-CM6A-LR.historical.r1i1p1f1.day.pr.gr.v20180803\",\n",
" \"c3s-cmip6.CMIP.KIOST.KIOST-ESM.historical.r1i1p1f1.day.pr.gr1.v20191202\",\n",
" \"c3s-cmip6.CMIP.MIROC.MIROC-ES2L.historical.r1i1p1f2.day.pr.gn.v20191129\",\n",
" \"c3s-cmip6.CMIP.MIROC.MIROC6.historical.r1i1p1f1.day.pr.gn.v20191016\",\n",
" \"c3s-cmip6.CMIP.MOHC.HadGEM3-GC31-LL.historical.r1i1p1f3.day.pr.gn.v20190624\",\n",
" \"c3s-cmip6.CMIP.MOHC.HadGEM3-GC31-MM.historical.r1i1p1f3.day.pr.gn.v20191207\",\n",
" \"c3s-cmip6.CMIP.MOHC.UKESM1-0-LL.historical.r1i1p1f2.day.pr.gn.v20190627\",\n",
" \"c3s-cmip6.CMIP.MPI-M.MPI-ESM1-2-HR.historical.r1i1p1f1.day.pr.gn.v20190710\",\n",
" \"c3s-cmip6.CMIP.MPI-M.MPI-ESM1-2-LR.historical.r1i1p1f1.day.pr.gn.v20190710\",\n",
" \"c3s-cmip6.CMIP.MRI.MRI-ESM2-0.historical.r1i1p1f1.day.pr.gn.v20190603\",\n",
" \"c3s-cmip6.CMIP.NCAR.CESM2-FV2.historical.r1i1p1f1.day.pr.gn.v20191120\",\n",
" \"c3s-cmip6.CMIP.NCAR.CESM2-WACCM-FV2.historical.r1i1p1f1.day.pr.gn.v20191120\",\n",
" \"c3s-cmip6.CMIP.NCAR.CESM2-WACCM.historical.r1i1p1f1.day.pr.gn.v20190415\",\n",
" \"c3s-cmip6.CMIP.NCAR.CESM2.historical.r1i1p1f1.day.pr.gn.v20190401\",\n",
" \"c3s-cmip6.CMIP.NCC.NorCPM1.historical.r1i1p1f1.day.pr.gn.v20200724\",\n",
" \"c3s-cmip6.CMIP.NCC.NorESM2-MM.historical.r1i1p1f1.day.pr.gn.v20191108\",\n",
" \"c3s-cmip6.CMIP.NIMS-KMA.KACE-1-0-G.historical.r1i1p1f1.day.pr.gr.v20190911\",\n",
" \"c3s-cmip6.CMIP.NOAA-GFDL.GFDL-ESM4.historical.r1i1p1f1.day.pr.gr1.v20190726\",\n",
" \"c3s-cmip6.CMIP.NUIST.NESM3.historical.r1i1p1f1.day.pr.gn.v20190812\",\n",
" \"c3s-cmip6.CMIP.SNU.SAM0-UNICON.historical.r1i1p1f1.day.pr.gn.v20190323\",\n",
"]\n",
"\n",
"days = \",\".join(f\"{day:02d}\" for day in range(1, 32))\n",
"months = \"jan,feb,mar,apr,may,jun,jul,aug,sep,oct,nov,dec\"\n",
"years = \",\".join(str(year) for year in range(1970, 1990))\n",
"time_components = f\"day:{days}|month:{months}|year:{years}\"\n",
"\n",
"request = {\n",
" \"inputs\": {\"pr\": collection},\n",
" \"steps\": {\n",
" \"subset_pr_1\": {\n",
" \"run\": \"subset\",\n",
" \"in\": {\n",
" \"collection\": \"inputs/pr\",\n",
" \"area\": \"88.5,20,93,24\",\n",
" \"time_components\": time_components,\n",
" \"time\": \"1970/1989\",\n",
" },\n",
" }\n",
" },\n",
" \"outputs\": {\"output\": \"subset_pr_1/output\"},\n",
" \"doc\": \"workflow\",\n",
"}\n",
"\n",
"assert len(collection) == 42\n",
"request"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Build the equivalent Rooki workflow\n",
"\n",
"Importing Rooki contacts the configured WPS service. Change `ROOK_URL` if the reproduction should run against another deployment."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import json\n",
"import os\n",
"\n",
"os.environ[\"ROOK_URL\"] = \"http://rook.dkrz.de/wps\"\n",
"\n",
"from rooki import operators as ops"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"pr = ops.Input(\"pr\", request[\"inputs\"][\"pr\"])\n",
"subset = ops.Subset(\n",
" pr,\n",
" area=request[\"steps\"][\"subset_pr_1\"][\"in\"][\"area\"],\n",
" time=request[\"steps\"][\"subset_pr_1\"][\"in\"][\"time\"],\n",
" time_components=request[\"steps\"][\"subset_pr_1\"][\"in\"][\"time_components\"],\n",
")\n",
"\n",
"serialized_request = json.loads(subset._serialise())\n",
"assert serialized_request == request\n",
"serialized_request"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Reproduce the error\n",
"\n",
"The next cell submits the full 42-dataset request. It is disabled by default because it may be expensive or trigger the reported server-side error. Set `RUN_REQUEST` to `True` only against the deployment being tested."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from time import perf_counter\n",
"\n",
"RUN_REQUEST = False\n",
"\n",
"if RUN_REQUEST:\n",
" started_at = perf_counter()\n",
" resp = subset.orchestrate()\n",
" elapsed_seconds = perf_counter() - started_at\n",
" print(f\"Orchestration time: {elapsed_seconds:.1f} seconds\")\n",
" print(f\"Succeeded: {resp.ok}\")\n",
" print(f\"Status: {resp.status}\")\n",
" display(resp)\n",
"else:\n",
" print(\"Request not submitted; set RUN_REQUEST = True to reproduce the error.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Observed failure\n",
"\n",
"The WPS response contains the following exception:\n",
"\n",
"```xml\n",
"<ows:ExceptionText>Process error: Resulting object does not have monotonic global indexes along dimension time</ows:ExceptionText>\n",
"```\n",
"\n",
"This error is raised while combining files when their decoded `time` coordinates cannot form one globally monotonic index. Possible data-level causes include overlapping or duplicate timestamps, files supplied out of temporal order, or inconsistent time encodings/calendars. The message alone does not identify which of the 42 datasets causes the failure. Running the same subset for each dataset separately is the most direct way to isolate the problematic collection before inspecting its source-file time coordinates."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Inspect a successful response without loading data\n",
"\n",
"If the workflow succeeds, inspect the Metalink size and output URLs without downloading or opening the NetCDF results."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"if RUN_REQUEST and resp.ok:\n",
" print(\n",
" f\"Total output size from Metalink: {resp.size:,} bytes \"\n",
" f\"({resp.size_in_mb:.2f} MiB / {resp.size_in_gb:.3f} GiB)\"\n",
" )\n",
" print(\"\\nOutput URLs (not downloaded):\")\n",
" for url in resp.download_urls():\n",
" print(url)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.12"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Loading
Loading