Skip to content

Add validate-sfile, a checker for SW4 velocity models - #137

Open
lispandfound wants to merge 6 commits into
nzvm/config-plumbingfrom
nzvm/sfile-validator
Open

Add validate-sfile, a checker for SW4 velocity models#137
lispandfound wants to merge 6 commits into
nzvm/config-plumbingfrom
nzvm/sfile-validator

Conversation

@lispandfound

@lispandfound lispandfound commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Adds an sfile validator, which runs a number of checks over an SW4 velocity model, namely:

  • All required attributes and groups exist, and the declared ngrid count matches the number of supplied grids.
  • The types of the datasets match what SW4 is expecting to read. This is more important that it looks because SW4 inherits the C-style coding culture of never checking anything. So, if a double is read into a float memory buffer, it will silently write into garbage and stack smash.
  • NaN and inf checks for all arrays.
  • All interfaces between mesh refinements are checked to ensure they are in increasing order of depth and don't clip each other, or that the topographic gradients are not too high.
  • Check that the material density is greater than zero.
  • Check Vp, Vs, Qp, Qs > 0 and Vp/Vs >= 1.
  • That each grid is at least two gridpoints thick.
  • That the grid corners of the velocity model align with geographic coordinates.

@lispandfound
lispandfound force-pushed the nzvm/sfile-validator branch 3 times, most recently from 5bb11e3 to 9d5f569 Compare September 8, 2026 22:56
@lispandfound

Copy link
Copy Markdown
Contributor Author

@claude review

@claude

claude Bot commented Sep 8, 2026

Copy link
Copy Markdown

Claude encountered an error —— View job


I'll analyze this and get back to you.

@lispandfound

Copy link
Copy Markdown
Contributor Author

@claude review

@claude

claude Bot commented Sep 8, 2026

Copy link
Copy Markdown

Claude finished @lispandfound's task in 2m 35s —— View job


Review complete

  • Gather context (diff, repo guidelines)
  • Review workflow/scripts/validate_sfile.py for correctness bugs
  • Check pyproject.toml change
  • Post findings as inline comments
  • Post summary comment

Findings: 2 (High: 2, Medium: 0, Low: 0)

  1. Unhandled crash on malformed Z_interfaces datasets (_check_cross, ~line 649) — re-reads interface datasets directly instead of reusing the already-validated arrays dict, so a non-2D interface (already flagged as an error upstream) crashes the whole script with an unhandled ValueError instead of a clean failure report.
  2. Dataset dtype is never actually checked (_check_dataset, ~line 514) — the PR description's headline motivation is catching dtype mismatches (double read into a float buffer → stack smash), but no code anywhere inspects ds.dtype; only value ranges/NaN/Inf are checked, so this specific failure mode currently passes validation silently.

pyproject.toml change (new validate-sfile entry point) looks correct, no issues there.

Comment thread workflow/scripts/validate_sfile.py Outdated
Comment thread workflow/scripts/validate_sfile.py Outdated
An sfile that SW4 accepts at load time can still crash it thousands of
steps in, or -- worse -- run to completion having quietly solved the
wrong problem. The failures that cost the most time are all detectable
up front from the file alone:

  Structure    required groups/attributes, dataset counts match ngrids
  Attributes   types, value ranges, NaN/Inf
  Z_interfaces strict monotonicity at every (i, j), zero/negative
               thickness layers (which trigger an hv=0 division in
               SW4), large topographic gradients, agreement between the
               Min/max depth attribute and the data
  Material     Rho > 0 (zero density is the most common SW4 crash),
               Cp > 0, Cs >= 0, Qp/Qs > 0 under attenuation, Vp/Vs >= 1,
               nk >= 2 (nk=1 gives 0*inf=NaN in SW4's interpolation)
  Boundaries   grid corners in geographic coordinates

Interfaces stored at different resolutions are nearest-resampled onto a
common grid rather than skipped, because crossed and pinched layers
cluster at exactly the grid-refinement boundaries where the resolutions
differ. Nearest, not bilinear: it preserves every stored node value, so
a crossing sitting on a single coarse node is never smoothed away.

Material datasets are scanned in row chunks -- they do not fit in
memory for any real domain.

Standalone: imports nothing from `workflow`, and exits non-zero if any
check reports an error, so it can gate a run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lispandfound

Copy link
Copy Markdown
Contributor Author

@claude review

@claude

claude Bot commented Sep 10, 2026

Copy link
Copy Markdown

Claude finished @lispandfound's task in 2m 18s —— View job


Review complete

  • Gather context (diff, prior review comments)
  • Check if previously flagged issues were addressed
  • Review full diff for new correctness bugs
  • Post findings

The file has been substantially rewritten since the last review (798 → 1668 lines). Both previously flagged issues are resolved:

  • The _check_cross/_check_z_interfaces crash on malformed Z_interfaces is gone — the new Sfile.layers/interface_arrays cached properties consistently filter out non-2D interfaces before any thickness/ordering math runs.
  • Dtype checking is still not implemented, but this was already flagged in the previous review round and remains unaddressed, so not re-reported here per instructions.

New findings: 1 (High: 1)

  1. Zero Vs is now flagged as an error, contradicting the tool's own zero_allowed design (_report_variable, workflow/scripts/validate_sfile.py:1215-1231) — Cs's VarSpec sets zero_allowed=True and documents zero as legal ("as it is for Vs in fluid cells"), but the zero_allowed branch yields Finding.error for n_zero instead of a warning. This is a regression from the pre-refactor code, which correctly warned rather than errored on zero Vs. As written, any legitimate model with fluid-cell layers (Vs=0) will now fail validation outright.

Posted as an inline comment with a suggested fix.

Comment thread workflow/scripts/validate_sfile.py Outdated
lispandfound and others added 2 commits September 10, 2026 17:10
The tests hand read_sfile a synthetic tree -- a dict carrying an attrs
mapping, with numpy arrays for datasets -- rather than an HDF5 file, since
that is the whole surface the validator uses. One test writes the same trees
out with h5py and checks both routes report the same findings, so the stand-in
cannot quietly drift.

Files are grouped by what the validator owes the user: BROKEN models must be
rejected, SUSPICIOUS ones must warn without being rejected, and INCOMPLETE
ones must be skipped rather than crashed on. Assertions are on severity, never
on message wording, which is rendering rather than contract.

_describe_corners only guarded the forward projection. A non-finite azimuth,
or a non-finite grid spacing via the extent, projects an origin fine but gives
NaN corners, which nztm_to_wgs_depth rejects -- so the ValueError escaped the
check and validate_path reported it as "cannot read sfile", losing every other
finding. Both projections are now inside the try, and the skip covers it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T1bWi7ju7iw669gKEqkcDT
@lispandfound
lispandfound marked this pull request as ready for review September 10, 2026 20:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant