Skip to content

Add reference-design example notebook - #4

Draft
hamed wants to merge 1 commit into
mainfrom
agent/example-notebook
Draft

Add reference-design example notebook#4
hamed wants to merge 1 commit into
mainfrom
agent/example-notebook

Conversation

@hamed

@hamed hamed commented Aug 9, 2026

Copy link
Copy Markdown
Owner

Summary

  • add an executed, download-free quickstart notebook that recreates the reference design's educational bootstrap flow
  • preserve the exact toy-data contract and direct at(), box_plot(), and plot_pr() usage
  • demonstrate a single PR plot, two-model comparison, and row × column cohort facets with confidence bands
  • link the quickstart from the README
  • fix plotting facet titles and hue legends exposed by the notebook
  • support PyArrow 25's updated sort-key API in seeded Spark sampling

Why

The project grew from an educational reference notebook, but users of the installable package need the same workflow expressed through the public library API rather than copied function definitions. The quickstart preserves that teaching sequence while replacing only the network- and training-dependent Kaggle/CatBoost section with deterministic imbalanced predictions.

The supporting fixes are required for the reference calls to work on the current dependency stack: Seaborn was not collecting the box-plot hue legend, no-hue PR plots created an empty legend, one-dimensional facets used invalid title templates, and PyArrow 25 moved null placement into each sort key.

User impact

Users can run one complete example from bootstrap resampling through operating-point uncertainty, PR confidence bands, model comparison, cohort facets, and an arbitrary per-replica metric. Existing public APIs are unchanged.

Validation

  • pytest -q: 95 passed
  • ruff check .
  • ruff format --check .
  • executed examples/quickstart.ipynb end to end
  • visually inspected all seven rendered figures
  • verified the reference toy contract: totals 12/12/1, dTP=2 at 0.95, eleven PR rows, and precision=.81 selects threshold 0.88

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant