Presence verification workflow from the taxa view

[mihow]

Presence verification workflow from the taxa view — a one-click drilldown from a taxon row to an identification modal over that taxon's most relevant occurrence, designed for sweeping unverified taxa quickly to produce "verified presence" data per project / station / session / capture set.

Follow-up to #1316 / #1317 (per-taxon verified_count + verified=true|false filter, shipped) and adjacent to #1319 (model-agreement stats).

User story

As a project owner I want to confirm that every taxon my pipeline reports is genuinely present, without verifying every single occurrence. I want to open one example occurrence per unverified taxon, accept or correct the ID, and move to the next row. Using filters I should be able to repeat that sweep per station, per session, or per capture set, so I can produce a presence/absence matrix without doing abundance-level verification.

This is presence, not abundance or trend. The goal is a count of "unique taxa whose presence has been confirmed by a human under this filter," derived from verified_count >= 1 per row.

Surfaces

Taxa list (/projects/<id>/taxa)

• New Example column showing an occurrence thumbnail per row. Clicking the thumbnail opens the identification modal over the taxa list (nested-route pattern), the same way clicking an occurrence in the session detail view does today.
• Thumbnail choice when the row is unverified: prefer the best-scoring unverified occurrence under the active filter (most likely to be a clean ID, fastest to verify). Fall back to the latest occurrence if no unverified ones exist (or just the existing cover_image_url).
• The column is not sortable — best_determination_score ordering already covers that intent.
• After the modal closes with a successful verification, the row re-renders with verified_count++ and the thumbnail rolls to the next-best unverified occurrence (or disappears if the taxon is now fully verified).
• Visual de-emphasis on verified rows so the user's eye lands on what's left to do (dim row / muted thumbnail / "verified ✓" badge in the Verified column).

Identification modal (existing component, reused)

• The modal is the same one that opens over the session detail view today. No new modal — verify reuse, not parallel implementation.
• Entry from the taxa list should default to a state where verification is the obvious next action. If the modal has tabs (Fields, Charts, …), open with the most-direct-to-verify tab focused. If the modal does not yet have a dedicated tab/CTA for verification, this ticket adds one. Worth confirming with a quick FE audit before scoping the UI work.
• Modal close returns the user to the taxa list. Optional follow-on: a "Verify next" button inside the modal advances to the next unverified taxon without closing — keeps the sweep cadence high.

Filter combinations that produce presence matrices

The existing taxa-list filters already drive the rollup. Combining ?verified=false with one of the join filters gives the user a focused queue:

• ?verified=false — every project taxon that has zero human-verified occurrences.
• ?verified=false&deployment=<id> — unverified taxa at a given station (presence per station).
• ?verified=false&event=<id> — unverified taxa for a given session / monitoring night (presence per night).
• ?verified=false&collection=<id> — unverified taxa in a capture set.

Sweeping each filter combination produces a per-(taxon × dimension) presence map.

Implementation guidance

Backend

The taxa list already returns verified_count and respects the ?verified= filter via TaxonQuerySet.with_verification_counts, reusing the same occurrence_filters Q across the dispatch branches. Adding the example-occurrence column means picking one occurrence per row under the active filter.

The naive approach — a per-taxon correlated Subquery selecting one occurrence by score — runs into the same patterns covered in docs/claude/reference/hierarchical-rollup-query-performance.md. The right shape depends on the path:

• Default / verified / event / deployment / ordering paths. Add a fourth correlated Subquery annotation alongside occurrences_count / best_determination_score / last_detected in with_observation_counts_subqueries, selecting the chosen occurrence's id. The composite (determination_id, project_id, event_id, determination_score) index serves the ordering. Returning the unverified example would need an Exists exclusion of Identification inside the subquery — verify this still hits the composite index in EXPLAIN ANALYZE before merging.
• ?collection=<id> path. The detections join turns correlated subqueries into per-row scans. Extend the conditional-aggregation branch (with_observation_counts_aggregated) with one of:
  • A separate Python pass that materialises {taxon_id: example_occurrence_id} once and applies it as a CASE-from-map (same pattern as the verification rollup — sparse if filtered, dense if not).
  • A lateral join (OuterRef + Subquery with ORDER BY ... LIMIT 1) accepting the per-row scan cost, gated behind a query param so the column only appears when the user opts in.
• Membership filter. No change — the row is already restricted to observed taxa.

Field shape on the response: example_occurrence: { id, detection_id, image_url, score, verified } so the frontend can render the thumbnail, deep-link the modal, and badge verified rows correctly. Occurrence.best_detection (already maintained, picks via BEST_MACHINE_PREDICTION_ORDER) is the natural source for the image.

Selecting which occurrence to surface

Three candidate signals — pick deliberately, document on the field:

1. Best-scoring unverified. Optimises for fastest verification (cleanest predicted ID). Default for unverified rows.
2. Latest occurrence. Optimises for recency / does the model still report this taxon? Good for already-verified rows ("still showing up?").
3. Latest unverified. Mix of the two — recent AND actionable.

Suggestion: hybrid — best-scoring unverified for unverified rows, latest for verified rows. Document in the API and on the column header tooltip.

Frontend

• Reuse the existing occurrence modal (over the session detail view today). Audit ui/src/pages/sessions/... to find the nested-route entry pattern, then wire the same hook on ui/src/pages/species/....
• Tooltip on the thumbnail: "Verify one occurrence of this taxon."
• After successful verification: invalidate the React Query cache for taxa + the specific taxon's occurrences?taxon=<id>&verified=true count so the row updates without a full reload.
• Empty state when the active filter has zero unverified taxa: "All taxa verified under this filter."

Performance budget

The taxa list endpoint cold-page should stay under ~2s for production-scale projects (~1k taxa / 17k occurrences) — current state without this column is ~1.1s default / ~1.4s under ?collection=. Adding the example-occurrence subquery shouldn't push past the budget if:

• The subquery is index-served on the default path (verified composite index covers (determination_id, project_id, event_id, determination_score)).
• The collection path uses CASE-from-map or is gated behind an opt-in flag.

Bench before merge on the same project used for #1317 timings.

Gotchas to preserve (cited from docs/claude/reference/hierarchical-rollup-query-performance.md)

• Correlated subqueries are fine on the default path but break under detections joins. Same dispatch rule as with_observation_counts_*.
• CASE-from-map only works for sparse driving sets. The unverified-occurrence set is sparse; the full observed-occurrence set is not.
• Don't include the default taxa filter inside a conditional-aggregate filter — measured 0.3s → 182s in Add verification status to Taxa views #1317.
• Detection fan-out under ?collection= requires pk in .values(...) + .distinct().

Optional — link existing columns to their source occurrence

The taxa list already returns last_detected and best_determination_score per row. Both are derived from a specific occurrence — last_detected from the most recent matching occurrence, best_determination_score from the highest-scoring one. Today those are static values.

Optional extension to this ticket: surface the source occurrence id alongside each value and turn the cell into a link that opens the identification modal for that occurrence, same pattern as the new Example column.

Backend: the subqueries powering these annotations already pick a single row (Max(...)). Add a parallel Subquery returning the matching occurrence id (.order_by("-timestamp").values("id")[:1] for last seen, .order_by("-determination_score").values("id")[:1] for best score), with the same hybrid dispatch (correlated Subquery on the default path, CASE-from-map / lateral on ?collection=). Field shape: last_detected_occurrence_id, best_scoring_occurrence_id.

Frontend: wrap the existing cell values in the same modal-route link the Example column uses. No new UX — just three entry points to the same modal per row (Example thumbnail + Last seen value + Best score value).

This is a small additive change that compounds with the main ticket: a user can verify one occurrence per taxon by clicking any of the three columns, depending on which signal they trust (best-scoring for "easiest to verify," most-recent for "is it still showing up," example for the curated pick). Worth bundling if cheap; defer if the example column already covers the workflow.

Out of scope (queued)

• Bulk verify ("mark all in this filter as verified one click") — too dangerous without per-occurrence inspection; presence verification is the smallest unit that should require a human look.
• A dedicated unverified-taxa queue page distinct from the taxa list. The taxa list with ?verified=false already is that queue once this column exists.
• Project-summary widget ("X of Y unique taxa have presence-verified") — natural follow-up; out of this ticket.
• Audit-trail / per-user verification credit — verification already produces an Identification row with the user attached; surfacing per-user counts is a separate UX problem.
• Presence-per-week / time-bucketed view. The ?event=<id> filter already gets you per-session; a calendar-style matrix is its own ticket.

Test plan

Backend:

• example_occurrence populated under each path (default / event / deployment / collection / verified).
• Unverified row returns a best-scoring unverified occurrence; verified row returns latest.
• Filter combinations (?verified=false&deployment=X) return example occurrences that satisfy both filters.
• No N+1 — query count under realistic page size stays bounded.
• Cold-page timing under budget on a production-scale project.

Frontend:

• Thumbnail renders, falls back gracefully when no image is available.
• Click opens the identification modal over the taxa list (URL change, list still visible behind).
• Modal close returns to taxa list; verified row updates without full reload.
• "Verify next" advances cleanly across rows (if added).
• Empty state when no unverified taxa exist under the filter.

References

• Add guidance and stats about which taxa need verification #1316 — original verification UX scope.
• Add verification status to Taxa views #1317 — verified_count + verified filter + GIN index, the foundation this builds on.
• Add model-agreement stats to Taxa views #1319 — model-agreement stats follow-up (sibling, not blocker).
• docs/claude/reference/hierarchical-rollup-query-performance.md — query patterns, fan-out dedup, sparse vs dense, GIN gotchas.
• ami/main/models.py — TaxonQuerySet.with_observation_counts_subqueries, TaxonQuerySet.with_observation_counts_aggregated, TaxonQuerySet.with_verification_counts, Occurrence.best_detection.
• ami/main/api/views.py — TaxonViewSet.get_taxa_observed (dispatch), TaxonViewSet.get_occurrence_filters (active-filter Q).