Skip to content

Collect per-template deployment counts - #7

Open
ak40u wants to merge 2 commits into
TinyboxSoftware:mainfrom
ak40u:feat/deployment-metrics
Open

Collect per-template deployment counts#7
ak40u wants to merge 2 commits into
TinyboxSoftware:mainfrom
ak40u:feat/deployment-metrics

Conversation

@ak40u

@ak40u ak40u commented Aug 5, 2026

Copy link
Copy Markdown

Problem

workspaceTemplates reports health as a rounded percentage. A percentage alone does not say how much activity it covers: a template sitting at 88% could have had two failed deploys or twenty, and nothing stored says which.

Change

Collect templateMetrics alongside the existing queries. It carries the counters behind the percentage — totalDeployments, deploymentsLast90Days, activeDeployments — plus supportHealth and per-template earnings.

  • New table template_deployment_snapshots. Deploy counts are a different population from the project counts already stored — they include redeploys of an existing install — so they get their own table instead of columns on template_snapshots, and no existing view or dashboard changes.
  • New view template_deployment_activity differences consecutive snapshots into deployments_added, and reports health_change beside it.
  • ensureSchema now drives off a REQUIRED_TABLES list rather than a hardcoded 3, so the schema file re-runs once on existing databases to add the new table. Every statement in it is idempotent.

Two traps, both hit in practice

I originally stored a failed_deployments column computed as total_deployments * (100 - health) / 100, and a failures_added delta on top of it. Both are wrong, and the first two days of real data showed it:

  1. The counter and the percentage cover different periods. Deploys are counted for the lifetime of the template; health is a percentage over a rolling window whose size is not published. When the window empties, the formula reports zero failures for deploys that did fail. One of my templates went from "6 failed" to "0 failed" without a single deploy in between — a change of minus six failures in a day.

  2. templateMetrics returns templateHealth: 100 when it has nothing to report, which is indistinguishable from a genuine 100%. The public health field is null in that same case. That is what the health_reported column is derived from, and the view returns NULL health rather than a default presented as a measurement.

So this PR deliberately reports no failure count. What it does report is what holds up: deployments_added (the counter is monotonic, so the difference is real) and health_change next to it. A health drop against a positive deployments_added is the signal worth chasing.

Batching

templateMetrics takes one template id per call, so the query is built with an alias per template. Sending all of them at once fails:

Query exceeds maximum Breadth (Breadth: 570, max: 500)

Each template costs ten nodes (its own plus nine fields), so requests go out in batches of 40 — under the cap with room for a field to be added later.

Verification

Run against a workspace of 57 templates across two daily collections: two batched requests per run, 189 deploys tracked, rows written and the view returning as expected. Deltas are NULL on the first snapshot, which is correct — there is no prior row to compare against.

Unit tests cover the query builder (alias/variable agreement, selected fields, batch size against the breadth cap). The database tests need a live Postgres and were not run in this environment.

ak40u added 2 commits August 5, 2026 13:53
workspaceTemplates only carries a rounded health percentage, which cannot
answer how many deploys failed in a given period: health is computed over a
rolling window, so it climbs back up as old failures age out of it.

Collect templateMetrics alongside it. It reports the deploy counts behind
that percentage, and storing them turns health into a number of failed
deploys - both as failed_deployments per snapshot and, via the
template_deployment_failures view, as the deploys and failures added between
consecutive runs.

The query takes one template id per call, so it is built with an alias per
template and sent in batches of 40: Railway rejects a query whose breadth
exceeds 500 and each template costs ten nodes.

Deploy counts are a different population from the existing project counts -
they include redeploys of an existing install - so they go in their own table
rather than as columns on template_snapshots.
Deploys are a lifetime counter; health is a percentage over a rolling window
whose size Railway does not publish. Multiplying them is unsound, and the
first two snapshots showed it: a template whose six deploys aged out of the
window went from "6 failed" to "0 failed" with no deploy in between - minus
six failures in a day.

templateMetrics also answers templateHealth 100 when it has nothing to
report, which reads exactly like a genuine 100%. The public health field is
null in that case, so health_reported is derived from it and the view returns
NULL health rather than a default dressed up as a measurement.

The view keeps what holds up - deployments_added off the monotonic counter,
and health_change beside it - and is renamed to template_deployment_activity
since it no longer reports failures.
@ak40u ak40u changed the title Collect per-template deployment counts and failures Collect per-template deployment counts Aug 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant