Skip to content

Commit 1577cd2

Browse files
committed
feat(stovepipe): Define issue-tracking handoff
1 parent f75843b commit 1577cd2

20 files changed

Lines changed: 688 additions & 5 deletions

‎stovepipe/core/messagequeue/README.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,3 +11,13 @@ Payloads are defined in proto3 (`proto/`, generated into `protopb/`) and seriali
1111
- **buildsignal** (`TopicKeyBuildSignal`, `BuildSignal`) — build publishes the build id here after triggering; buildsignal holds the delivery between polls until the build reaches a terminal status. Partitioned by build id, so each build's poll loop is an independent partition.
1212
- **buildanalysis** (`TopicKeyBuildAnalysis`, `BuildAnalysis`) — identifies a persisted build for issue detection, partitioned by build id. Build publishes a start notice and buildsignal publishes a completion notice with a distinct message id. See [build analysis](../../controller/buildanalysis/README.md) for polling and persistence behavior.
1313
- **record** (`TopicKeyRecord`, `Record`) — buildsignal publishes the request id after persisting the terminal outcome; record reloads the request and records its validation fact. Partitioned by request id.
14+
15+
## Issue tracking handoff
16+
17+
`IssueTrack` binds to `TopicKeyIssueTrack` and identifies a persisted build and its queue. Notifications are partitioned by queue so both running and final issue-state updates share one serialized path, independently of build-partitioned analysis polling. The message carries identifiers rather than a failure collection; per-build observations are read through `BuildIssueStore.List`.
18+
19+
Publish only after detection and observation persistence succeed. With `finalize` unset, the notification permits opening newly detected issues, not resolution by absence. With `finalize` set, it requests final reconciliation after a successful terminal analysis. Publish that notification even when no failures were detected; terminal Build status alone does not replace the handoff. Publication retries must also follow duplicate observation writes so a crash between persistence and publication does not strand work.
20+
21+
Per-build observations accumulate historical evidence; finalization does not turn them into a replacement snapshot or override the detector's absence semantics. The reconciler must establish the authoritative final failure set and applicable validation scope before resolving issues, reject older requests, and ignore running notifications after finalization. Notification identity must distinguish new observations from finalization while remaining stable on publication retries. Business behavior follows the payload, not a parsed message ID.
22+
23+
Only the handoff contract is defined. No publisher, consumer, topic registration, reconciliation progress, or writes to open issues and lifecycle history are enabled.

‎stovepipe/core/messagequeue/messagequeue.go‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,9 @@ type (
5353
// BuildAnalysis identifies the build whose issue observations are polled.
5454
BuildAnalysis = protopb.BuildAnalysis
5555

56+
// IssueTrack identifies persisted build observations for queue-scoped reconciliation.
57+
IssueTrack = protopb.IssueTrack
58+
5659
// Record is the payload buildsignal publishes to the record stage once a
5760
// build reaches a terminal status: the build id to record.
5861
Record = protopb.Record

‎stovepipe/core/messagequeue/messagequeue_test.go‎

Lines changed: 30 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -80,7 +80,7 @@ func TestWireFormat(t *testing.T) {
8080
// no topic_keys option names an unknown key.
8181
func TestTopicKeysBindEveryTopicKey(t *testing.T) {
8282
bound := map[string]int{}
83-
for _, m := range []proto.Message{&ProcessRequest{}, &BuildRequest{}, &BuildSignal{}, &BuildAnalysis{}, &Record{}} {
83+
for _, m := range []proto.Message{&ProcessRequest{}, &BuildRequest{}, &BuildSignal{}, &BuildAnalysis{}, &IssueTrack{}, &Record{}} {
8484
keys := TopicKeys(m)
8585
require.NotEmpty(t, keys, "message must declare a non-empty topic_keys option")
8686
for _, key := range keys {
@@ -93,6 +93,7 @@ func TestTopicKeysBindEveryTopicKey(t *testing.T) {
9393
TopicKeyBuild,
9494
TopicKeyBuildSignal,
9595
TopicKeyBuildAnalysis,
96+
TopicKeyIssueTrack,
9697
TopicKeyRecord,
9798
}
9899

@@ -117,3 +118,31 @@ func TestBuildAnalysisRoundTrip(t *testing.T) {
117118
assert.True(t, proto.Equal(analysis, got))
118119
assert.JSONEq(t, `{"id":"build/42","queue_name":"validation/main"}`, string(data))
119120
}
121+
122+
func TestIssueTrackRoundTrip(t *testing.T) {
123+
for _, tt := range []struct {
124+
name string
125+
finalize bool
126+
wire string
127+
}{
128+
{name: "observations", wire: `{"build_id":"build/42","queue_name":"validation/main"}`},
129+
{name: "final analysis", finalize: true, wire: `{"build_id":"build/42","queue_name":"validation/main","finalize":true}`},
130+
} {
131+
t.Run(tt.name, func(t *testing.T) {
132+
notification := &IssueTrack{BuildId: "build/42", QueueName: "validation/main", Finalize: tt.finalize}
133+
data, err := Marshal(notification)
134+
require.NoError(t, err)
135+
assert.JSONEq(t, tt.wire, string(data))
136+
137+
got := &IssueTrack{}
138+
require.NoError(t, Unmarshal(data, got))
139+
assert.True(t, proto.Equal(notification, got))
140+
})
141+
}
142+
}
143+
144+
func TestIssueTrackToleratesUnknownFields(t *testing.T) {
145+
got := &IssueTrack{}
146+
require.NoError(t, Unmarshal([]byte(`{"build_id":"build/42","queue_name":"validation/main","finalize":true,"future_field":42}`), got))
147+
assert.True(t, proto.Equal(&IssueTrack{BuildId: "build/42", QueueName: "validation/main", Finalize: true}, got))
148+
}

‎stovepipe/core/messagequeue/proto/BUILD.bazel‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ exports_files(
33
"build.proto",
44
"buildanalysis.proto",
55
"buildsignal.proto",
6+
"issuetrack.proto",
67
"process.proto",
78
"record.proto",
89
],
Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
// Copyright (c) 2026 Uber Technologies, Inc.
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
15+
syntax = "proto3";
16+
17+
package uber.stovepipe.messagequeue;
18+
19+
import "api/base/messagequeue/proto/messagequeue.proto";
20+
21+
option go_package = "github.com/uber/submitqueue/stovepipe/core/messagequeue/protopb";
22+
option java_multiple_files = true;
23+
option java_outer_classname = "IssueTrackProto";
24+
option java_package = "com.uber.submitqueue.stovepipe.messagequeue";
25+
26+
// IssueTrack requests queue-scoped reconciliation of persisted build observations.
27+
// Publish only after successful detection and observation persistence; partition by queue.
28+
message IssueTrack {
29+
option (uber.base.messagequeue.topic_keys) = "issuetrack";
30+
31+
// build_id is the required opaque identifier of the persisted build.
32+
string build_id = 1;
33+
// queue_name is the required queue namespace of the build.
34+
string queue_name = 2;
35+
// finalize requests final reconciliation after successful terminal detection
36+
// and persistence, including when no failures were detected.
37+
// Otherwise the notification permits opening issues, not resolution by absence.
38+
bool finalize = 3;
39+
}

‎stovepipe/core/messagequeue/protopb/BUILD.bazel‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ go_library(
66
"build.pb.go",
77
"buildanalysis.pb.go",
88
"buildsignal.pb.go",
9+
"issuetrack.pb.go",
910
"process.pb.go",
1011
"record.pb.go",
1112
],

‎stovepipe/core/messagequeue/protopb/issuetrack.pb.go‎

Lines changed: 167 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎stovepipe/core/messagequeue/topics.go‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,9 @@ const (
4545
// TopicKeyBuildAnalysis carries build IDs for issue detection, partitioned by build ID.
4646
TopicKeyBuildAnalysis TopicKey = "buildanalysis"
4747

48+
// TopicKeyIssueTrack carries persisted observation notifications, partitioned by queue.
49+
TopicKeyIssueTrack TopicKey = "issuetrack"
50+
4851
// TopicKeyRecord carries a request whose build reached a terminal status from
4952
// buildsignal to the record stage. The buildsignal controller publishes a
5053
// Record (the request id) here once, and only once, a build reaches a

‎stovepipe/entity/build_issue.go‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ package entity
1616

1717
// BuildIssue identifies a validation failure by task and optional unit.
1818
type BuildIssue struct {
19-
// TaskID identifies the logical validation task, stable across builds and retries.
19+
// TaskID identifies the logical validation task, stable across builds and retries; it must be nonempty.
2020
TaskID string `json:"task_id"`
2121
// UnitID identifies the failed unit within the task; empty for a task-wide issue.
2222
// Repeated observations of the same condition use the same TaskID and UnitID.

‎stovepipe/extension/storage/README.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,9 @@ Same design space as [`submitqueue/extension/storage`](../../../submitqueue/exte
2828

2929
## Build issue observations
3030

31-
`BuildIssueStore` accumulates immutable observations identified by build, task, and unit within the bound queue. The build ID is supplied separately from the detected issue. Repeated observations return `ErrAlreadyExists`, which callers may treat as successful persistence. Empty unit IDs represent task-wide observations.
31+
`BuildIssueStore` accumulates immutable observations identified by build, task, and unit within the bound queue. The build ID is supplied separately from the detected issue. Repeated observations return `ErrAlreadyExists`, which callers may treat as successful persistence. Build and task IDs must be nonempty; empty unit IDs represent task-wide observations.
32+
33+
Listing fixes the queue/build primary-key prefix and returns bounded pages in ascending bytewise task/unit order. A task/unit cursor continues after the last returned key; the zero cursor starts the range, and a unit without a task is invalid. Pages are not a snapshot across calls: concurrent observations can affect membership. Empty pages represent no observations in that range, not build success or completed analysis.
3234

3335
Each observation is stored separately, not in a serialized collection. Observations are historical evidence; they do not track whether an issue is currently open or resolved. No update, deletion, or absence-based recovery is implied by this store.
3436

0 commit comments

Comments
 (0)