Skip to content

Wisplog as single source of truth for chat history#136

Open
mcintyre94 wants to merge 4 commits into
mainfrom
wisplog-source-of-truth
Open

Wisplog as single source of truth for chat history#136
mcintyre94 wants to merge 4 commits into
mainfrom
wisplog-source-of-truth

Conversation

@mcintyre94

Copy link
Copy Markdown
Owner

Summary

Eliminates SwiftData message persistence entirely. The on-sprite wisplog file (/home/sprite/.wisp/chats/UUID.wisplog) is now the canonical record of conversation history — written in real-time by tee -a during streaming, read on demand when needed.

Removed:

  • SpriteChat.messagesData, streamEventUUIDsData, lastSessionComplete
  • PersistedChatMessage.swift (whole file)
  • persistMessages (14 call sites)
  • restoreFromSessionFile
  • linkToolResults (only needed for SwiftData round-trip)
  • 1-second periodic persist in processExecStream
  • All SwiftData-persistence tests

Changed:

  • parseWispLog now returns eventUUIDs: Set<String> so reattachToExec can seed processedEventUUIDs from the wisplog, preventing duplicate content when the exec session replays events already shown
  • reconnectIfNeeded simplified: execSessionId set → reattach; messages empty → load wisplog; otherwise → restore draft
  • reattachToExec pre-loads the wisplog before connecting to the exec WebSocket (instant history display during reconnect), then streams live events with UUID deduplication
  • saveSession drops the isComplete parameter; the result event clears execSessionId directly, so execSessionId == nil signals a completed session
  • DashboardView reconnect query: execSessionId != nil instead of !lastSessionComplete
  • Fork path: single on-sprite awk command copies the parent wisplog up to the fork point into the new chat's wisplog (replaces saveMessages + PersistedChatMessage serialisation)
  • importJSONLSession: calls loadFromWispLog after jq conversion (already wrote the wisplog, no separate persist needed)
  • SwiftData schema migration: WispSchema.swift with a SchemaV1 → SchemaV2 lightweight migration to drop the removed columns

Trade-offs

  • Cold-open latency: opening an old chat now requires a sprite round-trip (cat the wisplog) instead of an instant SwiftData read. The sprite is typically warm when you're actively using a chat.
  • Checkpoint annotations (checkpointId/checkpointComment on ChatMessage): in-memory only, not persisted. Can be fixed later by adding a wisp_checkpoint event type to the wisplog format.

Test plan

  • Send a message, let it complete — history displayed correctly
  • Background app mid-stream, return — reconnects and shows in-progress response
  • Background app, Claude finishes while backgrounded — history loads from wisplog on return
  • Open old completed chat — wisplog loads, full history shown
  • Fork from checkpoint — new chat shows prior history (from awk wisplog copy)
  • Load chat from Claude history — JSONL import converts and displays via wisplog
  • Fresh install / SwiftData cleared — wisplog load restores history seamlessly

🤖 Generated with Claude Code

Eliminates SwiftData message persistence entirely, replacing it with
the on-sprite wisplog file as the canonical record of conversation history.

Key changes:

- SpriteChat: remove messagesData, streamEventUUIDsData, lastSessionComplete
- PersistedChatMessage.swift: deleted
- WispSchema.swift: add SchemaV1→V2 lightweight migration for column drops
- parseWispLog: returns (messages, sessionId, eventUUIDs) so reattach can
  seed processedEventUUIDs from the wisplog and avoid replaying duplicates
- loadFromWispLog: seeds processedEventUUIDs, sets firstMessagePreview,
  calls saveSession/restoreUndeliveredDraft to keep exec state consistent
- reconnectIfNeeded: simplified — if execSessionId set, reattach; else if
  messages empty, load wisplog; else restoreUndeliveredDraft
- reattachToExec: pre-loads wisplog before attaching to exec WebSocket,
  seeds UUID set from the wisplog to prevent duplicate content; on timeout
  reloads wisplog instead of calling restoreFromSessionFile
- restoreFromSessionFile: deleted
- executeClaudeCommand: removes all persistMessages calls and the 1-second
  periodic persist; on disconnect schedules reconnect without persisting
- processExecStream: removes periodic persist, sessionInfo calls saveSession
  without isComplete
- loadSession: no longer loads messages from SwiftData; metadata only
- saveSession: removes isComplete parameter; result events clear execSessionId
  directly so execSessionId==nil signals a completed session
- DashboardView: queries execSessionId != nil instead of !lastSessionComplete
- fork path: replaces saveMessages with an on-sprite awk command that copies
  the parent wisplog up to the fork point into the new chat's wisplog
- importJSONLSession: calls loadFromWispLog after jq conversion instead of
  persistMessages
- SpriteChatMigration: no longer copies messagesData
- Tests: remove all SwiftData persistence tests; update parseWispLog callers
  for the new 3-tuple return; update migration and checkpoint tests

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@claude

claude Bot commented May 1, 2026

Copy link
Copy Markdown

test connection

@claude

claude Bot commented May 1, 2026

Copy link
Copy Markdown

Code review — 2 bugs and 1 CLAUDE.md violation found.


Bug 1: loadFromWispLog incorrectly clears execSessionId for in-progress sessions

if let last = messages.last, last.role == .user {
restoreUndeliveredDraft(modelContext: modelContext)
} else {
execSessionId = nil
saveSession(modelContext: modelContext)
}
}

The else branch fires whenever the last wisplog message is not a .user message — but an in-progress session (exec still running, Claude mid-stream) also ends with a partial assistant tail. loadFromWispLog clears execSessionId and calls saveSession for a live session. reattachToExec then hits the guard immediately after and bails:

// If the wisplog showed a complete session (execSessionId cleared by loadFromWispLog),
// the exec is already done — no need to attach.
guard self.execSessionId != nil else {
if !Task.isCancelled { status = .idle }
return
}
isReplaying = true

Impact: A user who backgrounds the app mid-stream and returns will have their in-flight Claude response silently abandoned. The execSessionId is also cleared in SwiftData, so subsequent dashboard reconnects won't pick this chat up.

Root cause: parseWispLog returns no signal about whether a result event was seen, so the caller can't distinguish a completed session from a still-running one — both end with an assistant tail. Fix: return whether a result event was observed, and only clear execSessionId when confirmed complete.


Bug 2: processedEventUUIDs blocks the exec replay from rebuilding the removed content

// Strip any trailing incomplete assistant from the wisplog load; the exec replay
// will rebuild it. A complete turn ends with a result event, so if the last message
// is assistant it may be partial.
if let last = messages.last, last.role == .assistant {
messages.removeLast()
}
let assistantMessage = ChatMessage(role: .assistant)
messages.append(assistantMessage)
currentAssistantMessage = assistantMessage

loadFromWispLog (called just above at line ~1281) sets processedEventUUIDs = seenUUIDs — which includes the UUIDs of every event that built the trailing assistant message being removed here. When the exec WebSocket replays those events, handleOrSkip finds their UUIDs already in processedEventUUIDs and skips them without emitting content.

func processExecStream(
events: AsyncThrowingStream<ExecEvent, Error>,
modelContext: ModelContext
) async -> StreamResult {
var receivedData = false
let timeoutTask = Task {
try await Task.sleep(for: .seconds(30))
if !receivedData {
logger.warning("No exec data received in 30s")
}
}
func handleOrSkip(_ parsedEvent: ClaudeStreamEvent) {
if let uuid = parsedEvent.uuid, processedEventUUIDs.contains(uuid) {
switch parsedEvent {
case .system(let se):
receivedSystemEvent = true
sessionId = se.sessionId
modelName = se.model
case .result(let re):
receivedResultEvent = true
sessionId = re.sessionId
default: break
}
return
}
if let uuid = parsedEvent.uuid {
processedEventUUIDs.insert(uuid)
}
if isReplayingLiveService {
applyReplayBuffer()
isReplaying = false
isReplayingLiveService = false
if case .reconnecting = status { status = .streaming }
}
handleEvent(parsedEvent, modelContext: modelContext)
}

Impact: The comment 'the exec replay will rebuild it' is incorrect. The new empty assistant bubble will only receive truly new events arriving after the wisplog snapshot. In the common case where the wisplog is fully caught up, the bubble stays empty for the rest of the turn.

Fix options: (a) before removing the last message, remove its events' UUIDs from processedEventUUIDs, or (b) don't pre-seed processedEventUUIDs from the wisplog at all. Note: this bug is only reachable once Bug 1 is fixed.


CLAUDE.md violation: Missing test for parseWispLog's new eventUUIDs return value

"""
let (messages, sessionId, _) = ChatViewModel.parseWispLog(ndjson)
#expect(messages.count == 2)

Per CLAUDE.md (https://github.com/mcintyre94/wisp/blob/472df5f3c7e131bf6e5cd7ce17cd93ac45b1c7d4/CLAUDE.md): 'Add new unit tests when adding or modifying logic (models, parsers, utilities, view models)'.

The PR adds eventUUIDs: Set as a new return value from parseWispLog, collecting event UUIDs from the wisplog. This output directly drives deduplication in loadFromWispLog (processedEventUUIDs = seenUUIDs). All test call sites discard it with _ — no test verifies that events with UUIDs are collected, events without UUIDs don't break the parse, or that empty input returns an empty set.


Disregard the earlier 'test connection' comment — posted during review tooling setup.

mcintyre94 and others added 3 commits May 1, 2026 22:43
The file was deleted from disk but its four references (PBXBuildFile,
PBXFileReference, group entry, Sources build phase entry) remained in
project.pbxproj, causing the CI build to fail with "Build input file
cannot be found".

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Bug 1: loadFromWispLog cleared execSessionId for in-progress sessions.
Both completed and in-progress sessions end with an assistant tail in the
wisplog, so the caller had no way to distinguish them. Add isComplete: Bool
to parseWispLog's return (set only when a result event is observed) and
gate the execSessionId clear + saveSession on isComplete.

Bug 2: reattachToExec stripped the trailing assistant then seeded
processedEventUUIDs from the wisplog — meaning the exec replay skipped
the events that would have rebuilt the removed content, leaving the new
bubble permanently empty. Fix: keep the trailing assistant as the streaming
target; processedEventUUIDs correctly prevents the replay from duplicating
content already shown, and only truly new events extend it.

Test: add parseWispLog_isCompleteAndEventUUIDs covering the new isComplete
field and UUID collection, addressing the CLAUDE.md test requirement.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
The file was created on disk but never registered in project.pbxproj,
causing 'cannot find WispMigrationPlan in scope' build errors in CI.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant