diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 366f1de..5d78d08 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -5,8 +5,8 @@ `README.md` is a generated single-page rendering of the spec. Do not edit it directly. Edit the source files and rebuild: -- `template.md` - the document head (title, definitions, role versioning) plus - the assembly order +- `template.md` - the document head (title, protocol overview, definitions, + role versioning) plus the assembly order - `connection.md`, `messaging.md`, `pairing.md`, `management.md` - `roles//.md` - one file per role version diff --git a/README.md b/README.md index f75e12d..f97e625 100644 --- a/README.md +++ b/README.md @@ -10,6 +10,81 @@ Sendspin is a multi-room music experience protocol. The goal of the protocol is to orchestrate all devices that make up the music listening experience. This includes outputting audio on multiple speakers simultaneously, screens and lights visualizing the audio or album art, and wall tablets providing media controls. +## Protocol overview + +A typical session, from handshake through playback to disconnect: + +```mermaid +sequenceDiagram + participant Client + participant Server + + Note over Client,Server: Noise handshake complete (see Communication) + + Server->>Client: server/hello (name) + Client->>Server: client/hello (roles and capabilities) + Server->>Client: server/activate (activities, active_roles) + + loop Continuous clock sync + Client->>Server: client/time (client clock) + Server->>Client: server/time (timing + offset info) + end + + Note over Client,Server: Clock synchronization established + Client->>Server: client/state (available: true, player: volume, muted) + + alt Stream starts + Server->>Client: stream/start (codec, format details) + end + + Server->>Client: group/update (playback_state, group_id, group_name) + Server->>Client: server/state (metadata, controller, color) + + loop During playback + alt Player role + Server->>Client: binary Type 4 (audio chunks with timestamps) + end + alt Artwork role + Server->>Client: binary Types 8-11 (artwork channels 0-3) + end + alt Visualizer role + Server->>Client: binary Types 16-20 (loudness, beat, f_peak, spectrum, peak) + end + end + + alt Player requests format change + Client->>Server: stream/request-format (codec, sample_rate, etc) + Server->>Client: stream/start (player: new format) + end + + alt Seek operation + Server->>Client: stream/clear (roles: [player, visualizer]) + end + + alt Track jump (skip to different track) + Server->>Client: stream/clear (roles: [player, visualizer]) + end + + alt Controller role + Client->>Server: client/command (controller: play/pause/seek/volume/switch/etc) + end + + alt State changes + Client->>Server: client/state (state and/or player changes) + end + + alt Server commands player + Server->>Client: server/command (player: volume, mute) + end + + Server->>Client: stream/end (ends all role streams) + + alt Graceful disconnect + Client->>Server: client/goodbye (reason) + Note over Client,Server: Server initiates disconnect + end +``` + ## Definitions - **Sendspin Server** - orchestrates all devices, generates audio streams, manages players and clients, provides metadata @@ -299,130 +374,6 @@ Each [`server/time`](#server--client-servertime) response provides the four time A player MUST NOT report `available: true` until its time filter has converged enough to begin scheduling playback. A source MUST NOT report `available: true` until its time filter has converged enough to timestamp captured audio. -## Playback Synchronization - -This section defines rules that require all implementations to provide a good experience, keeping playback seamlessly synchronized between speakers. While implementations can choose their own strategy, this section describes the minimal requirements that must be met by players. For a recommended strategy that is compliant, see the [Suggested correction strategy](#suggested-correction-strategy) subsection below. - -### Correction Quality - -- **Inaudible corrections:** In steady state, individual corrections MUST NOT produce audible noise, warble, or distortion during normal listening. -- **Maximum speed deviation:** The effective playback speed MUST stay within ±0.5% of normal speed, measured as a sliding average over 150 ms. This bounds continuous (steady-state) correction. A discrete one-shot resynchronization after a disturbance (startup, buffer underrun, or an error too large to correct smoothly) is not a speed deviation and is exempt; such events MUST be rare. - -### Sync Accuracy - -Sync accuracy is measured at the audio output, against what the time-filter predicts the local time should be (not against the true server clock). Use of the [time-filter](#clock-synchronization) is required to meet these minimum standards. The error is the absolute difference between when a sample actually plays in the client's local clock and the local time the time-filter predicts for that sample's server timestamp. - -Each client is responsible for maintaining its own synchronization with the server's timestamps. - -- **Accuracy floor:** In steady state, implementations MUST keep this error within ±1 ms. The only exception is the one-shot resynchronization exempted from the speed cap above, which MUST be rare. -- **Accuracy target:** Implementations SHOULD aim for ±0.5 ms. -- Clients subtract their [`static_delay_ms`](#client--server-clientstate-player-object) from server timestamps before scheduling playback. -- Audio chunks may arrive with timestamps in the past due to network delays or buffering; clients should drop these late chunks to maintain sync. - -### Startup Behavior - -- **No startup warble:** During startup, the client MUST NOT produce audible pitch modulation, warble, or other transient artifacts in the audio output. - -### Server Audio Send Constraints - -- **Chunk duration bounds:** A server MUST NOT send an audio chunk longer than 150 ms, and SHOULD NOT send one shorter than 15 ms (the final chunk of a stream or the chunk before a format change MAY be shorter). -- The server sends audio to late-joining clients with future timestamps only, allowing them to buffer and start playback in sync with existing clients. -- After sending [`stream/start`](#server--client-streamstart) or [`stream/clear`](#server--client-streamclear) messages, servers must schedule the first audio timestamp far enough in the future to satisfy each player's lead (see [Server behavior](#client--server-clienthello-playerv1-support-object)). For live streams the buffer cannot grow after playback begins, so the lead must already be reached before the first chunk plays. -- Servers factor in each client's [`static_delay_ms`](#client--server-clientstate-player-object) when calculating how far ahead to send audio, keeping effective buffer headroom constant. - -### Suggested correction strategy - -This is one valid correction strategy for clients with the `player` role: discrete sample deletion and insertion. It is an example, not a requirement. New implementers can use it as a starting point, especially where CPU or memory is limited: it needs no interpolation and leaves the audio bit-exact except at the moments it corrects. - -Other strategies are allowed and encouraged as long as they meet the rules in this section. For example, asynchronous sample-rate conversion (ASRC) continuously resamples the stream to track the clock, trading CPU/DSP load for lower steady-state distortion than discrete frame drops. - -#### Sample deletion and insertion - -The player renders decoded frames at their server timestamps translated to local time by the time-filter, and corrects accumulated drift by occasionally deleting or duplicating whole frames. At realistic clock drift these corrections are small and infrequent (a few per second) and individually inaudible. A "frame" is one sample across all channels (e.g. one stereo pair). - -**Soft correction.** Per decoded chunk: - -1. Measure the time error between when the chunk is scheduled to play (its server timestamp via the time-filter) and where the renderer will reach it in the output buffer. -2. If the absolute error is below the dead band (~100 µs), output the chunk unchanged. -3. Otherwise correct by `N` frames: if playback is running late (the chunk reaches the output after its scheduled local time), drop `N` frames to catch up; if running early, duplicate `N` frames to wait. Residual error beyond the step carries to the next chunk. - -**Choosing N.** Use the smallest `N` that keeps up with drift, scaled to hold the step duration constant across sample rates: `N = round(21 µs × sample_rate_hz / 1,000,000)` (N=1 at 44.1 and 48 kHz, 2 at 96 kHz, 4 at 192 kHz). A chunk's correction MUST NOT exceed the ±0.5% speed cap, so `N ≤ floor(0.005 × samples_in_chunk)`. Keep `N` small; at realistic drift any `N` in this range stays masked. - -**Drop** removes the `N` frames and lets the neighbouring frames abut. **Duplicate** repeats a boundary frame `N` times. The output is the original samples with `N` removed or `N` repeated, bit-exact everywhere else. - -**Large errors and startup.** When the error would otherwise exceed the ±1 ms floor, or on startup, [`stream/start`](#server--client-streamstart), [`stream/clear`](#server--client-streamclear), or recovery from underrun, snap to the correct position in one shot instead of soft-correcting: if playback is late, drop a leading prefix equal to the excess; if early, insert silence of the equivalent duration. This is a deliberate discontinuity and MUST be rare. - -```mermaid -sequenceDiagram - participant Client - participant Server - - Note over Client,Server: Noise handshake complete (see Communication) - - Server->>Client: server/hello (name) - Client->>Server: client/hello (roles and capabilities) - Server->>Client: server/activate (activities, active_roles) - - loop Continuous clock sync - Client->>Server: client/time (client clock) - Server->>Client: server/time (timing + offset info) - end - - Note over Client,Server: Clock synchronization established - Client->>Server: client/state (available: true, player: volume, muted) - - alt Stream starts - Server->>Client: stream/start (codec, format details) - end - - Server->>Client: group/update (playback_state, group_id, group_name) - Server->>Client: server/state (metadata, controller, color) - - loop During playback - alt Player role - Server->>Client: binary Type 4 (audio chunks with timestamps) - end - alt Artwork role - Server->>Client: binary Types 8-11 (artwork channels 0-3) - end - alt Visualizer role - Server->>Client: binary Types 16-20 (loudness, beat, f_peak, spectrum, peak) - end - end - - alt Player requests format change - Client->>Server: stream/request-format (codec, sample_rate, etc) - Server->>Client: stream/start (player: new format) - end - - alt Seek operation - Server->>Client: stream/clear (roles: [player, visualizer]) - end - - alt Track jump (skip to different track) - Server->>Client: stream/clear (roles: [player, visualizer]) - end - - alt Controller role - Client->>Server: client/command (controller: play/pause/seek/volume/switch/etc) - end - - alt State changes - Client->>Server: client/state (state and/or player changes) - end - - alt Server commands player - Server->>Client: server/command (player: volume, mute) - end - - Server->>Client: stream/end (ends all role streams) - - alt Graceful disconnect - Client->>Server: client/goodbye (reason) - Note over Client,Server: Server initiates disconnect - end -``` - ## Core messages This section describes the fundamental messages that establish communication between clients and the server. These messages handle initial handshakes, ongoing clock synchronization, stream lifecycle management, and role-based state updates and commands. @@ -1274,6 +1225,59 @@ Binary messages SHOULD be rejected if there is no active stream or the client is The timestamp indicates when the first audio sample in this chunk should be output. Clients must translate this server timestamp to their local clock using the offset computed from clock synchronization, subtracting their [`static_delay_ms`](#client--server-clientstate-player-object) from the timestamp. Clients should compensate for any known processing delays (e.g., DAC latency, audio buffer delays, amplifier delays) by accounting for these delays when submitting audio to the hardware. +## Playback Synchronization + +This section defines rules that require all implementations to provide a good experience, keeping playback seamlessly synchronized between speakers. While implementations can choose their own strategy, this section describes the minimal requirements that must be met by players. For a recommended strategy that is compliant, see the [Suggested correction strategy](#suggested-correction-strategy) subsection below. + +### Correction Quality + +- **Inaudible corrections:** In steady state, individual corrections MUST NOT produce audible noise, warble, or distortion during normal listening. +- **Maximum speed deviation:** The effective playback speed MUST stay within ±0.5% of normal speed, measured as a sliding average over 150 ms. This bounds continuous (steady-state) correction. A discrete one-shot resynchronization after a disturbance (startup, buffer underrun, or an error too large to correct smoothly) is not a speed deviation and is exempt; such events MUST be rare. + +### Sync Accuracy + +Sync accuracy is measured at the audio output, against what the time-filter predicts the local time should be (not against the true server clock). Use of the [time-filter](#clock-synchronization) is required to meet these minimum standards. The error is the absolute difference between when a sample actually plays in the client's local clock and the local time the time-filter predicts for that sample's server timestamp. + +Each client is responsible for maintaining its own synchronization with the server's timestamps. + +- **Accuracy floor:** In steady state, implementations MUST keep this error within ±1 ms. The only exception is the one-shot resynchronization exempted from the speed cap above, which MUST be rare. +- **Accuracy target:** Implementations SHOULD aim for ±0.5 ms. +- Clients subtract their [`static_delay_ms`](#client--server-clientstate-player-object) from server timestamps before scheduling playback. +- Audio chunks may arrive with timestamps in the past due to network delays or buffering; clients should drop these late chunks to maintain sync. + +### Startup Behavior + +- **No startup warble:** During startup, the client MUST NOT produce audible pitch modulation, warble, or other transient artifacts in the audio output. + +### Server Audio Send Constraints + +- **Chunk duration bounds:** A server MUST NOT send an audio chunk longer than 150 ms, and SHOULD NOT send one shorter than 15 ms (the final chunk of a stream or the chunk before a format change MAY be shorter). +- The server sends audio to late-joining clients with future timestamps only, allowing them to buffer and start playback in sync with existing clients. +- After sending [`stream/start`](#server--client-streamstart) or [`stream/clear`](#server--client-streamclear) messages, servers must schedule the first audio timestamp far enough in the future to satisfy each player's lead (see [Server behavior](#client--server-clienthello-playerv1-support-object)). For live streams the buffer cannot grow after playback begins, so the lead must already be reached before the first chunk plays. +- Servers factor in each client's [`static_delay_ms`](#client--server-clientstate-player-object) when calculating how far ahead to send audio, keeping effective buffer headroom constant. + +### Suggested correction strategy + +This is one valid correction strategy for clients with the `player` role: discrete sample deletion and insertion. It is an example, not a requirement. New implementers can use it as a starting point, especially where CPU or memory is limited: it needs no interpolation and leaves the audio bit-exact except at the moments it corrects. + +Other strategies are allowed and encouraged as long as they meet the rules in this section. For example, asynchronous sample-rate conversion (ASRC) continuously resamples the stream to track the clock, trading CPU/DSP load for lower steady-state distortion than discrete frame drops. + +#### Sample deletion and insertion + +The player renders decoded frames at their server timestamps translated to local time by the time-filter, and corrects accumulated drift by occasionally deleting or duplicating whole frames. At realistic clock drift these corrections are small and infrequent (a few per second) and individually inaudible. A "frame" is one sample across all channels (e.g. one stereo pair). + +**Soft correction.** Per decoded chunk: + +1. Measure the time error between when the chunk is scheduled to play (its server timestamp via the time-filter) and where the renderer will reach it in the output buffer. +2. If the absolute error is below the dead band (~100 µs), output the chunk unchanged. +3. Otherwise correct by `N` frames: if playback is running late (the chunk reaches the output after its scheduled local time), drop `N` frames to catch up; if running early, duplicate `N` frames to wait. Residual error beyond the step carries to the next chunk. + +**Choosing N.** Use the smallest `N` that keeps up with drift, scaled to hold the step duration constant across sample rates: `N = round(21 µs × sample_rate_hz / 1,000,000)` (N=1 at 44.1 and 48 kHz, 2 at 96 kHz, 4 at 192 kHz). A chunk's correction MUST NOT exceed the ±0.5% speed cap, so `N ≤ floor(0.005 × samples_in_chunk)`. Keep `N` small; at realistic drift any `N` in this range stays masked. + +**Drop** removes the `N` frames and lets the neighbouring frames abut. **Duplicate** repeats a boundary frame `N` times. The output is the original samples with `N` removed or `N` repeated, bit-exact everywhere else. + +**Large errors and startup.** When the error would otherwise exceed the ±1 ms floor, or on startup, [`stream/start`](#server--client-streamstart), [`stream/clear`](#server--client-streamclear), or recovery from underrun, snap to the correct position in one shot instead of soft-correcting: if playback is late, drop a leading prefix equal to the excess; if early, insert silence of the equivalent duration. This is a deliberate discontinuity and MUST be rare. + ## Source messages This section describes messages specific to clients with the `source` role, which capture audio from a local input (e.g., AUX/line-in, turntable preamp, Bluetooth receiver, or microphone) and stream it to the server. Unlike other roles, a source sends audio to the server; the server remains the single place that resamples, transcodes, mixes, buffers, and distributes audio to output players. Sources stay simple: they capture and encode audio, optionally report basic signal presence (line sensing), and stream timestamped audio frames. diff --git a/messaging.md b/messaging.md index ad55948..8766514 100644 --- a/messaging.md +++ b/messaging.md @@ -111,130 +111,6 @@ Each [`server/time`](#server--client-servertime) response provides the four time A player MUST NOT report `available: true` until its time filter has converged enough to begin scheduling playback. A source MUST NOT report `available: true` until its time filter has converged enough to timestamp captured audio. -## Playback Synchronization - -This section defines rules that require all implementations to provide a good experience, keeping playback seamlessly synchronized between speakers. While implementations can choose their own strategy, this section describes the minimal requirements that must be met by players. For a recommended strategy that is compliant, see the [Suggested correction strategy](#suggested-correction-strategy) subsection below. - -### Correction Quality - -- **Inaudible corrections:** In steady state, individual corrections MUST NOT produce audible noise, warble, or distortion during normal listening. -- **Maximum speed deviation:** The effective playback speed MUST stay within ±0.5% of normal speed, measured as a sliding average over 150 ms. This bounds continuous (steady-state) correction. A discrete one-shot resynchronization after a disturbance (startup, buffer underrun, or an error too large to correct smoothly) is not a speed deviation and is exempt; such events MUST be rare. - -### Sync Accuracy - -Sync accuracy is measured at the audio output, against what the time-filter predicts the local time should be (not against the true server clock). Use of the [time-filter](#clock-synchronization) is required to meet these minimum standards. The error is the absolute difference between when a sample actually plays in the client's local clock and the local time the time-filter predicts for that sample's server timestamp. - -Each client is responsible for maintaining its own synchronization with the server's timestamps. - -- **Accuracy floor:** In steady state, implementations MUST keep this error within ±1 ms. The only exception is the one-shot resynchronization exempted from the speed cap above, which MUST be rare. -- **Accuracy target:** Implementations SHOULD aim for ±0.5 ms. -- Clients subtract their [`static_delay_ms`](#client--server-clientstate-player-object) from server timestamps before scheduling playback. -- Audio chunks may arrive with timestamps in the past due to network delays or buffering; clients should drop these late chunks to maintain sync. - -### Startup Behavior - -- **No startup warble:** During startup, the client MUST NOT produce audible pitch modulation, warble, or other transient artifacts in the audio output. - -### Server Audio Send Constraints - -- **Chunk duration bounds:** A server MUST NOT send an audio chunk longer than 150 ms, and SHOULD NOT send one shorter than 15 ms (the final chunk of a stream or the chunk before a format change MAY be shorter). -- The server sends audio to late-joining clients with future timestamps only, allowing them to buffer and start playback in sync with existing clients. -- After sending [`stream/start`](#server--client-streamstart) or [`stream/clear`](#server--client-streamclear) messages, servers must schedule the first audio timestamp far enough in the future to satisfy each player's lead (see [Server behavior](#client--server-clienthello-playerv1-support-object)). For live streams the buffer cannot grow after playback begins, so the lead must already be reached before the first chunk plays. -- Servers factor in each client's [`static_delay_ms`](#client--server-clientstate-player-object) when calculating how far ahead to send audio, keeping effective buffer headroom constant. - -### Suggested correction strategy - -This is one valid correction strategy for clients with the `player` role: discrete sample deletion and insertion. It is an example, not a requirement. New implementers can use it as a starting point, especially where CPU or memory is limited: it needs no interpolation and leaves the audio bit-exact except at the moments it corrects. - -Other strategies are allowed and encouraged as long as they meet the rules in this section. For example, asynchronous sample-rate conversion (ASRC) continuously resamples the stream to track the clock, trading CPU/DSP load for lower steady-state distortion than discrete frame drops. - -#### Sample deletion and insertion - -The player renders decoded frames at their server timestamps translated to local time by the time-filter, and corrects accumulated drift by occasionally deleting or duplicating whole frames. At realistic clock drift these corrections are small and infrequent (a few per second) and individually inaudible. A "frame" is one sample across all channels (e.g. one stereo pair). - -**Soft correction.** Per decoded chunk: - -1. Measure the time error between when the chunk is scheduled to play (its server timestamp via the time-filter) and where the renderer will reach it in the output buffer. -2. If the absolute error is below the dead band (~100 µs), output the chunk unchanged. -3. Otherwise correct by `N` frames: if playback is running late (the chunk reaches the output after its scheduled local time), drop `N` frames to catch up; if running early, duplicate `N` frames to wait. Residual error beyond the step carries to the next chunk. - -**Choosing N.** Use the smallest `N` that keeps up with drift, scaled to hold the step duration constant across sample rates: `N = round(21 µs × sample_rate_hz / 1,000,000)` (N=1 at 44.1 and 48 kHz, 2 at 96 kHz, 4 at 192 kHz). A chunk's correction MUST NOT exceed the ±0.5% speed cap, so `N ≤ floor(0.005 × samples_in_chunk)`. Keep `N` small; at realistic drift any `N` in this range stays masked. - -**Drop** removes the `N` frames and lets the neighbouring frames abut. **Duplicate** repeats a boundary frame `N` times. The output is the original samples with `N` removed or `N` repeated, bit-exact everywhere else. - -**Large errors and startup.** When the error would otherwise exceed the ±1 ms floor, or on startup, [`stream/start`](#server--client-streamstart), [`stream/clear`](#server--client-streamclear), or recovery from underrun, snap to the correct position in one shot instead of soft-correcting: if playback is late, drop a leading prefix equal to the excess; if early, insert silence of the equivalent duration. This is a deliberate discontinuity and MUST be rare. - -```mermaid -sequenceDiagram - participant Client - participant Server - - Note over Client,Server: Noise handshake complete (see Communication) - - Server->>Client: server/hello (name) - Client->>Server: client/hello (roles and capabilities) - Server->>Client: server/activate (activities, active_roles) - - loop Continuous clock sync - Client->>Server: client/time (client clock) - Server->>Client: server/time (timing + offset info) - end - - Note over Client,Server: Clock synchronization established - Client->>Server: client/state (available: true, player: volume, muted) - - alt Stream starts - Server->>Client: stream/start (codec, format details) - end - - Server->>Client: group/update (playback_state, group_id, group_name) - Server->>Client: server/state (metadata, controller, color) - - loop During playback - alt Player role - Server->>Client: binary Type 4 (audio chunks with timestamps) - end - alt Artwork role - Server->>Client: binary Types 8-11 (artwork channels 0-3) - end - alt Visualizer role - Server->>Client: binary Types 16-20 (loudness, beat, f_peak, spectrum, peak) - end - end - - alt Player requests format change - Client->>Server: stream/request-format (codec, sample_rate, etc) - Server->>Client: stream/start (player: new format) - end - - alt Seek operation - Server->>Client: stream/clear (roles: [player, visualizer]) - end - - alt Track jump (skip to different track) - Server->>Client: stream/clear (roles: [player, visualizer]) - end - - alt Controller role - Client->>Server: client/command (controller: play/pause/seek/volume/switch/etc) - end - - alt State changes - Client->>Server: client/state (state and/or player changes) - end - - alt Server commands player - Server->>Client: server/command (player: volume, mute) - end - - Server->>Client: stream/end (ends all role streams) - - alt Graceful disconnect - Client->>Server: client/goodbye (reason) - Note over Client,Server: Server initiates disconnect - end -``` - ## Core messages This section describes the fundamental messages that establish communication between clients and the server. These messages handle initial handshakes, ongoing clock synchronization, stream lifecycle management, and role-based state updates and commands. diff --git a/roles/player/v1.md b/roles/player/v1.md index 3ed1bdd..e8ba4c2 100644 --- a/roles/player/v1.md +++ b/roles/player/v1.md @@ -111,3 +111,56 @@ Binary messages SHOULD be rejected if there is no active stream or the client is - Rest of bytes: encoded audio frame The timestamp indicates when the first audio sample in this chunk should be output. Clients must translate this server timestamp to their local clock using the offset computed from clock synchronization, subtracting their [`static_delay_ms`](#client--server-clientstate-player-object) from the timestamp. Clients should compensate for any known processing delays (e.g., DAC latency, audio buffer delays, amplifier delays) by accounting for these delays when submitting audio to the hardware. + +## Playback Synchronization + +This section defines rules that require all implementations to provide a good experience, keeping playback seamlessly synchronized between speakers. While implementations can choose their own strategy, this section describes the minimal requirements that must be met by players. For a recommended strategy that is compliant, see the [Suggested correction strategy](#suggested-correction-strategy) subsection below. + +### Correction Quality + +- **Inaudible corrections:** In steady state, individual corrections MUST NOT produce audible noise, warble, or distortion during normal listening. +- **Maximum speed deviation:** The effective playback speed MUST stay within ±0.5% of normal speed, measured as a sliding average over 150 ms. This bounds continuous (steady-state) correction. A discrete one-shot resynchronization after a disturbance (startup, buffer underrun, or an error too large to correct smoothly) is not a speed deviation and is exempt; such events MUST be rare. + +### Sync Accuracy + +Sync accuracy is measured at the audio output, against what the time-filter predicts the local time should be (not against the true server clock). Use of the [time-filter](../../messaging.md#clock-synchronization) is required to meet these minimum standards. The error is the absolute difference between when a sample actually plays in the client's local clock and the local time the time-filter predicts for that sample's server timestamp. + +Each client is responsible for maintaining its own synchronization with the server's timestamps. + +- **Accuracy floor:** In steady state, implementations MUST keep this error within ±1 ms. The only exception is the one-shot resynchronization exempted from the speed cap above, which MUST be rare. +- **Accuracy target:** Implementations SHOULD aim for ±0.5 ms. +- Clients subtract their [`static_delay_ms`](#client--server-clientstate-player-object) from server timestamps before scheduling playback. +- Audio chunks may arrive with timestamps in the past due to network delays or buffering; clients should drop these late chunks to maintain sync. + +### Startup Behavior + +- **No startup warble:** During startup, the client MUST NOT produce audible pitch modulation, warble, or other transient artifacts in the audio output. + +### Server Audio Send Constraints + +- **Chunk duration bounds:** A server MUST NOT send an audio chunk longer than 150 ms, and SHOULD NOT send one shorter than 15 ms (the final chunk of a stream or the chunk before a format change MAY be shorter). +- The server sends audio to late-joining clients with future timestamps only, allowing them to buffer and start playback in sync with existing clients. +- After sending [`stream/start`](../../messaging.md#server--client-streamstart) or [`stream/clear`](../../messaging.md#server--client-streamclear) messages, servers must schedule the first audio timestamp far enough in the future to satisfy each player's lead (see [Server behavior](#client--server-clienthello-playerv1-support-object)). For live streams the buffer cannot grow after playback begins, so the lead must already be reached before the first chunk plays. +- Servers factor in each client's [`static_delay_ms`](#client--server-clientstate-player-object) when calculating how far ahead to send audio, keeping effective buffer headroom constant. + +### Suggested correction strategy + +This is one valid correction strategy for clients with the `player` role: discrete sample deletion and insertion. It is an example, not a requirement. New implementers can use it as a starting point, especially where CPU or memory is limited: it needs no interpolation and leaves the audio bit-exact except at the moments it corrects. + +Other strategies are allowed and encouraged as long as they meet the rules in this section. For example, asynchronous sample-rate conversion (ASRC) continuously resamples the stream to track the clock, trading CPU/DSP load for lower steady-state distortion than discrete frame drops. + +#### Sample deletion and insertion + +The player renders decoded frames at their server timestamps translated to local time by the time-filter, and corrects accumulated drift by occasionally deleting or duplicating whole frames. At realistic clock drift these corrections are small and infrequent (a few per second) and individually inaudible. A "frame" is one sample across all channels (e.g. one stereo pair). + +**Soft correction.** Per decoded chunk: + +1. Measure the time error between when the chunk is scheduled to play (its server timestamp via the time-filter) and where the renderer will reach it in the output buffer. +2. If the absolute error is below the dead band (~100 µs), output the chunk unchanged. +3. Otherwise correct by `N` frames: if playback is running late (the chunk reaches the output after its scheduled local time), drop `N` frames to catch up; if running early, duplicate `N` frames to wait. Residual error beyond the step carries to the next chunk. + +**Choosing N.** Use the smallest `N` that keeps up with drift, scaled to hold the step duration constant across sample rates: `N = round(21 µs × sample_rate_hz / 1,000,000)` (N=1 at 44.1 and 48 kHz, 2 at 96 kHz, 4 at 192 kHz). A chunk's correction MUST NOT exceed the ±0.5% speed cap, so `N ≤ floor(0.005 × samples_in_chunk)`. Keep `N` small; at realistic drift any `N` in this range stays masked. + +**Drop** removes the `N` frames and lets the neighbouring frames abut. **Duplicate** repeats a boundary frame `N` times. The output is the original samples with `N` removed or `N` repeated, bit-exact everywhere else. + +**Large errors and startup.** When the error would otherwise exceed the ±1 ms floor, or on startup, [`stream/start`](../../messaging.md#server--client-streamstart), [`stream/clear`](../../messaging.md#server--client-streamclear), or recovery from underrun, snap to the correct position in one shot instead of soft-correcting: if playback is late, drop a leading prefix equal to the excess; if early, insert silence of the equivalent duration. This is a deliberate discontinuity and MUST be rare. diff --git a/template.md b/template.md index 58bf989..49303d2 100644 --- a/template.md +++ b/template.md @@ -11,6 +11,81 @@ Sendspin is a multi-room music experience protocol. The goal of the protocol is to orchestrate all devices that make up the music listening experience. This includes outputting audio on multiple speakers simultaneously, screens and lights visualizing the audio or album art, and wall tablets providing media controls. +## Protocol overview + +A typical session, from handshake through playback to disconnect: + +```mermaid +sequenceDiagram + participant Client + participant Server + + Note over Client,Server: Noise handshake complete (see Communication) + + Server->>Client: server/hello (name) + Client->>Server: client/hello (roles and capabilities) + Server->>Client: server/activate (activities, active_roles) + + loop Continuous clock sync + Client->>Server: client/time (client clock) + Server->>Client: server/time (timing + offset info) + end + + Note over Client,Server: Clock synchronization established + Client->>Server: client/state (available: true, player: volume, muted) + + alt Stream starts + Server->>Client: stream/start (codec, format details) + end + + Server->>Client: group/update (playback_state, group_id, group_name) + Server->>Client: server/state (metadata, controller, color) + + loop During playback + alt Player role + Server->>Client: binary Type 4 (audio chunks with timestamps) + end + alt Artwork role + Server->>Client: binary Types 8-11 (artwork channels 0-3) + end + alt Visualizer role + Server->>Client: binary Types 16-20 (loudness, beat, f_peak, spectrum, peak) + end + end + + alt Player requests format change + Client->>Server: stream/request-format (codec, sample_rate, etc) + Server->>Client: stream/start (player: new format) + end + + alt Seek operation + Server->>Client: stream/clear (roles: [player, visualizer]) + end + + alt Track jump (skip to different track) + Server->>Client: stream/clear (roles: [player, visualizer]) + end + + alt Controller role + Client->>Server: client/command (controller: play/pause/seek/volume/switch/etc) + end + + alt State changes + Client->>Server: client/state (state and/or player changes) + end + + alt Server commands player + Server->>Client: server/command (player: volume, mute) + end + + Server->>Client: stream/end (ends all role streams) + + alt Graceful disconnect + Client->>Server: client/goodbye (reason) + Note over Client,Server: Server initiates disconnect + end +``` + ## Definitions - **Sendspin Server** - orchestrates all devices, generates audio streams, manages players and clients, provides metadata