Repository navigation
avm2,audio: Implement SampleDataEvent - #14203
torokati44 wants to merge 12 commits into
Conversation
27be9ab to
8e7465b
Compare
|
Most NSFW video games depend on this commit, here still no any SFX, but now they panics after the end of the first loop (need to wait for the issue reproduction)
upd: here one my favorite chiptune site, music is playing http://zxtunes.com/author.php?id=446&sort=playing :D |
8e7465b to
ee06332
Compare
ee06332 to
30b855a
Compare
30b855a to
de195ac
Compare
de195ac to
59c88bb
Compare
|
TBD:
|
danielhjacobs
added
the
waiting-on-author
aa09e74 to
0610048
Compare
0610048 to
1812e00
Compare
0ef97dd to
0e4032c
Compare
c79f6c3 to
15658ba
Compare
15658ba to
be0cb66
Compare
54db6b2 to
f9b7d60
Compare
|
@copilot try to create a new PR inspired by this, implemented on top of current |
12d9dad to
3c8f9e2
Compare
| @@ -120,14 +122,15 @@ pub enum SoundLoadingState { | |||
| New, | |||
| Loading, | |||
| Loaded, | |||
| Generated, | |||
| } | |||
There was a problem hiding this comment.
Are SoundData and SoundLoadingState supposed to mirror each other like this? @kjarosh
I'm asking you because you added the latter in #19586 .
(And now that I'm reading it, maybe EDIT: lol I read the diff wrong when writing this, there's no split like thisSoundData::NotLoaded and SoundData::Loading should be collapsed into one - it's probably split as the result of another glitchy rebase from eons ago...)
There was a problem hiding this comment.
Well, it depends on whether sound behaves the same way in AVM1 and AVM2. If it does and we have a lot of evidence of that, perhaps it's a good idea to unify the structures between AVM1 and AVM2 to some extent?
There was a problem hiding this comment.
It's strange though because SoundData is used from both avm1 and avm2, while SoundLoadingState is only used for/by avm2.
There was a problem hiding this comment.
Is this point still relevant?
There was a problem hiding this comment.
Well, there's still two Generated variants added, to SoundData and SoundLoadingState... 🤔
3c8f9e2 to
3e1acb3
Compare
1f22180 to
81c2eb7
Compare
| s.load(new URLRequest("missing.mp3")); | ||
| trace("Load succeeded unexpectedly"); | ||
| } catch (error:Error) { | ||
| trace("An error occurred: " + error.message); |
There was a problem hiding this comment.
Can this trace error.getStackTrace() instead of error.message?
There was a problem hiding this comment.
It could, but then 😐
Diff < left / right > :
false
An error occurred:
Error: Error #2037: Functions called in incorrect sequence, or earlier call was unsuccessful.
> at flash.media::Sound/_load()
at flash.media::Sound/load()
at Test()
How should I add that intermediate method, @kjarosh ?
There was a problem hiding this comment.
I think I got it, but now I'm wondering why the indirection, what should the outer (AS) method do...
Lord-McSweeney
left a comment
Lord-McSweeney
left a comment
There was a problem hiding this comment.
What happens if event.data is set to null at the end of a SampleDataEvent event handler?
cb8cee0 to
1e6ee2f
Compare
|
Great, now there are two failing tests, @Lord-McSweeney...! 😝 |
05d3bf2 to
ba08101
Compare
The `sound: Option<SoundHandle>` field of `SoundInstance` was `Some` for event sounds and `None` for stream sounds. Replace it with an explicit `SoundInstanceSourceData` enum, so that other kinds of sound sources can be added with their own data. No behavior change.
Add `AudioBackend` methods to start a generated (synthesized) sound stream, whose samples are supplied from the main thread by handle, as 44100 Hz stereo `f32` frames. The backend reports how many of them are still unplayed, so the caller can decide when to supply more, and the stream ends once it's marked as finished and has drained. In the mixer, the samples are passed to the audio thread in a queue shared with the stream, which moves them in bulk to a local playout buffer, to take the lock rarely. An atomic counter tracks the number of unplayed frames (including those in the playout buffer), which also lets the audio thread skip locking when the shared queue is empty.
Add a native `sample_data_event` constructor on `EventObject`, taking the `data` ByteArray to pass to the listeners, register the `sampleData` common string, and mark `SampleDataEvent._data` as `[Ruffle(NativeAccessible)]`, so the samples written by ActionScript can be read back natively.
Add a `Generated` sound instance source to `AudioManager`, holding the AS3 `Sound` object, and the `data` ByteArray passed to its listeners. Each frame, `SampleDataEvent`s are dispatched to every generated sound until enough of its audio is buffered in the backend: two frames' worth, but at least 150 ms (to absorb frame timing jitter at high frame rates) and at most 2 s (to limit latency and the number of events dispatched per frame at very low frame rates). As in Flash Player, the same ByteArray is used for every event of a sound, only its position is reset, so the samples supplied are the ones written up to the position, not the length. `SampleDataEvent.position` starts at the given start position, and advances by the samples supplied. Supplying fewer than 2048 samples ends the sound once its buffered audio has drained; if the listener replaced `event.data` with something other than a ByteArray, that counts as supplying no samples. The sound instance is looked up again before and after each dispatch, so no more events are sent to a sound once a listener has stopped it (even with `SoundMixer.stopAll()`), and the samples written by that listener are discarded.
Rename `SoundData::NotLoaded` to `Loading`, and add the `Empty` (initial) and `Generated` states, with a matching `SoundLoadingState::Generated`. Calling `Sound.play()` on a `Sound` that `load()` was never called on makes it a generated sound, synthesized by its `SampleDataEvent` listeners. Every such call starts a new, independent stream with its own events, returning `null` if too many sounds are playing. The `startTime` argument offsets the `SampleDataEvent.position` of the stream, and `sndTransform` is applied to it; `loops` is ignored for now. Calling `Sound.load()` on a generated sound throws error ruffle-rs#2037, as in Flash Player.
Plays an unloaded `Sound` with a `SAMPLE_DATA` listener, synthesizing a 0.5-amplitude 440 Hz sine for ~0.5 seconds and then stopping. Audio volume assertions verify the mixed output peaks at ~0.5 while the tone plays, and is completely silent once the generator stops supplying samples - exercising the full generated-sound `SampleDataEvent` path end-to-end.
Covers ByteArray reuse across events, varying and short final block sizes, zero samples, stereo data, multiple concurrent generated sounds, the `startTime` argument of `Sound.play()`, and stopping the sound with `SoundChannel.stop()` and `SoundMixer.stopAll()`.
ba08101 to
8cb23ed
Compare
Fixes: #14156
Also fixes z0r.de loops: 5885, 7024 (issue #7879), 7312
Probably fixes (to be tested): #9171, #13655
It's quite rough (full of
unwrap()s for example, and overall unfinished) but it "just started working".Mostly pushing to show that it's in progress - any ideas on how to improve the architecture are welcome.
An unexpected (to me at least) problem is that sound synthesis in AS3 needs to do quite a lot of math really fast, so on my machine, even in release mode, the content linked above could barely keep up with real time sound generation, or not at all. Profiling and optimization is yet to be done (probably related: #7920).