[stb_vorbis] Fix setup_malloc, setup_temp_malloc, vendor, comment, and multiplicands bounding. by AliceLR · Pull Request #1960 · nothings/stb

AliceLR · 2026-06-03T20:59:58Z

Marking as a draft for now since it's poorly tested, but this should fix #1933, #1947, and the second case in the comment #1168 (comment). However, it's not clear if this is the fix you actually want (particularly re: the definition of uint64 and various type-corrections).

@nothings I've noticed several dubious submissions duplicating parts of bug reports and patches I submitted years ago. Is there anything I can do to help?

Expands the setup_malloc size parameter, and various calculations that the Vorbis spec allows to exceed 32 bits, to uint64. setup_malloc and setup_temp_malloc now reject allocations that exceed INT_MAX, as the underlying memory fields are still 32 bit signed integers.

AliceLR · 2026-06-05T00:20:08Z

I've replaced the simple typedef for uint64 with a more robust version copied from libxmp's common.h. This fragment is probably non-copyrightable.
I removed the creeping type corrections and reduced this patch down to what should be the bare minimum to fix the affected setup_malloc, setup_temp_malloc, and setup_temp_free calls.

Expands the setup_malloc size parameter, and various calculations that the Vorbis spec allows to exceed 32 bits, to uint64. setup_malloc and setup_temp_malloc still reject allocations that exceed INT_MAX, as the underlying memory fields are still 32 bit signed integers. Mainstream P/R: nothings/stb#1960 Reference issue: libxmp#996 Also see: nothings/stb#1947 Also see: nothings/stb#1933 Also see: nothings/stb#1168 (comment)

Expands the setup_malloc size parameter, and various calculations that the Vorbis spec allows to exceed 32 bits, to uint64. setup_malloc and setup_temp_malloc still reject allocations that exceed INT_MAX, as the underlying memory fields are still 32 bit signed integers. Mainstream P/R: nothings/stb#1960 Reference issue: #996 Also see: nothings/stb#1947 Also see: nothings/stb#1933 Also see: nothings/stb#1168 (comment) Co-authored-by: AliceLR <petrifiedrowan@gmail.com>

Expands the setup_malloc size parameter, and various calculations that the Vorbis spec allows to exceed 32 bits, to uint64. setup_malloc and setup_temp_malloc still reject allocations that exceed INT_MAX, as the underlying memory fields are still 32 bit signed integers. Reference issue: libxmp/libxmp#996 Reference issue: nothings#1947 Reference issue: nothings#1933 Also see : nothings#1168 (comment) Mainstream P/R : nothings#1960

Expands the setup_malloc size parameter, and various calculations that the Vorbis spec allows to exceed 32 bits, to uint64. setup_malloc and setup_temp_malloc still reject allocations that exceed INT_MAX, as the underlying memory fields are still 32 bit signed integers. Reference issue: libxmp/libxmp#996 Reference issue: nothings/stb#1947 Reference issue: nothings/stb#1933 Also see : nothings/stb#1168 (comment) Mainstream P/R : nothings/stb#1960 (cherry picked from commit 19ae8a8)

Expands the setup_malloc size parameter, and various calculations that the Vorbis spec allows to exceed 32 bits, to uint64. setup_malloc and setup_temp_malloc still reject allocations that exceed INT_MAX, as the underlying memory fields are still 32 bit signed integers. Reference issue: libxmp/libxmp#996 Reference issue: nothings/stb#1947 Reference issue: nothings/stb#1933 Also see : nothings/stb#1168 (comment) Mainstream P/R : nothings/stb#1960

Expands the setup_malloc size parameter, and various calculations that the Vorbis spec allows to exceed 32 bits, to uint64. setup_malloc and setup_temp_malloc still reject allocations that exceed INT_MAX, as the underlying memory fields are still 32 bit signed integers. Reference issue: libxmp/libxmp#996 Reference issue: nothings/stb#1947 Reference issue: nothings/stb#1933 Also see : nothings/stb#1168 (comment) Mainstream P/R : nothings/stb#1960 (cherry picked from commit 369279b)

Expands the setup_malloc size parameter, and various calculations that the Vorbis spec allows to exceed 32 bits, to uint64. setup_malloc and setup_temp_malloc still reject allocations that exceed INT_MAX, as the underlying memory fields are still 32 bit signed integers. Reference issue: libxmp/libxmp#996 Reference issue: nothings/stb#1947 Reference issue: nothings/stb#1933 Also see : nothings/stb#1168 (comment) Mainstream P/R : nothings/stb#1960

Expands the setup_malloc size parameter, and various calculations that the Vorbis spec allows to exceed 32 bits, to uint64. setup_malloc and setup_temp_malloc still reject allocations that exceed INT_MAX, as the underlying memory fields are still 32 bit signed integers. Reference issue: libxmp/libxmp#996 Reference issue: nothings/stb#1947 Reference issue: nothings/stb#1933 Also see : nothings/stb#1168 (comment) Mainstream P/R : nothings/stb#1960 (cherry picked from commit 369279b)

Expands the setup_malloc size parameter, and various calculations that the Vorbis spec allows to exceed 32 bits, to uint64. setup_malloc and setup_temp_malloc still reject allocations that exceed INT_MAX, as the underlying memory fields are still 32 bit signed integers. Reference issue: libxmp/libxmp#996 Reference issue: nothings/stb#1947 Reference issue: nothings/stb#1933 Also see : nothings/stb#1168 (comment) Mainstream P/R : nothings/stb#1960

Expands the setup_malloc size parameter, and various calculations that the Vorbis spec allows to exceed 32 bits, to uint64. setup_malloc and setup_temp_malloc still reject allocations that exceed INT_MAX, as the underlying memory fields are still 32 bit signed integers. Reference issue: libxmp/libxmp#996 Reference issue: nothings/stb#1947 Reference issue: nothings/stb#1933 Also see : nothings/stb#1168 (comment) Mainstream P/R : nothings/stb#1960 (cherry picked from commit 52afb20)

Backported from libsdl-org/SDL_mixer@369279b xpands the setup_malloc size parameter, and various calculations that the Vorbis spec allows to exceed 32 bits, to uint64. setup_malloc and setup_temp_malloc still reject allocations that exceed INT_MAX, as the underlying memory fields are still 32 bit signed integers. Reference issue: libxmp/libxmp#996 Reference issue: nothings/stb#1947 Reference issue: nothings/stb#1933 Also see : nothings/stb#1168 (comment) Mainstream P/R : nothings/stb#1960

nothings · 2026-10-25T05:09:17Z

Without putting too much thought into it, I think my preference would be that it's okay to use 64-bit to compute the products but to still bail if the size is larger than 32-bit, since it's surely a problematic file anyway, i.e. 64-bit locals okay, but not in structs.

AliceLR · 2026-10-25T05:11:34Z

Without putting too much thought into it, I think my preference would be that it's okay to use 64-bit to compute the products but to still bail if the size is larger than 32-bit, since it's surely a problematic file anyway, i.e. 64-bit locals okay, but not in structs.

That sounds reasonable. I will update this patch when I get a chance.

nothings · 2026-10-25T05:13:38Z

Also, the whole use of floor(pow) and floor(exp(log())) for integer powers, seems crazy to me, it should probably be done by repeated multiplication, or at least rounding with floor(...+0.5)? Weird that I did it that way in the first place.

nothings · 2026-10-25T05:16:23Z

Yeah, maybe better to just use #2021 .

AliceLR · 2026-10-25T06:45:16Z

Yeah, maybe better to just use #2021 .

That's also reasonable IMO; it is essentially what I would have implemented if you had rejected uint64 before that PR was opened.

AliceLR · 2026-10-25T07:27:00Z

I've pushed a second commit that limits uint64 to the bounding only. Aside from (I think) covering a few more integer overflow edge cases, it's not looking that different from #2021.

AliceLR · 2026-10-26T02:34:17Z

Removed uint64 altogether per #1933 (comment).

rygorous · 2026-10-26T09:45:55Z

At long last merged into dev branch, will be in the next release.

AliceLR mentioned this pull request Jun 3, 2026

Integer overflow → heap buffer overflow in OGG Vorbis loader (stb_vorbis codebook multiplicands) libxmp/libxmp#996

Closed

sezero mentioned this pull request Jun 4, 2026

Do an official SDL3 release. icculus/SDL_sound#125

Closed

AliceLR force-pushed the fix-setup-malloc branch from f5b1507 to 66fc4c6 Compare June 5, 2026 00:13

AliceLR marked this pull request as ready for review June 5, 2026 00:20

sezero mentioned this pull request Jun 5, 2026

stb_vorbis: Increase setup_malloc() size parameter to uint64 libxmp/libxmp#999

Merged

This was referenced Oct 25, 2026

stb_vorbis: prevent setup allocation size truncation #2019

Draft

stb_vorbis: reject header sizes that overflow setup_malloc() #2021

Open

uint64 multiplicands bounding, take 2 (please squash).

b895b0f

AliceLR force-pushed the fix-setup-malloc branch from 7b777aa to b895b0f Compare October 25, 2026 07:18

AliceLR mentioned this pull request Oct 26, 2026

stb_vorbis: integer overflow in codebook multiplicands allocation #1933

Open

AliceLR changed the title ~~[stb_vorbis] Increase setup_malloc size parameter to uint64.~~ [stb_vorbis] Fix setup_malloc, setup_temp_malloc, vendor, comment, and multiplicands bounding. Oct 26, 2026

Remove uint64 altogether and use CODEBOOK_LIMIT instead (squash).

4a74952

rygorous added 1 stb_vorbis 5 merged-dev Merged into development branch labels Oct 26, 2026

AliceLR mentioned this pull request Oct 27, 2026

stb_vorbis: slow loads caused by sorting c->sorted_codewords. #1249

Open

Navigation Menu

Repository navigation

[stb_vorbis] Fix setup_malloc, setup_temp_malloc, vendor, comment, and multiplicands bounding. - #1960

[stb_vorbis] Fix setup_malloc, setup_temp_malloc, vendor, comment, and multiplicands bounding.#1960
AliceLR wants to merge 3 commits into
nothings:masterfrom
AliceLR:fix-setup-malloc

AliceLR commented Jun 3, 2026

Uh oh!

AliceLR commented Jun 5, 2026

Uh oh!

nothings commented Oct 25, 2026

Uh oh!

AliceLR commented Oct 25, 2026

Uh oh!

nothings commented Oct 25, 2026

Uh oh!

nothings commented Oct 25, 2026

Uh oh!

AliceLR commented Oct 25, 2026

Uh oh!

AliceLR commented Oct 25, 2026

Uh oh!

AliceLR commented Oct 26, 2026

Uh oh!

rygorous commented Oct 26, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

AliceLR commented Jun 3, 2026

Uh oh!

AliceLR commented Jun 5, 2026

Uh oh!

nothings commented Oct 25, 2026

Uh oh!

AliceLR commented Oct 25, 2026

Uh oh!

nothings commented Oct 25, 2026

Uh oh!

nothings commented Oct 25, 2026

Uh oh!

AliceLR commented Oct 25, 2026

Uh oh!

AliceLR commented Oct 25, 2026

Uh oh!

AliceLR commented Oct 26, 2026

Uh oh!

rygorous commented Oct 26, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants