GLideNHQ: zstd compression for texture cache

[GhostlyDark]

Supersedes #2960. Based on #2968 (may want to merge that one first).

Compared to the previous PR, this is based on the vcpkg introduction and gets rid of the zstd.c and zstd.h source files.

We still have to deal with the zlib symbol conflicts as it stands, so recompiling the Qt5 binaries is a must (with -system-zlib instead of -qt-zlib during configure step). I am unfortunately still very much unable to get things to compile, but binary edited the Qt5Core.lib to "fix" the symbol conflicts for the zstd_test branch of my fork, with the edited Qt5 prebuilt uploaded there. From my testing, this does not cause any regressions and can be confirmed using the builds here:
https://github.com/GhostlyDark/GLideN64/actions/runs/27233155351

Note that I didn't touch the debug Qt5Cored.lib files as CI isn't using it and Qt5 should preferably be recompiled anyway.

---

This adds zstd compression support by using a zlibWrapper to support both zlib and zstd compression streams.
Closes #2913.

Initial size comparisons (HTS cache) for OoT Reloaded texture pack:
uncompressed: 28.5 GB (30.694.669.834 bytes)
zlib-level1: 8.28 GB (8 897 605 579 bytes)
zstd-level1: 8.69 GB (9 334 399 841 bytes)
zstd-level9: 6.80 GB (7 307 463 076 bytes)

Do note that level9 in this case is just the level9 compression level of zlib translated by the wrapper, not the actual zstd compression level (which ranges up to 22).

An HTS file using zstd outperforms any zlib compressed one, reducing stutter quite noticeably. I did however notice that uncompressed cache files still perform better (tested on a RAMDisk to rule out drive speed bottlenecks). The zlibWrapper causes some overhead, so the wrapper itself introduces a performance degradation that otherwise wouldn't be there (unknown by how much exactly). It should be possible to use the zstd API directly instead, which would involve recognizing the correct compression format and handling the zstd compression with new code - something I don't feel comfortable tackling myself at the time of writing.

Numbers from the zstd repo, which I believe to be somewhat accurate:

[gonetz]

Why do we need that file?

[GhostlyDark]

It's the equivalent to libpng.a and libz.a.

[gonetz]

Ok. I supposed that it should be system, but some platforms really use .a libs from GLideNHQ/lib folder.
I already forgot these details :(

[gonetz]

recompiling the Qt5 binaries is a must (with -system-zlib instead of -qt-zlib during configure step)

I just tried to configure QT build with -system-zlibinstead of-qt-zlib` and got this error:
ERROR: Feature 'system-zlib' was enabled, but the pre-condition 'libs.zlib' failed.
I didn't find a way to fix/bypass it, but I have no zlib symbol conflicts with my current QT static libs.
Could you test it on your side?

[GhostlyDark]

recompiling the Qt5 binaries is a must (with -system-zlib instead of -qt-zlib during configure step)

I just tried to configure QT build with -system-zlibinstead of-qt-zlib` and got this error: ERROR: Feature 'system-zlib' was enabled, but the pre-condition 'libs.zlib' failed. I didn't find a way to fix/bypass it, but I have no zlib symbol conflicts with my current QT static libs. Could you test it on your side?

The conflict still exists. You can confirm this by creating a cache file using a build from this PR, then trying to load that exact cache file with GLideN64 built from master. If the HD textures load on the master build, then the cache file is compressed with zlib. If it's compressed with zstd, the HD textures won't load on the master branch as support for that compression isn't in there.

I assume some of the symbols are kicked out during the compilation process, as a "proper" build of GLideN64 with zstd fully working is larger than one where the conflict kicked in: 11.300.352 vs 11.451.904 bytes (comparing PJ64 x86 Legacy QT).

If we can't get around that -qt-zlib nonsense, then we would need to get rid of the zstd_zlibwrapper and use the zstd API directly, using zstd to compress the cache and decompressing either with zstd or zlib depending on how the cache file has been created (for compatibility).

[gonetz]

I'll build QT without -qt-zlib and check if it breaks anything. But not today, it is 0:28 AM already.

[gonetz]

I'll build QT without -qt-zlib and check if it breaks anything.

Done, but it didn't solve the problem.

However, it was tricky, but I built QT libs with -system-zlib.
And it seems that it solves the problem with zlib symbol conflicts.
I created a cache file with the branch build. Cache size has changed.
master branch build can't open that cache file.
I hope, it is because zstd compression is finally works.
I updated qt_build release with the new QT build:
https://github.com/gonetz/GLideN64/releases/tag/qt_build

@GhostlyDark please check that zstd compression really works with the updated QT libs.

[GhostlyDark]

I'll build QT without -qt-zlib and check if it breaks anything.

Done, but it didn't solve the problem.

However, it was tricky, but I built QT libs with -system-zlib. And it seems that it solves the problem with zlib symbol conflicts. I created a cache file with the branch build. Cache size has changed. master branch build can't open that cache file. I hope, it is because zstd compression is finally works. I updated qt_build release with the new QT build: https://github.com/gonetz/GLideN64/releases/tag/qt_build

@GhostlyDark please check that zstd compression really works with the updated QT libs.

Yes, it works with the updated QT prebuilts as intended.