Skip to content

feat(qairt): enable HTP multicore for QAIRT bundles #1313

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
David Qian (Davidqian123) merged 1 commit into from
Sep 9, 2026
Merged

feat(qairt): enable HTP multicore for QAIRT bundles #1313

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions notes/run.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,31 @@ QnnHtp.dll (looked in the folder itself and lib/aarch64-windows-msvc): <path>`.

</details>

### HTP multicore (qairt)

@Davidqian123 David Qian (Davidqian123) Sep 9, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you simplify this section a little bit, the most important concept we should explain here is how to configrue multicore and run with multicore.


By default a QAIRT model executes on **one** NSP core. The knob is the bundle's
`htp_backend_ext_config.json`: the QAIRT core counts the `devices[].cores` entries
and, when more than one is listed, sets
`QNN_HTP_GRAPH_CONFIG_OPTION_NUM_CORES` on every loaded graph
(`ModelConfig::num_cores` in the plugin; see
[geniex-qairt docs § HTP Backend Config and Multicore Execution](../third-party/geniex-qairt/docs/README.md#5-htp-backend-config-and-multicore-execution)).

What you can and cannot change at load time:

- **Load time**: the `NUM_CORES` graph config, per-core `perf_profile`, and
`rpc_control_latency` — all read from the JSON when the model loads.
- **Generation time**: graph partitioning baked into the context binaries.
Binaries compiled single-core may not speed up (or may reject the config)
even when more cores are requested.

At init the runtime logs the device-reported core count and the effective core
count (`--log info`), warns and clamps when the JSON requests more cores
than the SoC exposes, and warns without failing when the driver reports
multicore unavailable (QNN verbose trace: `Error code 1000 ... key = 304` /
`Multicore support is unavailable`). Most current Snapdragon mobile/compute
SoCs expose a single NSP core; multi-NSP parts (e.g. certain automotive SoCs)
report `numCores > 1` in QNN platform info.

### Build and run locally

```bash
Expand Down
Loading