Spatz mxcore

[DiyouS]

HW

• Clean up core complex
  • Separate CC sources from snitch_cluster subdir to snitch_cc subdir
  • Move tracer logic to snitch_tracer module
  • Move SSR subsystem logic to snitch_ssr_subsystem module
  • Move address-based reqrsp demux logic to new reqrsp_demux_mapped module
• Clean up interfaces
  • Legacy reqrsp_interface interface was effectively an LSU interface. Separate and rename this interface to lsu_interface.
  • reqrsp_interface proper is now an abstract interface, from which different concrete reqrsp interfaces can be created (e.g. dca_interface, fpu_interface, lsu_interface, etc.)
  • Generalize reqrsp IPs, VIPs and TBs to accept/test a generic reqrsp interface
  • Implement tcdm_interface and mem_interface based entirely on generic reqrsp_interface
• Conditionally add Spatz into core complex
  • XIF from core is demuxed to Spatz and external coprocessor port
  • TCDM ports are appended to CC's already existing TCDM ports
  • Spatz LSU port is muxed with Snitch and FPSS LSUs
• Rename Xf16 to Zfh extension everywhere
• Move hart_id_t from snitch_cluster_pkg to snitch_pkg
• Rename riscv_instr to snitch_riscv_instr to prevent name conflicts
• Separate reqrsp_interface and LSU IP/interface to dedicated repos. Allows to break the circular snitch_cluster → spatz → snitch_cluster(for LSU and reqrsp_interface) dependency, which now becomes snitch_cluster → spatz → LSU/reqrsp_interface

SW

• Add minimal Spatz test
• Add Spatz kernels
• Rename Xf16 to Zfh extension everywhere
• Only perform inter-cluster reduction in snrt_global_all_to_all_reduction if there is more than one cluster
• Add support for uint16 types to data_utils
• Add missing SNRT_SUPPORTS_SMALLFLOAT guard on naive FP16 GEMM

Make

• Replace DEBUG=ON for enabling traces with TRACE=OFF for disabling them (on by default). This allows to run performance experiments without having to run the simulation with +acc.
• Template toolchain.mk and derive toolchain flags directly from cluster configuration (SN_MCPU flag thus becomes obsolete)
  • This requires to break the Makefile read phase into multiple explicit stages, e.g. toolchain.mk has to have been built and read before the bootrom generation targets can be evaluated
• Update system integration docs to reflect toolchain.mk addition
• Redirect Verilator build stderr (in addition to stdout) output to log file
• Create separate Make goal for Verilator filelist
• Split bootrom goal into separate subgoals

CI

• Waive explicit-parameter-storage-type in riscv_instr.sv
• Upload Yosys artifacts on CI job failure
• Print calculated base SHA in CI for simplified debugging
• Fix base-sha job to pick up the upstream main branch even on forks
• Bump make-prerequisites-changed action to fix undetected file changes (see Convert prerequisites to workspace-relative paths before diffing colluca/make-prerequisites-changed#1)

Misc

• Checkout Bender dependencies before Python setup (Spatz needs to be checked out before its Python dependencies can be installed)
• Replace VCD with FST tracing in Verilator and enable tracing only if DEBUG=ON
• Add agentic waveform debugging flow based on tsunami
  • Bump required Python to >=3.12

TODO

• Add Spatz configuration and tests to CI
• Test in Gwaihir
• Bump to use new toolchain 22.1.7-pulp-0.2.0

[colluca]

Apart from the minor comments/changes that follow, I think we should make one more major change, specifically in the integration of the software.

While this could potentially become a huge effort, to keep it reasonable I would aim only for the following two points:

1. Instead of a separate sw/spatz folder, move its contents respectively to the sw/runtime directory and to the subdirectories of sw/kernels. If a specific kernel already exists, there is no need to merge the original and the Spatz version, at this stage. Let's simply keep two adjacent directories, e.g. axpy and spatz_axpy.
2. Ideally we should align with the data generation and verification infrastructure used by the other kernels. In the case of different implementations of the same kernel, e.g. axpy and spatz_axpy, to avoid copy-pasting the same scripts around we can symlink them.

[colluca]

Ideally, one would extend the generation script to:

1. enable invoking from any working directory, and
2. override the path where the output is to be generated.

These are merely suggestions, not strictly needed for this PR.
But particularly the latter is a safety feature that can avoid future headaches, e.g. if you're working on Spatz and the cluster simultaneously, and don't want that reconfiguring Spatz in the cluster repo overwrites the file in the Spatz repo, potentially affecting the parallel work on Spatz.

[colluca]

One more question which just came to my mind, why exactly do we need riscv_instr.sv in Snitch to include the vector instructions if we are not decoding them here?

Similarly, why do we need to use Snitch's riscv_instr.sv instead of Spatz' own file, within Spatz?

Or is the issue simply that we can't compile two packages with the same name or something like that?

[DiyouS]

One more question which just came to my mind, why exactly do we need riscv_instr.sv in Snitch to include the vector instructions if we are not decoding them here?

Similarly, why do we need to use Snitch's riscv_instr.sv instead of Spatz' own file, within Spatz?

Or is the issue simply that we can't compile two packages with the same name or something like that?

Because currently this file overwrites the one in spatz_vpu, and Spatz's decoder needs to decode these instructions. And the one in Spatz does not contain some features snitch cluster needed, e.g. SSR, post-increment

[colluca]

Because currently this file overwrites the one in spatz_vpu, and Spatz's decoder needs to decode these instructions. And the one in Spatz does not contain some features snitch cluster needed, e.g. SSR, post-increment

What I mean to ask is: what goes wrong when each repo has its own riscv_instr.sv file and no override is done (either through Bender or manually copying the file)? Does Snitch's file end up taking precedence during compilation such that also Spatz picks it up? Is that what you mean by "overwrites"? And if so, how is that a problem if manually overwriting Spatz' file we are achieving the same?

The only condition in which I see issues is if Spatz' overwrites Snitch's during compilation.

[DiyouS]

Because currently this file overwrites the one in spatz_vpu, and Spatz's decoder needs to decode these instructions. And the one in Spatz does not contain some features snitch cluster needed, e.g. SSR, post-increment

What I mean to ask is: what goes wrong when each repo has its own riscv_instr.sv file and no override is done (either through Bender or manually copying the file)? Does Snitch's file end up taking precedence during compilation such that also Spatz picks it up? Is that what you mean by "overwrites"? And if so, how is that a problem if manually overwriting Spatz' file we are achieving the same?

The only condition in which I see issues is if Spatz' overwrites Snitch's during compilation.

I think this is what happens in Gwaihir that Spatz' riscv_instr.sv is compiled later and override Snitch's during compilation

[colluca]

The only condition in which I see issues is if Spatz' overwrites Snitch's during compilation.

I think this is what happens in Gwaihir that Spatz' riscv_instr.sv is compiled later and override Snitch's during compilation

Okay, this is what we need to clarify. I believe Bender determines the compilation order according to the dependency DAG, so what you say shouldn't be the case. But I think the way Bender is used in Gwaihir is for it to produce a single compilation command. In that case, probably it can't handle name conflicts (e.g. multiple packages with the same name), and it might be that it filters out all but the first file with the same basename.

If this is the correct explanation, then the best solution would just be to namespace the riscv_instr.sv file and package: snitch_riscv_instr.sv in Snitch, and spatz_riscv_instr.sv in Spatz.