Conversation
This commit adds comprehensive support for AMD APU architectures,
specifically targeting the MI300A with energy monitoring capabilities
via the ROCm SMI rsmi_dev_energy_count_get() API.
Major changes:
- Created new AMD_APU directory structure under src/variorum/AMD_APU/
- config_amd_apu.{c,h}: APU detection and function pointer registration
- amd_apu_power_features.{c,h}: Layer 1 ROCm-SMI API wrappers including
energy monitoring (get_energy_data, get_energy_json)
- mi300aAPU.{c,h}: Layer 2 MI300A-specific wrappers
- CMakeLists.txt: Build configuration for AMD_APU
- Updated config_architecture.{c,h}:
- Added amd_apu_arch_e enum with AMD_MI300A = 1
- Added P_AMD_APU_IDX to supported_platforms_e
- Integrated APU detection and function pointer registration
- Increased MAX_PLATFORMS from 2 to 3
- Updated build system:
- src/CMakeLists.txt: Added VARIORUM_WITH_AMD_APU option
- src/variorum/CMakeLists.txt: Added AMD_APU subdirectory and build rules
- src/CMake/Setup3rdParty.cmake: Include FindROCM.cmake for APU support
Energy monitoring features:
- variorum_get_energy_json(): Returns cumulative energy counters in microjoules
- variorum_print_energy(): Prints energy data in standard Variorum format
- Uses ROCm SMI rsmi_dev_energy_count_get() API
- Gracefully handles unsupported devices (RSMI_STATUS_NOT_SUPPORTED)
The implementation follows Variorum's three-layer architecture pattern
and mirrors the existing AMD_GPU structure for consistency.
Co-Authored-By: Claude <noreply@anthropic.com>
1) Update AMD APU to use ROCm 7 APIs exclusively
Simplified power monitoring implementation to use only ROCm 7+ APIs,
removing legacy ROCm 5 compatibility code.
Changes:
- Removed ROCM_VERSION_MAJOR preprocessor checks
- Removed rocm-core/rocm_version.h include (not needed)
- Use rsmi_dev_power_get() exclusively (recommended for ROCm 7+)
- This API automatically tries current power first, falls back to average
- Returns RSMI_POWER_TYPE indicating which type was retrieved
- Provides best compatibility across APU generations
- Removed deprecated rsmi_dev_power_ave_get() calls
Benefits:
- Cleaner, simpler code without version conditionals
- Uses recommended ROCm 7 APIs
- Better forward compatibility
- Automatically adapts to device capabilities
The rsmi_dev_power_get() API automatically handles the differences
between devices that support current vs average power, making it
the preferred choice for ROCm 7+.
2) Implement complete power and thermal monitoring APIs
Added full implementations for power and thermal monitoring functions
in AMD APU support, mirroring the AMD_GPU patterns.
Power Monitoring:
- get_power_data(): Print power consumption per device
- Uses rsmi_dev_power_get() for ROCm 6+ or rsmi_dev_power_ave_get() for ROCm 5
- Reports power in Watts (converted from microwatts)
- Handles RSMI_STATUS_NOT_SUPPORTED gracefully
- get_json_power_data(): JSON format power data
- Per-socket and per-device power in watts
- Aggregates total node power (power_node_watts)
- Compatible with multi-socket systems
Thermal Monitoring:
- get_thermals_data(): Print temperature per device
- Uses rsmi_dev_temp_metric_get() with RSMI_TEMP_TYPE_EDGE
- Reports edge temperature (die edge) in Celsius
- Converts from millidegrees to Celsius
- get_thermals_json(): JSON format thermal data
- Per-device temperatures in Celsius
- Organized by socket
Output Format Examples:
_AMD_APU_POWER_USAGE Host Socket DeviceID Power Timestamp_sec
_AMD_APU_TEMPERATURE Host Socket DeviceID Temperature Timestamp_sec
JSON: {"socket_0": {"power_apu_watts": {"APU_0": 150.5}, "APU": {"temp_celsius_apu_0": 65.2}}}
All functions include proper error handling, ROCm version compatibility,
and follow Variorum's established formatting patterns.
3) Fix: Correct rsmi_dev_energy_count_get() API signature
Updated to use the correct 4-parameter API signature for rsmi_dev_energy_count_get():
- Parameter 1: uint32_t dv_ind (device index)
- Parameter 2: uint64_t *power (energy counter in microjoules)
- Parameter 3: float *counter_resolution (resolution in microjoules)
- Parameter 4: uint64_t *timestamp (timestamp in nanoseconds)
Changes:
- Fixed get_energy_data() to use all 4 parameters
- Fixed get_energy_json() to use all 4 parameters
- Updated JSON output to include counter_resolution
- Changed timestamp from float to uint64_t (nanoseconds)
- Updated JSON field names to reflect units (_ns, _uJ)
- Added comments documenting each parameter
Reference: ROCm SMI Lib documentation v6.0.0
https://rocm.docs.amd.com/projects/rocm_smi_lib/en/docs-6.0.0/.doxygen/docBin/html/group__PowerQuer.html#ga1f61b24edaca83a0e395a34c466fcf86
4) Fix: Add VARIORUM_WITH_AMD_APU to variorum_config.h.in
The P_AMD_APU_IDX constant was undefined during compilation because
the VARIORUM_WITH_AMD_APU preprocessor macro wasn't being defined.
This adds the CMake configuration variable to variorum_config.h.in
so it gets properly defined when AMD_APU support is enabled.
Co-Authored-By: Claude <noreply@anthropic.com>
more than 2 not possible
|
Spent a few hours debugging the segfault with (Caveat: Not sure I trust claude here, but I think it is heading in the right direction based on my independent debugging of this as well. Worth looking into a local build of ================ Problem: Fix:
Option 1 is simpler and more robust long-term since Variorum doesn't use hwloc's GPU features at all; option 2 is worth it only if something does need hwloc's GPU-locality API later. |
|
Update from 7/21.
|
|
Latest TODO
Update 7/17 end of hackathon:
--> On tioga, you were able to collect energy with sudo using RSMI API (only CPU or both CPU and GPU)
This commit adds comprehensive support for AMD APU architectures, specifically targeting the MI300A with energy monitoring capabilities via the ROCm SMI rsmi_dev_energy_count_get() API.
Major changes:
Created new AMD_APU directory structure under src/variorum/AMD_APU/
config_amd_apu.{c,h}: APU detection and function pointer registration
amd_apu_power_features.{c,h}: Layer 1 ROCm-SMI API wrappers including energy monitoring (get_energy_data, get_energy_json)
mi300aAPU.{c,h}: Layer 2 MI300A-specific wrappers
CMakeLists.txt: Build configuration for AMD_APU
Updated config_architecture.{c,h}:
Added amd_apu_arch_e enum with AMD_MI300A = 1
Added P_AMD_APU_IDX to supported_platforms_e
Integrated APU detection and function pointer registration
Increased MAX_PLATFORMS from 2 to 3
Updated build system:
src/CMakeLists.txt: Added VARIORUM_WITH_AMD_APU option
src/variorum/CMakeLists.txt: Added AMD_APU subdirectory and build rules
src/CMake/Setup3rdParty.cmake: Include FindROCM.cmake for APU support
Energy monitoring features:
The implementation follows Variorum's three-layer architecture pattern and mirrors the existing AMD_GPU structure for consistency.
Description
Please provide a short summary of the change and tag any issues being fixed. Please also include relevant motivation
and context. List any dependencies that may be required for this change.
Fixes #
Type of change
How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Please provide hardware architecture specs and
instructions so we can reproduce.
Checklist:
./scripts/check-code-format.shand confirm my code code follows the style guidelines of variorum-DENABLE_WARNINGS=ON)Thank you for taking the time to contribute to Variorum!