engine/schema: fix new systemvm template is not registered during upgrade if hypervisor is not KVM by weizhouapache · Pull Request #12952 · apache/cloudstack

weizhouapache · 2026-04-02T15:07:43Z

Description

This PR fixes the issue that the new systemvm template is not registered for non-KVM hypervisor types

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
Build/CI
Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

…rade if hypervisor is not KVM

weizhouapache · 2026-04-02T15:07:56Z

@blueorangutan package

blueorangutan · 2026-04-02T15:10:05Z

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-04-02T15:18:20Z

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 17343

weizhouapache · 2026-04-02T15:23:30Z

@blueorangutan package

blueorangutan · 2026-04-02T15:24:05Z

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

sureshanaparti · 2026-04-02T15:24:14Z

engine/schema/src/main/java/com/cloud/upgrade/SystemVmTemplateRegistration.java

+            new Pair<>(Hypervisor.HypervisorType.XenServer, CPU.CPUArch.getDefault()),
+            new Pair<>(Hypervisor.HypervisorType.Hyperv, CPU.CPUArch.getDefault()),
+            new Pair<>(Hypervisor.HypervisorType.LXC, CPU.CPUArch.getDefault()),
+            new Pair<>(Hypervisor.HypervisorType.Ovm3, CPU.CPUArch.getDefault())


@weizhouapache when arch type is null, it's defaulted to amd64, is null arch issue here?

cloudstack/engine/schema/src/main/java/com/cloud/upgrade/SystemVmTemplateRegistration.java

Lines 402 to 408 in e2497cf

protected static String getHypervisorArchKey(Hypervisor.HypervisorType hypervisorType, CPU.CPUArch arch) {

if (Hypervisor.HypervisorType.KVM.equals(hypervisorType)) {

return String.format("%s-%s", hypervisorType.name().toLowerCase(),

arch == null ? CPU.CPUArch.amd64.getType() : arch.getType());

}

return hypervisorType.name().toLowerCase();

}

also, make note of the changes in the below PR in main (no need of forward merging the changes here to main)

https://github.com/apache/cloudstack/pull/11656/changes#diff-a9c9a38684718059c060de404bf9529de96e502f1e81b30793e6a32f725042a9

@sureshanaparti
in another place

the arch of MetadataTemplateDetails is from hypervisorType, which is null for non-KVM

NewTemplateMap.put(key, new MetadataTemplateDetails( hypervisorType.first(), section.get("templatename"), section.get("filename"), section.get("downloadurl"), section.get("checksum"), hypervisorType.second(), section.get("guestos")));

but in hypervisorsInUse, the arch is amd64/x86_64 (default arch of clusters)

cloudstack/engine/schema/src/main/java/com/cloud/upgrade/SystemVmTemplateRegistration.java

Line 1068 in e2497cf

hypervisorsInUse = clusterDao.listDistinctHypervisorsArchAcrossClusters(null);

cluster arch

mysql> SELECT DISTINCT hypervisor_type,arch from cluster; +-----------------+--------+ | hypervisor_type | arch | +-----------------+--------+ | VMware | x86_64 | +-----------------+--------+ 1 row in set (0.00 sec)

so the check always return false (line 1034)

cloudstack/engine/schema/src/main/java/com/cloud/upgrade/SystemVmTemplateRegistration.java

Lines 1032 to 1034 in e2497cf

boolean isHypervisorArchMatchMetadata = hypervisorsInUse.stream()

.anyMatch(p -> p.first().equals(templateDetails.getHypervisorType())

&& Objects.equals(p.second(), templateDetails.getArch()));

also, make note of the changes in the below PR in main (no need of forward merging the changes here to main)

https://github.com/apache/cloudstack/pull/11656/changes#diff-a9c9a38684718059c060de404bf9529de96e502f1e81b30793e6a32f725042a9

good point @sureshanaparti
so the issue should not appear in 4.22, great

ok @weizhouapache, how about replacing hypervisorType.second() == null ? CPU.CPUArch.amd64.getType() : hypervisorType.second().getType() at

cloudstack/engine/schema/src/main/java/com/cloud/upgrade/SystemVmTemplateRegistration.java

Line 837 in e2497cf

hypervisorType.second(),

maybe, move it to method and use the call same method here and in getHypervisorArchKey()

private String getCPUArchType(CPU.CPUArch arch) { if (arch == null) { return CPU.CPUArch.amd64.getType(); } return arch.getType(); }

codecov · 2026-04-02T15:36:38Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 16.26%. Comparing base (e2497cf) to head (57e1850).
⚠️ Report is 1 commits behind head on 4.20.

Additional details and impacted files

@@            Coverage Diff            @@
##               4.20   #12952   +/-   ##
=========================================
  Coverage     16.26%   16.26%           
- Complexity    13433    13434    +1     
=========================================
  Files          5665     5665           
  Lines        500530   500530           
  Branches      60787    60787           
=========================================
+ Hits          81395    81417   +22     
+ Misses       410044   410021   -23     
- Partials       9091     9092    +1

Flag	Coverage Δ
uitests	`4.15% <ø> (ø)`
unittests	`17.12% <ø> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

blueorangutan · 2026-04-02T16:31:55Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17344

weizhouapache · 2026-04-02T18:04:28Z

@blueorangutan test

blueorangutan · 2026-04-02T18:06:04Z

@weizhouapache a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

shwstppr

code lgtm

blueorangutan · 2026-04-03T10:10:11Z

[SF] Trillian test result (tid-15809)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 54910 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12952-t15809-kvm-ol8.zip
Smoke tests completed. 141 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

RosiKyu

LGTM

RosiKyu

MS Oracle Linux 8 Hypervisor VMware7:

Pre-upgrade template: systemvmtemplate-4.20.0-x86_64-vmware.ova
Post-upgrade Successfully installed log shows 4.20.2 to new directory 202
Secondary storage has new directory 202 with systemvmtemplate-4.20.2 files
DB has new template record systemvm-vmware-4.20.2-x86_64 with 4.20.2 URL
CPVM v-8-VM created from systemvm-vmware-4.20.2-x86_64
SSVM s-9-VM created from systemvm-vmware-4.20.2-x86_64
VR r-15-VM created from systemvm-vmware-4.20.2-x86_64

Directory 8 - systemvmtemplate-4.20.0-x86_64-vmware - initial template, seeded at env build time
Directory 202 - systemvmtemplate-4.20.2-x86_64-vmware - new template registered post-upgrade

[root@sl-nestednfs ~]# tree /acs/secondary/ref-trl-11496-v-Mol8-rositsa-kyuchukova/ref-trl-11496-v-Mol8-rositsa-kyuchukova-sec1/template/tmpl/
/acs/secondary/ref-trl-11496-v-Mol8-rositsa-kyuchukova/ref-trl-11496-v-Mol8-rositsa-kyuchukova-sec1/template/tmpl/
└── 1
    ├── 202
    │   ├── 9cd384be-becd-49aa-9673-5399326f2965.ova
    │   ├── systemvmtemplate-4.20.2-x86_64-vmware-disk1.vmdk
    │   ├── systemvmtemplate-4.20.2-x86_64-vmware.mf
    │   ├── systemvmtemplate-4.20.2-x86_64-vmware.ovf
    │   └── template.properties
    ├── 7
    │   ├── 0654e777-1660-3d44-a937-431b53f15ed7.ova
    │   ├── centos53-x64.ova
    │   ├── centos53-x64.ova.meta
    │   ├── CentOS5.3-x86_64-disk1.vmdk
    │   ├── CentOS5.3-x86_64.mf
    │   ├── CentOS5.3-x86_64.ovf
    │   ├── CentOS5.3-x86_64.ovf.orig
    │   └── template.properties
    └── 8
        ├── f60c4852-dd50-4251-afdd-ecef5a8c251f.ova
        ├── systemvmtemplate-4.20.0-x86_64-vmware-disk1.vmdk
        ├── systemvmtemplate-4.20.0-x86_64-vmware.mf
        ├── systemvmtemplate-4.20.0-x86_64-vmware.ovf
        └── template.properties

4 directories, 18 files

DB results:

mysql> SELECT t.name, t.url, ts.state, ts.download_state, ts.download_pct FROM vm_template t JOIN template_store_ref ts ON t.id = ts.template_id WHERE t.type='SYSTEM' AND t.removed IS NULL AND ts.destroyed=0;
+-------------------------------+-----------------------------------------------------------------------------------------+-------+----------------+--------------+
| name                          | url                                                                                     | state | download_state | download_pct |
+-------------------------------+-----------------------------------------------------------------------------------------+-------+----------------+--------------+
| SystemVM Template (vSphere)   | https://download.cloudstack.org/systemvm/4.20/systemvmtemplate-4.20.0-x86_64-vmware.ova | Ready | DOWNLOADED     |          100 |
| systemvm-vmware-4.20.2-x86_64 | https://download.cloudstack.org/systemvm/4.20/systemvmtemplate-4.20.2-x86_64-vmware.ova | Ready | DOWNLOADED     |          100 |
+-------------------------------+-----------------------------------------------------------------------------------------+-------+----------------+--------------+
2 rows in set (0.00 sec)

SSVMs (recreated post-upgrade):
v-8-VM (CPVM) - systemvm-vmware-4.20.2-x86_64
s-9-VM (SSVM) - systemvm-vmware-4.20.2-x86_64
VR (created post-upgrade):

r-15-VM - systemvm-vmware-4.20.2-x86_64

All three are using the new 4.20.2 template. The fix works correctly - new template registered, new template record created in DB, and all System VMs are being deployed from it.

mysql> SELECT vi.name, vi.type, vi.created, t.name as template_name, t.url 
    -> FROM vm_instance vi 
    -> JOIN vm_template t ON vi.vm_template_id = t.id 
    -> WHERE vi.type IN ('ConsoleProxy', 'SecondaryStorageVm') AND vi.state = 'Running';
+--------+--------------------+---------------------+-------------------------------+-----------------------------------------------------------------------------------------+
| name   | type               | created             | template_name                 | url                                                                                     |
+--------+--------------------+---------------------+-------------------------------+-----------------------------------------------------------------------------------------+
| v-8-VM | ConsoleProxy       | 2026-04-05 16:57:07 | systemvm-vmware-4.20.2-x86_64 | https://download.cloudstack.org/systemvm/4.20/systemvmtemplate-4.20.2-x86_64-vmware.ova |
| s-9-VM | SecondaryStorageVm | 2026-04-05 16:57:07 | systemvm-vmware-4.20.2-x86_64 | https://download.cloudstack.org/systemvm/4.20/systemvmtemplate-4.20.2-x86_64-vmware.ova |
+--------+--------------------+---------------------+-------------------------------+-----------------------------------------------------------------------------------------+
2 rows in set (0.00 sec)

mysql> SELECT vi.name, vi.type, vi.created, t.name as template_name, t.url 
    -> FROM vm_instance vi 
    -> JOIN vm_template t ON vi.vm_template_id = t.id 
    -> WHERE vi.type = 'DomainRouter' AND vi.state = 'Running';
+---------+--------------+---------------------+-------------------------------+-----------------------------------------------------------------------------------------+
| name    | type         | created             | template_name                 | url                                                                                     |
+---------+--------------+---------------------+-------------------------------+-----------------------------------------------------------------------------------------+
| r-15-VM | DomainRouter | 2026-04-06 03:53:23 | systemvm-vmware-4.20.2-x86_64 | https://download.cloudstack.org/systemvm/4.20/systemvmtemplate-4.20.2-x86_64-vmware.ova |
+---------+--------------+---------------------+-------------------------------+-----------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

Results from MS logs:

[root@ref-trl-11496-v-Mol8-rositsa-kyuchukova-mgmt1 ~]# zgrep "Successfully installed system VM template" /var/log/cloudstack/management/management-server.log.2026-04-05.gz
+ echo 'Successfully installed system VM template /usr/share/cloudstack-management/templates/systemvm/systemvmtemplate-4.20.2-x86_64-vmware.ova and template.properties to /tmp/tmp4879423099328784851/template/tmpl/1/202'
Successfully installed system VM template /usr/share/cloudstack-management/templates/systemvm/systemvmtemplate-4.20.2-x86_64-vmware.ova and template.properties to /tmp/tmp4879423099328784851/template/tmpl/1/202

abh1sar · 2026-04-06T09:15:15Z

@weizhouapache @sureshanaparti
Are the pending comments answered? Can this be merged as is?

sureshanaparti · 2026-04-06T11:16:35Z

@weizhouapache @sureshanaparti Are the pending comments answered? Can this be merged as is?

minor refactor pending, we can deal with it later. all good @abh1sar

engine/schema: fix new systemvm template is not registered during upg…

0f75453

…rade if hypervisor is not KVM

fix test failure

c0cbf79

sureshanaparti reviewed Apr 2, 2026

View reviewed changes

same code as main

57e1850

shwstppr approved these changes Apr 3, 2026

View reviewed changes

abh1sar added this to the 4.20.3 milestone Apr 3, 2026

RosiKyu self-assigned this Apr 5, 2026

RosiKyu reviewed Apr 6, 2026

View reviewed changes

weizhouapache marked this pull request as ready for review April 6, 2026 07:31

RosiKyu reviewed Apr 6, 2026

View reviewed changes

sureshanaparti approved these changes Apr 6, 2026

View reviewed changes

abh1sar merged commit 6f1aa96 into apache:4.20 Apr 6, 2026
24 of 26 checks passed

	protected static String getHypervisorArchKey(Hypervisor.HypervisorType hypervisorType, CPU.CPUArch arch) {
	if (Hypervisor.HypervisorType.KVM.equals(hypervisorType)) {
	return String.format("%s-%s", hypervisorType.name().toLowerCase(),
	arch == null ? CPU.CPUArch.amd64.getType() : arch.getType());
	}
	return hypervisorType.name().toLowerCase();
	}

	boolean isHypervisorArchMatchMetadata = hypervisorsInUse.stream()
	.anyMatch(p -> p.first().equals(templateDetails.getHypervisorType())
	&& Objects.equals(p.second(), templateDetails.getArch()));

Conversation

weizhouapache commented Apr 2, 2026

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

weizhouapache commented Apr 2, 2026

Uh oh!

blueorangutan commented Apr 2, 2026

Uh oh!

blueorangutan commented Apr 2, 2026

Uh oh!

weizhouapache commented Apr 2, 2026

Uh oh!

blueorangutan commented Apr 2, 2026

Uh oh!

sureshanaparti Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

sureshanaparti Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

weizhouapache Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

weizhouapache Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

sureshanaparti Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

blueorangutan commented Apr 2, 2026

Uh oh!

weizhouapache commented Apr 2, 2026

Uh oh!

blueorangutan commented Apr 2, 2026

Uh oh!

shwstppr left a comment

Choose a reason for hiding this comment

Uh oh!

blueorangutan commented Apr 3, 2026

Uh oh!

RosiKyu left a comment

Choose a reason for hiding this comment

Uh oh!

RosiKyu left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

abh1sar commented Apr 6, 2026

Uh oh!

sureshanaparti commented Apr 6, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

weizhouapache Apr 2, 2026 •

edited

Loading

sureshanaparti Apr 2, 2026 •

edited

Loading

codecov bot commented Apr 2, 2026 •

edited

Loading

RosiKyu left a comment •

edited

Loading