Hi,
Please have a look at virtio-mem which provides memory hotplug for VMs. It is available in Linux, QEMU, cloud-hypervisor and libvirt: https://libvirt.org/kbase/memorydevices.html#virtio-mem-model Another reason to use virtio-mem rather than ACPI memory hotplug is to keep complexity out of ACPI tables, in order to simplify remote attestation which requires measuring or verifying the firmware tables.
For example running QEMU with the following allows to hotplug 4G of memory to the VM:
-m 512M,maxmem=1T -object memory-backend-ram,id=mem0,size=4G -device virtio-mem-pci,id=vm0,memdev=mem0,node=0
Then at runtime QEMU monitor can plug and unplug memory:
(qemu) qom-get vm0 size (qemu) qom-set vm0 requested-size 1G
(qemu) qom-set vm0 requested-size 0
This works for a Realm VM, with a small change to the Linux guest: https://jpbrucker.net/git/linux/commit/?h=cca/v4-hotplug&id=6b8768385fa4... (I'm not sure it's correct yet but may be worth adding to the initial guest support.)
The host adds memory to the guest with RMI_GRANULE_DELEGATE+RMI_DATA_CREATE_UNKNOWN, and removes it with RMI_DATA_DESTROY+RMI_GRANULE_UNDELEGATE which ensures that the pages are wiped before being returned to the host.
Virtual device hotplug works out of the box for a Realm VM. The VM needs to have root ports allowing hotplug (see https://www.libvirt.org/pci-hotplug.html#aarch64-architecture ), and the guest kernel must have PCIe hotplug enabled. For example this adds a root port in QEMU:
-device pcie-root-port,chassis=1,id=pcie.1,bus=pcie.0
Then in the monitor add a virtio-net device:
(qemu) netdev_add user,id=net1 (qemu) device_add virtio-net-pci,netdev=net1,bus=pcie.1,id=hp0 [ 300.003234] pcieport 0000:00:03.0: pciehp: Slot(0): Button press: will power on in 5 sec ... [ 300.798772] virtio-pci 0000:01:00.0: enabling device (0000 -> 0002)
# lspci 01:00.0 Ethernet controller: Red Hat, Inc. Virtio 1.0 network device (rev 01)
And remove it:
(qemu) device_del hp0
The security model of hotplug is equivalent to regular PCI support in a Realm: the guest should only interact with devices whose driver has been hardened against untrusted hosts, and with devices authenticated via CMA-SPDM.
Thanks, Jean