Dear Suzuki, Thanks for you response,
Just want to give you more information and measurements that may help. At the beginning, I run FVP (FVP_Base_RevC-2xAEMvA_11.25_15_Linux64). Normal world hypervisor is booted and new realm VM is created using: nice -n -20 taskset -c 1 lkvm run --realm -c 1 -m 350 -k /root/VM_image/Image -i /root/VM_image/VM-fs.cpio --irqchip=gicv3 After booting realm's kernel, 5 machine learning inference is executed while realm computation are local to its VM (the model and input data are provisioned into VM-fs.cpio). The realm is then terminated and a new realm VM is created. This process is repeated six times while each time I measure number of instructions executed by the core 1 between specific points in the VM lifecycle using GenericTrace mixed with MTITracePlugin. I measure these parameters in each run of VMs: 1-Numbers of instructions executed during each inference (and its average over 5 inference as well as standard deviation (std)) 2-Numbers of instructions executed during realm creation (from the point lkvm is executed until command line access to the VM file system). This measure is breaked down into two parts, before and after the activation point of VM (if it is a realm VM). I get these number of instructions in a 6 consecutive runs of realm VM (all in million number of instructions):
1) mean of 5 inference: 8128 std of inference: 189 Realm creation: 41715 Realm_statrup_Before_Activation = 19580 Realm_statrup_After_Activation = 22146 2) mean of 5 inference: 448 std of inference: 2 Realm creation: 23446 Realm_statrup_Before_Activation = 19531 Realm_statrup_After_Activation = 3917 3) mean of 5 inference: 313 std of inference: 1 Realm creation: 23126 Realm_statrup_Before_Activation = 19515 Realm_statrup_After_Activation = 3611 4) mean of 5 inference: 450 std of inference: 1 Realm creation: 23442 Realm_statrup_Before_Activation = 19515 Realm_statrup_After_Activation = 3928 5) mean of 5 inference: 449 std of inference: 1 Realm creation: 23438 Realm_statrup_Before_Activation = 19516 Realm_statrup_After_Activation = 3923 6) mean of 5 inference: 6968 std of inference: 34 Realm creation: 40140 Realm_statrup_Before_Activation = 19516 Realm_statrup_After_Activation = 20634
I also run six normal world VM in between each two realm VM experiments, I roughly get this very stable numbers: mean of 5 inference: 271 std of inference: 1 Realm creation: 859
Based on the numbers: a) Number of instructions executed before realm activation is very stable in different experiments (between 19516 to 19580), but whenever the number of instructions of realm_creation_after_activation increases, mean of inference also increases. We can claim that the reason of increase happens after realm activation point and continues during the realm's runtime. b) As the std of inference is always small (compared to the mean of inference), we can claim that the realm behaviour remains the same during its runtime regardless of its expected/unexpected behaviour. c) The normal world numbers are very stable during different runs of normal world VM experiment. I use the same filesystem for the both experiments.
Looking forward to hearing back from you. Sina