Hi,
I have a couple of questions related to notification. Recently, we are working on kernel upgrade and I found this log while booting.
| genirq: Flags mismatch irq 6. 00004401 (ARM-FFA-NPI) vs. 00004400 (IPI) | ARM FF-A: Error registering percpu NPI nIRQ 6 : -16 | ARM FF-A: Notification setup failed -16, not enabled
This does not affect the other functionalities when I tested using xtest of OP-TEE.
And, I found a commit [1] related this log in linux kernel and it said
| If the firmware returns incorrect SRI/NRI number, we fail to set it up | in the kernel which is absolutely fine.
So I checked hafnium and I found that hafnium returns "5" as HF_NOTIFICATION_PENDING_INTID [2] and it conflicts with IPI numbers (IPI_TIMER) [3].
So the question is : 1. HF_NOTIFICATION_PENDING_INTID seems to be conflicted. Do you have any plan to change its number greater than MAX_IPI? 2. Or, Is there the other way to avoid IRQ number conflicts? I mean, Did I miss something to configure? 3. Actually, I haven't find any use-case of notification yet. so notification is still working in progress?
[1] https://lore.kernel.org/linux-arm-kernel/171396205191.700527.181573515429570... [2] https://github.com/TF-Hafnium/hafnium/blob/main/inc/vmapi/hf/types.h#L62 [3] https://elixir.bootlin.com/linux/v6.14.1/source/arch/arm64/kernel/smp.c#L72
Hi,
Thanks for your observations.
My own perspective is that normal world shall not try to enquire about the NPI virtual interrupt id used by hafnium to inject into secure partitions. NPI virtual interrupt id shall be defined by an Hypervisor and SPMC independently and don't necessarily need to match. An hypervisor uses whichever value it defines to inject into VMs. Similarly for an SPMC and secure partitions.
There may have been other reasons to design the driver this way because of presence of an hypervisor (and the driver running in a VM) vs a system without an hypervisor.
We're discussing this internally and will come back to you with a more precise answer.
Meanwhile the error message seems to suggest the returned value is INTID 6 ? Which doesn't correlate with hafnium HF_NOTIFICATION_PENDING_INTID = 5?
Which commit hash or tag are you using for hafnium?
Regards, Olivier.
________________________________ From: Sungbae Yoo via Hafnium hafnium@lists.trustedfirmware.org Sent: 17 April 2025 11:16 To: hafnium@lists.trustedfirmware.org hafnium@lists.trustedfirmware.org Cc: Jon Hunter jonathanh@nvidia.com; Byungkuk Seo bseo@nvidia.com Subject: [Hafnium] Notification setup failed log in the latest Linux
Hi,
I have a couple of questions related to notification. Recently, we are working on kernel upgrade and I found this log while booting.
| genirq: Flags mismatch irq 6. 00004401 (ARM-FFA-NPI) vs. 00004400 (IPI) | ARM FF-A: Error registering percpu NPI nIRQ 6 : -16 | ARM FF-A: Notification setup failed -16, not enabled
This does not affect the other functionalities when I tested using xtest of OP-TEE.
And, I found a commit [1] related this log in linux kernel and it said
| If the firmware returns incorrect SRI/NRI number, we fail to set it up | in the kernel which is absolutely fine.
So I checked hafnium and I found that hafnium returns "5" as HF_NOTIFICATION_PENDING_INTID [2] and it conflicts with IPI numbers (IPI_TIMER) [3].
So the question is : 1. HF_NOTIFICATION_PENDING_INTID seems to be conflicted. Do you have any plan to change its number greater than MAX_IPI? 2. Or, Is there the other way to avoid IRQ number conflicts? I mean, Did I miss something to configure? 3. Actually, I haven't find any use-case of notification yet. so notification is still working in progress?
[1] https://lore.kernel.org/linux-arm-kernel/171396205191.700527.181573515429570... [2] https://github.com/TF-Hafnium/hafnium/blob/main/inc/vmapi/hf/types.h#L62 [3] https://elixir.bootlin.com/linux/v6.14.1/source/arch/arm64/kernel/smp.c#L72
-- Hafnium mailing list -- hafnium@lists.trustedfirmware.org To unsubscribe send an email to hafnium-leave@lists.trustedfirmware.org
Hi,
On this actually after checking the driver
* Meanwhile the error message seems to suggest the returned value is INTID 6 ? Which doesn't correlate with hafnium HF_NOTIFICATION_PENDING_INTID = 5?
Value 6 may be the return value from irq_create_of_mapping [1], not the value returned by FFA_FEATURES(FFA_FEAT_NOTIFICATION_PENDING_INT) [2]
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/driv... [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/driv...
Regards, Olivier.
________________________________ From: Olivier Deprez via Hafnium hafnium@lists.trustedfirmware.org Sent: 17 April 2025 13:01 To: hafnium@lists.trustedfirmware.org hafnium@lists.trustedfirmware.org; Sungbae Yoo sungbaey@nvidia.com Cc: Jon Hunter jonathanh@nvidia.com; Byungkuk Seo bseo@nvidia.com; Sudeep Holla Sudeep.Holla@arm.com; Bertrand Marquis Bertrand.Marquis@arm.com Subject: [Hafnium] Re: Notification setup failed log in the latest Linux
Hi,
Thanks for your observations.
My own perspective is that normal world shall not try to enquire about the NPI virtual interrupt id used by hafnium to inject into secure partitions. NPI virtual interrupt id shall be defined by an Hypervisor and SPMC independently and don't necessarily need to match. An hypervisor uses whichever value it defines to inject into VMs. Similarly for an SPMC and secure partitions.
There may have been other reasons to design the driver this way because of presence of an hypervisor (and the driver running in a VM) vs a system without an hypervisor.
We're discussing this internally and will come back to you with a more precise answer.
Meanwhile the error message seems to suggest the returned value is INTID 6 ? Which doesn't correlate with hafnium HF_NOTIFICATION_PENDING_INTID = 5?
Which commit hash or tag are you using for hafnium?
Regards, Olivier.
________________________________ From: Sungbae Yoo via Hafnium hafnium@lists.trustedfirmware.org Sent: 17 April 2025 11:16 To: hafnium@lists.trustedfirmware.org hafnium@lists.trustedfirmware.org Cc: Jon Hunter jonathanh@nvidia.com; Byungkuk Seo bseo@nvidia.com Subject: [Hafnium] Notification setup failed log in the latest Linux
Hi,
I have a couple of questions related to notification. Recently, we are working on kernel upgrade and I found this log while booting.
| genirq: Flags mismatch irq 6. 00004401 (ARM-FFA-NPI) vs. 00004400 (IPI) | ARM FF-A: Error registering percpu NPI nIRQ 6 : -16 | ARM FF-A: Notification setup failed -16, not enabled
This does not affect the other functionalities when I tested using xtest of OP-TEE.
And, I found a commit [1] related this log in linux kernel and it said
| If the firmware returns incorrect SRI/NRI number, we fail to set it up | in the kernel which is absolutely fine.
So I checked hafnium and I found that hafnium returns "5" as HF_NOTIFICATION_PENDING_INTID [2] and it conflicts with IPI numbers (IPI_TIMER) [3].
So the question is : 1. HF_NOTIFICATION_PENDING_INTID seems to be conflicted. Do you have any plan to change its number greater than MAX_IPI? 2. Or, Is there the other way to avoid IRQ number conflicts? I mean, Did I miss something to configure? 3. Actually, I haven't find any use-case of notification yet. so notification is still working in progress?
[1] https://lore.kernel.org/linux-arm-kernel/171396205191.700527.181573515429570... [2] https://github.com/TF-Hafnium/hafnium/blob/main/inc/vmapi/hf/types.h#L62 [3] https://elixir.bootlin.com/linux/v6.14.1/source/arch/arm64/kernel/smp.c#L72
-- Hafnium mailing list -- hafnium@lists.trustedfirmware.org To unsubscribe send an email to hafnium-leave@lists.trustedfirmware.org -- Hafnium mailing list -- hafnium@lists.trustedfirmware.org To unsubscribe send an email to hafnium-leave@lists.trustedfirmware.org
On Thu, Apr 17, 2025 at 12:06:16PM +0100, Olivier Deprez wrote:
Hi, On this actually after checking the driver * Meanwhile the error message seems to suggest the returned value is INTID 6 ? Which doesn't correlate with hafnium HF_NOTIFICATION_PENDING_INTID = 5? Value 6 may be the return value from irq_create_of_mapping [1], not the value returned by FFA_FEATURES(FFA_FEAT_NOTIFICATION_PENDING_INT) [2]
Yes that's correct.
----------------------------------------------------------------------From: Olivier Deprez via Hafnium hafnium@lists.trustedfirmware.org Sent: 17 April 2025 13:01 To: hafnium@lists.trustedfirmware.org hafnium@lists.trustedfirmware.org; Sungbae Yoo sungbaey@nvidia.com Cc: Jon Hunter jonathanh@nvidia.com; Byungkuk Seo bseo@nvidia.com; Sudeep Holla Sudeep.Holla@arm.com; Bertrand Marquis Bertrand.Marquis@arm.com Subject: [Hafnium] Re: Notification setup failed log in the latest Linux Hi,
Thanks for your observations.
My own perspective is that normal world shall not try to enquire about the NPI virtual interrupt id used by hafnium to inject into secure partitions. NPI virtual interrupt id shall be defined by an Hypervisor and SPMC independently and don't necessarily need to match. An hypervisor uses whichever value it defines to inject into VMs. Similarly for an SPMC and secure partitions.
Indeed.
There may have been other reasons to design the driver this way because of presence of an hypervisor (and the driver running in a VM) vs a system without an hypervisor.
I thought this was fixed in Hafnium with Commit 17c9ad327bfa ("fix(ff-a): report NPI and MEI unsupported to normal world")
I don't see any issue with upstream hafnium and hence wanted to confirm if the above commit is present or not on the setup where the issue is observed.
We're discussing this internally and will come back to you with a more precise answer.
We can do SRI XOR NPI in the driver, but I don't recall the details as why it wasn't done that way. I do remember at the time we added this, we were not clear about it.
Not sure if Bertrand remembers anything more especially if this was needed by Xen ?
hafnium@lists.trustedfirmware.org
-
Olivier Deprez -
Sudeep Holla -
Sungbae Yoo