This series is a follow-up to the discussion that has started here [1].
When memory is lent to the Secure world via FF-A, CPU speculative accesses from NS to the lent pages can still occur as long as it retains a cacheable mapping to it.
Ideally, lent memory would be "no-map" but that would mean giving up MiBs of useful memory, so let's try to do better with the help of a CMA pool.
On arm64, modifying the direct map at runtime is generally restricted because the linear map defaults to block mapping and splitting blocks at runtime may trigger fatal page fault, unless the CPU implements BBML3 or the entire direct map was mapped at page granularity from boot. Forcing last-level mappings system-wide incurs a severe penalty we want to avoid. Instead, this series introduces targeted last-level mappings for designated memory regions, along with the "arm,ffa-lend-pool" CMA driver to manage unmapping and remapping on lend/reclaim transitions:
1. memblock: - Introduce MEMBLOCK_PTEMAP to force PTE mappings only for a specific region.
2. set_memory infrastructure: - Introduce can_set_direct_map_range() to check if a specific address range is mapped with last-level entries and can be modified safely. - Introduce __set_direct_map_*() variants that bypass redundant checks when the caller has already validated the range.
3. "arm,ffa-lend-pool" driver - Introduce the "arm,ffa-lend-pool" CMA reserved-memory driver, which unmaps pages prior to lending (ffa_prepare_lend()) and restores them when reclaimed (ffa_lend_reclaimed()).
4. Optee support - Hook OP-TEE dynamic protected memory pools to "arm,ffa-lend-pool" for both SMC (via DT memory-region phandle) and FF-A (via ffa_lend_pool_attach()) transports.
Testing: ========
Tested with QEMU v8 using OP-TEE OS (built with CFG_CORE_DYN_PROTMEM=y) under both SMC and FF-A transports [2]
static void dump_direct_map(const char *label) { printf("\n=== %s ===\n", label); fflush(stdout); system("sed -n '/Linear Mapping start/,/Linear Mapping end/p' /sys/kernel/debug/kernel_page_tables"); fflush(stdout); }
int main(int argc, char *argv[]) { int heap_fd; int dmabuf_fd; struct dma_heap_allocation_data data = { 0 }; size_t size = 1024 * 1024; /* 1MB */
if (argc > 1) size = strtoul(argv[1], NULL, 0);
dump_direct_map("BEFORE ALLOCATION");
heap_fd = open("/dev/dma_heap/protected,secure-video", O_RDWR); if (heap_fd < 0) { perror("open /dev/dma_heap/protected,secure-video"); return 1; }
printf("\nOpened /dev/dma_heap/protected,secure-video\n"); printf("Allocating %zu bytes of protected memory via DMA heap...\n", size);
data.len = size; data.fd_flags = O_RDWR | O_CLOEXEC; if (ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &data) < 0) { perror("ioctl DMA_HEAP_IOCTL_ALLOC"); close(heap_fd); return 1; }
dmabuf_fd = data.fd; printf("Successfully allocated %zu bytes! dmabuf_fd = %d\n", size, dmabuf_fd);
dump_direct_map("DURING LEND (EXPECT HOLE IN DIRECT MAP)");
printf("\nReleasing dmabuf_fd...\n"); close(dmabuf_fd); close(heap_fd);
dump_direct_map("AFTER RECLAIM (RESTORED DIRECT MAP)");
return 0; }
[1] https://lore.kernel.org/all/20260807-tegra-vpr-v4-7-5510d16af89e@nvidia.com/ [2] https://optee.readthedocs.io/en/latest/building/gits/build.html#qemu-v8
Changelog:
v2: - Rename MEMBLOCK_LLMAP to MEMBLOCK_PTEMAP (Mike) - Warn on conflicting MEMBLOCK_NOMAP and MEMBLOCK_PTEMAP flags (Mike) - Drop DT "ll-map" property and mark MEMBLOCK_PTEMAP from ffa_lend_pool_setup() (Rob, Thierry) - Drop "reusable" and "no-map" property checks from ffa_lend_pool_setup() (Rob) - dt-bindings: Document arm,ffa-lend-pool (Rob) - Rework the Optee support with a separate ffa_lend_pool.c file. - use walk_kernel_page_table_range_lockless() for can_set_direct_map_range() to fix folded page table (Sashiko) - Add missing TLB flush in ffa_prepare_lend() - Disable hibernation - Rebase on linux-next (next-20260918) to use the new set_direct_map*() ranges (Mike)
v1 (https://lore.kernel.org/all/20260902104712.2399797-1-vdonnefort@google.com/)
Vincent Donnefort (8): memblock: Introduce MEMBLOCK_PTEMAP arm64: Introduce can_set_direct_map_range() arm64: Introduce __set_direct_map*() arm64: Add support for MEMBLOCK_PTEMAP firmware: arm_ffa: Introduce ffa-lend-pool optee: Add support for arm,ffa-lend-pool dt-bindings: reserved-memory: Add Arm FF-A lend pool dt-bindings: firmware: optee: Add memory-region property
.../arm/firmware/linaro,optee-tz.yaml | 5 + .../reserved-memory/arm,ffa-lend-pool.yaml | 55 ++++ arch/arm64/include/asm/set_memory.h | 7 + arch/arm64/mm/mmu.c | 23 +- arch/arm64/mm/pageattr.c | 72 ++++- drivers/firmware/arm_ffa/Kconfig | 5 + drivers/firmware/arm_ffa/Makefile | 1 + drivers/firmware/arm_ffa/lend_pool.c | 261 ++++++++++++++++++ drivers/tee/optee/Makefile | 1 + drivers/tee/optee/core.c | 6 + drivers/tee/optee/ffa_abi.c | 13 +- drivers/tee/optee/ffa_lend_pool.c | 72 +++++ drivers/tee/optee/optee_private.h | 4 + drivers/tee/optee/protmem.c | 20 +- drivers/tee/optee/smc_abi.c | 21 +- include/linux/arm_ffa.h | 23 ++ include/linux/memblock.h | 18 ++ mm/memblock.c | 30 ++ 18 files changed, 608 insertions(+), 29 deletions(-) create mode 100644 Documentation/devicetree/bindings/reserved-memory/arm,ffa-lend-pool.yaml create mode 100644 drivers/firmware/arm_ffa/lend_pool.c create mode 100644 drivers/tee/optee/ffa_lend_pool.c
base-commit: 3f2425f5b5bbbdd991ca9cdfd5502e68d8895998
Keeping PTE-level mappings is interesting on some architectures as it allows mapping/unmapping pages from the kernel direct map without the risk of splitting blocks which, under the break-before-make rule, may trigger page-faults the kernel can't handle.
However, mapping the entire direct map at PTE-level is costly. So instead, create a new memblock flag MEMBLOCK_PTEMAP to selectively apply a PTE-level mapping.
Signed-off-by: Vincent Donnefort vdonnefort@google.com --- include/linux/memblock.h | 18 ++++++++++++++++++ mm/memblock.c | 30 ++++++++++++++++++++++++++++++ 2 files changed, 48 insertions(+)
diff --git a/include/linux/memblock.h b/include/linux/memblock.h index aa845f488327..13263055996d 100644 --- a/include/linux/memblock.h +++ b/include/linux/memblock.h @@ -52,6 +52,7 @@ extern unsigned long long max_possible_pfn; * kernel that we know is good to use. It is the only memory that * allocations may happen from in this phase. * @MEMBLOCK_RSRV_HUGETLB: memory is reserved for hugetlb pages + * @MEMBLOCK_PTEMAP: memory region to be mapped using PTE-level mapping */ enum memblock_flags { MEMBLOCK_NONE = 0x0, /* No special request */ @@ -63,6 +64,7 @@ enum memblock_flags { MEMBLOCK_RSRV_KERN = 0x20, /* memory reserved for kernel use */ MEMBLOCK_KHO_SCRATCH = 0x40, /* scratch memory for kexec handover */ MEMBLOCK_RSRV_HUGETLB = 0x80, /* memory reserved for hugetlb pages */ + MEMBLOCK_PTEMAP = 0x100,/* PTE-level mapping */ };
/** @@ -160,6 +162,8 @@ int memblock_reserved_mark_noinit(phys_addr_t base, phys_addr_t size); int memblock_reserved_mark_kern(phys_addr_t base, phys_addr_t size); int memblock_mark_kho_scratch(phys_addr_t base, phys_addr_t size); int memblock_clear_kho_scratch(phys_addr_t base, phys_addr_t size); +int memblock_mark_ptemap(phys_addr_t base, phys_addr_t size); +int memblock_clear_ptemap(phys_addr_t base, phys_addr_t size);
void memblock_free(void *ptr, size_t size);
@@ -285,8 +289,16 @@ static inline bool memblock_is_mirror(struct memblock_region *m) return m->flags & MEMBLOCK_MIRROR; }
+static inline void memblock_warn_invalid_map_flags(struct memblock_region *m) +{ + enum memblock_flags map_flags = MEMBLOCK_NOMAP | MEMBLOCK_PTEMAP; + + WARN_ON_ONCE((m->flags & map_flags) == map_flags); +} + static inline bool memblock_is_nomap(struct memblock_region *m) { + memblock_warn_invalid_map_flags(m); return m->flags & MEMBLOCK_NOMAP; }
@@ -305,6 +317,12 @@ static inline bool memblock_is_kho_scratch(struct memblock_region *m) return m->flags & MEMBLOCK_KHO_SCRATCH; }
+static inline bool memblock_is_ptemap(struct memblock_region *m) +{ + memblock_warn_invalid_map_flags(m); + return m->flags & MEMBLOCK_PTEMAP; +} + int memblock_search_pfn_nid(unsigned long pfn, unsigned long *start_pfn, unsigned long *end_pfn); void __next_mem_pfn_range(int *idx, int nid, unsigned long *out_start_pfn, diff --git a/mm/memblock.c b/mm/memblock.c index ea0de4b5f356..8a87d4d2516d 100644 --- a/mm/memblock.c +++ b/mm/memblock.c @@ -1204,6 +1204,35 @@ __init int memblock_clear_kho_scratch(phys_addr_t base, phys_addr_t size) MEMBLOCK_KHO_SCRATCH); }
+/** + * memblock_mark_ptemap - Mark a memory region with flag MEMBLOCK_PTEMAP. + * @base: the base phys addr of the region + * @size: the size of the region + * + * If supported by the architecture, such region is mapped using PTE-level + * mappings in the kernel direct map. + * + * Return: 0 on success, -errno on failure. + */ +int __init_memblock memblock_mark_ptemap(phys_addr_t base, phys_addr_t size) +{ + return memblock_setclr_flag(&memblock.memory, base, size, 1, + MEMBLOCK_PTEMAP); +} + +/** + * memblock_clear_ptemap - Clear flag MEMBLOCK_PTEMAP for a specified region. + * @base: the base phys addr of the region + * @size: the size of the region + * + * Return: 0 on success, -errno on failure. + */ +int __init_memblock memblock_clear_ptemap(phys_addr_t base, phys_addr_t size) +{ + return memblock_setclr_flag(&memblock.memory, base, size, 0, + MEMBLOCK_PTEMAP); +} + static bool should_skip_region(struct memblock_type *type, struct memblock_region *m, int nid, int flags) @@ -2880,6 +2909,7 @@ static const char * const flagname[] = { [ilog2(MEMBLOCK_RSRV_KERN)] = "RSV_KERN", [ilog2(MEMBLOCK_KHO_SCRATCH)] = "KHO_SCRATCH", [ilog2(MEMBLOCK_RSRV_HUGETLB)] = "RSV_HUGETLB", + [ilog2(MEMBLOCK_PTEMAP)] = "PTEMAP", };
static int memblock_debug_show(struct seq_file *m, void *private)
For systems where can_set_direct_map() is false, it is still possible that a subregion of the direct map can be modified if it is mapped at the last-level. Add an implementation for can_set_direct_map_range(). It falls back to a page table walk to verify the mapping level if can_set_direct_map() is false.
Signed-off-by: Vincent Donnefort vdonnefort@google.com --- arch/arm64/include/asm/set_memory.h | 3 ++ arch/arm64/mm/pageattr.c | 50 +++++++++++++++++++++++++++++ 2 files changed, 53 insertions(+)
diff --git a/arch/arm64/include/asm/set_memory.h b/arch/arm64/include/asm/set_memory.h index 0091ba12200e..80bb94a340fd 100644 --- a/arch/arm64/include/asm/set_memory.h +++ b/arch/arm64/include/asm/set_memory.h @@ -9,6 +9,9 @@ bool can_set_direct_map(void); #define can_set_direct_map can_set_direct_map
+bool can_set_direct_map_range(struct page *page, unsigned long numpages); +#define can_set_direct_map_range can_set_direct_map_range + int set_memory_valid(unsigned long addr, int numpages, int enable);
int set_direct_map_invalid_noflush(struct page *page, unsigned int numpages); diff --git a/arch/arm64/mm/pageattr.c b/arch/arm64/mm/pageattr.c index 132938b32eb1..5fcdb292905f 100644 --- a/arch/arm64/mm/pageattr.c +++ b/arch/arm64/mm/pageattr.c @@ -102,6 +102,56 @@ bool can_set_direct_map(void) arm64_kfence_can_set_direct_map() || is_realm_world(); }
+static int can_set_direct_map_pud_entry(pud_t *pud, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + if (pud_leaf(pudp_get(pud))) + return -EINVAL; + + return 0; +} + +static int can_set_direct_map_pmd_entry(pmd_t *pmd, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + if (pmd_leaf(pmdp_get(pmd))) + return -EINVAL; + + return 0; +} + +static int can_set_direct_map_pte_hole(unsigned long addr, unsigned long next, + int depth, struct mm_walk *walk) +{ + return -EINVAL; +} + +static const struct mm_walk_ops can_set_direct_map_ops = { + .pud_entry = can_set_direct_map_pud_entry, + .pmd_entry = can_set_direct_map_pmd_entry, + .pte_hole = can_set_direct_map_pte_hole, +}; + +bool can_set_direct_map_range(struct page *page, unsigned long numpages) +{ + unsigned long addr = (unsigned long)page_address(page); + unsigned long end = addr + numpages * PAGE_SIZE; + + if (can_set_direct_map()) + return true; + + if (!numpages) + return false; + + /* + * If !can_set_direct_map() then no one can split blocks and it is safe + * to walk the page-table lockless. + */ + return !walk_kernel_page_table_range_lockless(addr, end, + &can_set_direct_map_ops, + NULL, NULL); +} + static int update_range_prot(unsigned long start, unsigned long size, pgprot_t set_mask, pgprot_t clear_mask) {
Add implementation for the unsafe functions __set_direct_map*(). They do not verify for can_set_direct_map() and expect the caller to do so beforehand.
Signed-off-by: Vincent Donnefort vdonnefort@google.com --- arch/arm64/include/asm/set_memory.h | 4 ++++ arch/arm64/mm/pageattr.c | 22 ++++++++++++++++------ 2 files changed, 20 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/include/asm/set_memory.h b/arch/arm64/include/asm/set_memory.h index 80bb94a340fd..5e372a02b825 100644 --- a/arch/arm64/include/asm/set_memory.h +++ b/arch/arm64/include/asm/set_memory.h @@ -16,6 +16,10 @@ int set_memory_valid(unsigned long addr, int numpages, int enable);
int set_direct_map_invalid_noflush(struct page *page, unsigned int numpages); int set_direct_map_default_noflush(struct page *page, unsigned int numpages); +int __set_direct_map_invalid_noflush(struct page *page, unsigned int numpages); +#define __set_direct_map_invalid_noflush __set_direct_map_invalid_noflush +int __set_direct_map_default_noflush(struct page *page, unsigned int numpages); +#define __set_direct_map_default_noflush __set_direct_map_default_noflush bool kernel_page_present(struct page *page);
int set_memory_encrypted(unsigned long addr, int numpages); diff --git a/arch/arm64/mm/pageattr.c b/arch/arm64/mm/pageattr.c index 5fcdb292905f..3aaba7f988ce 100644 --- a/arch/arm64/mm/pageattr.c +++ b/arch/arm64/mm/pageattr.c @@ -301,28 +301,38 @@ int set_memory_valid(unsigned long addr, int numpages, int enable) __pgprot(PTE_PRESENT_VALID_KERNEL)); }
-int set_direct_map_invalid_noflush(struct page *page, unsigned int numpages) +int __set_direct_map_invalid_noflush(struct page *page, unsigned int numpages) { pgprot_t clear_mask = __pgprot(PTE_PRESENT_VALID_KERNEL); pgprot_t set_mask = __pgprot(PTE_PRESENT_INVALID);
+ return update_range_prot((unsigned long)page_address(page), + PAGE_SIZE * numpages, set_mask, clear_mask); +} + +int set_direct_map_invalid_noflush(struct page *page, unsigned int numpages) +{ if (!can_set_direct_map()) return 0;
+ return __set_direct_map_invalid_noflush(page, numpages); +} + +int __set_direct_map_default_noflush(struct page *page, unsigned int numpages) +{ + pgprot_t set_mask = __pgprot(PTE_PRESENT_VALID_KERNEL | PTE_WRITE); + pgprot_t clear_mask = __pgprot(PTE_PRESENT_INVALID | PTE_RDONLY); + return update_range_prot((unsigned long)page_address(page), PAGE_SIZE * numpages, set_mask, clear_mask); }
int set_direct_map_default_noflush(struct page *page, unsigned int numpages) { - pgprot_t set_mask = __pgprot(PTE_PRESENT_VALID_KERNEL | PTE_WRITE); - pgprot_t clear_mask = __pgprot(PTE_PRESENT_INVALID | PTE_RDONLY); - if (!can_set_direct_map()) return 0;
- return update_range_prot((unsigned long)page_address(page), - PAGE_SIZE * numpages, set_mask, clear_mask); + return __set_direct_map_default_noflush(page, numpages); }
static int __set_memory_enc_dec(unsigned long addr,
With the MEMBLOCK_PTEMAP flag, force a PTE-level mapping.
Signed-off-by: Vincent Donnefort vdonnefort@google.com --- arch/arm64/mm/mmu.c | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-)
diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 79d90226fd5d..6b22b43a5f77 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1184,9 +1184,8 @@ static void __init map_mem(void) phys_addr_t init_begin = __pa_symbol(__init_begin); phys_addr_t init_end = __pa_symbol(__init_end); phys_addr_t kernel_end = __pa_symbol(__bss_stop); - phys_addr_t start, end; int flags = NO_EXEC_MAPPINGS; - u64 i; + struct memblock_region *r;
/* * Setting hierarchical PXNTable attributes on table entries covering @@ -1226,9 +1225,13 @@ static void __init map_mem(void) flags);
/* map all the memory banks */ - for_each_mem_range(i, &start, &end) { + for_each_mem_region(r) { + phys_addr_t start, end; + + if (memblock_is_nomap(r)) + continue; /* - * for_each_mem_range may return sub-page-aligned boundaries + * for_each_mem_region may return sub-page-aligned boundaries * after memblock_mark_nomap() splits regions at byte precision. * __create_pgd_mapping_locked aligns phys down to PAGE_MASK, * which could accidentally map no-map memory on the boundary. @@ -1237,17 +1240,23 @@ static void __init map_mem(void) * regions. The cost is at most one page of unmapped gap at * each boundary. */ - start = PAGE_ALIGN(start); - end = end & PAGE_MASK; + start = PAGE_ALIGN(r->base); + end = (r->base + r->size) & PAGE_MASK; if (start >= end) continue; + + int rflags = flags; + + if (memblock_is_ptemap(r)) + rflags |= NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; + /* * The linear map must allow allocation tags reading/writing * if MTE is present. Otherwise, it has the same attributes as * PAGE_KERNEL. */ __map_memblock(start, end, pgprot_tagged(PAGE_KERNEL), - flags); + rflags); } }
When memory is lent to the Secure world via FF-A, fatal CPU speculative reads from Non-Secure can still occur as long as it retains a cacheable mapping to that memory. Introduce the "arm,ffa-lend-pool" reserved-memory CMA driver to unmap pages before lending (ffa_prepare_lend()) and restore them upon reclaim (ffa_lend_reclaimed()). Devices bind to the pool via the "memory-region" DT property or via ffa_lend_pool_attach().
reserved-memory { #address-cells = <0x2>; #size-cells = <0x2>; ranges;
ffa_lend: ffa-lend-pool { compatible = "arm,ffa-lend-pool"; reusable; size = <0x0 0x4000000>; }; };
Signed-off-by: Vincent Donnefort vdonnefort@google.com --- drivers/firmware/arm_ffa/Kconfig | 5 + drivers/firmware/arm_ffa/Makefile | 1 + drivers/firmware/arm_ffa/lend_pool.c | 261 +++++++++++++++++++++++++++ include/linux/arm_ffa.h | 23 +++ 4 files changed, 290 insertions(+) create mode 100644 drivers/firmware/arm_ffa/lend_pool.c
diff --git a/drivers/firmware/arm_ffa/Kconfig b/drivers/firmware/arm_ffa/Kconfig index 5e3ae5cf82e8..66dbf74c37c3 100644 --- a/drivers/firmware/arm_ffa/Kconfig +++ b/drivers/firmware/arm_ffa/Kconfig @@ -19,3 +19,8 @@ config ARM_FFA_SMCCC bool default ARM_FFA_TRANSPORT depends on ARM64 && HAVE_ARM_SMCCC_DISCOVERY + +config ARM_FFA_LEND_POOL + bool + default y + depends on ARM_FFA_TRANSPORT && CMA && OF_RESERVED_MEM diff --git a/drivers/firmware/arm_ffa/Makefile b/drivers/firmware/arm_ffa/Makefile index 168990a7e792..5ea3019c407b 100644 --- a/drivers/firmware/arm_ffa/Makefile +++ b/drivers/firmware/arm_ffa/Makefile @@ -6,3 +6,4 @@ ffa-core-objs := $(ffa-bus-y) ffa-module-objs := $(ffa-driver-y) $(ffa-transport-y) obj-$(CONFIG_ARM_FFA_TRANSPORT) = ffa-core.o obj-$(CONFIG_ARM_FFA_TRANSPORT) += ffa-module.o +obj-$(CONFIG_ARM_FFA_LEND_POOL) += lend_pool.o diff --git a/drivers/firmware/arm_ffa/lend_pool.c b/drivers/firmware/arm_ffa/lend_pool.c new file mode 100644 index 000000000000..e4348f5c618b --- /dev/null +++ b/drivers/firmware/arm_ffa/lend_pool.c @@ -0,0 +1,261 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Arm FF-A Reserved Memory CMA driver for Memory Lending + * + * Prevents CPU speculative reads to secure memory by unmapping it from the + * kernel direct map. This works if rodata=full or if all CPUs in the system + * support BBML3 or if a reserved-memory is declared for this driver + * "arm,ffa_lend_pool". + * + * Copyright (C) 2026 Google LLC + * Author: Vincent Donnefort vdonnefort@google.com + */ + +#include <linux/arm_ffa.h> +#include <linux/cleanup.h> +#include <linux/cma.h> +#include <linux/dma-map-ops.h> +#include <linux/init.h> +#include <linux/memblock.h> +#include <linux/mm.h> +#include <linux/of.h> +#include <linux/of_reserved_mem.h> +#include <linux/rcupdate.h> +#include <linux/suspend.h> +#include <linux/set_memory.h> +#include <linux/xarray.h> + +#include <linux/cacheflush.h> +#include <asm/tlbflush.h> + +static DEFINE_XARRAY(ffa_lend_pool_devices); +static atomic_t ffa_lend_pool_active; + +#define FFA_LEND_POOL_DISABLED -1 + +static bool ffa_lend_pool_can_set_direct_map(struct device *dev, struct page *page, u64 nr_pages) +{ + phys_addr_t addr, base, end; + size_t size; + + if (can_set_direct_map()) + return true; + + if (!dev) + return false; + + guard(rcu)(); + + if (xa_load(&ffa_lend_pool_devices, (unsigned long)dev) != dev) + return false; + + if (WARN_ON_ONCE(!dev->cma_area)) + return false; + + addr = page_to_phys(page); + size = nr_pages << PAGE_SHIFT; + base = cma_get_base(dev->cma_area); + end = base + cma_get_size(dev->cma_area); + + return addr >= base && (addr + size) <= end; +} + +/** + * ffa_prepare_lend() - Prepare a memory region to be lent in FF-A + * @dev: Device attached to the lend pool + * @page: First page of the memory region + * @nr_pages: Number of pages + * + * When memory is lent via FF-A, TrustZone transitions it to the secure state. + * As long as Arm CPUs retain a valid mapping to that now-secure memory, they + * can speculatively read it, which is fatal on some systems. + * + * ffa_prepare_lend() prevents this by unmapping the memory range from the + * kernel's direct map. + * + * Return: 0 on success, negative error code otherwise. + */ +int ffa_prepare_lend(struct device *dev, struct page *page, u64 nr_pages) +{ + struct page *p, *end; + unsigned long addr; + int ret; + + if (!ffa_lend_pool_can_set_direct_map(dev, page, nr_pages)) + return -ENODEV; + + /* provides full ordering */ + if (!atomic_add_unless(&ffa_lend_pool_active, 1, FFA_LEND_POOL_DISABLED)) + return -EBUSY; + + end = page + nr_pages; + for (p = page; p < end; p++) + flush_dcache_page(p); + + ret = __set_direct_map_invalid_noflush(page, nr_pages); + if (ret) { + atomic_dec(&ffa_lend_pool_active); + return ret; + } + + addr = (unsigned long)page_address(page); + flush_tlb_kernel_range(addr, addr + (nr_pages << PAGE_SHIFT)); + + return 0; +} +EXPORT_SYMBOL_GPL(ffa_prepare_lend); + +/** + * ffa_lend_reclaimed() - Restore a reclaimed FF-A memory region + * @dev: Device attached to the lend pool + * @page: First page of the memory region + * @nr_pages: Number of pages + * + * Restores a memory range into the kernel's direct mapping. It must be called + * after a successful FF-A memory reclaim invocation. + */ +void ffa_lend_reclaimed(struct device *dev, struct page *page, u64 nr_pages) +{ + if (!ffa_lend_pool_can_set_direct_map(dev, page, nr_pages)) + return; + + __set_direct_map_default_noflush(page, nr_pages); + + atomic_dec_return_release(&ffa_lend_pool_active); +} +EXPORT_SYMBOL_GPL(ffa_lend_reclaimed); + +static int ffa_lend_pool_pm_notify(struct notifier_block *nb, unsigned long mode, void *data) +{ + /* Prevent hibernation which would try to access lent memory */ + switch (mode) { + case PM_HIBERNATION_PREPARE: + if (atomic_cmpxchg_acquire(&ffa_lend_pool_active, 0, FFA_LEND_POOL_DISABLED)) + return notifier_from_errno(-EBUSY); + break; + case PM_POST_HIBERNATION: + atomic_set(&ffa_lend_pool_active, 0); + } + + return NOTIFY_DONE; +} + +static struct notifier_block ffa_lend_pool_pm_nb = { + .notifier_call = ffa_lend_pool_pm_notify, +}; + +static const struct reserved_mem_ops ffa_lend_pool_ops; + +static struct reserved_mem *ffa_lend_pool_get_rmem(void) +{ + struct reserved_mem *rmem; + struct device_node *np; + + np = of_find_compatible_node(NULL, NULL, "arm,ffa-lend-pool"); + if (!np) + return NULL; + + rmem = of_reserved_mem_lookup(np); + of_node_put(np); + + if (WARN_ON_ONCE(rmem && rmem->ops != &ffa_lend_pool_ops)) + return NULL; + + return rmem; +} + +/** + * ffa_lend_pool_attach() - Attach a device to the FF-A lend pool + * @dev: Device to attach + * + * FF-A devices are dynamically discovered and might not have an associated + * device tree node with a "memory-region" phandle. In that case, drivers must + * use this function to attach to the "arm,ffa-lend-pool" reserved memory + * region. + * + * Return: 0 on success, negative error code otherwise. + */ +int ffa_lend_pool_attach(struct device *dev) +{ + struct reserved_mem *rmem = ffa_lend_pool_get_rmem(); + + return rmem ? rmem->ops->device_init(rmem, dev) : -ENODEV; +} +EXPORT_SYMBOL_GPL(ffa_lend_pool_attach); + +/** + * ffa_lend_pool_detach() - Detach a device from the FF-A lend pool + * @dev: Device to detach + * + * Releases the device from the "arm,ffa-lend-pool" reserved memory region. + */ +void ffa_lend_pool_detach(struct device *dev) +{ + struct reserved_mem *rmem = ffa_lend_pool_get_rmem(); + + if (rmem) + rmem->ops->device_release(rmem, dev); +} +EXPORT_SYMBOL_GPL(ffa_lend_pool_detach); + +static int __init ffa_lend_pool_setup(unsigned long node, struct reserved_mem *rmem) +{ + struct cma *cma; + int ret; + + /* Limited by ffa_lend_pool_active */ + if (rmem->size / PAGE_SIZE > INT_MAX) + return -E2BIG; + + if (!IS_ALIGNED(rmem->base | rmem->size, CMA_MIN_ALIGNMENT_BYTES)) { + pr_err("FF-A lend pool: incorrect alignment of CMA region\n"); + return -EINVAL; + } + + ret = memblock_mark_ptemap(rmem->base, rmem->size); + if (ret) + return ret; + + ret = cma_init_reserved_mem(rmem->base, rmem->size, 0, rmem->name, &cma); + if (ret) { + pr_err("FF-A lend pool: unable to setup CMA region (%d)\n", ret); + memblock_clear_ptemap(rmem->base, rmem->size); + return ret; + } + + register_pm_notifier(&ffa_lend_pool_pm_nb); + rmem->priv = cma; + + return 0; +} + +static int ffa_lend_pool_device_init(struct reserved_mem *rmem, struct device *dev) +{ + int ret; + + if (!can_set_direct_map_range(pfn_to_page(PHYS_PFN(rmem->base)), rmem->size / PAGE_SIZE)) { + pr_err("FF-A lend pool: reserved memory cannot be unmapped in direct map\n"); + return -EINVAL; + } + + dev->cma_area = rmem->priv; + + ret = xa_err(xa_store(&ffa_lend_pool_devices, (unsigned long)dev, dev, GFP_KERNEL)); + if (ret) + dev->cma_area = NULL; + + return ret; +} + +static void ffa_lend_pool_device_release(struct reserved_mem *rmem, struct device *dev) +{ + xa_erase(&ffa_lend_pool_devices, (unsigned long)dev); + dev->cma_area = NULL; +} + +static const struct reserved_mem_ops ffa_lend_pool_ops = { + .node_init = ffa_lend_pool_setup, + .device_init = ffa_lend_pool_device_init, + .device_release = ffa_lend_pool_device_release, +}; +RESERVEDMEM_OF_DECLARE(ffa_lend_pool, "arm,ffa-lend-pool", &ffa_lend_pool_ops); diff --git a/include/linux/arm_ffa.h b/include/linux/arm_ffa.h index e71d83ee0aef..8608847931cc 100644 --- a/include/linux/arm_ffa.h +++ b/include/linux/arm_ffa.h @@ -519,4 +519,27 @@ struct ffa_ops { const struct ffa_notifier_ops *notifier_ops; };
+struct page; + +#if IS_ENABLED(CONFIG_ARM_FFA_LEND_POOL) +int ffa_lend_pool_attach(struct device *dev); +void ffa_lend_pool_detach(struct device *dev); +int ffa_prepare_lend(struct device *dev, struct page *page, u64 nr_pages); +void ffa_lend_reclaimed(struct device *dev, struct page *page, u64 nr_pages); +#else +static inline int ffa_lend_pool_attach(struct device *dev) +{ + return -ENODEV; +} +static inline void ffa_lend_pool_detach(struct device *dev) +{ +} +static inline int ffa_prepare_lend(struct device *dev, struct page *page, u64 nr_pages) +{ + return -ENODEV; +} +static inline void ffa_lend_reclaimed(struct device *dev, struct page *page, u64 nr_pages) +{ +} +#endif #endif /* _LINUX_ARM_FFA_H */
Hook OP-TEE dynamically allocated protected memory pools to the "arm,ffa-lend-pool" driver. While the SMC transport platform device resolves the pool through its DT "memory-region" property, the FF-A transport lacks a device tree node and binds via ffa_lend_pool_attach().
Signed-off-by: Vincent Donnefort vdonnefort@google.com --- drivers/tee/optee/Makefile | 1 + drivers/tee/optee/core.c | 6 +++ drivers/tee/optee/ffa_abi.c | 13 +++++- drivers/tee/optee/ffa_lend_pool.c | 72 +++++++++++++++++++++++++++++++ drivers/tee/optee/optee_private.h | 4 ++ drivers/tee/optee/protmem.c | 20 +++++---- drivers/tee/optee/smc_abi.c | 21 ++++++--- 7 files changed, 121 insertions(+), 16 deletions(-) create mode 100644 drivers/tee/optee/ffa_lend_pool.c
diff --git a/drivers/tee/optee/Makefile b/drivers/tee/optee/Makefile index ad7049c1c107..4986d863b9f7 100644 --- a/drivers/tee/optee/Makefile +++ b/drivers/tee/optee/Makefile @@ -9,6 +9,7 @@ optee-objs += supp.o optee-objs += device.o optee-objs += smc_abi.o optee-objs += ffa_abi.o +optee-objs += ffa_lend_pool.o
# for tracing framework to find optee_trace.h CFLAGS_smc_abi.o := -I$(src) diff --git a/drivers/tee/optee/core.c b/drivers/tee/optee/core.c index a52c1f498b99..39f315cea830 100644 --- a/drivers/tee/optee/core.c +++ b/drivers/tee/optee/core.c @@ -201,6 +201,12 @@ void optee_remove_common(struct optee *optee) /* Unregister OP-TEE specific client devices on TEE bus */ optee_unregister_devices();
+ /* + * Must follow optee_unregister_devices(). Clients require the lend pool + * linkage to successfully free their memory. + */ + optee_lend_pool_unregister(optee); + optee_notif_uninit(optee); optee_shm_arg_cache_uninit(optee); teedev_close_context(optee->ctx); diff --git a/drivers/tee/optee/ffa_abi.c b/drivers/tee/optee/ffa_abi.c index 633715b98625..e979dbc9a547 100644 --- a/drivers/tee/optee/ffa_abi.c +++ b/drivers/tee/optee/ffa_abi.c @@ -1042,13 +1042,21 @@ static int optee_ffa_protmem_pool_init(struct optee *optee, u32 sec_caps) int rc = 0;
if (sec_caps & OPTEE_FFA_SEC_CAP_PROTMEM) { + rc = optee_lend_pool_register(optee); + if (rc) + return rc; + pool = optee_protmem_alloc_dyn_pool(optee, id); - if (IS_ERR(pool)) + if (IS_ERR(pool)) { + optee_lend_pool_unregister(optee); return PTR_ERR(pool); + }
rc = tee_device_register_dma_heap(optee->teedev, id, pool); - if (rc) + if (rc) { + optee_lend_pool_unregister(optee); pool->ops->destroy_pool(pool); + } }
return rc; @@ -1172,6 +1180,7 @@ static int optee_ffa_probe(struct ffa_device *ffa_dev)
err_unregister_devices: optee_unregister_devices(); + optee_lend_pool_unregister(optee); if (optee->ffa.bottom_half_value != U32_MAX) notif_ops->notify_relinquish(ffa_dev, optee->ffa.bottom_half_value); diff --git a/drivers/tee/optee/ffa_lend_pool.c b/drivers/tee/optee/ffa_lend_pool.c new file mode 100644 index 000000000000..b865c37efd5a --- /dev/null +++ b/drivers/tee/optee/ffa_lend_pool.c @@ -0,0 +1,72 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Support for the arm,ffa-lend-pool, unmaps lent memory from the host stage-1 + * to mitigate for CPU speculative read of Secure memory + * + * Copyright (C) 2026 Google LLC. + * Author: Vincent Donnefort vdonnefort@google.com + */ + +#include <linux/arm_ffa.h> +#include <linux/errno.h> +#include <linux/of_reserved_mem.h> + +#include "optee_private.h" + +static struct device_node *optee_dev_node(struct optee *optee) +{ + return dev_of_node(optee->teedev->dev.parent); +} + +static struct device *optee_device(struct optee *optee) +{ + return &optee->teedev->dev; +} + +static int optee_lend_pool_err(int err) +{ + /* + * Registration of the arm,ffa-lend-pool reserved-memory is optional as + * another (although less performant) stage-2 mitigation might be in + * place. + */ + if (err == -ENODEV) + return 0; + + return err; +} + +int optee_lend_pool_register(struct optee *optee) +{ + struct device_node *np = optee_dev_node(optee); + struct device *dev = optee_device(optee); + int ret; + + if (np) + ret = of_reserved_mem_device_init_by_idx(dev, np, 0); + else + ret = ffa_lend_pool_attach(dev); + + return optee_lend_pool_err(ret); +} + +void optee_lend_pool_unregister(struct optee *optee) +{ + struct device_node *np = optee_dev_node(optee); + struct device *dev = optee_device(optee); + + if (np) + of_reserved_mem_device_release(dev); + else + ffa_lend_pool_detach(dev); +} + +int optee_lend_pool_prepare(struct optee *optee, struct page *page, u64 nr_pages) +{ + return optee_lend_pool_err(ffa_prepare_lend(optee_device(optee), page, nr_pages)); +} + +void optee_lend_pool_reclaimed(struct optee *optee, struct page *page, u64 nr_pages) +{ + ffa_lend_reclaimed(optee_device(optee), page, nr_pages); +} diff --git a/drivers/tee/optee/optee_private.h b/drivers/tee/optee/optee_private.h index aefe1e6f5689..f26723e81573 100644 --- a/drivers/tee/optee/optee_private.h +++ b/drivers/tee/optee/optee_private.h @@ -325,6 +325,10 @@ void optee_supp_uninit(struct optee_supp *supp); void optee_supp_release(struct optee_supp *supp); struct tee_protmem_pool *optee_protmem_alloc_dyn_pool(struct optee *optee, enum tee_dma_heap_id id); +int optee_lend_pool_register(struct optee *optee); +void optee_lend_pool_unregister(struct optee *optee); +int optee_lend_pool_prepare(struct optee *optee, struct page *page, u64 nr_pages); +void optee_lend_pool_reclaimed(struct optee *optee, struct page *page, u64 nr_pages);
int optee_supp_recv(struct tee_context *ctx, u32 *func, u32 *num_params, struct tee_param *param); diff --git a/drivers/tee/optee/protmem.c b/drivers/tee/optee/protmem.c index be3abf6e8aa6..b8ee372d20c0 100644 --- a/drivers/tee/optee/protmem.c +++ b/drivers/tee/optee/protmem.c @@ -42,19 +42,16 @@ static int init_dyn_protmem(struct optee_protmem_dyn_pool *rp) goto err_null_protmem; }
- /* - * TODO unmap the memory range since the physical memory will - * become inaccesible after the lend_protmem() call. - * - * If the platform supports a hypervisor at EL2, it will unmap the - * intermediate physical memory for us and stop cache pre-fetch of - * the memory. - */ + rc = optee_lend_pool_prepare(rp->optee, phys_to_page(rp->protmem->paddr), + rp->page_count); + if (rc) + goto err_put_shm; + rc = rp->optee->ops->lend_protmem(rp->optee, rp->protmem, rp->mem_attrs, rp->mem_attr_count, rp->use_case); if (rc) - goto err_put_shm; + goto err_lend_pool_reclaimed; rp->protmem->flags |= TEE_SHM_DYNAMIC;
rp->gen_pool = gen_pool_create(PAGE_SHIFT, -1); @@ -76,6 +73,9 @@ static int init_dyn_protmem(struct optee_protmem_dyn_pool *rp) rp->gen_pool = NULL; err_reclaim: rp->optee->ops->reclaim_protmem(rp->optee, rp->protmem); +err_lend_pool_reclaimed: + optee_lend_pool_reclaimed(rp->optee, phys_to_page(rp->protmem->paddr), + rp->page_count); err_put_shm: tee_shm_put(rp->protmem); err_null_protmem: @@ -112,6 +112,8 @@ static void release_dyn_protmem(struct optee_protmem_dyn_pool *rp) rp->gen_pool = NULL;
rp->optee->ops->reclaim_protmem(rp->optee, rp->protmem); + optee_lend_pool_reclaimed(rp->optee, phys_to_page(rp->protmem->paddr), + rp->page_count); rp->protmem->flags &= ~TEE_SHM_DYNAMIC;
WARN(refcount_read(&rp->protmem->refcount) != 1, "Unexpected refcount"); diff --git a/drivers/tee/optee/smc_abi.c b/drivers/tee/optee/smc_abi.c index b8a2bdac3208..e46eb881cf90 100644 --- a/drivers/tee/optee/smc_abi.c +++ b/drivers/tee/optee/smc_abi.c @@ -1522,7 +1522,6 @@ static void optee_smc_remove(struct platform_device *pdev) optee_disable_shm_cache(optee);
optee_smc_notif_uninit_irq(optee); - optee_remove_common(optee);
if (optee->smc.memremaped_shm) @@ -1714,14 +1713,25 @@ static int optee_protmem_pool_init(struct optee *optee)
if (protm) pool = static_protmem_pool_init(optee); - if (dyn_protm && IS_ERR(pool)) + if (dyn_protm && IS_ERR(pool)) { + rc = optee_lend_pool_register(optee); + if (rc) + return rc; + pool = optee_protmem_alloc_dyn_pool(optee, heap_id); + if (IS_ERR(pool)) { + optee_lend_pool_unregister(optee); + return PTR_ERR(pool); + } + } if (IS_ERR(pool)) return PTR_ERR(pool);
rc = tee_device_register_dma_heap(optee->teedev, heap_id, pool); - if (rc) + if (rc) { + optee_lend_pool_unregister(optee); pool->ops->destroy_pool(pool); + }
return rc; } @@ -1833,14 +1843,14 @@ static int optee_probe(struct platform_device *pdev) (sec_caps & OPTEE_SMC_SEC_CAP_RPMB_PROBE)) optee->in_kernel_rpmb_routing = true;
- teedev = tee_device_alloc(&optee_clnt_desc, NULL, pool, optee); + teedev = tee_device_alloc(&optee_clnt_desc, &pdev->dev, pool, optee); if (IS_ERR(teedev)) { rc = PTR_ERR(teedev); goto err_free_optee; } optee->teedev = teedev;
- teedev = tee_device_alloc(&optee_supp_desc, NULL, pool, optee); + teedev = tee_device_alloc(&optee_supp_desc, &pdev->dev, pool, optee); if (IS_ERR(teedev)) { rc = PTR_ERR(teedev); goto err_unreg_teedev; @@ -1932,6 +1942,7 @@ static int optee_probe(struct platform_device *pdev) optee_disable_shm_cache(optee); optee_smc_notif_uninit_irq(optee); optee_unregister_devices(); + optee_lend_pool_unregister(optee); err_notif_uninit: optee_notif_uninit(optee); err_close_ctx:
Add bindings for the "arm,ffa-lend-pool" reserved-memory CMA pool.
Signed-off-by: Vincent Donnefort vdonnefort@google.com --- .../reserved-memory/arm,ffa-lend-pool.yaml | 55 +++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100644 Documentation/devicetree/bindings/reserved-memory/arm,ffa-lend-pool.yaml
diff --git a/Documentation/devicetree/bindings/reserved-memory/arm,ffa-lend-pool.yaml b/Documentation/devicetree/bindings/reserved-memory/arm,ffa-lend-pool.yaml new file mode 100644 index 000000000000..c02d36e1e418 --- /dev/null +++ b/Documentation/devicetree/bindings/reserved-memory/arm,ffa-lend-pool.yaml @@ -0,0 +1,55 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/reserved-memory/arm,ffa-lend-pool.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Arm FF-A Memory Lend Pool + +maintainers: + - Vincent Donnefort vdonnefort@google.com + +description: | + When memory is lent to the Secure world via FF-A, CPU speculative accesses + from Non-Secure to the lent pages can still occur as long as it retains a + cacheable mapping to it. + + Ideally, lent memory would be "no-map", but that would mean giving up MiBs of + useful memory. Instead, "arm,ffa-lend-pool" defines a reserved-memory CMA pool + to manage unmapping pages prior to lending and restoring them when reclaimed. + +allOf: + - $ref: reserved-memory.yaml + +properties: + compatible: + const: arm,ffa-lend-pool + + no-map: false + +required: + - compatible + - reusable + +unevaluatedProperties: false + +examples: + - | + / { + compatible = "foo"; + model = "foo"; + #address-cells = <2>; + #size-cells = <2>; + + reserved-memory { + #address-cells = <2>; + #size-cells = <2>; + ranges; + + ffa_lend: ffa-lend-pool { + compatible = "arm,ffa-lend-pool"; + reusable; + size = <0x0 0x4000000>; + }; + }; + };
On Mon, 21 Sep 2026 12:00:49 +0100, Vincent Donnefort wrote:
Add bindings for the "arm,ffa-lend-pool" reserved-memory CMA pool.
Signed-off-by: Vincent Donnefort vdonnefort@google.com
.../reserved-memory/arm,ffa-lend-pool.yaml | 55 +++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100644 Documentation/devicetree/bindings/reserved-memory/arm,ffa-lend-pool.yaml
My bot found errors running 'make dt_binding_check' on your patch:
yamllint warnings/errors:
dtschema/dtc warnings/errors: Documentation/devicetree/bindings/reserved-memory/arm,ffa-lend-pool.example.dtb: ffa-lend-pool (arm,ffa-lend-pool): {'compatible': ['arm,ffa-lend-pool'], 'reusable': True, 'size': 67108864, '$nodename': ['ffa-lend-pool']} is valid under each of {'if': {'required': ['iommu-addresses']}, 'then': {'required': ['reg']}}, {'required': ['size']} from schema $id: http://devicetree.org/schemas/reserved-memory/arm,ffa-lend-pool.yaml Documentation/devicetree/bindings/reserved-memory/arm,ffa-lend-pool.example.dtb: ffa-lend-pool (arm,ffa-lend-pool): Unevaluated properties are not allowed ('reusable', 'size' were unexpected) from schema $id: http://devicetree.org/schemas/reserved-memory/arm,ffa-lend-pool.yaml
doc reference errors (make refcheckdocs):
See https://patchwork.kernel.org/project/devicetree/patch/20260921110050.3977591...
The base for the series is generally the latest rc1. A different dependency should be noted in *this* patch.
If you already ran 'make dt_binding_check' and didn't see the above error(s), then make sure 'yamllint' is installed and dt-schema is up to date:
pip3 install dtschema --upgrade
Please check and re-submit after running the above command yourself. Note that DT_SCHEMA_FILES can be set to your schema file to speed up checking your schema. However, it must be unset to test all examples with your schema.
Allow reserved-memory via memory-region to support the arm,ffa-lend-pool.
Signed-off-by: Vincent Donnefort vdonnefort@google.com --- .../devicetree/bindings/arm/firmware/linaro,optee-tz.yaml | 5 +++++ 1 file changed, 5 insertions(+)
diff --git a/Documentation/devicetree/bindings/arm/firmware/linaro,optee-tz.yaml b/Documentation/devicetree/bindings/arm/firmware/linaro,optee-tz.yaml index 26ebcf4565f3..7fddea9cfdbc 100644 --- a/Documentation/devicetree/bindings/arm/firmware/linaro,optee-tz.yaml +++ b/Documentation/devicetree/bindings/arm/firmware/linaro,optee-tz.yaml @@ -41,6 +41,11 @@ properties: HVC #0, register assignments register assignments are specified in drivers/tee/optee/optee_smc.h
+ memory-region: + maxItems: 1 + description: | + Phandle to a reserved-memory node for the OP-TEE lend pool. + required: - compatible - method
Hi Vincent,
On Mon, 21 Sep 2026 at 12:00:42 +0100, Vincent Donnefort wrote:
This series is a follow-up to the discussion that has started here [1].
When memory is lent to the Secure world via FF-A, CPU speculative accesses from NS to the lent pages can still occur as long as it retains a cacheable mapping to it.
Ideally, lent memory would be "no-map" but that would mean giving up MiBs of useful memory, so let's try to do better with the help of a CMA pool.
On arm64, modifying the direct map at runtime is generally restricted because the linear map defaults to block mapping and splitting blocks at runtime may trigger fatal page fault, unless the CPU implements BBML3 or the entire direct map was mapped at page granularity from boot. Forcing last-level mappings system-wide incurs a severe penalty we want to avoid. Instead, this series introduces targeted last-level mappings for designated memory regions, along with the "arm,ffa-lend-pool" CMA driver to manage unmapping and remapping on lend/reclaim transitions:
- memblock:
- Introduce MEMBLOCK_PTEMAP to force PTE mappings only for a specific region.
- set_memory infrastructure:
- Introduce can_set_direct_map_range() to check if a specific address range is mapped with last-level entries and can be modified safely.
- Introduce __set_direct_map_*() variants that bypass redundant checks when the caller has already validated the range.
- "arm,ffa-lend-pool" driver
- Introduce the "arm,ffa-lend-pool" CMA reserved-memory driver, which unmaps pages prior to lending (ffa_prepare_lend()) and restores them when reclaimed (ffa_lend_reclaimed()).
- Optee support
- Hook OP-TEE dynamic protected memory pools to "arm,ffa-lend-pool" for both SMC (via DT memory-region phandle) and FF-A (via ffa_lend_pool_attach()) transports.
Thanks for your proposal in trying to solve this hard problem of unmapping pages from kernel linear map. I remember discussing this problem last year at LPC too.
Have you had a chance to look at a more generic MM proposal around this issue here [1]?
One of the major concern for me with your proposal is tying the protected memory allocation to fixed sized platform specific pool size based on DT. Then the cost of granular mappings/unmapping if the platforms choose to enlarge these pools.
As you maybe aware one of the major use-cases here for protected DMAbufs is the secure media pipeline use-case which is memory intensive workload. IMO, the solution proposed at [1] seems to address it although people have flagged rough edges there but should be addressable.
Can you try a port of [1] for arm64 since the author did all the work with x86 as reference?
[1] https://lore.kernel.org/all/20260726-page_alloc-unmapped-v3-0-6f5729aa9832@g...
-Sumit
Testing:
Tested with QEMU v8 using OP-TEE OS (built with CFG_CORE_DYN_PROTMEM=y) under both SMC and FF-A transports [2]
static void dump_direct_map(const char *label) { printf("\n=== %s ===\n", label); fflush(stdout); system("sed -n '/Linear Mapping start/,/Linear Mapping end/p' /sys/kernel/debug/kernel_page_tables"); fflush(stdout); }
int main(int argc, char *argv[]) { int heap_fd; int dmabuf_fd; struct dma_heap_allocation_data data = { 0 }; size_t size = 1024 * 1024; /* 1MB */
if (argc > 1) size = strtoul(argv[1], NULL, 0);
dump_direct_map("BEFORE ALLOCATION");
heap_fd = open("/dev/dma_heap/protected,secure-video", O_RDWR); if (heap_fd < 0) { perror("open /dev/dma_heap/protected,secure-video"); return 1; }
printf("\nOpened /dev/dma_heap/protected,secure-video\n"); printf("Allocating %zu bytes of protected memory via DMA heap...\n", size);
data.len = size; data.fd_flags = O_RDWR | O_CLOEXEC; if (ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &data) < 0) { perror("ioctl DMA_HEAP_IOCTL_ALLOC"); close(heap_fd); return 1; }
dmabuf_fd = data.fd; printf("Successfully allocated %zu bytes! dmabuf_fd = %d\n", size, dmabuf_fd);
dump_direct_map("DURING LEND (EXPECT HOLE IN DIRECT MAP)");
printf("\nReleasing dmabuf_fd...\n"); close(dmabuf_fd); close(heap_fd);
dump_direct_map("AFTER RECLAIM (RESTORED DIRECT MAP)");
return 0; }
[1] https://lore.kernel.org/all/20260807-tegra-vpr-v4-7-5510d16af89e@nvidia.com/ [2] https://optee.readthedocs.io/en/latest/building/gits/build.html#qemu-v8
Changelog:
v2:
- Rename MEMBLOCK_LLMAP to MEMBLOCK_PTEMAP (Mike)
- Warn on conflicting MEMBLOCK_NOMAP and MEMBLOCK_PTEMAP flags (Mike)
- Drop DT "ll-map" property and mark MEMBLOCK_PTEMAP from ffa_lend_pool_setup() (Rob, Thierry)
- Drop "reusable" and "no-map" property checks from ffa_lend_pool_setup() (Rob)
- dt-bindings: Document arm,ffa-lend-pool (Rob)
- Rework the Optee support with a separate ffa_lend_pool.c file.
- use walk_kernel_page_table_range_lockless() for can_set_direct_map_range() to fix folded page table (Sashiko)
- Add missing TLB flush in ffa_prepare_lend()
- Disable hibernation
- Rebase on linux-next (next-20260918) to use the new set_direct_map*() ranges (Mike)
v1 (https://lore.kernel.org/all/20260902104712.2399797-1-vdonnefort@google.com/)
Vincent Donnefort (8): memblock: Introduce MEMBLOCK_PTEMAP arm64: Introduce can_set_direct_map_range() arm64: Introduce __set_direct_map*() arm64: Add support for MEMBLOCK_PTEMAP firmware: arm_ffa: Introduce ffa-lend-pool optee: Add support for arm,ffa-lend-pool dt-bindings: reserved-memory: Add Arm FF-A lend pool dt-bindings: firmware: optee: Add memory-region property
.../arm/firmware/linaro,optee-tz.yaml | 5 + .../reserved-memory/arm,ffa-lend-pool.yaml | 55 ++++ arch/arm64/include/asm/set_memory.h | 7 + arch/arm64/mm/mmu.c | 23 +- arch/arm64/mm/pageattr.c | 72 ++++- drivers/firmware/arm_ffa/Kconfig | 5 + drivers/firmware/arm_ffa/Makefile | 1 + drivers/firmware/arm_ffa/lend_pool.c | 261 ++++++++++++++++++ drivers/tee/optee/Makefile | 1 + drivers/tee/optee/core.c | 6 + drivers/tee/optee/ffa_abi.c | 13 +- drivers/tee/optee/ffa_lend_pool.c | 72 +++++ drivers/tee/optee/optee_private.h | 4 + drivers/tee/optee/protmem.c | 20 +- drivers/tee/optee/smc_abi.c | 21 +- include/linux/arm_ffa.h | 23 ++ include/linux/memblock.h | 18 ++ mm/memblock.c | 30 ++ 18 files changed, 608 insertions(+), 29 deletions(-) create mode 100644 Documentation/devicetree/bindings/reserved-memory/arm,ffa-lend-pool.yaml create mode 100644 drivers/firmware/arm_ffa/lend_pool.c create mode 100644 drivers/tee/optee/ffa_lend_pool.c
base-commit: 3f2425f5b5bbbdd991ca9cdfd5502e68d8895998
2.55.0.1082.g2b9226bbc0-goog
On Tue, Sep 22, 2026 at 11:27:10AM +0530, Sumit Garg wrote:
Hi Vincent,
On Mon, 21 Sep 2026 at 12:00:42 +0100, Vincent Donnefort wrote:
This series is a follow-up to the discussion that has started here [1].
When memory is lent to the Secure world via FF-A, CPU speculative accesses from NS to the lent pages can still occur as long as it retains a cacheable mapping to it.
Ideally, lent memory would be "no-map" but that would mean giving up MiBs of useful memory, so let's try to do better with the help of a CMA pool.
On arm64, modifying the direct map at runtime is generally restricted because the linear map defaults to block mapping and splitting blocks at runtime may trigger fatal page fault, unless the CPU implements BBML3 or the entire direct map was mapped at page granularity from boot. Forcing last-level mappings system-wide incurs a severe penalty we want to avoid. Instead, this series introduces targeted last-level mappings for designated memory regions, along with the "arm,ffa-lend-pool" CMA driver to manage unmapping and remapping on lend/reclaim transitions:
- memblock:
- Introduce MEMBLOCK_PTEMAP to force PTE mappings only for a specific region.
- set_memory infrastructure:
- Introduce can_set_direct_map_range() to check if a specific address range is mapped with last-level entries and can be modified safely.
- Introduce __set_direct_map_*() variants that bypass redundant checks when the caller has already validated the range.
- "arm,ffa-lend-pool" driver
- Introduce the "arm,ffa-lend-pool" CMA reserved-memory driver, which unmaps pages prior to lending (ffa_prepare_lend()) and restores them when reclaimed (ffa_lend_reclaimed()).
- Optee support
- Hook OP-TEE dynamic protected memory pools to "arm,ffa-lend-pool" for both SMC (via DT memory-region phandle) and FF-A (via ffa_lend_pool_attach()) transports.
Thanks for your proposal in trying to solve this hard problem of unmapping pages from kernel linear map. I remember discussing this problem last year at LPC too.
Have you had a chance to look at a more generic MM proposal around this issue here [1]?
One of the major concern for me with your proposal is tying the protected memory allocation to fixed sized platform specific pool size based on DT. Then the cost of granular mappings/unmapping if the platforms choose to enlarge these pools.
As you maybe aware one of the major use-cases here for protected DMAbufs is the secure media pipeline use-case which is memory intensive workload. IMO, the solution proposed at [1] seems to address it although people have flagged rough edges there but should be addressable.
Can you try a port of [1] for arm64 since the author did all the work with x86 as reference?
[1] https://lore.kernel.org/all/20260726-page_alloc-unmapped-v3-0-6f5729aa9832@g...
There's a lot to unpack and I'm not sure I fully understand all of it, but from reading through that series it seems like the primary use-case is to allow things like alloc_pages() to unmap pages from the kernel's linear map. However what Vincent is trying to solve (and there's some overlap with the VPR work on Tegra) is how to remove pages from the linear map that are part of a fixed carveout region.
I suspect there's some overlap, and we might be able to reuse things like the folio_{zap,restore}_direct_map(), but given that we already have a set of struct page objects, the two approaches seem to be fairly orthogonal at a higher level.
Thierry
On Tue, Sep 22, 2026 at 11:27:10AM +0530, Sumit Garg wrote:
On Mon, 21 Sep 2026 at 12:00:42 +0100, Vincent Donnefort wrote:
This series is a follow-up to the discussion that has started here [1].
When memory is lent to the Secure world via FF-A, CPU speculative accesses from NS to the lent pages can still occur as long as it retains a cacheable mapping to it.
Ideally, lent memory would be "no-map" but that would mean giving up MiBs of useful memory, so let's try to do better with the help of a CMA pool.
On arm64, modifying the direct map at runtime is generally restricted because the linear map defaults to block mapping and splitting blocks at runtime may trigger fatal page fault, unless the CPU implements BBML3 or the entire direct map was mapped at page granularity from boot. Forcing last-level mappings system-wide incurs a severe penalty we want to avoid. Instead, this series introduces targeted last-level mappings for designated memory regions, along with the "arm,ffa-lend-pool" CMA driver to manage unmapping and remapping on lend/reclaim transitions:
- memblock:
- Introduce MEMBLOCK_PTEMAP to force PTE mappings only for a specific region.
- set_memory infrastructure:
- Introduce can_set_direct_map_range() to check if a specific address range is mapped with last-level entries and can be modified safely.
- Introduce __set_direct_map_*() variants that bypass redundant checks when the caller has already validated the range.
- "arm,ffa-lend-pool" driver
- Introduce the "arm,ffa-lend-pool" CMA reserved-memory driver, which unmaps pages prior to lending (ffa_prepare_lend()) and restores them when reclaimed (ffa_lend_reclaimed()).
- Optee support
- Hook OP-TEE dynamic protected memory pools to "arm,ffa-lend-pool" for both SMC (via DT memory-region phandle) and FF-A (via ffa_lend_pool_attach()) transports.
Thanks for your proposal in trying to solve this hard problem of unmapping pages from kernel linear map. I remember discussing this problem last year at LPC too.
Have you had a chance to look at a more generic MM proposal around this issue here [1]?
One of the major concern for me with your proposal is tying the protected memory allocation to fixed sized platform specific pool size based on DT. Then the cost of granular mappings/unmapping if the platforms choose to enlarge these pools.
As you maybe aware one of the major use-cases here for protected DMAbufs is the secure media pipeline use-case which is memory intensive workload. IMO, the solution proposed at [1] seems to address it although people have flagged rough edges there but should be addressable.
Can you try a port of [1] for arm64 since the author did all the work with x86 as reference?
[1] https://lore.kernel.org/all/20260726-page_alloc-unmapped-v3-0-6f5729aa9832@g...
I spoke to Brendan at LPC (?) last year but this doesn't really work for arm64 because it relies on being able to unmap arbitrary parts of the linear map, which isn't generally possible unless you force pte-level mappings for everything, which is prohibitive for perf/power.
Vincent's series tackles that by using a pool so that only that part of memory requires the pte-level mappings in the linear map. That's the whole point of it, so I don't think it makes sense to drop it in favour of Brendan's approach (which doesn't work).
Will
Adding Brendan to this thread.
Will,
Looks like you dropped me from your reply.
On Tue, 22 Sep 2026 at 15:55:59 +0100, Will Deacon wrote:
On Tue, Sep 22, 2026 at 11:27:10AM +0530, Sumit Garg wrote:
On Mon, 21 Sep 2026 at 12:00:42 +0100, Vincent Donnefort wrote:
This series is a follow-up to the discussion that has started here [1].
When memory is lent to the Secure world via FF-A, CPU speculative accesses from NS to the lent pages can still occur as long as it retains a cacheable mapping to it.
Ideally, lent memory would be "no-map" but that would mean giving up MiBs of useful memory, so let's try to do better with the help of a CMA pool.
On arm64, modifying the direct map at runtime is generally restricted because the linear map defaults to block mapping and splitting blocks at runtime may trigger fatal page fault, unless the CPU implements BBML3 or the entire direct map was mapped at page granularity from boot. Forcing last-level mappings system-wide incurs a severe penalty we want to avoid. Instead, this series introduces targeted last-level mappings for designated memory regions, along with the "arm,ffa-lend-pool" CMA driver to manage unmapping and remapping on lend/reclaim transitions:
- memblock:
- Introduce MEMBLOCK_PTEMAP to force PTE mappings only for a specific region.
- set_memory infrastructure:
- Introduce can_set_direct_map_range() to check if a specific address range is mapped with last-level entries and can be modified safely.
- Introduce __set_direct_map_*() variants that bypass redundant checks when the caller has already validated the range.
- "arm,ffa-lend-pool" driver
- Introduce the "arm,ffa-lend-pool" CMA reserved-memory driver, which unmaps pages prior to lending (ffa_prepare_lend()) and restores them when reclaimed (ffa_lend_reclaimed()).
- Optee support
- Hook OP-TEE dynamic protected memory pools to "arm,ffa-lend-pool" for both SMC (via DT memory-region phandle) and FF-A (via ffa_lend_pool_attach()) transports.
Thanks for your proposal in trying to solve this hard problem of unmapping pages from kernel linear map. I remember discussing this problem last year at LPC too.
Have you had a chance to look at a more generic MM proposal around this issue here [1]?
One of the major concern for me with your proposal is tying the protected memory allocation to fixed sized platform specific pool size based on DT. Then the cost of granular mappings/unmapping if the platforms choose to enlarge these pools.
As you maybe aware one of the major use-cases here for protected DMAbufs is the secure media pipeline use-case which is memory intensive workload. IMO, the solution proposed at [1] seems to address it although people have flagged rough edges there but should be addressable.
Can you try a port of [1] for arm64 since the author did all the work with x86 as reference?
[1] https://lore.kernel.org/all/20260726-page_alloc-unmapped-v3-0-6f5729aa9832@g...
I spoke to Brendan at LPC (?) last year but this doesn't really work for arm64 because it relies on being able to unmap arbitrary parts of the linear map, which isn't generally possible unless you force pte-level mappings for everything, which is prohibitive for perf/power.
As per the cover letter it's about allocating pages that are not present in the direct map. This essentially fits the protected DMABufs use-case where we don't want any kernel mapping to exist at any time. Bufers allocated from protected DMABufs are only meant to be accessed by the TEE implementation or HW accelerators like in the secure media pipeline.
Vincent's series tackles that by using a pool so that only that part of memory requires the pte-level mappings in the linear map. That's the whole point of it, so I don't think it makes sense to drop it in favour of Brendan's approach (which doesn't work).
For protected DMABufs, we even don't require any pte-level mappings in linear map. The whole idea of mapping and unmapping is just a bottleneck for performance sensitive secure media pipeline use-cases. That's why if we can support __GFP_UNMAPPED with the memory allocator on arm64 would be the best fit for this use-case.
Now the question is why arm64 can't support __GFP_UNMAPPED similar to how Brendan is doing it for x86?
-Sumit
On Sat, Sep 26, 2026 at 05:43:08PM +0530, Sumit Garg wrote:
Looks like you dropped me from your reply.
Huh, that's weird. If I hit reply-all ('g') in mutt, it moves the entire CC list to To: and drops you. I suspect the same thing happened to Thierry when he replied here:
https://lore.kernel.org/all/arJcN60nagFKXj2Q@orome/
I can't figure out why that's happening :/ I manually tried to fix the To: and Cc: lines in this message.
On Tue, 22 Sep 2026 at 15:55:59 +0100, Will Deacon wrote:
I spoke to Brendan at LPC (?) last year but this doesn't really work for arm64 because it relies on being able to unmap arbitrary parts of the linear map, which isn't generally possible unless you force pte-level mappings for everything, which is prohibitive for perf/power.
As per the cover letter it's about allocating pages that are not present in the direct map. This essentially fits the protected DMABufs use-case where we don't want any kernel mapping to exist at any time. Bufers allocated from protected DMABufs are only meant to be accessed by the TEE implementation or HW accelerators like in the secure media pipeline.
That sounds like it should probably build on top of this series, then. You could allocate the DMABufs from a CMA region that has no linear alias.
Vincent's series tackles that by using a pool so that only that part of memory requires the pte-level mappings in the linear map. That's the whole point of it, so I don't think it makes sense to drop it in favour of Brendan's approach (which doesn't work).
For protected DMABufs, we even don't require any pte-level mappings in linear map. The whole idea of mapping and unmapping is just a bottleneck for performance sensitive secure media pipeline use-cases. That's why if we can support __GFP_UNMAPPED with the memory allocator on arm64 would be the best fit for this use-case.
The point is that you will require pte-level mappings for the linear map if you want to unmap from it at runtime on arm64. If you don't, then you can end up needing to split a block mapping (e.g. pmd-level) when you decide to unmap only part of it and there isn't a safe way to do that on arm64 without transiently unmapping the entire block, which can lead to fatal translation faults during that window.
Now the question is why arm64 can't support __GFP_UNMAPPED similar to how Brendan is doing it for x86?
Because arm64 != x86? Presumably x86 doesn't have the strict break-before-make requirements we have on arm64.
Will
On Tue, 29 Sep 2026 at 12:33:07 +0100, Will Deacon wrote:
On Sat, Sep 26, 2026 at 05:43:08PM +0530, Sumit Garg wrote:
Looks like you dropped me from your reply.
Huh, that's weird. If I hit reply-all ('g') in mutt, it moves the entire CC list to To: and drops you. I suspect the same thing happened to Thierry when he replied here:
https://lore.kernel.org/all/arJcN60nagFKXj2Q@orome/
I can't figure out why that's happening :/ I manually tried to fix the To: and Cc: lines in this message.
Thanks.
On Tue, 22 Sep 2026 at 15:55:59 +0100, Will Deacon wrote:
I spoke to Brendan at LPC (?) last year but this doesn't really work for arm64 because it relies on being able to unmap arbitrary parts of the linear map, which isn't generally possible unless you force pte-level mappings for everything, which is prohibitive for perf/power.
As per the cover letter it's about allocating pages that are not present in the direct map. This essentially fits the protected DMABufs use-case where we don't want any kernel mapping to exist at any time. Bufers allocated from protected DMABufs are only meant to be accessed by the TEE implementation or HW accelerators like in the secure media pipeline.
That sounds like it should probably build on top of this series, then.
The main problem I see with this series is it tries to move from the generic CMA allocator to a reserved FF-A CMA pool which again comes from the DT. We already support reserved pool allocations via "no-map" which are discovered dynamically from OP-TEE.
If we can rather support a generic CMA like allocator for arm64 which can support unmapped allocations then I am all for it. These platform specific reserved pools isn't a scalable solution.
You could allocate the DMABufs from a CMA region that has no linear alias.
Allocation from generic CMA pool is already the case here:
tee_shm_alloc_dma_mem() -> dma_alloc_pages()
Do you mean it's just fine to unmap buffers allocated from generic CMA region since it doesn't have any linear alias?
Vincent's series tackles that by using a pool so that only that part of memory requires the pte-level mappings in the linear map. That's the whole point of it, so I don't think it makes sense to drop it in favour of Brendan's approach (which doesn't work).
For protected DMABufs, we even don't require any pte-level mappings in linear map. The whole idea of mapping and unmapping is just a bottleneck for performance sensitive secure media pipeline use-cases. That's why if we can support __GFP_UNMAPPED with the memory allocator on arm64 would be the best fit for this use-case.
The point is that you will require pte-level mappings for the linear map if you want to unmap from it at runtime on arm64. If you don't, then you can end up needing to split a block mapping (e.g. pmd-level) when you decide to unmap only part of it and there isn't a safe way to do that on arm64 without transiently unmapping the entire block, which can lead to fatal translation faults during that window.
I agree with you here, what I am rather trying to find if on arm64 there is any possibilty to have a generic allocator for unmapped pages. That's the use-case here as well as what Brendan't patch-set was trying to address.
Now the question is why arm64 can't support __GFP_UNMAPPED similar to how Brendan is doing it for x86?
Because arm64 != x86? Presumably x86 doesn't have the strict break-before-make requirements we have on arm64.
Sure, sounds like you are eluding to that the generic MM flag __GFP_UNMAPPED being proposed can't be supported on arm64, right?
-Sumit
op-tee@lists.trustedfirmware.org
-
Rob Herring (Arm) -
Sumit Garg -
Thierry Reding -
Vincent Donnefort -
Will Deacon