Shared-memory allocator services (e.g., Android's gralloc) allocate dma-bufs on behalf of client processes in different cgroups. Today those allocations are charged to the allocator's own cgroup, leaving client cgroups with no visibility into their actual memory consumption and defeating resource limits.
This series adds memory cgroup accounting for dma-buf allocations, building on T.J. Mercier's earlier series [1] and incorporating feedback from previous versions and LPC discussions [2].
Patch 1 introduces the MEMCG_DMABUF stat and reserve/commit/cancel and uncharge helpers. Rather than charging at the page level via __GFP_ACCOUNT (which would require every dma-buf exporter to implement charging, and makes cross-cgroup transfer impractical without access to backing pages), we track charges externally on struct dma_buf via a MEMCG_DMABUF stat counter and route all accounting through a single path, removing the __GFP_ACCOUNT flag from system-heap allocations to avoid double counting.
Patch 2 adds a charge_fd field to DMA_HEAP_IOCTL_ALLOC, accepting either a pidfd or a cgroupfd to identify the target cgroup at allocation time. If the target cgroup cannot accommodate the charge, the allocation returns -ENOMEM.
Patches 3-6 add access control for cross-cgroup charging. Letting userspace decide who to charge needs to be a privileged operation because it has material impact on OOM decission and resource limits. We gate it via LSM hooks for the two charge_fd flavours: security_dma_buf_charge_task() gates the pidfd path (target identified by task credentials) and security_dma_buf_charge_cgroup() gates the cgroupfd path (target identified by cgroup directory label), with SELinux implementations under a dma_buf object class.
Patch 7 adds DMA_BUF_IOCTL_XFER_CHARGE, a general ioctl on any dma-buf fd to transfer charges post-allocation. This covers the double-hop allocator case (SurfaceFlinger -> gralloc -> kernel) and works for buffers from any exporter (DRM, V4L2, etc.). A new memcg_account flag is added for dma_buf_export() to ensure that transfer/charge is only applied for buffers backed by reclaimable system memory. If the receiving cgroup lacks capacity for the charge, the ioctl returns -ENOMEM and the original charge is preserved.
Patch 8 adds selftests.
Known limitations:
- Single-owner model: a dma-buf carries one memcg charge regardless of how many processes share it. - Only memcg accounting. CMA heaps would additionally need dmem controller integration. - If a transfer of an uncharged buffer fails, it remains uncharged. The allocator (userspace) is expected to handle this (retry, free, ...). - The LSM hooks are intentionally narrow. Broader cgroup access controls (e.g. for the memfd/kexec handover case [2]) could be a separate effort reusing the same pattern. - Only system heap sets memcg_account today. Other exporters (DRM, V4L2, etc.) can opt in by setting the flag, but are not covered by this series.
[1] https://lore.kernel.org/cgroups/20230109213809.418135-1-tjmercier@google.com... [2] https://lpc.events/event/20/contributions/2416/
Signed-off-by: Albert Esteve aesteve@redhat.com --- Changes in v3: - Split mem_cgroup_charge_dmabuf() into reserve/commit/cancel three-phase API - Extract mem_cgroup_get_from_fd() helper shared by allocate and xfer paths - Rename __padding to flags in struct dma_buf_transfer_charge - Fix UAF race: move mod_memcg_state() before cmpxchg in transfer - Add memcg_account opt-in flag; xfer returns -EOPNOTSUPP if unset - Wrap dma_buf_charge_cgroup hook in #ifdef CONFIG_CGROUPS - Fix TOCTOU in SELinux charge_cgroup xattr length handling - Link to v2: https://lore.kernel.org/r/20261021-v2_20230123_tjmercier_google_com-v2-0-ef6...
Changes in v2: - Move charge_pid_fd to charge_fd, accepting cgroupfd in addition to pidfd per Christian König. - New DMA_BUF_IOCTL_XFER_CHARGE ioctl for post-allocation charge transfer on any dma-buf, not just dma-heap buffers per Christian König and T.J. Mercier. - Moved __GFP_ACCOUNT removal to patch 1 per Barry Song. - dma-buf charge/uncharge helpers take struct dma_buf * directly to avoid the need for caller to manage size calculations. - Added test_xfer_charge_cgroupfd_self to dmabuf-heap selftests. - Added LSM and SELinux hooks for cgroupfd-based charging. - Rebased to v7.3-rc7. - Link to v1: https://lore.kernel.org/r/20260512-v2_20230123_tjmercier_google_com-v1-0-632...
--- Albert Esteve (7): dma-heap: charge dma-buf memory via explicit memcg security: dma-buf: Add dma_buf_charge_task LSM hook security: dma-buf: Add dma_buf_charge_cgroup LSM hook selinux: Restrict cross-cgroup dma-buf charging selinux: Restrict dma-buf direct cgroup charging dma-buf: transfer memcg charges via DMA_BUF_IOCTL_XFER_CHARGE selftests/dmabuf-heaps: Add dma-buf memcg accounting tests
T.J. Mercier (1): memcg: Add MEMCG_DMABUF accounting infrastructure
Documentation/admin-guide/cgroup-v2.rst | 5 + drivers/dma-buf/dma-buf.c | 46 ++++ drivers/dma-buf/dma-heap.c | 28 +- drivers/dma-buf/heaps/cma_heap.c | 3 +- drivers/dma-buf/heaps/system_heap.c | 22 +- drivers/tee/tee_heap.c | 3 +- include/linux/dma-buf.h | 9 + include/linux/dma-heap.h | 3 +- include/linux/lsm_hook_defs.h | 4 + include/linux/memcontrol.h | 52 ++++ include/linux/security.h | 18 ++ include/uapi/linux/dma-buf.h | 26 ++ include/uapi/linux/dma-heap.h | 7 + mm/memcontrol.c | 192 ++++++++++++++ security/security.c | 34 +++ security/selinux/hooks.c | 44 ++++ security/selinux/include/classmap.h | 1 + tools/testing/selftests/cgroup/Makefile | 2 +- tools/testing/selftests/cgroup/test_memcontrol.c | 290 ++++++++++++++++++++- tools/testing/selftests/dmabuf-heaps/config | 1 + tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c | 227 +++++++++++++++- tools/testing/selftests/dmabuf-heaps/vmtest.sh | 226 ++++++++++++++++ 22 files changed, 1228 insertions(+), 15 deletions(-) --- base-commit: 24a19b953e2a3d0ea5a8c4951a0ab8f32720c907 change-id: 20260508-v2_20230123_tjmercier_google_com-f44fcfb16530
Best regards,
From: "T.J. Mercier" tjmercier@google.com
Introduce a MEMCG_DMABUF stat counter and helpers for reserve/commit/cancel and uncharging dma-buf memory against a memory cgroup:
The charge is tracked on struct dma_buf via a new memcg pointer. The page count is derived internally from PAGE_ALIGN(dmabuf->size), and the memcg reference is stored into (or cleared from) dmabuf->memcg by the helpers themselves, removing the need for callers to manage those details. The release path in dma_buf_release() is reduced to a single mem_cgroup_uncharge_dmabuf() call.
This decouples accounting from the backing page allocator, avoiding the need for every exporter to pass __GFP_ACCOUNT and enabling cross-cgroup charge transfer, which is impractical with page-level accounting.
The next patch wires up the charging policy in the dma-heap ioctl.
Signed-off-by: T.J. Mercier tjmercier@google.com Signed-off-by: Albert Esteve aesteve@redhat.com --- Documentation/admin-guide/cgroup-v2.rst | 4 ++ drivers/dma-buf/dma-buf.c | 3 ++ include/linux/dma-buf.h | 4 ++ include/linux/memcontrol.h | 36 ++++++++++++++ mm/memcontrol.c | 87 +++++++++++++++++++++++++++++++++ 5 files changed, 134 insertions(+)
diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index 86a2a0099178..a3265a4a953e 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -1662,6 +1662,10 @@ The following nested keys are defined. Amount of memory used for storing in-kernel data structures.
+ dmabuf (npn) + Amount of memory used for exported DMA buffers allocated by the cgroup. + Stays with the allocating cgroup regardless of how the buffer is shared. + workingset_refault_anon Number of refaults of previously evicted anonymous pages.
diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c index 4c9add51f9ef..f31f49ebd0af 100644 --- a/drivers/dma-buf/dma-buf.c +++ b/drivers/dma-buf/dma-buf.c @@ -14,6 +14,7 @@ #include <linux/fs.h> #include <linux/slab.h> #include <linux/dma-buf.h> +#include <linux/memcontrol.h> #include <linux/dma-fence.h> #include <linux/dma-fence-unwrap.h> #include <linux/anon_inodes.h> @@ -180,6 +181,8 @@ static void dma_buf_release(struct dentry *dentry) */ BUG_ON(dmabuf->cb_in.active || dmabuf->cb_out.active);
+ mem_cgroup_uncharge_dmabuf(dmabuf); + dmabuf->ops->release(dmabuf);
if (dmabuf->resv == (struct dma_resv *)&dmabuf[1]) diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index d15b2b31d3c9..23ec0ff23fba 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -27,6 +27,7 @@ struct device; struct dma_buf; struct dma_buf_attachment; +struct mem_cgroup;
/** * struct dma_buf_ops - operations possible on struct dma_buf @@ -429,6 +430,9 @@ struct dma_buf {
__poll_t active; } cb_in, cb_out; + + /** @memcg: the cgroup to which this buffer is currently attributed */ + struct mem_cgroup *memcg; };
/** diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 7d1c0ce189a8..4c8711fcf4fd 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -10,6 +10,9 @@
#ifndef _LINUX_MEMCONTROL_H #define _LINUX_MEMCONTROL_H + +struct dma_buf; + #include <linux/cgroup.h> #include <linux/vm_event_item.h> #include <linux/hardirq.h> @@ -40,6 +43,7 @@ enum memcg_stat_item { MEMCG_ZSWAP_B, MEMCG_ZSWAPPED, MEMCG_ZSWAP_INCOMP, + MEMCG_DMABUF, MEMCG_NR_STAT, };
@@ -667,6 +671,12 @@ int mem_cgroup_charge_hugetlb(struct folio* folio, gfp_t gfp); int mem_cgroup_swapin_charge_folio(struct folio *folio, unsigned short id, struct mm_struct *mm, gfp_t gfp);
+bool mem_cgroup_reserve_dmabuf(struct mem_cgroup *memcg, gfp_t gfp, + unsigned int nr_pages); + +void mem_cgroup_commit_dmabuf(struct dma_buf *dmabuf, struct mem_cgroup *memcg, + unsigned int nr_pages); + void __mem_cgroup_uncharge(struct folio *folio);
/** @@ -682,6 +692,10 @@ static inline void mem_cgroup_uncharge(struct folio *folio) __mem_cgroup_uncharge(folio); }
+void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages); + +void mem_cgroup_uncharge_dmabuf(struct dma_buf *dmabuf); + void __mem_cgroup_uncharge_folios(struct folio_batch *folios); static inline void mem_cgroup_uncharge_folios(struct folio_batch *folios) { @@ -1167,10 +1181,32 @@ static inline int mem_cgroup_swapin_charge_folio(struct folio *folio, return 0; }
+static inline bool mem_cgroup_reserve_dmabuf(struct mem_cgroup *memcg, + gfp_t gfp, + unsigned int nr_pages) +{ + return true; +} + +static inline void mem_cgroup_commit_dmabuf(struct dma_buf *dmabuf, + struct mem_cgroup *memcg, + unsigned int nr_pages) +{ +} + +static inline void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, + unsigned int nr_pages) +{ +} + static inline void mem_cgroup_uncharge(struct folio *folio) { }
+static inline void mem_cgroup_uncharge_dmabuf(struct dma_buf *dmabuf) +{ +} + static inline void mem_cgroup_uncharge_folios(struct folio_batch *folios) { } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 856a7d07586c..bbd46ba40350 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -69,6 +69,7 @@ #include <net/ip.h> #include "slab.h" #include "memcontrol-v1.h" +#include <linux/dma-buf.h>
#include <linux/uaccess.h>
@@ -442,6 +443,7 @@ static const unsigned int memcg_stat_items[] = { MEMCG_ZSWAP_B, MEMCG_ZSWAPPED, MEMCG_ZSWAP_INCOMP, + MEMCG_DMABUF, };
#define NR_MEMCG_NODE_STAT_ITEMS ARRAY_SIZE(memcg_node_stat_items) @@ -1625,6 +1627,7 @@ static const struct memory_stat memory_stats[] = { #ifdef CONFIG_HUGETLB_PAGE { "hugetlb", NR_HUGETLB }, #endif + { "dmabuf", MEMCG_DMABUF },
/* The memory events */ { "workingset_refault_anon", WORKINGSET_REFAULT_ANON }, @@ -5658,6 +5661,90 @@ void mem_cgroup_flush_workqueue(void) flush_workqueue(memcg_wq); }
+/** + * mem_cgroup_reserve_dmabuf - charge a memcg for a dma-buf allocation + * @memcg: memcg to charge (may be %NULL) + * @gfp_mask: reclaim mode + * @nr_pages: number of pages to charge + * + * Charges @nr_pages against @memcg. Call this before allocating pages so + * the cgroup limit is enforced up front. On success the caller must either + * call mem_cgroup_commit_dmabuf() to finalize the charge, or + * mem_cgroup_cancel_dmabuf() to undo it. + * + * Returns %true on success, %false if the charge would exceed the limit. + */ +bool mem_cgroup_reserve_dmabuf(struct mem_cgroup *memcg, gfp_t gfp_mask, + unsigned int nr_pages) +{ + if (!memcg || mem_cgroup_disabled()) + return true; + + if (try_charge(memcg, gfp_mask, nr_pages)) + return false; + + return true; +} + +/** + * mem_cgroup_commit_dmabuf - finalize a dma-buf charge after export + * @dmabuf: the exported dma-buf + * @memcg: memcg previously charged via mem_cgroup_reserve_dmabuf() + * @nr_pages: same value passed to mem_cgroup_reserve_dmabuf() + * + * Updates the MEMCG_DMABUF counter and stores @memcg in @dmabuf->memcg, + * transferring the caller's reference. + */ +void mem_cgroup_commit_dmabuf(struct dma_buf *dmabuf, struct mem_cgroup *memcg, + unsigned int nr_pages) +{ + if (!memcg || mem_cgroup_disabled()) + return; + + mod_memcg_state(memcg, MEMCG_DMABUF, nr_pages); + dmabuf->memcg = memcg; +} + +/** + * mem_cgroup_cancel_dmabuf - undo a dma-buf charge + * @memcg: memcg to uncharge (may be %NULL) + * @nr_pages: same value passed to mem_cgroup_reserve_dmabuf() + * + * Returns @nr_pages of charge to @memcg. Use when the allocation fails + * after mem_cgroup_reserve_dmabuf() but before mem_cgroup_commit_dmabuf(). + */ +void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages) +{ + if (!memcg) + return; + + if (!mem_cgroup_is_root(memcg)) + refill_stock(memcg, nr_pages); +} + +/** + * mem_cgroup_uncharge_dmabuf - uncharge a dma-buf from its memcg + * @dmabuf: buffer to uncharge + * + * Reverses a successful mem_cgroup_commit_dmabuf(). Drops the memcg + * reference stored in @dmabuf->memcg and clears the pointer. + * Safe to call when @dmabuf->memcg is %NULL. + */ +void mem_cgroup_uncharge_dmabuf(struct dma_buf *dmabuf) +{ + unsigned int nr_pages; + struct mem_cgroup *memcg = dmabuf->memcg; + + if (!memcg) + return; + + nr_pages = PAGE_ALIGN(dmabuf->size) >> PAGE_SHIFT; + mod_memcg_state(memcg, MEMCG_DMABUF, -nr_pages); + mem_cgroup_cancel_dmabuf(memcg, nr_pages); + mem_cgroup_put(memcg); + dmabuf->memcg = NULL; +} + static int __init cgroup_memory(char *s) { char *token;
On embedded platforms a central process often allocates dma-buf memory on behalf of client applications. Without a way to attribute the charge to the requesting client's cgroup, the cost lands on the allocator, making per-cgroup memory limits ineffective for the actual consumers.
Add charge_fd to struct dma_heap_allocation_data. When set to a valid pidfd or cgroupfd, DMA_HEAP_IOCTL_ALLOC resolves the target task's or cgroup's memcg and charges the buffer there via mem_cgroup_reserve_dmabuf()/mem_cgroup_commit_dmabuf() inside dma_heap_ioctl_allocate(). Without charge_fd, and with the mem_accounting module parameter enabled, the buffer is charged to the allocator's own cgroup.
Usage examples:
1. Central allocator charging to a client at allocation time. The allocator knows the client's PID (e.g., from binder's sender_pid) and uses pidfd to attribute the charge:
pid_t client_pid = txn->sender_pid; int pidfd = pidfd_open(client_pid, 0);
struct dma_heap_allocation_data alloc = { .len = buffer_size, .fd_flags = O_RDWR | O_CLOEXEC, .charge_fd = pidfd, }; ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &alloc); close(pidfd); /* alloc.fd is now charged to client's cgroup */
2. Default allocation (no pidfd, mem_accounting=1). When charge_fd is not set and the mem_accounting module parameter is enabled, the buffer is charged to the allocator's own cgroup:
struct dma_heap_allocation_data alloc = { .len = buffer_size, .fd_flags = O_RDWR | O_CLOEXEC, }; ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &alloc); /* charged to current process's cgroup */
Current limitations:
- Single-owner model: a dma-buf carries one memcg charge regardless of how many processes share it. Means only the first owner (and exporter) of the shared buffer bears the charge. - Only memcg accounting supported. While this makes sense for system heap buffers, other heaps (e.g., CMA heaps) will require selectively charging also for the dmem controller.
Signed-off-by: Albert Esteve aesteve@redhat.com --- Documentation/admin-guide/cgroup-v2.rst | 5 ++-- drivers/dma-buf/dma-heap.c | 28 +++++++++++++++++--- drivers/dma-buf/heaps/cma_heap.c | 3 ++- drivers/dma-buf/heaps/system_heap.c | 21 +++++++++++---- drivers/tee/tee_heap.c | 3 ++- include/linux/dma-heap.h | 3 ++- include/linux/memcontrol.h | 7 +++++ include/uapi/linux/dma-heap.h | 7 +++++ mm/memcontrol.c | 47 +++++++++++++++++++++++++++++++++ 9 files changed, 111 insertions(+), 13 deletions(-)
diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index a3265a4a953e..66ddba678784 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -1663,8 +1663,9 @@ The following nested keys are defined. structures.
dmabuf (npn) - Amount of memory used for exported DMA buffers allocated by the cgroup. - Stays with the allocating cgroup regardless of how the buffer is shared. + Amount of memory used for exported DMA buffers allocated by or on + behalf of the cgroup. Stays with the allocating cgroup regardless + of how the buffer is shared.
workingset_refault_anon Number of refaults of previously evicted anonymous pages. diff --git a/drivers/dma-buf/dma-heap.c b/drivers/dma-buf/dma-heap.c index 8a6c2b6e0cc6..7bd416ae2c23 100644 --- a/drivers/dma-buf/dma-heap.c +++ b/drivers/dma-buf/dma-heap.c @@ -10,6 +10,8 @@ #include <linux/device.h> #include <linux/dma-buf.h> #include <linux/dma-heap.h> +#include <linux/memcontrol.h> +#include <linux/sched/mm.h> #include <linux/err.h> #include <linux/export.h> #include <linux/list.h> @@ -78,6 +80,7 @@ static struct dma_buf *dma_heap_ioctl_allocate(struct file *file, void *data) struct dma_heap_allocation_data *heap_allocation = data; struct dma_heap *heap = file->private_data; struct dma_buf *dmabuf; + struct mem_cgroup *memcg = NULL; int fd; size_t len;
@@ -90,15 +93,34 @@ static struct dma_buf *dma_heap_ioctl_allocate(struct file *file, void *data) if (heap_allocation->heap_flags & ~DMA_HEAP_VALID_HEAP_FLAGS) return ERR_PTR(-EINVAL);
+ if (heap_allocation->__padding != 0) + return ERR_PTR(-EINVAL); + + if (heap_allocation->charge_fd) { + memcg = mem_cgroup_get_from_fd(heap_allocation->charge_fd); + if (IS_ERR(memcg)) + return ERR_CAST(memcg); + } + + if (!memcg && mem_accounting) + memcg = get_mem_cgroup_from_mm(current->mm); + len = PAGE_ALIGN(heap_allocation->len); - if (!len) + if (!len) { + mem_cgroup_put(memcg); return ERR_PTR(-EINVAL); + }
dmabuf = heap->ops->allocate(heap, len, heap_allocation->fd_flags, - heap_allocation->heap_flags); + heap_allocation->heap_flags, memcg);
- if (IS_ERR(dmabuf)) + if (IS_ERR(dmabuf)) { + mem_cgroup_put(memcg); return dmabuf; + } + + if (memcg && !dmabuf->memcg) + mem_cgroup_put(memcg);
fd = get_unused_fd_flags(heap_allocation->fd_flags); if (fd < 0) { diff --git a/drivers/dma-buf/heaps/cma_heap.c b/drivers/dma-buf/heaps/cma_heap.c index 3fb4b946c91a..04de88b468a8 100644 --- a/drivers/dma-buf/heaps/cma_heap.c +++ b/drivers/dma-buf/heaps/cma_heap.c @@ -281,7 +281,8 @@ static const struct dma_buf_ops cma_heap_buf_ops = { static struct dma_buf *cma_heap_allocate(struct dma_heap *heap, unsigned long len, u32 fd_flags, - u64 heap_flags) + u64 heap_flags, + struct mem_cgroup *memcg) { struct cma_heap *cma_heap = dma_heap_get_drvdata(heap); struct cma_heap_buffer *buffer; diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c index c8959eadc71d..adeea8cd328b 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -17,6 +17,7 @@ #include <linux/err.h> #include <linux/highmem.h> #include <linux/mem_encrypt.h> +#include <linux/memcontrol.h> #include <linux/mm.h> #include <linux/set_memory.h> #include <linux/module.h> @@ -388,8 +389,6 @@ static struct page *alloc_largest_available(unsigned long size, if (max_order < orders[i]) continue; flags = order_flags[i]; - if (mem_accounting) - flags |= __GFP_ACCOUNT; page = alloc_pages(flags, orders[i]); if (!page) continue; @@ -401,11 +400,13 @@ static struct page *alloc_largest_available(unsigned long size, static struct dma_buf *system_heap_allocate(struct dma_heap *heap, unsigned long len, u32 fd_flags, - u64 heap_flags) + u64 heap_flags, + struct mem_cgroup *memcg) { struct system_heap_buffer *buffer; DEFINE_DMA_BUF_EXPORT_INFO(exp_info); unsigned long size_remaining = len; + unsigned int nr_pages = PAGE_ALIGN(len) >> PAGE_SHIFT; unsigned int max_order = orders[0]; struct system_heap_priv *priv = dma_heap_get_drvdata(heap); bool cc_shared = priv->cc_shared; @@ -416,10 +417,15 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, struct page *page, *tmp_page; int i, ret = -ENOMEM;
- buffer = kzalloc_obj(*buffer); - if (!buffer) + if (!mem_cgroup_reserve_dmabuf(memcg, GFP_KERNEL, nr_pages)) return ERR_PTR(-ENOMEM);
+ buffer = kzalloc_obj(*buffer); + if (!buffer) { + ret = -ENOMEM; + goto cancel_charge; + } + INIT_LIST_HEAD(&buffer->attachments); mutex_init(&buffer->lock); buffer->heap = heap; @@ -478,6 +484,9 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, ret = PTR_ERR(dmabuf); goto free_pages; } + + mem_cgroup_commit_dmabuf(dmabuf, memcg, nr_pages); + return dmabuf;
free_pages: @@ -498,6 +507,8 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, list_for_each_entry_safe(page, tmp_page, &pages, lru) __free_pages(page, compound_order(page)); kfree(buffer); +cancel_charge: + mem_cgroup_cancel_dmabuf(memcg, nr_pages);
return ERR_PTR(ret); } diff --git a/drivers/tee/tee_heap.c b/drivers/tee/tee_heap.c index 007c00156520..83e2fbd6960b 100644 --- a/drivers/tee/tee_heap.c +++ b/drivers/tee/tee_heap.c @@ -167,7 +167,8 @@ static const struct dma_buf_ops tee_heap_buf_ops = {
static struct dma_buf *tee_dma_heap_alloc(struct dma_heap *heap, unsigned long len, u32 fd_flags, - u64 heap_flags) + u64 heap_flags, + struct mem_cgroup *memcg) { struct tee_dma_heap *h = dma_heap_get_drvdata(heap); DEFINE_DMA_BUF_EXPORT_INFO(exp_info); diff --git a/include/linux/dma-heap.h b/include/linux/dma-heap.h index 648328a64b27..2d69f8432bd2 100644 --- a/include/linux/dma-heap.h +++ b/include/linux/dma-heap.h @@ -23,7 +23,8 @@ struct dma_heap_ops { struct dma_buf *(*allocate)(struct dma_heap *heap, unsigned long len, u32 fd_flags, - u64 heap_flags); + u64 heap_flags, + struct mem_cgroup *memcg); };
/** diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 4c8711fcf4fd..6b30baa79fa9 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -671,6 +671,8 @@ int mem_cgroup_charge_hugetlb(struct folio* folio, gfp_t gfp); int mem_cgroup_swapin_charge_folio(struct folio *folio, unsigned short id, struct mm_struct *mm, gfp_t gfp);
+struct mem_cgroup *mem_cgroup_get_from_fd(int fd); + bool mem_cgroup_reserve_dmabuf(struct mem_cgroup *memcg, gfp_t gfp, unsigned int nr_pages);
@@ -1181,6 +1183,11 @@ static inline int mem_cgroup_swapin_charge_folio(struct folio *folio, return 0; }
+static inline struct mem_cgroup *mem_cgroup_get_from_fd(int fd) +{ + return ERR_PTR(-EOPNOTSUPP); +} + static inline bool mem_cgroup_reserve_dmabuf(struct mem_cgroup *memcg, gfp_t gfp, unsigned int nr_pages) diff --git a/include/uapi/linux/dma-heap.h b/include/uapi/linux/dma-heap.h index a4cf716a49fa..bfcd1c25bc7e 100644 --- a/include/uapi/linux/dma-heap.h +++ b/include/uapi/linux/dma-heap.h @@ -29,6 +29,11 @@ * handle to the allocated dma-buf * @fd_flags: file descriptor flags used when allocating * @heap_flags: flags passed to heap + * @charge_fd: optional pidfd or cgroupfd whose cgroup should be + * charged for this allocation; 0 means charge the calling + * process's cgroup when the mem_accounting module parameter + * is enabled, or skip charging otherwise + * @__padding: reserved, must be zero * * Provided by userspace as an argument to the ioctl */ @@ -37,6 +42,8 @@ struct dma_heap_allocation_data { __u32 fd; __u32 fd_flags; __u64 heap_flags; + __u32 charge_fd; + __u32 __padding; };
#define DMA_HEAP_IOC_MAGIC 'H' diff --git a/mm/memcontrol.c b/mm/memcontrol.c index bbd46ba40350..f66254f52b99 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5661,6 +5661,53 @@ void mem_cgroup_flush_workqueue(void) flush_workqueue(memcg_wq); }
+/** + * mem_cgroup_get_from_fd - resolve a memcg from a cgroupfd or pidfd + * @fd: file descriptor (cgroupfd or pidfd) + * + * Tries @fd as a cgroupfd first; if that fails, tries it as a pidfd. + * On success returns a referenced memcg pointer. On failure returns + * an ERR_PTR. + */ +struct mem_cgroup *mem_cgroup_get_from_fd(int fd) +{ + struct cgroup_subsys_state *css; + struct mem_cgroup *memcg; + struct cgroup *cgrp; + struct task_struct *task; + struct mm_struct *mm; + unsigned int pidfd_flags; + + cgrp = cgroup_get_from_fd(fd); + if (!IS_ERR(cgrp)) { + rcu_read_lock(); + css = cgroup_css(cgrp, &memory_cgrp_subsys); + if (!css || !css_tryget_online(css)) { + rcu_read_unlock(); + cgroup_put(cgrp); + return ERR_PTR(-ENOENT); + } + + memcg = mem_cgroup_from_css(css); + rcu_read_unlock(); + cgroup_put(cgrp); + return memcg; + } + + task = pidfd_get_task(fd, &pidfd_flags); + if (IS_ERR(task)) + return ERR_CAST(task); + + mm = get_task_mm(task); + put_task_struct(task); + if (!mm) + return ERR_PTR(-ESRCH); + + memcg = get_mem_cgroup_from_mm(mm); + mmput(mm); + return memcg; +} + /** * mem_cgroup_reserve_dmabuf - charge a memcg for a dma-buf allocation * @memcg: memcg to charge (may be %NULL)
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider: - [Critical] dma-heap: missing permission check in mem_cgroup_get_from_fd() - [High] dma-buf: system_heap: 32-bit nr_pages truncation in system_heap_allocate() - [Medium] mm: memcontrol: missing EXPORT_SYMBOL_GPL() on dmabuf memcg helpers - [Medium] mm: memcontrol: mem_cgroup_get_from_fd() returns NULL when memcg is disabled --
commit 961ee31303129349ee7fa6988bef4bc244f22a61 Author: Albert Esteve aesteve@redhat.com
dma-heap: charge dma-buf memory via explicit memcg
This commit adds a charge_fd field to struct dma_heap_allocation_data so DMA_HEAP_IOCTL_ALLOC can resolve a target cgroup from a pidfd or cgroupfd via mem_cgroup_get_from_fd() and charge system heap allocations to that memcg.
[ ... ]
diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c index c8959eadc71d1..adeea8cd328b8 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -17,6 +17,7 @@ #include <linux/err.h> #include <linux/highmem.h> #include <linux/mem_encrypt.h> +#include <linux/memcontrol.h> #include <linux/mm.h> #include <linux/set_memory.h> #include <linux/module.h> @@ -388,8 +389,6 @@ static struct page *alloc_largest_available(unsigned long size, if (max_order < orders[i]) continue; flags = order_flags[i];
if (mem_accounting) page = alloc_pages(flags, orders[i]); if (!page) continue;flags |= __GFP_ACCOUNT;@@ -401,11 +400,13 @@ static struct page *alloc_largest_available(unsigned long size, static struct dma_buf *system_heap_allocate(struct dma_heap *heap, unsigned long len, u32 fd_flags,
u64 heap_flags)
u64 heap_flags,struct mem_cgroup *memcg){ struct system_heap_buffer *buffer; DEFINE_DMA_BUF_EXPORT_INFO(exp_info); unsigned long size_remaining = len;
- unsigned int nr_pages = PAGE_ALIGN(len) >> PAGE_SHIFT;
[Severity: High] Can nr_pages truncate on 64-bit systems when a caller passes a large heap_allocation->len via DMA_HEAP_IOCTL_ALLOC?
In dma_heap_ioctl_allocate(), heap_allocation->len is page-aligned into a size_t len and only checked for zero before being passed to system_heap_allocate() as an unsigned long:
drivers/dma-buf/dma-heap.c:dma_heap_ioctl_allocate() { ... len = PAGE_ALIGN(heap_allocation->len); if (!len) { mem_cgroup_put(memcg); return ERR_PTR(-EINVAL); }
dmabuf = heap->ops->allocate(heap, len, heap_allocation->fd_flags, heap_allocation->heap_flags, memcg); }
In system_heap_allocate(), size_remaining keeps the full 64-bit len, while nr_pages is stored in a 32-bit unsigned int. Because __GFP_ACCOUNT is removed from alloc_largest_available(), cgroup enforcement now relies solely on mem_cgroup_reserve_dmabuf(memcg, GFP_KERNEL, nr_pages).
If userspace passes len >= (1ULL << (32 + PAGE_SHIFT)) (for example, 16 TiB with 4 KiB pages), nr_pages wraps to 0 (or a small value), so mem_cgroup_reserve_dmabuf() succeeds without charging the cgroup while the while (size_remaining > 0) loop continues allocating uncharged physical pages until global memory exhaustion. In addition, values of nr_pages in [0x80000000U, 0xffffffffU] are passed by mem_cgroup_commit_dmabuf() to mod_memcg_state(), which takes a signed int val and would corrupt MEMCG_DMABUF statistics.
unsigned int max_order = orders[0]; struct system_heap_priv *priv = dma_heap_get_drvdata(heap); bool cc_shared = priv->cc_shared; @@ -416,10 +417,15 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, struct page *page, *tmp_page; int i, ret = -ENOMEM;
- buffer = kzalloc_obj(*buffer);
- if (!buffer)
- if (!mem_cgroup_reserve_dmabuf(memcg, GFP_KERNEL, nr_pages)) return ERR_PTR(-ENOMEM);
- buffer = kzalloc_obj(*buffer);
- if (!buffer) {
ret = -ENOMEM;goto cancel_charge;- }
- INIT_LIST_HEAD(&buffer->attachments); mutex_init(&buffer->lock); buffer->heap = heap;
@@ -478,6 +484,9 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, ret = PTR_ERR(dmabuf); goto free_pages; }
- mem_cgroup_commit_dmabuf(dmabuf, memcg, nr_pages);
- return dmabuf;
free_pages: @@ -498,6 +507,8 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, list_for_each_entry_safe(page, tmp_page, &pages, lru) __free_pages(page, compound_order(page)); kfree(buffer); +cancel_charge:
- mem_cgroup_cancel_dmabuf(memcg, nr_pages);
[Severity: Medium] Will building with CONFIG_DMABUF_HEAPS_SYSTEM=m and CONFIG_MEMCG=y fail at modpost with undefined symbol errors for mem_cgroup_reserve_dmabuf(), mem_cgroup_commit_dmabuf(), and mem_cgroup_cancel_dmabuf()?
In drivers/dma-buf/heaps/Kconfig, DMABUF_HEAPS_SYSTEM is tristate, so drivers/dma-buf/heaps/system_heap.c can be compiled as a loadable module (system_heap.ko). However, mem_cgroup_reserve_dmabuf(), mem_cgroup_commit_dmabuf(), and mem_cgroup_cancel_dmabuf() in mm/memcontrol.c do not have EXPORT_SYMBOL_GPL() annotations.
Should those symbols be exported (or should the reservation/commit/cancel calls live in the built-in drivers/dma-buf/dma-heap.c layer as described in the commit message)?
[ ... ]
diff --git a/mm/memcontrol.c b/mm/memcontrol.c index bbd46ba40350b..f66254f52b999 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5661,6 +5661,53 @@ void mem_cgroup_flush_workqueue(void) flush_workqueue(memcg_wq); } +/**
- mem_cgroup_get_from_fd - resolve a memcg from a cgroupfd or pidfd
- @fd: file descriptor (cgroupfd or pidfd)
- Tries @fd as a cgroupfd first; if that fails, tries it as a pidfd.
- On success returns a referenced memcg pointer. On failure returns
- an ERR_PTR.
- */
+struct mem_cgroup *mem_cgroup_get_from_fd(int fd) +{
- struct cgroup_subsys_state *css;
- struct mem_cgroup *memcg;
- struct cgroup *cgrp;
- struct task_struct *task;
- struct mm_struct *mm;
- unsigned int pidfd_flags;
- cgrp = cgroup_get_from_fd(fd);
- if (!IS_ERR(cgrp)) {
rcu_read_lock();css = cgroup_css(cgrp, &memory_cgrp_subsys);if (!css || !css_tryget_online(css)) {rcu_read_unlock();cgroup_put(cgrp);return ERR_PTR(-ENOENT);}memcg = mem_cgroup_from_css(css);rcu_read_unlock();cgroup_put(cgrp);return memcg;- }
- task = pidfd_get_task(fd, &pidfd_flags);
- if (IS_ERR(task))
return ERR_CAST(task);- mm = get_task_mm(task);
- put_task_struct(task);
- if (!mm)
return ERR_PTR(-ESRCH);- memcg = get_mem_cgroup_from_mm(mm);
- mmput(mm);
- return memcg;
+}
[Severity: Critical] What prevents an unprivileged user with access to a dma-heap device from passing a cgroupfd (opened with O_RDONLY | O_DIRECTORY or O_PATH on any world-readable /sys/fs/cgroup directory) or a pidfd (via pidfd_open() on PID 1 or another user's process) as charge_fd to DMA_HEAP_IOCTL_ALLOC?
Through dma_heap_ioctl_allocate() -> mem_cgroup_get_from_fd() and system_heap_allocate() -> mem_cgroup_reserve_dmabuf() -> try_charge_memcg(), charging to the root cgroup (where mem_cgroup_is_root(memcg) skips page counter charging) allows a restricted process to bypass its own memory.max and memory.high limits, while charging to another cgroup allows exhausting that victim cgroup's quota and triggering mem_cgroup_oom() kills inside it.
Later patches in the series ("security: dma-buf: Add dma_buf_charge_task LSM hook", "security: dma-buf: Add dma_buf_charge_cgroup LSM hook", "selinux: Restrict cross-cgroup dma-buf charging", and "selinux: Restrict dma-buf direct cgroup charging") add LSM hooks and SELinux checks, but there are no baseline DAC, capability, ptrace_may_access(), or cgroup write permission checks when CONFIG_SECURITY is disabled, when SELinux is not active, or when a cgroup directory has no explicit security.selinux xattr (where selinux_dma_buf_charge_cgroup() returns 0 on -ENODATA).
Also, when charging a remote memcg via try_charge_memcg() breaches that target cgroup's memory.high limit, try_charge_memcg() increments current->memcg_nr_pages_over_high and calls __mem_cgroup_handle_over_high(), which resolves get_mem_cgroup_from_mm(current->mm) and reclaims from the allocating process's own cgroup rather than the remotely charged memcg. Should remote charging require appropriate permissions over the target task or cgroup, and how should memory.high reclaim be handled when charging a foreign memcg?
[Severity: Medium] In mem_cgroup_get_from_fd(), what happens on the pidfd path when CONFIG_MEMCG=y and memory cgroups are disabled at boot via cgroup_disable=memory?
When mem_cgroup_disabled() is true, get_mem_cgroup_from_mm(mm) returns NULL, so mem_cgroup_get_from_fd() returns NULL instead of an ERR_PTR as documented in its kernel-doc comment.
In dma_heap_ioctl_allocate() (and in dma_buf_ioctl_xfer_charge() later in the series), IS_ERR(memcg) is false for NULL, so a pidfd charge_fd silently falls through while a cgroupfd charge_fd fails with -ENOENT (because cgroup_css() returns NULL when the subsystem is disabled). Should the pidfd path also return an ERR_PTR when get_mem_cgroup_from_mm(mm) returns NULL?
DMA_HEAP_IOCTL_ALLOC accepts a charge_fd field that, when set, causes the allocation to be charged to an arbitrary process's cgroup rather than the caller's.
Without an access-control point, any process that holds a handle to a dma-heap device node can charge unlimited memory to any other process's cgroup, potentially exhausting that cgroup's limit and triggering OOM kills independent of the victim's own activity or privileges.
Add security_dma_buf_charge_task(), called in mem_cgroup_get_from_fd() when charge_fd refers to another process. The hook receives the credentials of the allocating process (from) and the credentials of the process whose cgroup will be charged (to), giving security modules a controlled enforcement point for cross-cgroup dma-buf attribution policy.
When CONFIG_SECURITY is not set the hook compiles to an inline returning 0, adding no overhead to the fast path.
Signed-off-by: Albert Esteve aesteve@redhat.com --- include/linux/lsm_hook_defs.h | 1 + include/linux/security.h | 7 +++++++ mm/memcontrol.c | 11 +++++++++++ security/security.c | 16 ++++++++++++++++ 4 files changed, 35 insertions(+)
diff --git a/include/linux/lsm_hook_defs.h b/include/linux/lsm_hook_defs.h index 65c9609ec207..0f1138e16dbe 100644 --- a/include/linux/lsm_hook_defs.h +++ b/include/linux/lsm_hook_defs.h @@ -43,6 +43,7 @@ LSM_HOOK(int, 0, capset, struct cred *new, const struct cred *old, const kernel_cap_t *permitted) LSM_HOOK(int, 0, capable, const struct cred *cred, struct user_namespace *ns, int cap, unsigned int opts) +LSM_HOOK(int, 0, dma_buf_charge_task, const struct cred *from, const struct cred *to) LSM_HOOK(int, 0, quotactl, int cmds, int type, int id, const struct super_block *sb) LSM_HOOK(int, 0, quota_on, struct dentry *dentry) LSM_HOOK(int, 0, syslog, int type) diff --git a/include/linux/security.h b/include/linux/security.h index 153e9043058f..89a35eb07b8c 100644 --- a/include/linux/security.h +++ b/include/linux/security.h @@ -350,6 +350,7 @@ int security_capable(const struct cred *cred, struct user_namespace *ns, int cap, unsigned int opts); +int security_dma_buf_charge_task(const struct cred *from, const struct cred *to); int security_quotactl(int cmds, int type, int id, const struct super_block *sb); int security_quota_on(struct dentry *dentry); int security_syslog(int type); @@ -701,6 +702,12 @@ static inline int security_capable(const struct cred *cred, return cap_capable(cred, ns, cap, opts); }
+static inline int security_dma_buf_charge_task(const struct cred *from, + const struct cred *to) +{ + return 0; +} + static inline int security_quotactl(int cmds, int type, int id, const struct super_block *sb) { diff --git a/mm/memcontrol.c b/mm/memcontrol.c index f66254f52b99..ad195cfd1b5f 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -59,6 +59,7 @@ #include <linux/lockdep.h> #include <linux/resume_user_mode.h> #include <linux/psi.h> +#include <linux/security.h> #include <linux/seq_buf.h> #include <linux/sched/isolation.h> #include <linux/kmemleak.h> @@ -5672,11 +5673,13 @@ void mem_cgroup_flush_workqueue(void) struct mem_cgroup *mem_cgroup_get_from_fd(int fd) { struct cgroup_subsys_state *css; + const struct cred *tcred; struct mem_cgroup *memcg; struct cgroup *cgrp; struct task_struct *task; struct mm_struct *mm; unsigned int pidfd_flags; + int ret;
cgrp = cgroup_get_from_fd(fd); if (!IS_ERR(cgrp)) { @@ -5698,6 +5701,14 @@ struct mem_cgroup *mem_cgroup_get_from_fd(int fd) if (IS_ERR(task)) return ERR_CAST(task);
+ tcred = get_task_cred(task); + ret = security_dma_buf_charge_task(current_cred(), tcred); + put_cred(tcred); + if (ret) { + put_task_struct(task); + return ERR_PTR(ret); + } + mm = get_task_mm(task); put_task_struct(task); if (!mm) diff --git a/security/security.c b/security/security.c index 2ee276ab15c5..4a16e4076aab 100644 --- a/security/security.c +++ b/security/security.c @@ -660,6 +660,22 @@ int security_capable(const struct cred *cred, return call_int_hook(capable, cred, ns, cap, opts); }
+/** + * security_dma_buf_charge_task() - Check if cross-cgroup dma-heap charging is allowed + * @from: credentials of the allocating process + * @to: credentials of the process to charge + * + * Check whether the process with credentials @from is allowed to allocate + * dma-heap memory and charge it to the cgroup of the process with credentials + * @to. + * + * Return: Returns 0 if permission is granted. + */ +int security_dma_buf_charge_task(const struct cred *from, const struct cred *to) +{ + return call_int_hook(dma_buf_charge_task, from, to); +} + /** * security_quotactl() - Check if a quotactl() syscall is allowed for this fs * @cmds: commands
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] memcg: task cred and mm->owner mismatch in mem_cgroup_get_from_fd() --
commit 6c6031732d7f42644e3d99679d750fcc12907447 Author: Albert Esteve aesteve@redhat.com
security: dma-buf: Add dma_buf_charge_task LSM hook
This commit adds the security_dma_buf_charge_task() LSM hook and calls it in mem_cgroup_get_from_fd() when resolving a pidfd, allowing security modules to mediate cross-process dma-buf cgroup charging.
diff --git a/mm/memcontrol.c b/mm/memcontrol.c index f66254f52b999..ad195cfd1b5f6 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c
[ ... ]
@@ -5698,6 +5701,14 @@ struct mem_cgroup *mem_cgroup_get_from_fd(int fd) if (IS_ERR(task)) return ERR_CAST(task);
- tcred = get_task_cred(task);
- ret = security_dma_buf_charge_task(current_cred(), tcred);
- put_cred(tcred);
- if (ret) {
put_task_struct(task);return ERR_PTR(ret);- }
- mm = get_task_mm(task); put_task_struct(task); if (!mm)
[Severity: High] Can the cgroup returned by mem_cgroup_get_from_fd() diverge from the task credentials checked by security_dma_buf_charge_task()?
Here security_dma_buf_charge_task() validates get_task_cred(task), and then mem_cgroup_get_from_fd() resolves the target mem_cgroup via get_task_mm(task) and get_mem_cgroup_from_mm(mm):
mm/memcontrol.c:mem_cgroup_get_from_fd() { ... mm = get_task_mm(task); put_task_struct(task); if (!mm) return ERR_PTR(-ESRCH);
memcg = get_mem_cgroup_from_mm(mm); mmput(mm); return memcg; }
However, get_mem_cgroup_from_mm() resolves the mem_cgroup from mm->owner rather than from task, and falls back to root_mem_cgroup when mm->owner is NULL:
mm/memcontrol.c:get_mem_cgroup_from_mm() { ... rcu_read_lock(); do { memcg = mem_cgroup_from_task(rcu_dereference(mm->owner)); if (unlikely(!memcg)) memcg = root_mem_cgroup; } while (!css_tryget(&memcg->css)); rcu_read_unlock(); return memcg; }
If task exits after get_task_mm(task) increments mm->mm_users but before get_mem_cgroup_from_mm(mm) runs, exit_mm() -> mm_update_next_owner() finds no live owner with mm_users > 1 and clears mm->owner:
kernel/exit.c:mm_update_next_owner() { ... /* * We found no owner yet mm_users > 1: this implies that we are * most likely racing with swapoff (try_to_unuse()) or /proc or * ptrace or page migration (get_task_mm()). Mark owner as NULL. */ WRITE_ONCE(mm->owner, NULL); ... }
In that case, get_mem_cgroup_from_mm(mm) returns root_mem_cgroup, where try_charge() skips charging via mem_cgroup_is_root(memcg). A confined caller in DMA_HEAP_IOCTL_ALLOC or DMA_BUF_IOCTL_XFER_CHARGE could fork a short-lived helper child with matching credentials to pass security_dma_buf_charge_task() and race its exit to charge allocations to root_mem_cgroup, escaping its own cgroup memory limit.
Similarly, if task shares an mm_struct with another task via clone(CLONE_VM) that has different credentials or cgroup membership, mm->owner can refer to that other task while security_dma_buf_charge_task() only checks task's credentials.
Would it make sense to resolve the task and its credentials from mm->owner (or look up the mem_cgroup directly from task) so that the LSM check and the mem_cgroup lookup operate on the same task and cannot fall back to root_mem_cgroup when mm->owner is cleared?
Similar to how security_dma_buf_charge_task adds an access-control point for pidfd for the charge_fd of a DMA_HEAP_IOCTL_ALLOC ioctl, we need a new access-control point for the case where charge_fd represents a cgroupfd.
Add security_dma_buf_charge_cgroup(), called in mem_cgroup_get_from_fd() when charge_fd refers to a cgroup. The hook receives the credentials of the requesting process (cred) and the target cgroup to charge (cgrp), giving security modules a controlled enforcement point for cross-cgroup dma-buf attribution policy.
When CONFIG_SECURITY is not set the hook compiles to an inline returning 0, adding no overhead to the fast path.
Signed-off-by: Albert Esteve aesteve@redhat.com --- include/linux/lsm_hook_defs.h | 3 +++ include/linux/security.h | 11 +++++++++++ mm/memcontrol.c | 6 ++++++ security/security.c | 18 ++++++++++++++++++ 4 files changed, 38 insertions(+)
diff --git a/include/linux/lsm_hook_defs.h b/include/linux/lsm_hook_defs.h index 0f1138e16dbe..fbf7f17127db 100644 --- a/include/linux/lsm_hook_defs.h +++ b/include/linux/lsm_hook_defs.h @@ -44,6 +44,9 @@ LSM_HOOK(int, 0, capset, struct cred *new, const struct cred *old, LSM_HOOK(int, 0, capable, const struct cred *cred, struct user_namespace *ns, int cap, unsigned int opts) LSM_HOOK(int, 0, dma_buf_charge_task, const struct cred *from, const struct cred *to) +#ifdef CONFIG_CGROUPS +LSM_HOOK(int, 0, dma_buf_charge_cgroup, const struct cred *cred, struct cgroup *cgrp) +#endif LSM_HOOK(int, 0, quotactl, int cmds, int type, int id, const struct super_block *sb) LSM_HOOK(int, 0, quota_on, struct dentry *dentry) LSM_HOOK(int, 0, syslog, int type) diff --git a/include/linux/security.h b/include/linux/security.h index 89a35eb07b8c..8fb1539156b5 100644 --- a/include/linux/security.h +++ b/include/linux/security.h @@ -351,6 +351,9 @@ int security_capable(const struct cred *cred, int cap, unsigned int opts); int security_dma_buf_charge_task(const struct cred *from, const struct cred *to); +#ifdef CONFIG_CGROUPS +int security_dma_buf_charge_cgroup(const struct cred *cred, struct cgroup *cgrp); +#endif int security_quotactl(int cmds, int type, int id, const struct super_block *sb); int security_quota_on(struct dentry *dentry); int security_syslog(int type); @@ -708,6 +711,14 @@ static inline int security_dma_buf_charge_task(const struct cred *from, return 0; }
+#ifdef CONFIG_CGROUPS +static inline int security_dma_buf_charge_cgroup(const struct cred *cred, + struct cgroup *cgrp) +{ + return 0; +} +#endif + static inline int security_quotactl(int cmds, int type, int id, const struct super_block *sb) { diff --git a/mm/memcontrol.c b/mm/memcontrol.c index ad195cfd1b5f..7b841b905759 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5693,7 +5693,13 @@ struct mem_cgroup *mem_cgroup_get_from_fd(int fd)
memcg = mem_cgroup_from_css(css); rcu_read_unlock(); + + ret = security_dma_buf_charge_cgroup(current_cred(), cgrp); cgroup_put(cgrp); + if (ret) { + mem_cgroup_put(memcg); + return ERR_PTR(ret); + } return memcg; }
diff --git a/security/security.c b/security/security.c index 4a16e4076aab..a5dcf9dc4963 100644 --- a/security/security.c +++ b/security/security.c @@ -676,6 +676,24 @@ int security_dma_buf_charge_task(const struct cred *from, const struct cred *to) return call_int_hook(dma_buf_charge_task, from, to); }
+#ifdef CONFIG_CGROUPS +/** + * security_dma_buf_charge_cgroup() - Check if charging dma-buf memory to a cgroup is allowed + * @cred: credentials of the process requesting the charge + * @cgrp: target cgroup to charge + * + * Check whether the process with credentials @cred is allowed to charge + * dma-buf memory to @cgrp. Called when userspace passes a cgroupfd to + * identify the charge target instead of a pidfd. + * + * Return: Returns 0 if permission is granted. + */ +int security_dma_buf_charge_cgroup(const struct cred *cred, struct cgroup *cgrp) +{ + return call_int_hook(dma_buf_charge_cgroup, cred, cgrp); +} +#endif + /** * security_quotactl() - Check if a quotactl() syscall is allowed for this fs * @cmds: commands
The security_dma_buf_charge_task() hook allows security modules to control which processes may charge dma-buf allocations to another process's cgroup via the charge_fd field of DMA_HEAP_IOCTL_ALLOC. Without a policy implementation, the hook is a no-op and the restriction is not enforced.
On SELinux-managed systems any domain with access to a dma-heap device node can therefore exhaust another cgroup's memory budget without restriction.
Implement selinux_dma_buf_charge_task() using avc_has_perm() with a new dma_buf object class and a charge_task permission. Policy authors can then grant cross-cgroup charging selectively, for example:
allow allocator_app_t client_app_t:dma_buf charge_task;
Signed-off-by: Albert Esteve aesteve@redhat.com --- security/selinux/hooks.c | 7 +++++++ security/selinux/include/classmap.h | 1 + 2 files changed, 8 insertions(+)
diff --git a/security/selinux/hooks.c b/security/selinux/hooks.c index 30776b0107f2..8f39d91eabc9 100644 --- a/security/selinux/hooks.c +++ b/security/selinux/hooks.c @@ -2204,6 +2204,12 @@ static int selinux_capable(const struct cred *cred, struct user_namespace *ns, return cred_has_capability(cred, cap, opts, ns == &init_user_ns); }
+static int selinux_dma_buf_charge_task(const struct cred *from, const struct cred *to) +{ + return avc_has_perm(cred_sid(from), cred_sid(to), + SECCLASS_DMA_BUF, DMA_BUF__CHARGE_TASK, NULL); +} + static int selinux_quotactl(int cmds, int type, int id, const struct super_block *sb) { const struct cred *cred = current_cred(); @@ -7674,6 +7680,7 @@ static struct security_hook_list selinux_hooks[] __ro_after_init = { LSM_HOOK_INIT(capget, selinux_capget), LSM_HOOK_INIT(capset, selinux_capset), LSM_HOOK_INIT(capable, selinux_capable), + LSM_HOOK_INIT(dma_buf_charge_task, selinux_dma_buf_charge_task), LSM_HOOK_INIT(quotactl, selinux_quotactl), LSM_HOOK_INIT(quota_on, selinux_quota_on), LSM_HOOK_INIT(syslog, selinux_syslog), diff --git a/security/selinux/include/classmap.h b/security/selinux/include/classmap.h index 90cb61b16425..b4ebedc6e424 100644 --- a/security/selinux/include/classmap.h +++ b/security/selinux/include/classmap.h @@ -181,6 +181,7 @@ const struct security_class_mapping secclass_map[] = { { "user_namespace", { "create", NULL } }, { "memfd_file", { COMMON_FILE_PERMS, "execute_no_trans", "entrypoint", NULL } }, + { "dma_buf", { "charge_task", NULL } }, /* last one */ { NULL, {} } };
The security_dma_buf_charge_cgroup() hook allows security modules to control which processes may charge dma-buf to an arbitrary cgroup via cgroupfd passed in charge_fd. Without a policy implementation, the hook is a no-op and any process that can open a cgroup directory (effectively any process) can charge memory to it.
Implement selinux_dma_buf_charge_cgroup() using avc_has_perm() with the dma_buf object class and a charge_cgroup permission. The target SID is derived from the SELinux label on the cgroup's kernfs directory, so access control follows the cgroup filesystem labeling. Policy authors can then grant cross-cgroup charging selectively, for example:
allow allocator_app_t cgroup_t:dma_buf charge_cgroup;
The hook matches the fail-open semantics of selinux_kernfs_init_security(). This means unlabeled cgroups are unrestricted by default.
Signed-off-by: Albert Esteve aesteve@redhat.com --- security/selinux/hooks.c | 37 +++++++++++++++++++++++++++++++++++++ security/selinux/include/classmap.h | 2 +- 2 files changed, 38 insertions(+), 1 deletion(-)
diff --git a/security/selinux/hooks.c b/security/selinux/hooks.c index 8f39d91eabc9..2f99eaa910af 100644 --- a/security/selinux/hooks.c +++ b/security/selinux/hooks.c @@ -2210,6 +2210,40 @@ static int selinux_dma_buf_charge_task(const struct cred *from, const struct cre SECCLASS_DMA_BUF, DMA_BUF__CHARGE_TASK, NULL); }
+#ifdef CONFIG_CGROUPS +static int selinux_dma_buf_charge_cgroup(const struct cred *cred, struct cgroup *cgrp) +{ + u32 cgrp_sid, clen; + int rc; + char *context; + + rc = kernfs_xattr_get(cgrp->kn, XATTR_NAME_SELINUX, NULL, 0); + if (rc == -ENODATA) + return 0; + else if (rc < 0) + return rc; + + clen = (u32)rc; + context = kmalloc(clen, GFP_KERNEL); + if (!context) + return -ENOMEM; + + rc = kernfs_xattr_get(cgrp->kn, XATTR_NAME_SELINUX, context, clen); + if (rc < 0) { + kfree(context); + return rc; + } + + rc = security_context_to_sid(context, rc, &cgrp_sid, GFP_KERNEL); + kfree(context); + if (rc) + return rc; + + return avc_has_perm(cred_sid(cred), cgrp_sid, + SECCLASS_DMA_BUF, DMA_BUF__CHARGE_CGROUP, NULL); +} +#endif + static int selinux_quotactl(int cmds, int type, int id, const struct super_block *sb) { const struct cred *cred = current_cred(); @@ -7681,6 +7715,9 @@ static struct security_hook_list selinux_hooks[] __ro_after_init = { LSM_HOOK_INIT(capset, selinux_capset), LSM_HOOK_INIT(capable, selinux_capable), LSM_HOOK_INIT(dma_buf_charge_task, selinux_dma_buf_charge_task), +#ifdef CONFIG_CGROUPS + LSM_HOOK_INIT(dma_buf_charge_cgroup, selinux_dma_buf_charge_cgroup), +#endif LSM_HOOK_INIT(quotactl, selinux_quotactl), LSM_HOOK_INIT(quota_on, selinux_quota_on), LSM_HOOK_INIT(syslog, selinux_syslog), diff --git a/security/selinux/include/classmap.h b/security/selinux/include/classmap.h index b4ebedc6e424..90d1e04c3d27 100644 --- a/security/selinux/include/classmap.h +++ b/security/selinux/include/classmap.h @@ -181,7 +181,7 @@ const struct security_class_mapping secclass_map[] = { { "user_namespace", { "create", NULL } }, { "memfd_file", { COMMON_FILE_PERMS, "execute_no_trans", "entrypoint", NULL } }, - { "dma_buf", { "charge_task", NULL } }, + { "dma_buf", { "charge_task", "charge_cgroup", NULL } }, /* last one */ { NULL, {} } };
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] selinux: selinux_dma_buf_charge_cgroup() bypasses check on genfscon cgroups --
commit 6b412783b17727c382f1a67078326c58c39264e7 Author: Albert Esteve aesteve@redhat.com
selinux: Restrict dma-buf direct cgroup charging
This commit implements the selinux_dma_buf_charge_cgroup() LSM hook and adds the charge_cgroup permission to the dma_buf security class to control which processes may charge dma-buf memory to a cgroup. It reads the target security context from the cgroup kernfs node's security.selinux extended attribute and checks permission via avc_has_perm().
diff --git a/security/selinux/hooks.c b/security/selinux/hooks.c index 8f39d91eabc97..2f99eaa910af4 100644 --- a/security/selinux/hooks.c +++ b/security/selinux/hooks.c @@ -2210,6 +2210,40 @@ static int selinux_dma_buf_charge_task(const struct cred *from, const struct cre SECCLASS_DMA_BUF, DMA_BUF__CHARGE_TASK, NULL); } +#ifdef CONFIG_CGROUPS +static int selinux_dma_buf_charge_cgroup(const struct cred *cred, struct cgroup *cgrp) +{
- u32 cgrp_sid, clen;
- int rc;
- char *context;
- rc = kernfs_xattr_get(cgrp->kn, XATTR_NAME_SELINUX, NULL, 0);
- if (rc == -ENODATA)
return 0;
[Severity: High] Does returning 0 on -ENODATA here bypass the permission check for all cgroups that carry their default genfscon label (such as cgroup_t)?
In selinux_set_mnt_opts(), cgroup and cgroup2 filesystems are configured with SE_SBGENFS | SE_SBGENFS_XATTR. When a cgroup VFS inode is initialized in inode_doinit_with_dentry(), SELinux obtains the default SID from policy via selinux_genfs_get_sid() and then checks for an optional xattr override via inode_doinit_use_xattr():
security/selinux/hooks.c:inode_doinit_with_dentry() { ... rc = selinux_genfs_get_sid(dentry, sclass, sbsec->flags, &sid); if (rc) { dput(dentry); goto out; }
if ((sbsec->flags & SE_SBGENFS_XATTR) && (inode->i_opflags & IOP_XATTR)) { rc = inode_doinit_use_xattr(inode, dentry, sid, &sid); ... }
When no explicit setxattr() has been performed on the cgroup directory, inode_doinit_use_xattr() sees -ENODATA from the underlying kernfs node and falls back to def_sid in memory without writing a security.selinux xattr to cgrp->kn:
security/selinux/hooks.c:inode_doinit_use_xattr() { ... if (rc < 0) { kfree(context); if (rc != -ENODATA) { pr_warn("SELinux: %s: getxattr returned %d for dev=%s ino=%llu\n", __func__, -rc, inode->i_sb->s_id, inode->i_ino); return rc; } *sid = def_sid; return 0; } ... }
Likewise, selinux_kernfs_init_security() sees -ENODATA on the parent kernfs node and returns 0 without storing an xattr on newly created child cgroups.
As a result, unless a cgroup directory has been explicitly relabeled with setxattr(), kernfs_xattr_get(cgrp->kn, XATTR_NAME_SELINUX, NULL, 0) will return -ENODATA even though the cgroup is labeled cgroup_t via genfscon. When userspace passes a cgroup fd via DMA_HEAP_IOCTL_ALLOC or DMA_BUF_IOCTL_XFER_CHARGE, mem_cgroup_get_from_fd() invokes security_dma_buf_charge_cgroup() -> selinux_dma_buf_charge_cgroup(), which will return 0 without calling avc_has_perm(), leaving default-labeled cgroup_t cgroups unrestricted?
dma-heap already supports charging a specific cgroup at allocation time via charge_fd in DMA_HEAP_IOCTL_ALLOC. However, dma-bufs can originate from any exporter (DRM, V4L2, etc.), and the allocator may not know the final owner at allocation time.
Add DMA_BUF_IOCTL_XFER_CHARGE to transfer a dma-buf's memcg charge to a different cgroup after allocation. The target is identified by charge_fd, which may be a cgroupfd or a pidfd.
The transfer charges the new cgroup before uncharging the old one, preserving the original charge on failure. An atomic cmpxchg on dmabuf->memcg prevents races with concurrent transfers.
Not all dma-buf exporters back their buffers with reclaimable system memory. Some represent device registers, VRAM, or carved-out pools where size is not a meaningful memory cost. Add a memcg_account flag to dma_buf_export_info that exporters set when their buffers should participate in memcg accounting. The xfer ioctl returns -EOPNOTSUPP for buffers that do not set the flag. Currently only the system heap sets it, other exporters can opt in as needed.
Usage example: int cg_fd = open("/sys/fs/cgroup/clients/app1", O_RDONLY | O_DIRECTORY);
struct dma_buf_transfer_charge xfer = { .charge_fd = cg_fd, }; ioctl(dmabuf_fd, DMA_BUF_IOCTL_XFER_CHARGE, &xfer); /* charge now on app1's cgroup */ close(cg_fd);
Signed-off-by: Albert Esteve aesteve@redhat.com --- drivers/dma-buf/dma-buf.c | 43 +++++++++++++++++++++++++++++++++++++ drivers/dma-buf/heaps/system_heap.c | 1 + include/linux/dma-buf.h | 5 +++++ include/linux/memcontrol.h | 9 ++++++++ include/uapi/linux/dma-buf.h | 26 ++++++++++++++++++++++ mm/memcontrol.c | 41 +++++++++++++++++++++++++++++++++++ 6 files changed, 125 insertions(+)
diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c index f31f49ebd0af..07513ecedc10 100644 --- a/drivers/dma-buf/dma-buf.c +++ b/drivers/dma-buf/dma-buf.c @@ -540,6 +540,45 @@ static long dma_buf_import_sync_file(struct dma_buf *dmabuf, } #endif
+/* Transfer a dma-buf's memcg charge; see struct dma_buf_transfer_charge */ +static long dma_buf_xfer_charge(struct dma_buf *dmabuf, + const void __user *user_data) +{ + struct dma_buf_transfer_charge arg; + struct mem_cgroup *memcg; + int ret = 0; + + if (copy_from_user(&arg, user_data, sizeof(arg))) + return -EFAULT; + + if (!IS_ENABLED(CONFIG_MEMCG)) + return 0; + + if (arg.flags != 0) + return -EINVAL; + + if (!dmabuf->memcg_account) + return -EOPNOTSUPP; + + memcg = mem_cgroup_get_from_fd(arg.charge_fd); + if (IS_ERR(memcg)) + return PTR_ERR(memcg); + + if (dmabuf->memcg == memcg) + goto skip_transfer; + + if (!mem_cgroup_transfer_dmabuf_charge(dmabuf, memcg)) { + ret = -ENOMEM; + goto skip_transfer; + } + + return 0; + +skip_transfer: + mem_cgroup_put(memcg); + return ret; +} + static long dma_buf_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { @@ -590,6 +629,9 @@ static long dma_buf_ioctl(struct file *file, return dma_buf_import_sync_file(dmabuf, (const void __user *)arg); #endif
+ case DMA_BUF_IOCTL_XFER_CHARGE: + return dma_buf_xfer_charge(dmabuf, (const void __user *)arg); + default: return -ENOTTY; } @@ -750,6 +792,7 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info) dmabuf->size = exp_info->size; dmabuf->exp_name = exp_info->exp_name; dmabuf->owner = exp_info->owner; + dmabuf->memcg_account = exp_info->memcg_account; spin_lock_init(&dmabuf->name_lock); init_waitqueue_head(&dmabuf->poll); dmabuf->cb_in.poll = dmabuf->cb_out.poll = &dmabuf->poll; diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c index adeea8cd328b..a764167167ba 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -479,6 +479,7 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, exp_info.size = buffer->len; exp_info.flags = fd_flags; exp_info.priv = buffer; + exp_info.memcg_account = true; dmabuf = dma_buf_export(&exp_info); if (IS_ERR(dmabuf)) { ret = PTR_ERR(dmabuf); diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index 23ec0ff23fba..ac0ec88cab93 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -433,6 +433,9 @@ struct dma_buf {
/** @memcg: the cgroup to which this buffer is currently attributed */ struct mem_cgroup *memcg; + + /** @memcg_account: account buffer memory to cgroup */ + bool memcg_account; };
/** @@ -505,6 +508,7 @@ struct dma_buf_attachment { * @flags: mode flags for the file * @resv: reservation-object, NULL to allocate default one * @priv: Attach private data of allocator to this buffer + * @memcg_account: account buffer memory to cgroup * * This structure holds the information required to export the buffer. Used * with dma_buf_export() only. @@ -517,6 +521,7 @@ struct dma_buf_export_info { int flags; struct dma_resv *resv; void *priv; + bool memcg_account; };
/** diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 6b30baa79fa9..36fddafbef4e 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -679,6 +679,9 @@ bool mem_cgroup_reserve_dmabuf(struct mem_cgroup *memcg, gfp_t gfp, void mem_cgroup_commit_dmabuf(struct dma_buf *dmabuf, struct mem_cgroup *memcg, unsigned int nr_pages);
+bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf, + struct mem_cgroup *new_memcg); + void __mem_cgroup_uncharge(struct folio *folio);
/** @@ -1206,6 +1209,12 @@ static inline void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, { }
+static inline bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf, + struct mem_cgroup *new_memcg) +{ + return true; +} + static inline void mem_cgroup_uncharge(struct folio *folio) { } diff --git a/include/uapi/linux/dma-buf.h b/include/uapi/linux/dma-buf.h index e827c9d20c5d..ea957f4b1d38 100644 --- a/include/uapi/linux/dma-buf.h +++ b/include/uapi/linux/dma-buf.h @@ -168,6 +168,31 @@ struct dma_buf_import_sync_file { __s32 fd; };
+/** + * struct dma_buf_transfer_charge - Transfer a dma-buf's memcg charge + * + * Any holder of a dma-buf file descriptor can perform a + * DMA_BUF_IOCTL_XFER_CHARGE to attribute the buffer's memory + * cost to a specific cgroup, or move it from one cgroup to + * another. The ioctl charges the new cgroup before uncharging + * the old one atomically. On failure the original charge is + * preserved. If the buffer has no existing charge, a new + * charge is established. + */ +struct dma_buf_transfer_charge { + /** @flags: Reserved, must be zero */ + __u32 flags; /* unused for now */ + /** + * @charge_fd: File descriptor identifying the target cgroup. + * + * May be either a pidfd, in which case the charge moves to the + * memory cgroup of the referenced process, or a cgroupfd obtained + * by opening a cgroup v2 directory, in which case the charge moves + * directly to that cgroup's memory controller. + */ + __u32 charge_fd; +}; + #define DMA_BUF_BASE 'b' #define DMA_BUF_IOCTL_SYNC _IOW(DMA_BUF_BASE, 0, struct dma_buf_sync)
@@ -179,5 +204,6 @@ struct dma_buf_import_sync_file { #define DMA_BUF_SET_NAME_B _IOW(DMA_BUF_BASE, 1, __u64) #define DMA_BUF_IOCTL_EXPORT_SYNC_FILE _IOWR(DMA_BUF_BASE, 2, struct dma_buf_export_sync_file) #define DMA_BUF_IOCTL_IMPORT_SYNC_FILE _IOW(DMA_BUF_BASE, 3, struct dma_buf_import_sync_file) +#define DMA_BUF_IOCTL_XFER_CHARGE _IOW(DMA_BUF_BASE, 4, struct dma_buf_transfer_charge)
#endif diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 7b841b905759..3adc84fe6668 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5786,6 +5786,47 @@ void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages) refill_stock(memcg, nr_pages); }
+/** + * mem_cgroup_transfer_dmabuf_charge - move a dma-buf charge + * @dmabuf: buffer to transfer charge + * @new_memcg: target memcg; caller must hold a reference + * + * Transfers the charge for @dmabuf to @new_memcg and, on + * success, stores @new_memcg in @dmabuf->memcg. + * Returns %true on success, %false if the charge against @new_memcg + * failed or a concurrent transfer raced on @dmabuf->memcg. + * On failure the caller retains its reference to @new_memcg. + */ +bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf, + struct mem_cgroup *new_memcg) +{ + struct mem_cgroup *old_memcg = READ_ONCE(dmabuf->memcg); + unsigned int nr_pages = PAGE_ALIGN(dmabuf->size) >> PAGE_SHIFT; + + if (mem_cgroup_disabled()) + return true; + + if (!mem_cgroup_reserve_dmabuf(new_memcg, GFP_KERNEL, nr_pages)) + return false; + + mod_memcg_state(new_memcg, MEMCG_DMABUF, nr_pages); + + if (cmpxchg(&dmabuf->memcg, old_memcg, new_memcg) != old_memcg) { + mod_memcg_state(new_memcg, MEMCG_DMABUF, -nr_pages); + mem_cgroup_cancel_dmabuf(new_memcg, nr_pages); + return false; + } + + if (old_memcg) { + /* Uncharge old cgroup */ + mod_memcg_state(old_memcg, MEMCG_DMABUF, -nr_pages); + mem_cgroup_cancel_dmabuf(old_memcg, nr_pages); + mem_cgroup_put(old_memcg); + } + + return true; +} + /** * mem_cgroup_uncharge_dmabuf - uncharge a dma-buf from its memcg * @dmabuf: buffer to uncharge
On 10/27/26 16:02, Albert Esteve wrote:
dma-heap already supports charging a specific cgroup at allocation time via charge_fd in DMA_HEAP_IOCTL_ALLOC. However, dma-bufs can originate from any exporter (DRM, V4L2, etc.), and the allocator may not know the final owner at allocation time.
Add DMA_BUF_IOCTL_XFER_CHARGE to transfer a dma-buf's memcg charge to a different cgroup after allocation. The target is identified by charge_fd, which may be a cgroupfd or a pidfd.
The transfer charges the new cgroup before uncharging the old one, preserving the original charge on failure. An atomic cmpxchg on dmabuf->memcg prevents races with concurrent transfers.
Not all dma-buf exporters back their buffers with reclaimable system memory. Some represent device registers, VRAM, or carved-out pools where size is not a meaningful memory cost. Add a memcg_account flag to dma_buf_export_info that exporters set when their buffers should participate in memcg accounting. The xfer ioctl returns -EOPNOTSUPP for buffers that do not set the flag. Currently only the system heap sets it, other exporters can opt in as needed.
Usage example: int cg_fd = open("/sys/fs/cgroup/clients/app1", O_RDONLY | O_DIRECTORY);
struct dma_buf_transfer_charge xfer = { .charge_fd = cg_fd, }; ioctl(dmabuf_fd, DMA_BUF_IOCTL_XFER_CHARGE, &xfer); /* charge now on app1's cgroup */ close(cg_fd);
Signed-off-by: Albert Esteve aesteve@redhat.com
drivers/dma-buf/dma-buf.c | 43 +++++++++++++++++++++++++++++++++++++ drivers/dma-buf/heaps/system_heap.c | 1 + include/linux/dma-buf.h | 5 +++++ include/linux/memcontrol.h | 9 ++++++++ include/uapi/linux/dma-buf.h | 26 ++++++++++++++++++++++ mm/memcontrol.c | 41 +++++++++++++++++++++++++++++++++++ 6 files changed, 125 insertions(+)
diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c index f31f49ebd0af..07513ecedc10 100644 --- a/drivers/dma-buf/dma-buf.c +++ b/drivers/dma-buf/dma-buf.c @@ -540,6 +540,45 @@ static long dma_buf_import_sync_file(struct dma_buf *dmabuf, } #endif +/* Transfer a dma-buf's memcg charge; see struct dma_buf_transfer_charge */ +static long dma_buf_xfer_charge(struct dma_buf *dmabuf,
const void __user *user_data)+{
- struct dma_buf_transfer_charge arg;
- struct mem_cgroup *memcg;
- int ret = 0;
- if (copy_from_user(&arg, user_data, sizeof(arg)))
return -EFAULT;- if (!IS_ENABLED(CONFIG_MEMCG))
return 0;- if (arg.flags != 0)
return -EINVAL;- if (!dmabuf->memcg_account)
return -EOPNOTSUPP;- memcg = mem_cgroup_get_from_fd(arg.charge_fd);
- if (IS_ERR(memcg))
return PTR_ERR(memcg);- if (dmabuf->memcg == memcg)
goto skip_transfer;- if (!mem_cgroup_transfer_dmabuf_charge(dmabuf, memcg)) {
ret = -ENOMEM;goto skip_transfer;- }
- return 0;
+skip_transfer:
- mem_cgroup_put(memcg);
- return ret;
+}
static long dma_buf_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { @@ -590,6 +629,9 @@ static long dma_buf_ioctl(struct file *file, return dma_buf_import_sync_file(dmabuf, (const void __user *)arg); #endif
- case DMA_BUF_IOCTL_XFER_CHARGE:
return dma_buf_xfer_charge(dmabuf, (const void __user *)arg);- default: return -ENOTTY; }
@@ -750,6 +792,7 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info) dmabuf->size = exp_info->size; dmabuf->exp_name = exp_info->exp_name; dmabuf->owner = exp_info->owner;
- dmabuf->memcg_account = exp_info->memcg_account;
The initial charging should probably be done here as well.
This way DMA-buf heaps only needs to pass in the charge_fd and we can re-use the functionality for other DMA-buf exporters as well.
spin_lock_init(&dmabuf->name_lock); init_waitqueue_head(&dmabuf->poll); dmabuf->cb_in.poll = dmabuf->cb_out.poll = &dmabuf->poll; diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c index adeea8cd328b..a764167167ba 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -479,6 +479,7 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, exp_info.size = buffer->len; exp_info.flags = fd_flags; exp_info.priv = buffer;
- exp_info.memcg_account = true; dmabuf = dma_buf_export(&exp_info); if (IS_ERR(dmabuf)) { ret = PTR_ERR(dmabuf);
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index 23ec0ff23fba..ac0ec88cab93 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -433,6 +433,9 @@ struct dma_buf { /** @memcg: the cgroup to which this buffer is currently attributed */ struct mem_cgroup *memcg;
- /** @memcg_account: account buffer memory to cgroup */
- bool memcg_account;
}; /** @@ -505,6 +508,7 @@ struct dma_buf_attachment {
- @flags: mode flags for the file
- @resv: reservation-object, NULL to allocate default one
- @priv: Attach private data of allocator to this buffer
- @memcg_account: account buffer memory to cgroup
- This structure holds the information required to export the buffer. Used
- with dma_buf_export() only.
@@ -517,6 +521,7 @@ struct dma_buf_export_info { int flags; struct dma_resv *resv; void *priv;
- bool memcg_account;
}; /** diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 6b30baa79fa9..36fddafbef4e 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -679,6 +679,9 @@ bool mem_cgroup_reserve_dmabuf(struct mem_cgroup *memcg, gfp_t gfp, void mem_cgroup_commit_dmabuf(struct dma_buf *dmabuf, struct mem_cgroup *memcg, unsigned int nr_pages); +bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg);void __mem_cgroup_uncharge(struct folio *folio); /** @@ -1206,6 +1209,12 @@ static inline void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, { } +static inline bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)+{
- return true;
+}
static inline void mem_cgroup_uncharge(struct folio *folio) { } diff --git a/include/uapi/linux/dma-buf.h b/include/uapi/linux/dma-buf.h index e827c9d20c5d..ea957f4b1d38 100644 --- a/include/uapi/linux/dma-buf.h +++ b/include/uapi/linux/dma-buf.h @@ -168,6 +168,31 @@ struct dma_buf_import_sync_file { __s32 fd; }; +/**
- struct dma_buf_transfer_charge - Transfer a dma-buf's memcg charge
- Any holder of a dma-buf file descriptor can perform a
- DMA_BUF_IOCTL_XFER_CHARGE to attribute the buffer's memory
- cost to a specific cgroup, or move it from one cgroup to
- another. The ioctl charges the new cgroup before uncharging
- the old one atomically. On failure the original charge is
- preserved. If the buffer has no existing charge, a new
- charge is established.
- */
+struct dma_buf_transfer_charge {
- /** @flags: Reserved, must be zero */
- __u32 flags; /* unused for now */
- /**
* @charge_fd: File descriptor identifying the target cgroup.** May be either a pidfd, in which case the charge moves to the* memory cgroup of the referenced process, or a cgroupfd obtained* by opening a cgroup v2 directory, in which case the charge moves* directly to that cgroup's memory controller.*/- __u32 charge_fd;
+};
#define DMA_BUF_BASE 'b' #define DMA_BUF_IOCTL_SYNC _IOW(DMA_BUF_BASE, 0, struct dma_buf_sync) @@ -179,5 +204,6 @@ struct dma_buf_import_sync_file { #define DMA_BUF_SET_NAME_B _IOW(DMA_BUF_BASE, 1, __u64) #define DMA_BUF_IOCTL_EXPORT_SYNC_FILE _IOWR(DMA_BUF_BASE, 2, struct dma_buf_export_sync_file) #define DMA_BUF_IOCTL_IMPORT_SYNC_FILE _IOW(DMA_BUF_BASE, 3, struct dma_buf_import_sync_file) +#define DMA_BUF_IOCTL_XFER_CHARGE _IOW(DMA_BUF_BASE, 4, struct dma_buf_transfer_charge) #endif diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 7b841b905759..3adc84fe6668 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5786,6 +5786,47 @@ void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages) refill_stock(memcg, nr_pages); } +/**
- mem_cgroup_transfer_dmabuf_charge - move a dma-buf charge
- @dmabuf: buffer to transfer charge
- @new_memcg: target memcg; caller must hold a reference
- Transfers the charge for @dmabuf to @new_memcg and, on
- success, stores @new_memcg in @dmabuf->memcg.
- Returns %true on success, %false if the charge against @new_memcg
- failed or a concurrent transfer raced on @dmabuf->memcg.
- On failure the caller retains its reference to @new_memcg.
- */
+bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)
I would make that more generic instead of dmabuf specific. E.g. something like this:
bool mem_cgroup_transfer_charge(struct struct mem_cgroup **ptr_memcg, struct mem_cgroup *new_memcg, enum memcg_stat_item item, unsigned int nr_pages)
+{
- struct mem_cgroup *old_memcg = READ_ONCE(dmabuf->memcg);
- unsigned int nr_pages = PAGE_ALIGN(dmabuf->size) >> PAGE_SHIFT;
This way we can also document in the DMA-buf caller that this conversation must be ok when memcg_account is true.
Regards, Christian.
- if (mem_cgroup_disabled())
return true;- if (!mem_cgroup_reserve_dmabuf(new_memcg, GFP_KERNEL, nr_pages))
return false;- mod_memcg_state(new_memcg, MEMCG_DMABUF, nr_pages);
- if (cmpxchg(&dmabuf->memcg, old_memcg, new_memcg) != old_memcg) {
mod_memcg_state(new_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(new_memcg, nr_pages);return false;- }
- if (old_memcg) {
/* Uncharge old cgroup */mod_memcg_state(old_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(old_memcg, nr_pages);mem_cgroup_put(old_memcg);- }
- return true;
+}
/**
- mem_cgroup_uncharge_dmabuf - uncharge a dma-buf from its memcg
- @dmabuf: buffer to uncharge
On Wed, Oct 28, 2026 at 10:08:19AM +0100, Christian König wrote:
On 10/27/26 16:02, Albert Esteve wrote:
dma-heap already supports charging a specific cgroup at allocation time via charge_fd in DMA_HEAP_IOCTL_ALLOC. However, dma-bufs can originate from any exporter (DRM, V4L2, etc.), and the allocator may not know the final owner at allocation time.
Add DMA_BUF_IOCTL_XFER_CHARGE to transfer a dma-buf's memcg charge to a different cgroup after allocation. The target is identified by charge_fd, which may be a cgroupfd or a pidfd.
The transfer charges the new cgroup before uncharging the old one, preserving the original charge on failure. An atomic cmpxchg on dmabuf->memcg prevents races with concurrent transfers.
Not all dma-buf exporters back their buffers with reclaimable system memory. Some represent device registers, VRAM, or carved-out pools where size is not a meaningful memory cost. Add a memcg_account flag to dma_buf_export_info that exporters set when their buffers should participate in memcg accounting. The xfer ioctl returns -EOPNOTSUPP for buffers that do not set the flag. Currently only the system heap sets it, other exporters can opt in as needed.
Usage example: int cg_fd = open("/sys/fs/cgroup/clients/app1", O_RDONLY | O_DIRECTORY);
struct dma_buf_transfer_charge xfer = { .charge_fd = cg_fd, }; ioctl(dmabuf_fd, DMA_BUF_IOCTL_XFER_CHARGE, &xfer); /* charge now on app1's cgroup */ close(cg_fd);
Signed-off-by: Albert Esteve aesteve@redhat.com
drivers/dma-buf/dma-buf.c | 43 +++++++++++++++++++++++++++++++++++++ drivers/dma-buf/heaps/system_heap.c | 1 + include/linux/dma-buf.h | 5 +++++ include/linux/memcontrol.h | 9 ++++++++ include/uapi/linux/dma-buf.h | 26 ++++++++++++++++++++++ mm/memcontrol.c | 41 +++++++++++++++++++++++++++++++++++ 6 files changed, 125 insertions(+)
diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c index f31f49ebd0af..07513ecedc10 100644 --- a/drivers/dma-buf/dma-buf.c +++ b/drivers/dma-buf/dma-buf.c @@ -540,6 +540,45 @@ static long dma_buf_import_sync_file(struct dma_buf *dmabuf, } #endif +/* Transfer a dma-buf's memcg charge; see struct dma_buf_transfer_charge */ +static long dma_buf_xfer_charge(struct dma_buf *dmabuf,
const void __user *user_data)+{
- struct dma_buf_transfer_charge arg;
- struct mem_cgroup *memcg;
- int ret = 0;
- if (copy_from_user(&arg, user_data, sizeof(arg)))
return -EFAULT;- if (!IS_ENABLED(CONFIG_MEMCG))
return 0;- if (arg.flags != 0)
return -EINVAL;- if (!dmabuf->memcg_account)
return -EOPNOTSUPP;- memcg = mem_cgroup_get_from_fd(arg.charge_fd);
- if (IS_ERR(memcg))
return PTR_ERR(memcg);- if (dmabuf->memcg == memcg)
goto skip_transfer;- if (!mem_cgroup_transfer_dmabuf_charge(dmabuf, memcg)) {
ret = -ENOMEM;goto skip_transfer;- }
- return 0;
+skip_transfer:
- mem_cgroup_put(memcg);
- return ret;
+}
static long dma_buf_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { @@ -590,6 +629,9 @@ static long dma_buf_ioctl(struct file *file, return dma_buf_import_sync_file(dmabuf, (const void __user *)arg); #endif
- case DMA_BUF_IOCTL_XFER_CHARGE:
return dma_buf_xfer_charge(dmabuf, (const void __user *)arg);- default: return -ENOTTY; }
@@ -750,6 +792,7 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info) dmabuf->size = exp_info->size; dmabuf->exp_name = exp_info->exp_name; dmabuf->owner = exp_info->owner;
- dmabuf->memcg_account = exp_info->memcg_account;
The initial charging should probably be done here as well.
This way DMA-buf heaps only needs to pass in the charge_fd and we can re-use the functionality for other DMA-buf exporters as well.
The problem with doing the charge here is that it happens too late. That is the reasoning for this proposal to move all the charging logic into heap allocators before pages are allocated.
What we could do is fill the dmabuf->memcg field when the flag is set (i.e., committing into the dmabuf in the patch semantics). But for that we'd need adding the memcg we want to charge it to into exp_info in order to support charging to cgroups other than the allocator's (for the common case would be simpler as we can just look at current). Maybe that's what you meant?
spin_lock_init(&dmabuf->name_lock); init_waitqueue_head(&dmabuf->poll); dmabuf->cb_in.poll = dmabuf->cb_out.poll = &dmabuf->poll; diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c index adeea8cd328b..a764167167ba 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -479,6 +479,7 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, exp_info.size = buffer->len; exp_info.flags = fd_flags; exp_info.priv = buffer;
- exp_info.memcg_account = true; dmabuf = dma_buf_export(&exp_info); if (IS_ERR(dmabuf)) { ret = PTR_ERR(dmabuf);
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index 23ec0ff23fba..ac0ec88cab93 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -433,6 +433,9 @@ struct dma_buf { /** @memcg: the cgroup to which this buffer is currently attributed */ struct mem_cgroup *memcg;
- /** @memcg_account: account buffer memory to cgroup */
- bool memcg_account;
}; /** @@ -505,6 +508,7 @@ struct dma_buf_attachment {
- @flags: mode flags for the file
- @resv: reservation-object, NULL to allocate default one
- @priv: Attach private data of allocator to this buffer
- @memcg_account: account buffer memory to cgroup
- This structure holds the information required to export the buffer. Used
- with dma_buf_export() only.
@@ -517,6 +521,7 @@ struct dma_buf_export_info { int flags; struct dma_resv *resv; void *priv;
- bool memcg_account;
}; /** diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 6b30baa79fa9..36fddafbef4e 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -679,6 +679,9 @@ bool mem_cgroup_reserve_dmabuf(struct mem_cgroup *memcg, gfp_t gfp, void mem_cgroup_commit_dmabuf(struct dma_buf *dmabuf, struct mem_cgroup *memcg, unsigned int nr_pages); +bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg);void __mem_cgroup_uncharge(struct folio *folio); /** @@ -1206,6 +1209,12 @@ static inline void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, { } +static inline bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)+{
- return true;
+}
static inline void mem_cgroup_uncharge(struct folio *folio) { } diff --git a/include/uapi/linux/dma-buf.h b/include/uapi/linux/dma-buf.h index e827c9d20c5d..ea957f4b1d38 100644 --- a/include/uapi/linux/dma-buf.h +++ b/include/uapi/linux/dma-buf.h @@ -168,6 +168,31 @@ struct dma_buf_import_sync_file { __s32 fd; }; +/**
- struct dma_buf_transfer_charge - Transfer a dma-buf's memcg charge
- Any holder of a dma-buf file descriptor can perform a
- DMA_BUF_IOCTL_XFER_CHARGE to attribute the buffer's memory
- cost to a specific cgroup, or move it from one cgroup to
- another. The ioctl charges the new cgroup before uncharging
- the old one atomically. On failure the original charge is
- preserved. If the buffer has no existing charge, a new
- charge is established.
- */
+struct dma_buf_transfer_charge {
- /** @flags: Reserved, must be zero */
- __u32 flags; /* unused for now */
- /**
* @charge_fd: File descriptor identifying the target cgroup.** May be either a pidfd, in which case the charge moves to the* memory cgroup of the referenced process, or a cgroupfd obtained* by opening a cgroup v2 directory, in which case the charge moves* directly to that cgroup's memory controller.*/- __u32 charge_fd;
+};
#define DMA_BUF_BASE 'b' #define DMA_BUF_IOCTL_SYNC _IOW(DMA_BUF_BASE, 0, struct dma_buf_sync) @@ -179,5 +204,6 @@ struct dma_buf_import_sync_file { #define DMA_BUF_SET_NAME_B _IOW(DMA_BUF_BASE, 1, __u64) #define DMA_BUF_IOCTL_EXPORT_SYNC_FILE _IOWR(DMA_BUF_BASE, 2, struct dma_buf_export_sync_file) #define DMA_BUF_IOCTL_IMPORT_SYNC_FILE _IOW(DMA_BUF_BASE, 3, struct dma_buf_import_sync_file) +#define DMA_BUF_IOCTL_XFER_CHARGE _IOW(DMA_BUF_BASE, 4, struct dma_buf_transfer_charge) #endif diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 7b841b905759..3adc84fe6668 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5786,6 +5786,47 @@ void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages) refill_stock(memcg, nr_pages); } +/**
- mem_cgroup_transfer_dmabuf_charge - move a dma-buf charge
- @dmabuf: buffer to transfer charge
- @new_memcg: target memcg; caller must hold a reference
- Transfers the charge for @dmabuf to @new_memcg and, on
- success, stores @new_memcg in @dmabuf->memcg.
- Returns %true on success, %false if the charge against @new_memcg
- failed or a concurrent transfer raced on @dmabuf->memcg.
- On failure the caller retains its reference to @new_memcg.
- */
+bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)I would make that more generic instead of dmabuf specific. E.g. something like this:
bool mem_cgroup_transfer_charge(struct struct mem_cgroup **ptr_memcg, struct mem_cgroup *new_memcg, enum memcg_stat_item item, unsigned int nr_pages)
Ah yes, I like this generalization. And the implementation remains mostly the same. Thanks!
+{
- struct mem_cgroup *old_memcg = READ_ONCE(dmabuf->memcg);
- unsigned int nr_pages = PAGE_ALIGN(dmabuf->size) >> PAGE_SHIFT;
This way we can also document in the DMA-buf caller that this conversation must be ok when memcg_account is true.
Regards, Christian.
- if (mem_cgroup_disabled())
return true;- if (!mem_cgroup_reserve_dmabuf(new_memcg, GFP_KERNEL, nr_pages))
return false;- mod_memcg_state(new_memcg, MEMCG_DMABUF, nr_pages);
- if (cmpxchg(&dmabuf->memcg, old_memcg, new_memcg) != old_memcg) {
mod_memcg_state(new_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(new_memcg, nr_pages);return false;- }
- if (old_memcg) {
/* Uncharge old cgroup */mod_memcg_state(old_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(old_memcg, nr_pages);mem_cgroup_put(old_memcg);- }
- return true;
+}
/**
- mem_cgroup_uncharge_dmabuf - uncharge a dma-buf from its memcg
- @dmabuf: buffer to uncharge
On Wed, Oct 28, 2026 at 05:09:37PM +0100, Albert Esteve via OP-TEE wrote:
On Wed, Oct 28, 2026 at 10:08:19AM +0100, Christian König wrote:
On 10/27/26 16:02, Albert Esteve wrote:
dma-heap already supports charging a specific cgroup at allocation time via charge_fd in DMA_HEAP_IOCTL_ALLOC. However, dma-bufs can originate from any exporter (DRM, V4L2, etc.), and the allocator may not know the final owner at allocation time.
Add DMA_BUF_IOCTL_XFER_CHARGE to transfer a dma-buf's memcg charge to a different cgroup after allocation. The target is identified by charge_fd, which may be a cgroupfd or a pidfd.
The transfer charges the new cgroup before uncharging the old one, preserving the original charge on failure. An atomic cmpxchg on dmabuf->memcg prevents races with concurrent transfers.
Not all dma-buf exporters back their buffers with reclaimable system memory. Some represent device registers, VRAM, or carved-out pools where size is not a meaningful memory cost. Add a memcg_account flag to dma_buf_export_info that exporters set when their buffers should participate in memcg accounting. The xfer ioctl returns -EOPNOTSUPP for buffers that do not set the flag. Currently only the system heap sets it, other exporters can opt in as needed.
Usage example: int cg_fd = open("/sys/fs/cgroup/clients/app1", O_RDONLY | O_DIRECTORY);
struct dma_buf_transfer_charge xfer = { .charge_fd = cg_fd, }; ioctl(dmabuf_fd, DMA_BUF_IOCTL_XFER_CHARGE, &xfer); /* charge now on app1's cgroup */ close(cg_fd);
Signed-off-by: Albert Esteve aesteve@redhat.com
drivers/dma-buf/dma-buf.c | 43 +++++++++++++++++++++++++++++++++++++ drivers/dma-buf/heaps/system_heap.c | 1 + include/linux/dma-buf.h | 5 +++++ include/linux/memcontrol.h | 9 ++++++++ include/uapi/linux/dma-buf.h | 26 ++++++++++++++++++++++ mm/memcontrol.c | 41 +++++++++++++++++++++++++++++++++++ 6 files changed, 125 insertions(+)
diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c index f31f49ebd0af..07513ecedc10 100644 --- a/drivers/dma-buf/dma-buf.c +++ b/drivers/dma-buf/dma-buf.c @@ -540,6 +540,45 @@ static long dma_buf_import_sync_file(struct dma_buf *dmabuf, } #endif +/* Transfer a dma-buf's memcg charge; see struct dma_buf_transfer_charge */ +static long dma_buf_xfer_charge(struct dma_buf *dmabuf,
const void __user *user_data)+{
- struct dma_buf_transfer_charge arg;
- struct mem_cgroup *memcg;
- int ret = 0;
- if (copy_from_user(&arg, user_data, sizeof(arg)))
return -EFAULT;- if (!IS_ENABLED(CONFIG_MEMCG))
return 0;- if (arg.flags != 0)
return -EINVAL;- if (!dmabuf->memcg_account)
return -EOPNOTSUPP;- memcg = mem_cgroup_get_from_fd(arg.charge_fd);
- if (IS_ERR(memcg))
return PTR_ERR(memcg);- if (dmabuf->memcg == memcg)
goto skip_transfer;- if (!mem_cgroup_transfer_dmabuf_charge(dmabuf, memcg)) {
ret = -ENOMEM;goto skip_transfer;- }
- return 0;
+skip_transfer:
- mem_cgroup_put(memcg);
- return ret;
+}
static long dma_buf_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { @@ -590,6 +629,9 @@ static long dma_buf_ioctl(struct file *file, return dma_buf_import_sync_file(dmabuf, (const void __user *)arg); #endif
- case DMA_BUF_IOCTL_XFER_CHARGE:
return dma_buf_xfer_charge(dmabuf, (const void __user *)arg);- default: return -ENOTTY; }
@@ -750,6 +792,7 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info) dmabuf->size = exp_info->size; dmabuf->exp_name = exp_info->exp_name; dmabuf->owner = exp_info->owner;
- dmabuf->memcg_account = exp_info->memcg_account;
The initial charging should probably be done here as well.
This way DMA-buf heaps only needs to pass in the charge_fd and we can re-use the functionality for other DMA-buf exporters as well.
The problem with doing the charge here is that it happens too late. That is the reasoning for this proposal to move all the charging logic into heap allocators before pages are allocated.
What we could do is fill the dmabuf->memcg field when the flag is set (i.e., committing into the dmabuf in the patch semantics). But for that we'd need adding the memcg we want to charge it to into exp_info in order to support charging to cgroups other than the allocator's (for the common case would be simpler as we can just look at current). Maybe that's what you meant?
Actually if I do this we can remove the memcg_account flag and just uses exp_info->memcg as both the memcg to set *and* the flag when it is != NULL. Exporters that set the memcg info are implicitly flagging the dmabuf as chargeable (and transfereable).
The only thing is that mod_memcg_state() still needs nr_pages and we cannot use size as that varies from one exporter to another as you explained before. So that can be left for the exporter to call. WDYT?
spin_lock_init(&dmabuf->name_lock); init_waitqueue_head(&dmabuf->poll); dmabuf->cb_in.poll = dmabuf->cb_out.poll = &dmabuf->poll; diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c index adeea8cd328b..a764167167ba 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -479,6 +479,7 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, exp_info.size = buffer->len; exp_info.flags = fd_flags; exp_info.priv = buffer;
- exp_info.memcg_account = true; dmabuf = dma_buf_export(&exp_info); if (IS_ERR(dmabuf)) { ret = PTR_ERR(dmabuf);
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index 23ec0ff23fba..ac0ec88cab93 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -433,6 +433,9 @@ struct dma_buf { /** @memcg: the cgroup to which this buffer is currently attributed */ struct mem_cgroup *memcg;
- /** @memcg_account: account buffer memory to cgroup */
- bool memcg_account;
}; /** @@ -505,6 +508,7 @@ struct dma_buf_attachment {
- @flags: mode flags for the file
- @resv: reservation-object, NULL to allocate default one
- @priv: Attach private data of allocator to this buffer
- @memcg_account: account buffer memory to cgroup
- This structure holds the information required to export the buffer. Used
- with dma_buf_export() only.
@@ -517,6 +521,7 @@ struct dma_buf_export_info { int flags; struct dma_resv *resv; void *priv;
- bool memcg_account;
}; /** diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 6b30baa79fa9..36fddafbef4e 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -679,6 +679,9 @@ bool mem_cgroup_reserve_dmabuf(struct mem_cgroup *memcg, gfp_t gfp, void mem_cgroup_commit_dmabuf(struct dma_buf *dmabuf, struct mem_cgroup *memcg, unsigned int nr_pages); +bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg);void __mem_cgroup_uncharge(struct folio *folio); /** @@ -1206,6 +1209,12 @@ static inline void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, { } +static inline bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)+{
- return true;
+}
static inline void mem_cgroup_uncharge(struct folio *folio) { } diff --git a/include/uapi/linux/dma-buf.h b/include/uapi/linux/dma-buf.h index e827c9d20c5d..ea957f4b1d38 100644 --- a/include/uapi/linux/dma-buf.h +++ b/include/uapi/linux/dma-buf.h @@ -168,6 +168,31 @@ struct dma_buf_import_sync_file { __s32 fd; }; +/**
- struct dma_buf_transfer_charge - Transfer a dma-buf's memcg charge
- Any holder of a dma-buf file descriptor can perform a
- DMA_BUF_IOCTL_XFER_CHARGE to attribute the buffer's memory
- cost to a specific cgroup, or move it from one cgroup to
- another. The ioctl charges the new cgroup before uncharging
- the old one atomically. On failure the original charge is
- preserved. If the buffer has no existing charge, a new
- charge is established.
- */
+struct dma_buf_transfer_charge {
- /** @flags: Reserved, must be zero */
- __u32 flags; /* unused for now */
- /**
* @charge_fd: File descriptor identifying the target cgroup.** May be either a pidfd, in which case the charge moves to the* memory cgroup of the referenced process, or a cgroupfd obtained* by opening a cgroup v2 directory, in which case the charge moves* directly to that cgroup's memory controller.*/- __u32 charge_fd;
+};
#define DMA_BUF_BASE 'b' #define DMA_BUF_IOCTL_SYNC _IOW(DMA_BUF_BASE, 0, struct dma_buf_sync) @@ -179,5 +204,6 @@ struct dma_buf_import_sync_file { #define DMA_BUF_SET_NAME_B _IOW(DMA_BUF_BASE, 1, __u64) #define DMA_BUF_IOCTL_EXPORT_SYNC_FILE _IOWR(DMA_BUF_BASE, 2, struct dma_buf_export_sync_file) #define DMA_BUF_IOCTL_IMPORT_SYNC_FILE _IOW(DMA_BUF_BASE, 3, struct dma_buf_import_sync_file) +#define DMA_BUF_IOCTL_XFER_CHARGE _IOW(DMA_BUF_BASE, 4, struct dma_buf_transfer_charge) #endif diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 7b841b905759..3adc84fe6668 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5786,6 +5786,47 @@ void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages) refill_stock(memcg, nr_pages); } +/**
- mem_cgroup_transfer_dmabuf_charge - move a dma-buf charge
- @dmabuf: buffer to transfer charge
- @new_memcg: target memcg; caller must hold a reference
- Transfers the charge for @dmabuf to @new_memcg and, on
- success, stores @new_memcg in @dmabuf->memcg.
- Returns %true on success, %false if the charge against @new_memcg
- failed or a concurrent transfer raced on @dmabuf->memcg.
- On failure the caller retains its reference to @new_memcg.
- */
+bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)I would make that more generic instead of dmabuf specific. E.g. something like this:
bool mem_cgroup_transfer_charge(struct struct mem_cgroup **ptr_memcg, struct mem_cgroup *new_memcg, enum memcg_stat_item item, unsigned int nr_pages)
Ah yes, I like this generalization. And the implementation remains mostly the same. Thanks!
+{
- struct mem_cgroup *old_memcg = READ_ONCE(dmabuf->memcg);
- unsigned int nr_pages = PAGE_ALIGN(dmabuf->size) >> PAGE_SHIFT;
This way we can also document in the DMA-buf caller that this conversation must be ok when memcg_account is true.
Regards, Christian.
- if (mem_cgroup_disabled())
return true;- if (!mem_cgroup_reserve_dmabuf(new_memcg, GFP_KERNEL, nr_pages))
return false;- mod_memcg_state(new_memcg, MEMCG_DMABUF, nr_pages);
- if (cmpxchg(&dmabuf->memcg, old_memcg, new_memcg) != old_memcg) {
mod_memcg_state(new_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(new_memcg, nr_pages);return false;- }
- if (old_memcg) {
/* Uncharge old cgroup */mod_memcg_state(old_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(old_memcg, nr_pages);mem_cgroup_put(old_memcg);- }
- return true;
+}
/**
- mem_cgroup_uncharge_dmabuf - uncharge a dma-buf from its memcg
- @dmabuf: buffer to uncharge
On 10/28/26 17:09, Albert Esteve wrote:
On Wed, Oct 28, 2026 at 10:08:19AM +0100, Christian König wrote:
On 10/27/26 16:02, Albert Esteve wrote:
dma-heap already supports charging a specific cgroup at allocation time via charge_fd in DMA_HEAP_IOCTL_ALLOC. However, dma-bufs can originate from any exporter (DRM, V4L2, etc.), and the allocator may not know the final owner at allocation time.
Add DMA_BUF_IOCTL_XFER_CHARGE to transfer a dma-buf's memcg charge to a different cgroup after allocation. The target is identified by charge_fd, which may be a cgroupfd or a pidfd.
The transfer charges the new cgroup before uncharging the old one, preserving the original charge on failure. An atomic cmpxchg on dmabuf->memcg prevents races with concurrent transfers.
Not all dma-buf exporters back their buffers with reclaimable system memory. Some represent device registers, VRAM, or carved-out pools where size is not a meaningful memory cost. Add a memcg_account flag to dma_buf_export_info that exporters set when their buffers should participate in memcg accounting. The xfer ioctl returns -EOPNOTSUPP for buffers that do not set the flag. Currently only the system heap sets it, other exporters can opt in as needed.
Usage example: int cg_fd = open("/sys/fs/cgroup/clients/app1", O_RDONLY | O_DIRECTORY);
struct dma_buf_transfer_charge xfer = { .charge_fd = cg_fd, }; ioctl(dmabuf_fd, DMA_BUF_IOCTL_XFER_CHARGE, &xfer); /* charge now on app1's cgroup */ close(cg_fd);
Signed-off-by: Albert Esteve aesteve@redhat.com
drivers/dma-buf/dma-buf.c | 43 +++++++++++++++++++++++++++++++++++++ drivers/dma-buf/heaps/system_heap.c | 1 + include/linux/dma-buf.h | 5 +++++ include/linux/memcontrol.h | 9 ++++++++ include/uapi/linux/dma-buf.h | 26 ++++++++++++++++++++++ mm/memcontrol.c | 41 +++++++++++++++++++++++++++++++++++ 6 files changed, 125 insertions(+)
diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c index f31f49ebd0af..07513ecedc10 100644 --- a/drivers/dma-buf/dma-buf.c +++ b/drivers/dma-buf/dma-buf.c @@ -540,6 +540,45 @@ static long dma_buf_import_sync_file(struct dma_buf *dmabuf, } #endif +/* Transfer a dma-buf's memcg charge; see struct dma_buf_transfer_charge */ +static long dma_buf_xfer_charge(struct dma_buf *dmabuf,
const void __user *user_data)+{
- struct dma_buf_transfer_charge arg;
- struct mem_cgroup *memcg;
- int ret = 0;
- if (copy_from_user(&arg, user_data, sizeof(arg)))
return -EFAULT;- if (!IS_ENABLED(CONFIG_MEMCG))
return 0;- if (arg.flags != 0)
return -EINVAL;- if (!dmabuf->memcg_account)
return -EOPNOTSUPP;- memcg = mem_cgroup_get_from_fd(arg.charge_fd);
- if (IS_ERR(memcg))
return PTR_ERR(memcg);- if (dmabuf->memcg == memcg)
goto skip_transfer;- if (!mem_cgroup_transfer_dmabuf_charge(dmabuf, memcg)) {
ret = -ENOMEM;goto skip_transfer;- }
- return 0;
+skip_transfer:
- mem_cgroup_put(memcg);
- return ret;
+}
static long dma_buf_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { @@ -590,6 +629,9 @@ static long dma_buf_ioctl(struct file *file, return dma_buf_import_sync_file(dmabuf, (const void __user *)arg); #endif
- case DMA_BUF_IOCTL_XFER_CHARGE:
return dma_buf_xfer_charge(dmabuf, (const void __user *)arg);- default: return -ENOTTY; }
@@ -750,6 +792,7 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info) dmabuf->size = exp_info->size; dmabuf->exp_name = exp_info->exp_name; dmabuf->owner = exp_info->owner;
- dmabuf->memcg_account = exp_info->memcg_account;
The initial charging should probably be done here as well.
This way DMA-buf heaps only needs to pass in the charge_fd and we can re-use the functionality for other DMA-buf exporters as well.
The problem with doing the charge here is that it happens too late. That is the reasoning for this proposal to move all the charging logic into heap allocators before pages are allocated.
Oh, that is a really good point we need to talk about.
DMA-buf heaps is actually a rather simple use case, but we have way more complex things.
For example in some use cases the backing store of a DMA-buf is only allocated on first use and only then it needs to be comitted to it's memcg or dmem.
I don't mind implementing it for DMA-buf heaps first, but we should have the uAPI and it's handling so solid that it also works for future use cases.
What we could do is fill the dmabuf->memcg field when the flag is set (i.e., committing into the dmabuf in the patch semantics). But for that we'd need adding the memcg we want to charge it to into exp_info in order to support charging to cgroups other than the allocator's (for the common case would be simpler as we can just look at current). Maybe that's what you meant?
At least for the DMA-buf heaps use case that sounds reasonable for now.
Regards, Christian.
spin_lock_init(&dmabuf->name_lock); init_waitqueue_head(&dmabuf->poll); dmabuf->cb_in.poll = dmabuf->cb_out.poll = &dmabuf->poll; diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c index adeea8cd328b..a764167167ba 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -479,6 +479,7 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, exp_info.size = buffer->len; exp_info.flags = fd_flags; exp_info.priv = buffer;
- exp_info.memcg_account = true; dmabuf = dma_buf_export(&exp_info); if (IS_ERR(dmabuf)) { ret = PTR_ERR(dmabuf);
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index 23ec0ff23fba..ac0ec88cab93 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -433,6 +433,9 @@ struct dma_buf { /** @memcg: the cgroup to which this buffer is currently attributed */ struct mem_cgroup *memcg;
- /** @memcg_account: account buffer memory to cgroup */
- bool memcg_account;
}; /** @@ -505,6 +508,7 @@ struct dma_buf_attachment {
- @flags: mode flags for the file
- @resv: reservation-object, NULL to allocate default one
- @priv: Attach private data of allocator to this buffer
- @memcg_account: account buffer memory to cgroup
- This structure holds the information required to export the buffer. Used
- with dma_buf_export() only.
@@ -517,6 +521,7 @@ struct dma_buf_export_info { int flags; struct dma_resv *resv; void *priv;
- bool memcg_account;
}; /** diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 6b30baa79fa9..36fddafbef4e 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -679,6 +679,9 @@ bool mem_cgroup_reserve_dmabuf(struct mem_cgroup *memcg, gfp_t gfp, void mem_cgroup_commit_dmabuf(struct dma_buf *dmabuf, struct mem_cgroup *memcg, unsigned int nr_pages); +bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg);void __mem_cgroup_uncharge(struct folio *folio); /** @@ -1206,6 +1209,12 @@ static inline void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, { } +static inline bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)+{
- return true;
+}
static inline void mem_cgroup_uncharge(struct folio *folio) { } diff --git a/include/uapi/linux/dma-buf.h b/include/uapi/linux/dma-buf.h index e827c9d20c5d..ea957f4b1d38 100644 --- a/include/uapi/linux/dma-buf.h +++ b/include/uapi/linux/dma-buf.h @@ -168,6 +168,31 @@ struct dma_buf_import_sync_file { __s32 fd; }; +/**
- struct dma_buf_transfer_charge - Transfer a dma-buf's memcg charge
- Any holder of a dma-buf file descriptor can perform a
- DMA_BUF_IOCTL_XFER_CHARGE to attribute the buffer's memory
- cost to a specific cgroup, or move it from one cgroup to
- another. The ioctl charges the new cgroup before uncharging
- the old one atomically. On failure the original charge is
- preserved. If the buffer has no existing charge, a new
- charge is established.
- */
+struct dma_buf_transfer_charge {
- /** @flags: Reserved, must be zero */
- __u32 flags; /* unused for now */
- /**
* @charge_fd: File descriptor identifying the target cgroup.** May be either a pidfd, in which case the charge moves to the* memory cgroup of the referenced process, or a cgroupfd obtained* by opening a cgroup v2 directory, in which case the charge moves* directly to that cgroup's memory controller.*/- __u32 charge_fd;
+};
#define DMA_BUF_BASE 'b' #define DMA_BUF_IOCTL_SYNC _IOW(DMA_BUF_BASE, 0, struct dma_buf_sync) @@ -179,5 +204,6 @@ struct dma_buf_import_sync_file { #define DMA_BUF_SET_NAME_B _IOW(DMA_BUF_BASE, 1, __u64) #define DMA_BUF_IOCTL_EXPORT_SYNC_FILE _IOWR(DMA_BUF_BASE, 2, struct dma_buf_export_sync_file) #define DMA_BUF_IOCTL_IMPORT_SYNC_FILE _IOW(DMA_BUF_BASE, 3, struct dma_buf_import_sync_file) +#define DMA_BUF_IOCTL_XFER_CHARGE _IOW(DMA_BUF_BASE, 4, struct dma_buf_transfer_charge) #endif diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 7b841b905759..3adc84fe6668 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5786,6 +5786,47 @@ void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages) refill_stock(memcg, nr_pages); } +/**
- mem_cgroup_transfer_dmabuf_charge - move a dma-buf charge
- @dmabuf: buffer to transfer charge
- @new_memcg: target memcg; caller must hold a reference
- Transfers the charge for @dmabuf to @new_memcg and, on
- success, stores @new_memcg in @dmabuf->memcg.
- Returns %true on success, %false if the charge against @new_memcg
- failed or a concurrent transfer raced on @dmabuf->memcg.
- On failure the caller retains its reference to @new_memcg.
- */
+bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)I would make that more generic instead of dmabuf specific. E.g. something like this:
bool mem_cgroup_transfer_charge(struct struct mem_cgroup **ptr_memcg, struct mem_cgroup *new_memcg, enum memcg_stat_item item, unsigned int nr_pages)
Ah yes, I like this generalization. And the implementation remains mostly the same. Thanks!
+{
- struct mem_cgroup *old_memcg = READ_ONCE(dmabuf->memcg);
- unsigned int nr_pages = PAGE_ALIGN(dmabuf->size) >> PAGE_SHIFT;
This way we can also document in the DMA-buf caller that this conversation must be ok when memcg_account is true.
Regards, Christian.
- if (mem_cgroup_disabled())
return true;- if (!mem_cgroup_reserve_dmabuf(new_memcg, GFP_KERNEL, nr_pages))
return false;- mod_memcg_state(new_memcg, MEMCG_DMABUF, nr_pages);
- if (cmpxchg(&dmabuf->memcg, old_memcg, new_memcg) != old_memcg) {
mod_memcg_state(new_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(new_memcg, nr_pages);return false;- }
- if (old_memcg) {
/* Uncharge old cgroup */mod_memcg_state(old_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(old_memcg, nr_pages);mem_cgroup_put(old_memcg);- }
- return true;
+}
/**
- mem_cgroup_uncharge_dmabuf - uncharge a dma-buf from its memcg
- @dmabuf: buffer to uncharge
On Thu, Oct 29, 2026 at 03:16:12PM +0100, Christian König wrote:
On 10/28/26 17:09, Albert Esteve wrote:
On Wed, Oct 28, 2026 at 10:08:19AM +0100, Christian König wrote:
On 10/27/26 16:02, Albert Esteve wrote:
dma-heap already supports charging a specific cgroup at allocation time via charge_fd in DMA_HEAP_IOCTL_ALLOC. However, dma-bufs can originate from any exporter (DRM, V4L2, etc.), and the allocator may not know the final owner at allocation time.
Add DMA_BUF_IOCTL_XFER_CHARGE to transfer a dma-buf's memcg charge to a different cgroup after allocation. The target is identified by charge_fd, which may be a cgroupfd or a pidfd.
The transfer charges the new cgroup before uncharging the old one, preserving the original charge on failure. An atomic cmpxchg on dmabuf->memcg prevents races with concurrent transfers.
Not all dma-buf exporters back their buffers with reclaimable system memory. Some represent device registers, VRAM, or carved-out pools where size is not a meaningful memory cost. Add a memcg_account flag to dma_buf_export_info that exporters set when their buffers should participate in memcg accounting. The xfer ioctl returns -EOPNOTSUPP for buffers that do not set the flag. Currently only the system heap sets it, other exporters can opt in as needed.
Usage example: int cg_fd = open("/sys/fs/cgroup/clients/app1", O_RDONLY | O_DIRECTORY);
struct dma_buf_transfer_charge xfer = { .charge_fd = cg_fd, }; ioctl(dmabuf_fd, DMA_BUF_IOCTL_XFER_CHARGE, &xfer); /* charge now on app1's cgroup */ close(cg_fd);
Signed-off-by: Albert Esteve aesteve@redhat.com
drivers/dma-buf/dma-buf.c | 43 +++++++++++++++++++++++++++++++++++++ drivers/dma-buf/heaps/system_heap.c | 1 + include/linux/dma-buf.h | 5 +++++ include/linux/memcontrol.h | 9 ++++++++ include/uapi/linux/dma-buf.h | 26 ++++++++++++++++++++++ mm/memcontrol.c | 41 +++++++++++++++++++++++++++++++++++ 6 files changed, 125 insertions(+)
diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c index f31f49ebd0af..07513ecedc10 100644 --- a/drivers/dma-buf/dma-buf.c +++ b/drivers/dma-buf/dma-buf.c @@ -540,6 +540,45 @@ static long dma_buf_import_sync_file(struct dma_buf *dmabuf, } #endif +/* Transfer a dma-buf's memcg charge; see struct dma_buf_transfer_charge */ +static long dma_buf_xfer_charge(struct dma_buf *dmabuf,
const void __user *user_data)+{
- struct dma_buf_transfer_charge arg;
- struct mem_cgroup *memcg;
- int ret = 0;
- if (copy_from_user(&arg, user_data, sizeof(arg)))
return -EFAULT;- if (!IS_ENABLED(CONFIG_MEMCG))
return 0;- if (arg.flags != 0)
return -EINVAL;- if (!dmabuf->memcg_account)
return -EOPNOTSUPP;- memcg = mem_cgroup_get_from_fd(arg.charge_fd);
- if (IS_ERR(memcg))
return PTR_ERR(memcg);- if (dmabuf->memcg == memcg)
goto skip_transfer;- if (!mem_cgroup_transfer_dmabuf_charge(dmabuf, memcg)) {
ret = -ENOMEM;goto skip_transfer;- }
- return 0;
+skip_transfer:
- mem_cgroup_put(memcg);
- return ret;
+}
static long dma_buf_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { @@ -590,6 +629,9 @@ static long dma_buf_ioctl(struct file *file, return dma_buf_import_sync_file(dmabuf, (const void __user *)arg); #endif
- case DMA_BUF_IOCTL_XFER_CHARGE:
return dma_buf_xfer_charge(dmabuf, (const void __user *)arg);- default: return -ENOTTY; }
@@ -750,6 +792,7 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info) dmabuf->size = exp_info->size; dmabuf->exp_name = exp_info->exp_name; dmabuf->owner = exp_info->owner;
- dmabuf->memcg_account = exp_info->memcg_account;
The initial charging should probably be done here as well.
This way DMA-buf heaps only needs to pass in the charge_fd and we can re-use the functionality for other DMA-buf exporters as well.
The problem with doing the charge here is that it happens too late. That is the reasoning for this proposal to move all the charging logic into heap allocators before pages are allocated.
Oh, that is a really good point we need to talk about.
DMA-buf heaps is actually a rather simple use case, but we have way more complex things.
For example in some use cases the backing store of a DMA-buf is only allocated on first use and only then it needs to be comitted to it's memcg or dmem.
Yes. The idea is focus on heaps for now. But the uAPI is actually agnostic of the time when the charge happens. When we deal with lazy-allocation in the future we can just call the reserve/cancel helpers wherever is more appropiate. As long as the pages are not allocated, exporters do not set dmabuf->memcg, and XFER does not work (returns -EOPNOTSUPP).
I don't mind implementing it for DMA-buf heaps first, but we should have the uAPI and it's handling so solid that it also works for future use cases.
Agreed. I think we should cover all usecases with the current status. Your feedback helped a lot in getting there!
What we could do is fill the dmabuf->memcg field when the flag is set (i.e., committing into the dmabuf in the patch semantics). But for that we'd need adding the memcg we want to charge it to into exp_info in order to support charging to cgroups other than the allocator's (for the common case would be simpler as we can just look at current). Maybe that's what you meant?
At least for the DMA-buf heaps use case that sounds reasonable for now.
Thanks! Next version will have the flag removed and just use dmabuf->memcg as indicator, set within dma_buf_export(). Also, `nr_pages` will be computed within dma_buf_export(), the contract being that exporters setting exp_info.memcg assert that size reflects actual system memory bytes.
Regards, Christian.
spin_lock_init(&dmabuf->name_lock); init_waitqueue_head(&dmabuf->poll); dmabuf->cb_in.poll = dmabuf->cb_out.poll = &dmabuf->poll; diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c index adeea8cd328b..a764167167ba 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -479,6 +479,7 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, exp_info.size = buffer->len; exp_info.flags = fd_flags; exp_info.priv = buffer;
- exp_info.memcg_account = true; dmabuf = dma_buf_export(&exp_info); if (IS_ERR(dmabuf)) { ret = PTR_ERR(dmabuf);
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index 23ec0ff23fba..ac0ec88cab93 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -433,6 +433,9 @@ struct dma_buf { /** @memcg: the cgroup to which this buffer is currently attributed */ struct mem_cgroup *memcg;
- /** @memcg_account: account buffer memory to cgroup */
- bool memcg_account;
}; /** @@ -505,6 +508,7 @@ struct dma_buf_attachment {
- @flags: mode flags for the file
- @resv: reservation-object, NULL to allocate default one
- @priv: Attach private data of allocator to this buffer
- @memcg_account: account buffer memory to cgroup
- This structure holds the information required to export the buffer. Used
- with dma_buf_export() only.
@@ -517,6 +521,7 @@ struct dma_buf_export_info { int flags; struct dma_resv *resv; void *priv;
- bool memcg_account;
}; /** diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 6b30baa79fa9..36fddafbef4e 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -679,6 +679,9 @@ bool mem_cgroup_reserve_dmabuf(struct mem_cgroup *memcg, gfp_t gfp, void mem_cgroup_commit_dmabuf(struct dma_buf *dmabuf, struct mem_cgroup *memcg, unsigned int nr_pages); +bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg);void __mem_cgroup_uncharge(struct folio *folio); /** @@ -1206,6 +1209,12 @@ static inline void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, { } +static inline bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)+{
- return true;
+}
static inline void mem_cgroup_uncharge(struct folio *folio) { } diff --git a/include/uapi/linux/dma-buf.h b/include/uapi/linux/dma-buf.h index e827c9d20c5d..ea957f4b1d38 100644 --- a/include/uapi/linux/dma-buf.h +++ b/include/uapi/linux/dma-buf.h @@ -168,6 +168,31 @@ struct dma_buf_import_sync_file { __s32 fd; }; +/**
- struct dma_buf_transfer_charge - Transfer a dma-buf's memcg charge
- Any holder of a dma-buf file descriptor can perform a
- DMA_BUF_IOCTL_XFER_CHARGE to attribute the buffer's memory
- cost to a specific cgroup, or move it from one cgroup to
- another. The ioctl charges the new cgroup before uncharging
- the old one atomically. On failure the original charge is
- preserved. If the buffer has no existing charge, a new
- charge is established.
- */
+struct dma_buf_transfer_charge {
- /** @flags: Reserved, must be zero */
- __u32 flags; /* unused for now */
- /**
* @charge_fd: File descriptor identifying the target cgroup.** May be either a pidfd, in which case the charge moves to the* memory cgroup of the referenced process, or a cgroupfd obtained* by opening a cgroup v2 directory, in which case the charge moves* directly to that cgroup's memory controller.*/- __u32 charge_fd;
+};
#define DMA_BUF_BASE 'b' #define DMA_BUF_IOCTL_SYNC _IOW(DMA_BUF_BASE, 0, struct dma_buf_sync) @@ -179,5 +204,6 @@ struct dma_buf_import_sync_file { #define DMA_BUF_SET_NAME_B _IOW(DMA_BUF_BASE, 1, __u64) #define DMA_BUF_IOCTL_EXPORT_SYNC_FILE _IOWR(DMA_BUF_BASE, 2, struct dma_buf_export_sync_file) #define DMA_BUF_IOCTL_IMPORT_SYNC_FILE _IOW(DMA_BUF_BASE, 3, struct dma_buf_import_sync_file) +#define DMA_BUF_IOCTL_XFER_CHARGE _IOW(DMA_BUF_BASE, 4, struct dma_buf_transfer_charge) #endif diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 7b841b905759..3adc84fe6668 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5786,6 +5786,47 @@ void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages) refill_stock(memcg, nr_pages); } +/**
- mem_cgroup_transfer_dmabuf_charge - move a dma-buf charge
- @dmabuf: buffer to transfer charge
- @new_memcg: target memcg; caller must hold a reference
- Transfers the charge for @dmabuf to @new_memcg and, on
- success, stores @new_memcg in @dmabuf->memcg.
- Returns %true on success, %false if the charge against @new_memcg
- failed or a concurrent transfer raced on @dmabuf->memcg.
- On failure the caller retains its reference to @new_memcg.
- */
+bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)I would make that more generic instead of dmabuf specific. E.g. something like this:
bool mem_cgroup_transfer_charge(struct struct mem_cgroup **ptr_memcg, struct mem_cgroup *new_memcg, enum memcg_stat_item item, unsigned int nr_pages)
Ah yes, I like this generalization. And the implementation remains mostly the same. Thanks!
+{
- struct mem_cgroup *old_memcg = READ_ONCE(dmabuf->memcg);
- unsigned int nr_pages = PAGE_ALIGN(dmabuf->size) >> PAGE_SHIFT;
This way we can also document in the DMA-buf caller that this conversation must be ok when memcg_account is true.
Regards, Christian.
- if (mem_cgroup_disabled())
return true;- if (!mem_cgroup_reserve_dmabuf(new_memcg, GFP_KERNEL, nr_pages))
return false;- mod_memcg_state(new_memcg, MEMCG_DMABUF, nr_pages);
- if (cmpxchg(&dmabuf->memcg, old_memcg, new_memcg) != old_memcg) {
mod_memcg_state(new_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(new_memcg, nr_pages);return false;- }
- if (old_memcg) {
/* Uncharge old cgroup */mod_memcg_state(old_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(old_memcg, nr_pages);mem_cgroup_put(old_memcg);- }
- return true;
+}
/**
- mem_cgroup_uncharge_dmabuf - uncharge a dma-buf from its memcg
- @dmabuf: buffer to uncharge
On 10/29/26 17:23, Albert Esteve wrote:
On Thu, Oct 29, 2026 at 03:16:12PM +0100, Christian König wrote:
On 10/28/26 17:09, Albert Esteve wrote:
On Wed, Oct 28, 2026 at 10:08:19AM +0100, Christian König wrote:
On 10/27/26 16:02, Albert Esteve wrote:
dma-heap already supports charging a specific cgroup at allocation time via charge_fd in DMA_HEAP_IOCTL_ALLOC. However, dma-bufs can originate from any exporter (DRM, V4L2, etc.), and the allocator may not know the final owner at allocation time.
Add DMA_BUF_IOCTL_XFER_CHARGE to transfer a dma-buf's memcg charge to a different cgroup after allocation. The target is identified by charge_fd, which may be a cgroupfd or a pidfd.
The transfer charges the new cgroup before uncharging the old one, preserving the original charge on failure. An atomic cmpxchg on dmabuf->memcg prevents races with concurrent transfers.
Not all dma-buf exporters back their buffers with reclaimable system memory. Some represent device registers, VRAM, or carved-out pools where size is not a meaningful memory cost. Add a memcg_account flag to dma_buf_export_info that exporters set when their buffers should participate in memcg accounting. The xfer ioctl returns -EOPNOTSUPP for buffers that do not set the flag. Currently only the system heap sets it, other exporters can opt in as needed.
Usage example: int cg_fd = open("/sys/fs/cgroup/clients/app1", O_RDONLY | O_DIRECTORY);
struct dma_buf_transfer_charge xfer = { .charge_fd = cg_fd, }; ioctl(dmabuf_fd, DMA_BUF_IOCTL_XFER_CHARGE, &xfer); /* charge now on app1's cgroup */ close(cg_fd);
Signed-off-by: Albert Esteve aesteve@redhat.com
drivers/dma-buf/dma-buf.c | 43 +++++++++++++++++++++++++++++++++++++ drivers/dma-buf/heaps/system_heap.c | 1 + include/linux/dma-buf.h | 5 +++++ include/linux/memcontrol.h | 9 ++++++++ include/uapi/linux/dma-buf.h | 26 ++++++++++++++++++++++ mm/memcontrol.c | 41 +++++++++++++++++++++++++++++++++++ 6 files changed, 125 insertions(+)
diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c index f31f49ebd0af..07513ecedc10 100644 --- a/drivers/dma-buf/dma-buf.c +++ b/drivers/dma-buf/dma-buf.c @@ -540,6 +540,45 @@ static long dma_buf_import_sync_file(struct dma_buf *dmabuf, } #endif +/* Transfer a dma-buf's memcg charge; see struct dma_buf_transfer_charge */ +static long dma_buf_xfer_charge(struct dma_buf *dmabuf,
const void __user *user_data)+{
- struct dma_buf_transfer_charge arg;
- struct mem_cgroup *memcg;
- int ret = 0;
- if (copy_from_user(&arg, user_data, sizeof(arg)))
return -EFAULT;- if (!IS_ENABLED(CONFIG_MEMCG))
return 0;- if (arg.flags != 0)
return -EINVAL;- if (!dmabuf->memcg_account)
return -EOPNOTSUPP;- memcg = mem_cgroup_get_from_fd(arg.charge_fd);
- if (IS_ERR(memcg))
return PTR_ERR(memcg);- if (dmabuf->memcg == memcg)
goto skip_transfer;- if (!mem_cgroup_transfer_dmabuf_charge(dmabuf, memcg)) {
ret = -ENOMEM;goto skip_transfer;- }
- return 0;
+skip_transfer:
- mem_cgroup_put(memcg);
- return ret;
+}
static long dma_buf_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { @@ -590,6 +629,9 @@ static long dma_buf_ioctl(struct file *file, return dma_buf_import_sync_file(dmabuf, (const void __user *)arg); #endif
- case DMA_BUF_IOCTL_XFER_CHARGE:
return dma_buf_xfer_charge(dmabuf, (const void __user *)arg);- default: return -ENOTTY; }
@@ -750,6 +792,7 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info) dmabuf->size = exp_info->size; dmabuf->exp_name = exp_info->exp_name; dmabuf->owner = exp_info->owner;
- dmabuf->memcg_account = exp_info->memcg_account;
The initial charging should probably be done here as well.
This way DMA-buf heaps only needs to pass in the charge_fd and we can re-use the functionality for other DMA-buf exporters as well.
The problem with doing the charge here is that it happens too late. That is the reasoning for this proposal to move all the charging logic into heap allocators before pages are allocated.
Oh, that is a really good point we need to talk about.
DMA-buf heaps is actually a rather simple use case, but we have way more complex things.
For example in some use cases the backing store of a DMA-buf is only allocated on first use and only then it needs to be comitted to it's memcg or dmem.
Yes. The idea is focus on heaps for now. But the uAPI is actually agnostic of the time when the charge happens. When we deal with lazy-allocation in the future we can just call the reserve/cancel helpers wherever is more appropiate. As long as the pages are not allocated, exporters do not set dmabuf->memcg, and XFER does not work (returns -EOPNOTSUPP).
Yeah exactly that's the reason why I'm raising this topic. That approach won't work in all cases.
See userspace is not aware that the kernel is delaying allocation or that the kernel is allocating the backing store asynchroniously in a background worker for example.
So returning -EOPNOTSUPP when the backing store is not allocated yet and might be allocated later on is rather questionable uAPI because that is racy by design.
I don't mind implementing it for DMA-buf heaps first, but we should have the uAPI and it's handling so solid that it also works for future use cases.
Agreed. I think we should cover all usecases with the current status. Your feedback helped a lot in getting there!
I mean we can postpone that until we finally need it, but essentially we need to be able to distinct between charging, e.g. making sure the cgroup has enough free limit, and committing, e.g. allowing the resource to be reclaimed.
And then it is possible that when the memcg of a DMA-buf changes that only the charging changes, but there isn't any comitted backing store yet...
What we could do is fill the dmabuf->memcg field when the flag is set (i.e., committing into the dmabuf in the patch semantics). But for that we'd need adding the memcg we want to charge it to into exp_info in order to support charging to cgroups other than the allocator's (for the common case would be simpler as we can just look at current). Maybe that's what you meant?
At least for the DMA-buf heaps use case that sounds reasonable for now.
Thanks! Next version will have the flag removed and just use dmabuf->memcg as indicator, set within dma_buf_export(). Also, `nr_pages` will be computed within dma_buf_export(), the contract being that exporters setting exp_info.memcg assert that size reflects actual system memory bytes.
Thinking more about it I would also put changing dmabuf->memcg under the dmabuf->resv lock instead of doing it with cmpxchg.
We don't need it currently, but the dmabuf->resv lock is used for all state changes of the DMA-buf object and this operation is not performance critical in any way possible.
If there is a problem with locking order we at least get lockdep screaming at us very early and not when we want to extend the functionality in the future.
Regards, Christian.
Regards, Christian.
spin_lock_init(&dmabuf->name_lock); init_waitqueue_head(&dmabuf->poll); dmabuf->cb_in.poll = dmabuf->cb_out.poll = &dmabuf->poll; diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c index adeea8cd328b..a764167167ba 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -479,6 +479,7 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, exp_info.size = buffer->len; exp_info.flags = fd_flags; exp_info.priv = buffer;
- exp_info.memcg_account = true; dmabuf = dma_buf_export(&exp_info); if (IS_ERR(dmabuf)) { ret = PTR_ERR(dmabuf);
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index 23ec0ff23fba..ac0ec88cab93 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -433,6 +433,9 @@ struct dma_buf { /** @memcg: the cgroup to which this buffer is currently attributed */ struct mem_cgroup *memcg;
- /** @memcg_account: account buffer memory to cgroup */
- bool memcg_account;
}; /** @@ -505,6 +508,7 @@ struct dma_buf_attachment {
- @flags: mode flags for the file
- @resv: reservation-object, NULL to allocate default one
- @priv: Attach private data of allocator to this buffer
- @memcg_account: account buffer memory to cgroup
- This structure holds the information required to export the buffer. Used
- with dma_buf_export() only.
@@ -517,6 +521,7 @@ struct dma_buf_export_info { int flags; struct dma_resv *resv; void *priv;
- bool memcg_account;
}; /** diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 6b30baa79fa9..36fddafbef4e 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -679,6 +679,9 @@ bool mem_cgroup_reserve_dmabuf(struct mem_cgroup *memcg, gfp_t gfp, void mem_cgroup_commit_dmabuf(struct dma_buf *dmabuf, struct mem_cgroup *memcg, unsigned int nr_pages); +bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg);void __mem_cgroup_uncharge(struct folio *folio); /** @@ -1206,6 +1209,12 @@ static inline void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, { } +static inline bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)+{
- return true;
+}
static inline void mem_cgroup_uncharge(struct folio *folio) { } diff --git a/include/uapi/linux/dma-buf.h b/include/uapi/linux/dma-buf.h index e827c9d20c5d..ea957f4b1d38 100644 --- a/include/uapi/linux/dma-buf.h +++ b/include/uapi/linux/dma-buf.h @@ -168,6 +168,31 @@ struct dma_buf_import_sync_file { __s32 fd; }; +/**
- struct dma_buf_transfer_charge - Transfer a dma-buf's memcg charge
- Any holder of a dma-buf file descriptor can perform a
- DMA_BUF_IOCTL_XFER_CHARGE to attribute the buffer's memory
- cost to a specific cgroup, or move it from one cgroup to
- another. The ioctl charges the new cgroup before uncharging
- the old one atomically. On failure the original charge is
- preserved. If the buffer has no existing charge, a new
- charge is established.
- */
+struct dma_buf_transfer_charge {
- /** @flags: Reserved, must be zero */
- __u32 flags; /* unused for now */
- /**
* @charge_fd: File descriptor identifying the target cgroup.** May be either a pidfd, in which case the charge moves to the* memory cgroup of the referenced process, or a cgroupfd obtained* by opening a cgroup v2 directory, in which case the charge moves* directly to that cgroup's memory controller.*/- __u32 charge_fd;
+};
#define DMA_BUF_BASE 'b' #define DMA_BUF_IOCTL_SYNC _IOW(DMA_BUF_BASE, 0, struct dma_buf_sync) @@ -179,5 +204,6 @@ struct dma_buf_import_sync_file { #define DMA_BUF_SET_NAME_B _IOW(DMA_BUF_BASE, 1, __u64) #define DMA_BUF_IOCTL_EXPORT_SYNC_FILE _IOWR(DMA_BUF_BASE, 2, struct dma_buf_export_sync_file) #define DMA_BUF_IOCTL_IMPORT_SYNC_FILE _IOW(DMA_BUF_BASE, 3, struct dma_buf_import_sync_file) +#define DMA_BUF_IOCTL_XFER_CHARGE _IOW(DMA_BUF_BASE, 4, struct dma_buf_transfer_charge) #endif diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 7b841b905759..3adc84fe6668 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5786,6 +5786,47 @@ void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages) refill_stock(memcg, nr_pages); } +/**
- mem_cgroup_transfer_dmabuf_charge - move a dma-buf charge
- @dmabuf: buffer to transfer charge
- @new_memcg: target memcg; caller must hold a reference
- Transfers the charge for @dmabuf to @new_memcg and, on
- success, stores @new_memcg in @dmabuf->memcg.
- Returns %true on success, %false if the charge against @new_memcg
- failed or a concurrent transfer raced on @dmabuf->memcg.
- On failure the caller retains its reference to @new_memcg.
- */
+bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)I would make that more generic instead of dmabuf specific. E.g. something like this:
bool mem_cgroup_transfer_charge(struct struct mem_cgroup **ptr_memcg, struct mem_cgroup *new_memcg, enum memcg_stat_item item, unsigned int nr_pages)
Ah yes, I like this generalization. And the implementation remains mostly the same. Thanks!
+{
- struct mem_cgroup *old_memcg = READ_ONCE(dmabuf->memcg);
- unsigned int nr_pages = PAGE_ALIGN(dmabuf->size) >> PAGE_SHIFT;
This way we can also document in the DMA-buf caller that this conversation must be ok when memcg_account is true.
Regards, Christian.
- if (mem_cgroup_disabled())
return true;- if (!mem_cgroup_reserve_dmabuf(new_memcg, GFP_KERNEL, nr_pages))
return false;- mod_memcg_state(new_memcg, MEMCG_DMABUF, nr_pages);
- if (cmpxchg(&dmabuf->memcg, old_memcg, new_memcg) != old_memcg) {
mod_memcg_state(new_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(new_memcg, nr_pages);return false;- }
- if (old_memcg) {
/* Uncharge old cgroup */mod_memcg_state(old_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(old_memcg, nr_pages);mem_cgroup_put(old_memcg);- }
- return true;
+}
/**
- mem_cgroup_uncharge_dmabuf - uncharge a dma-buf from its memcg
- @dmabuf: buffer to uncharge
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider: - [High] dma-buf: unprivileged cgroup escape and OOM via dma_buf_xfer_charge() - [High] dma-buf: race in mem_cgroup_transfer_dmabuf_charge() double-charges memcg - [Medium] dma-buf: broken disabled-memcg handling in dma_buf_xfer_charge() --
commit 1660729e62b15fd915c69f26225530bd5ae30ce0 Author: Albert Esteve aesteve@redhat.com
dma-buf: transfer memcg charges via DMA_BUF_IOCTL_XFER_CHARGE
This commit adds DMA_BUF_IOCTL_XFER_CHARGE to allow transferring a dma-buf's memory cgroup charge to a cgroup identified by a cgroupfd or pidfd after allocation. It also introduces a memcg_account flag in dma_buf_export_info for exporters to opt in to memcg accounting, enabling it for the system heap.
diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c index f31f49ebd0af3..07513ecedc105 100644 --- a/drivers/dma-buf/dma-buf.c +++ b/drivers/dma-buf/dma-buf.c @@ -540,6 +540,45 @@ static long dma_buf_import_sync_file(struct dma_buf *dmabuf, } #endif +/* Transfer a dma-buf's memcg charge; see struct dma_buf_transfer_charge */ +static long dma_buf_xfer_charge(struct dma_buf *dmabuf,
const void __user *user_data)+{
- struct dma_buf_transfer_charge arg;
- struct mem_cgroup *memcg;
- int ret = 0;
- if (copy_from_user(&arg, user_data, sizeof(arg)))
return -EFAULT;- if (!IS_ENABLED(CONFIG_MEMCG))
return 0;- if (arg.flags != 0)
return -EINVAL;- if (!dmabuf->memcg_account)
return -EOPNOTSUPP;
[Severity: Medium] When CONFIG_MEMCG is disabled, should dma_buf_xfer_charge() still validate arg.flags and dmabuf->memcg_account, or return -EOPNOTSUPP?
Returning 0 before checking arg.flags != 0 and !dmabuf->memcg_account causes calls with invalid flags, non-accounting exporters, or invalid file descriptors to silently succeed under CONFIG_MEMCG=n, while also leaving the !CONFIG_MEMCG stubs for mem_cgroup_get_from_fd() (which returns ERR_PTR(-EOPNOTSUPP)) and mem_cgroup_transfer_dmabuf_charge() in include/linux/memcontrol.h unreachable.
Also, when CONFIG_MEMCG=y and cgroup_disable=memory is passed on the kernel command line (so mem_cgroup_disabled() is true at runtime), mem_cgroup_get_from_fd() behaves inconsistently across its two branches:
mm/memcontrol.c:mem_cgroup_get_from_fd() { cgrp = cgroup_get_from_fd(fd); if (!IS_ERR(cgrp)) { rcu_read_lock(); css = cgroup_css(cgrp, &memory_cgrp_subsys); if (!css || !css_tryget_online(css)) { rcu_read_unlock(); cgroup_put(cgrp); return ERR_PTR(-ENOENT); } ... memcg = get_mem_cgroup_from_mm(mm); mmput(mm); return memcg; }
For a cgroupfd, cgroup_css() is NULL and it returns ERR_PTR(-ENOENT), whereas for a pidfd, get_mem_cgroup_from_mm() returns NULL rather than an ERR_PTR, so dma_buf_xfer_charge() returns 0 for pidfds and -ENOENT for cgroupfds.
- memcg = mem_cgroup_get_from_fd(arg.charge_fd);
- if (IS_ERR(memcg))
return PTR_ERR(memcg);- if (dmabuf->memcg == memcg)
goto skip_transfer;- if (!mem_cgroup_transfer_dmabuf_charge(dmabuf, memcg)) {
ret = -ENOMEM;goto skip_transfer;- }
[Severity: High] Can an unprivileged holder of a dma-buf file descriptor use dma_buf_xfer_charge() to escape its own memory cgroup limit or trigger OOM kills in another cgroup?
In the call chain:
dma_buf_ioctl() -> dma_buf_xfer_charge() -> mem_cgroup_get_from_fd() -> mem_cgroup_transfer_dmabuf_charge() -> mem_cgroup_reserve_dmabuf(new_memcg, GFP_KERNEL, nr_pages) -> try_charge()
neither dma_buf_xfer_charge() nor mem_cgroup_get_from_fd() checks DAC or cgroup delegation/write permissions (such as inode_permission() with MAY_WRITE or cgroup_procs_write_permission()), capabilities, or ptrace_may_access() on either the source cgroup (dmabuf->memcg) being uncharged or the target cgroup/task (new_memcg) being charged.
Because /sys/fs/cgroup directories are normally world-readable (mode 0755, so any user can open them with O_RDONLY | O_DIRECTORY) and pidfd_open() can open a pidfd for PID 1 or any visible process without privilege, an unprivileged process can resolve a handle to the root cgroup or to another cgroup on the system.
When SELinux is not active, security_dma_buf_charge_cgroup() and security_dma_buf_charge_task() return 0. Even with SELinux enabled, selinux_dma_buf_charge_cgroup() returns 0 when the cgroup kernfs node has no explicit security.selinux xattr set:
security/selinux/hooks.c:selinux_dma_buf_charge_cgroup() { rc = kernfs_xattr_get(cgrp->kn, XATTR_NAME_SELINUX, NULL, 0); if (rc == -ENODATA) return 0; }
and neither hook checks permission to uncharge the existing dmabuf->memcg. As a result, a confined client receiving a pre-charged dma-buf fd can move the charge to the root cgroup to bypass its own memory.max limit, or move the charge into a victim cgroup where mem_cgroup_reserve_dmabuf() with GFP_KERNEL triggers direct reclaim and the memcg OOM killer.
diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 7b841b9057594..3adc84fe6668d 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5786,6 +5786,47 @@ void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages) refill_stock(memcg, nr_pages); } +/**
- mem_cgroup_transfer_dmabuf_charge - move a dma-buf charge
- @dmabuf: buffer to transfer charge
- @new_memcg: target memcg; caller must hold a reference
- Transfers the charge for @dmabuf to @new_memcg and, on
- success, stores @new_memcg in @dmabuf->memcg.
- Returns %true on success, %false if the charge against @new_memcg
- failed or a concurrent transfer raced on @dmabuf->memcg.
- On failure the caller retains its reference to @new_memcg.
- */
+bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)+{
- struct mem_cgroup *old_memcg = READ_ONCE(dmabuf->memcg);
- unsigned int nr_pages = PAGE_ALIGN(dmabuf->size) >> PAGE_SHIFT;
- if (mem_cgroup_disabled())
return true;- if (!mem_cgroup_reserve_dmabuf(new_memcg, GFP_KERNEL, nr_pages))
return false;- mod_memcg_state(new_memcg, MEMCG_DMABUF, nr_pages);
- if (cmpxchg(&dmabuf->memcg, old_memcg, new_memcg) != old_memcg) {
mod_memcg_state(new_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(new_memcg, nr_pages);return false;- }
[Severity: High] What happens if two threads concurrently call DMA_BUF_IOCTL_XFER_CHARGE on the same dma-buf?
In dma_buf_xfer_charge(), dmabuf->memcg == memcg is checked without synchronization before calling mem_cgroup_transfer_dmabuf_charge(), which then re-reads old_memcg = READ_ONCE(dmabuf->memcg) without checking whether old_memcg == new_memcg.
If thread 1 transfers the buffer to new_memcg after thread 2 passes the dmabuf->memcg == memcg check in dma_buf_xfer_charge() (or if both threads concurrently transfer the buffer to the same new_memcg), thread 2 sees old_memcg == new_memcg and still calls mem_cgroup_reserve_dmabuf(new_memcg, GFP_KERNEL, nr_pages) before uncharging old_memcg. That transiently charges 2 * nr_pages against new_memcg for the same buffer, which can trigger reclaim, memcg OOM kills, or -ENOMEM in new_memcg even though the buffer is already charged to it.
In addition, old_memcg is read before mem_cgroup_reserve_dmabuf(new_memcg, GFP_KERNEL, nr_pages), which can sleep in direct reclaim. If another thread updates dmabuf->memcg while mem_cgroup_reserve_dmabuf() is sleeping, the cmpxchg(&dmabuf->memcg, old_memcg, new_memcg) fails without retrying against the updated old_memcg and returns false, which dma_buf_xfer_charge() reports to userspace as -ENOMEM even though charging new_memcg succeeded.
Finally, if mem_cgroup_transfer_dmabuf_charge() is called when mem_cgroup_disabled() is true, returning true at the top without storing new_memcg in dmabuf->memcg or dropping new_memcg's reference would leak the caller's reference to new_memcg, since callers only call mem_cgroup_put() when this function returns false.
Add tests for the new charge_fd field in struct dma_heap_allocation_data.
When the charge_fd feature is absent (unpatched kernel), the probe in pidfd_alloc_supported() detects this and the tests are skipped gracefully.
Add vmtest.sh similar to other subsystem suites, to orchestrate building the selftests (optionally with a freshly compiled kernel) inside a virtme-ng VM, so the tests can be run without modifying the host system. Add a config fragment with required Kconfig symbols.
Also add test_memcg_dmabuf() to the existing test_memcontrol suite to verify end-to-end cross-cgroup accounting: a parent process opens a pidfd for a child in a separate cgroup, allocates a dma-buf via DMA_HEAP_IOCTL_ALLOC with that pidfd, and asserts that memory.stat dmabuf in the child's cgroup reflects the allocation. If the dmabuf key is missing (unpatched kernel) or /dev/dma_heap/system is absent, the test is skipped.
Assisted-by: LLM Signed-off-by: Albert Esteve aesteve@redhat.com --- tools/testing/selftests/cgroup/Makefile | 2 +- tools/testing/selftests/cgroup/test_memcontrol.c | 290 ++++++++++++++++++++- tools/testing/selftests/dmabuf-heaps/config | 1 + tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c | 227 +++++++++++++++- tools/testing/selftests/dmabuf-heaps/vmtest.sh | 226 ++++++++++++++++ 5 files changed, 742 insertions(+), 4 deletions(-)
diff --git a/tools/testing/selftests/cgroup/Makefile b/tools/testing/selftests/cgroup/Makefile index e01584c2189a..9edfc9f1de5c 100644 --- a/tools/testing/selftests/cgroup/Makefile +++ b/tools/testing/selftests/cgroup/Makefile @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -CFLAGS += -Wall -pthread +CFLAGS += -Wall -pthread $(KHDR_INCLUDES)
all: ${HELPER_PROGS}
diff --git a/tools/testing/selftests/cgroup/test_memcontrol.c b/tools/testing/selftests/cgroup/test_memcontrol.c index 0ed82347044e..0b41c3c6f62c 100644 --- a/tools/testing/selftests/cgroup/test_memcontrol.c +++ b/tools/testing/selftests/cgroup/test_memcontrol.c @@ -19,10 +19,20 @@ #include <errno.h> #include <sys/mman.h>
+#include <linux/dma-buf.h> +#include <linux/dma-heap.h> +#include <signal.h> +#include <sys/ioctl.h> + +#include "../pidfd/pidfd.h" #include "kselftest.h" #include "cgroup_util.h"
+#define DMA_HEAP_SYSTEM "/dev/dma_heap/system" +#define ONE_MEG (1024 * 1024) + #define MEMCG_SOCKSTAT_WAIT_RETRIES 30 +#define MEMCG_DMABUF_WAIT_RETRIES 15
static bool has_localevents; static bool has_recursiveprot; @@ -1764,6 +1774,267 @@ static int test_memcg_inotify_delete_dir(const char *root) return ret; }
+static int memcg_dmabuf_child(const char *cgroup, void *arg) +{ + pause(); + return 0; +} + +/* + * This test allocates a dma-buf via DMA_HEAP_IOCTL_ALLOC with a pidfd + * pointing to a child process in a separate cgroup, then checks that + * memory.stat[dmabuf] in the child's cgroup rises by the allocation size + * and returns to zero after the buffer fd is closed. + */ +static int test_memcg_dmabuf(const char *root) +{ + char *parent = NULL, *child_cg = NULL; + int ret = KSFT_FAIL; + int heap_fd = -1, dmabuf_fd = -1, pidfd = -1; + pid_t child_pid; + int child_status; + long dmabuf_stat; + struct dma_heap_allocation_data alloc = { + .len = ONE_MEG, + .fd_flags = O_RDWR | O_CLOEXEC, + }; + + if (access(DMA_HEAP_SYSTEM, R_OK | W_OK)) { + ret = KSFT_SKIP; + goto cleanup; + } + + parent = cg_name(root, "dmabuf_memcg_test"); + if (!parent) + goto cleanup; + + if (cg_create(parent)) + goto cleanup_parent; + + if (cg_write(parent, "cgroup.subtree_control", "+memory")) + goto cleanup_parent; + + child_cg = cg_name(parent, "child"); + if (!child_cg) + goto cleanup_parent; + + if (cg_create(child_cg)) + goto cleanup_parent; + + child_pid = cg_run_nowait(child_cg, memcg_dmabuf_child, NULL); + if (child_pid < 0) + goto cleanup_child; + + if (cg_wait_for_proc_count(child_cg, 1)) + goto cleanup_kill; + + pidfd = sys_pidfd_open(child_pid, 0); + if (pidfd < 0) { + ret = KSFT_SKIP; + goto cleanup_kill; + } + + heap_fd = open(DMA_HEAP_SYSTEM, O_RDWR); + if (heap_fd < 0) { + ret = KSFT_SKIP; + goto cleanup_pidfd; + } + + alloc.charge_fd = (__u32)pidfd; + if (ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &alloc) < 0) + goto cleanup_heap; + dmabuf_fd = (int)alloc.fd; + + dmabuf_stat = cg_read_key_long(child_cg, "memory.stat", "dmabuf "); + if (dmabuf_stat == -1) { + ret = KSFT_SKIP; + goto cleanup_dmabuf; + } + if (dmabuf_stat != ONE_MEG) + dmabuf_stat = cg_read_key_long_poll(child_cg, "memory.stat", + "dmabuf ", ONE_MEG, + MEMCG_DMABUF_WAIT_RETRIES, + DEFAULT_WAIT_INTERVAL_US); + if (dmabuf_stat != ONE_MEG) { + fprintf(stderr, "Expected dmabuf stat %d, got %ld\n", + ONE_MEG, dmabuf_stat); + goto cleanup_dmabuf; + } + + close(dmabuf_fd); + dmabuf_fd = -1; + + dmabuf_stat = cg_read_key_long_poll(child_cg, "memory.stat", + "dmabuf ", 0, + MEMCG_DMABUF_WAIT_RETRIES, + DEFAULT_WAIT_INTERVAL_US); + if (dmabuf_stat != 0) { + fprintf(stderr, "Expected dmabuf stat 0 after close, got %ld\n", + dmabuf_stat); + goto cleanup_heap; + } + + ret = KSFT_PASS; + +cleanup_dmabuf: + if (dmabuf_fd >= 0) + close(dmabuf_fd); +cleanup_heap: + close(heap_fd); +cleanup_pidfd: + close(pidfd); +cleanup_kill: + kill(child_pid, SIGTERM); + waitpid(child_pid, &child_status, 0); +cleanup_child: + cg_destroy(child_cg); + free(child_cg); +cleanup_parent: + cg_destroy(parent); + free(parent); +cleanup: + return ret; +} + +/* + * Allocate a dma-buf in cgroup A, set memory.max on cgroup B below the + * buffer size, then attempt DMA_BUF_IOCTL_XFER_CHARGE to B. The transfer + * must fail with ENOMEM and the charge must remain in A. + */ +static int test_memcg_dmabuf_xfer_at_limit(const char *root) +{ + char *parent = NULL, *cg_a = NULL, *cg_b = NULL; + int ret = KSFT_FAIL; + int heap_fd = -1, dmabuf_fd = -1, cg_b_fd = -1; + pid_t child_pid; + int child_status; + long stat_a; + struct dma_heap_allocation_data alloc = { + .len = ONE_MEG, + .fd_flags = O_RDWR | O_CLOEXEC, + }; + struct dma_buf_transfer_charge xfer = { 0 }; + + if (access(DMA_HEAP_SYSTEM, R_OK | W_OK)) { + ret = KSFT_SKIP; + goto cleanup; + } + + parent = cg_name(root, "dmabuf_xfer_limit_test"); + if (!parent) + goto cleanup; + + if (cg_create(parent)) + goto cleanup_parent; + + if (cg_write(parent, "cgroup.subtree_control", "+memory")) + goto cleanup_parent; + + cg_a = cg_name(parent, "cg_a"); + cg_b = cg_name(parent, "cg_b"); + if (!cg_a || !cg_b) + goto cleanup_cgs; + + if (cg_create(cg_a) || cg_create(cg_b)) + goto cleanup_cgs; + + if (cg_write(cg_b, "memory.max", "524288")) + goto cleanup_cgs; + + child_pid = cg_run_nowait(cg_a, memcg_dmabuf_child, NULL); + if (child_pid < 0) + goto cleanup_cgs; + + if (cg_wait_for_proc_count(cg_a, 1)) + goto cleanup_kill; + + heap_fd = open(DMA_HEAP_SYSTEM, O_RDWR); + if (heap_fd < 0) { + ret = KSFT_SKIP; + goto cleanup_kill; + } + + alloc.charge_fd = (__u32)sys_pidfd_open(child_pid, 0); + if ((int)alloc.charge_fd < 0) { + ret = KSFT_SKIP; + goto cleanup_heap; + } + + if (ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &alloc) < 0) { + close(alloc.charge_fd); + goto cleanup_heap; + } + close(alloc.charge_fd); + dmabuf_fd = (int)alloc.fd; + + stat_a = cg_read_key_long(cg_a, "memory.stat", "dmabuf "); + if (stat_a == -1) { + ret = KSFT_SKIP; + goto cleanup_dmabuf; + } + if (stat_a != ONE_MEG) + stat_a = cg_read_key_long_poll(cg_a, "memory.stat", + "dmabuf ", ONE_MEG, + MEMCG_DMABUF_WAIT_RETRIES, + DEFAULT_WAIT_INTERVAL_US); + if (stat_a != ONE_MEG) { + fprintf(stderr, "Charge not in A after alloc: expected %d, got %ld\n", + ONE_MEG, stat_a); + goto cleanup_dmabuf; + } + + cg_b_fd = open(cg_b, O_RDONLY | O_DIRECTORY); + if (cg_b_fd < 0) + goto cleanup_dmabuf; + + xfer.charge_fd = (__u32)cg_b_fd; + if (ioctl(dmabuf_fd, DMA_BUF_IOCTL_XFER_CHARGE, &xfer) == 0) { + fprintf(stderr, "xfer_charge should have failed at limit\n"); + goto cleanup_cgfd; + } + if (errno != ENOMEM) { + fprintf(stderr, "Expected ENOMEM, got %s\n", strerror(errno)); + goto cleanup_cgfd; + } + + stat_a = cg_read_key_long_poll(cg_a, "memory.stat", + "dmabuf ", ONE_MEG, + MEMCG_DMABUF_WAIT_RETRIES, + DEFAULT_WAIT_INTERVAL_US); + if (stat_a != ONE_MEG) { + fprintf(stderr, "Charge leaked from A: expected %d, got %ld\n", + ONE_MEG, stat_a); + goto cleanup_cgfd; + } + + ret = KSFT_PASS; + +cleanup_cgfd: + close(cg_b_fd); +cleanup_dmabuf: + if (dmabuf_fd >= 0) + close(dmabuf_fd); +cleanup_heap: + close(heap_fd); +cleanup_kill: + kill(child_pid, SIGTERM); + waitpid(child_pid, &child_status, 0); +cleanup_cgs: + if (cg_a) { + cg_destroy(cg_a); + free(cg_a); + } + if (cg_b) { + cg_destroy(cg_b); + free(cg_b); + } +cleanup_parent: + cg_destroy(parent); + free(parent); +cleanup: + return ret; +} + #define T(x) { x, #x } struct memcg_test { int (*fn)(const char *root); @@ -1785,13 +2056,19 @@ struct memcg_test { T(test_memcg_oom_group_score_events), T(test_memcg_inotify_delete_file), T(test_memcg_inotify_delete_dir), + T(test_memcg_dmabuf), + T(test_memcg_dmabuf_xfer_at_limit), }; #undef T
int main(int argc, char **argv) { char root[PATH_MAX]; - int i, proc_status; + int i, proc_status, count = ARRAY_SIZE(tests); + const char *filter = NULL; + + if (argc > 1) + filter = argv[1];
page_size = sysconf(_SC_PAGE_SIZE); if (page_size <= 0) @@ -1822,8 +2099,17 @@ int main(int argc, char **argv) ksft_exit_skip("Failed to query cgroup mount option\n"); has_localevents = proc_status;
- ksft_set_plan(ARRAY_SIZE(tests)); + if (filter) { + count = 0; + for (i = 0; i < ARRAY_SIZE(tests); i++) { + if (!strcmp(tests[i].name, filter)) + count++; + } + } + ksft_set_plan(count); for (i = 0; i < ARRAY_SIZE(tests); i++) { + if (filter && strcmp(tests[i].name, filter)) + continue; switch (tests[i].fn(root)) { case KSFT_PASS: ksft_test_result_pass("%s\n", tests[i].name); diff --git a/tools/testing/selftests/dmabuf-heaps/config b/tools/testing/selftests/dmabuf-heaps/config index be091f1cdfa0..94c8f33b71a2 100644 --- a/tools/testing/selftests/dmabuf-heaps/config +++ b/tools/testing/selftests/dmabuf-heaps/config @@ -1,3 +1,4 @@ +CONFIG_MEMCG=y CONFIG_DMABUF_HEAPS=y CONFIG_DMABUF_HEAPS_SYSTEM=y CONFIG_DRM_VGEM=y diff --git a/tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c b/tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c index 1d49df671919..90ef14d2ecba 100644 --- a/tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c +++ b/tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c @@ -3,6 +3,7 @@ #include <dirent.h> #include <errno.h> #include <fcntl.h> +#include <signal.h> #include <stdio.h> #include <stdlib.h> #include <stdint.h> @@ -10,11 +11,15 @@ #include <unistd.h> #include <sys/ioctl.h> #include <sys/mman.h> +#include <sys/syscall.h> #include <sys/types.h> +#include <sys/wait.h>
#include <linux/dma-buf.h> #include <linux/dma-heap.h> +#include <linux/limits.h> #include <drm/drm.h> +#include "../pidfd/pidfd.h" #include "kselftest.h"
#define DEVPATH "/dev/dma_heap" @@ -320,6 +325,8 @@ static int dmabuf_heap_alloc_newer(int fd, size_t len, unsigned int flags, __u32 fd; __u32 fd_flags; __u64 heap_flags; + __u32 charge_fd; + __u32 __padding; __u64 garbage1; __u64 garbage2; __u64 garbage3; @@ -328,6 +335,8 @@ static int dmabuf_heap_alloc_newer(int fd, size_t len, unsigned int flags, .fd = 0, .fd_flags = O_RDWR | O_CLOEXEC, .heap_flags = flags, + .charge_fd = 0, + .__padding = 0, .garbage1 = 0xffffffff, .garbage2 = 0x88888888, .garbage3 = 0x11111111, @@ -496,6 +505,218 @@ static void test_alloc_no_fd_leak_on_efault(char *heap_name) ksft_test_result(fd_before == fd_after, "fd leak on EFAULT: before=%d after=%d\n", fd_before, fd_after); + +out: + close(heap_fd); +} + +static int dmabuf_heap_alloc_pidfd(int fd, size_t len, unsigned int heap_flags, + unsigned int charge_fd, int *dmabuf_fd) +{ + struct dma_heap_allocation_data data = { + .len = len, + .fd = 0, + .fd_flags = O_RDWR | O_CLOEXEC, + .heap_flags = heap_flags, + .charge_fd = charge_fd, + }; + int ret; + + if (!dmabuf_fd) + return -EINVAL; + + ret = ioctl(fd, DMA_HEAP_IOCTL_ALLOC, &data); + if (ret < 0) + return ret; + *dmabuf_fd = (int)data.fd; + return ret; +} + +/* + * Probe whether the kernel honours charge_fd in DMA_HEAP_IOCTL_ALLOC. + */ +static bool pidfd_alloc_supported(int heap_fd) +{ + int devnull_fd, dmabuf_fd = -1, ret; + + devnull_fd = open("/dev/null", O_RDONLY); + if (devnull_fd < 0) + return false; + + ret = dmabuf_heap_alloc_pidfd(heap_fd, ONE_MEG, 0, devnull_fd, &dmabuf_fd); + if (dmabuf_fd >= 0) { + close(dmabuf_fd); + dmabuf_fd = -1; + } + close(devnull_fd); + return ret < 0; +} + +/* + * Test: allocate charging the calling process's own cgroup via a self pidfd. + */ +static void test_alloc_pidfd_self(char *heap_name) +{ + int heap_fd = -1, pidfd = -1, dmabuf_fd = -1, ret; + + heap_fd = dmabuf_heap_open(heap_name); + + if (!pidfd_alloc_supported(heap_fd)) { + ksft_test_result_skip("charge_fd not supported by this kernel\n"); + goto out; + } + + pidfd = sys_pidfd_open(getpid(), 0); + if (pidfd < 0) { + ksft_test_result_skip("pidfd_open not available\n"); + goto out; + } + + ret = dmabuf_heap_alloc_pidfd(heap_fd, ONE_MEG, 0, pidfd, &dmabuf_fd); + ksft_test_result(!ret, "Allocation with self pidfd %d\n", ret); + if (dmabuf_fd >= 0) + close(dmabuf_fd); + close(pidfd); +out: + close(heap_fd); +} + +/* + * Test: allocate charging a child process's cgroup via a child pidfd. + */ +static void test_alloc_pidfd_child(char *heap_name) +{ + int heap_fd = -1, pidfd = -1, dmabuf_fd = -1; + pid_t child_pid; + int status, ret; + + heap_fd = dmabuf_heap_open(heap_name); + + if (!pidfd_alloc_supported(heap_fd)) { + ksft_test_result_skip("charge_fd not supported by this kernel\n"); + goto out; + } + + child_pid = fork(); + if (child_pid == 0) { + pause(); + _exit(0); + } + if (child_pid < 0) + ksft_exit_fail_msg("fork failed: %s\n", strerror(errno)); + + pidfd = sys_pidfd_open(child_pid, 0); + if (pidfd < 0) { + kill(child_pid, SIGTERM); + waitpid(child_pid, &status, 0); + ksft_test_result_skip("pidfd_open for child failed\n"); + goto out; + } + + ret = dmabuf_heap_alloc_pidfd(heap_fd, ONE_MEG, 0, pidfd, &dmabuf_fd); + ksft_test_result(!ret, "Allocation with child pidfd %d\n", ret); + if (dmabuf_fd >= 0) + close(dmabuf_fd); + close(pidfd); + kill(child_pid, SIGTERM); + waitpid(child_pid, &status, 0); +out: + close(heap_fd); +} + +static int open_self_cgroupfd(void) +{ + char cgroup_path[PATH_MAX]; + char buf[PATH_MAX]; + FILE *f; + + f = fopen("/proc/self/cgroup", "r"); + if (!f) + return -1; + + cgroup_path[0] = '\0'; + while (fgets(buf, sizeof(buf), f)) { + char *p = buf; + + if (strncmp(p, "0::", 3) == 0) { + p += 3; + p[strcspn(p, "\n")] = '\0'; + snprintf(cgroup_path, sizeof(cgroup_path), + "/sys/fs/cgroup%s", p); + break; + } + } + fclose(f); + + if (!cgroup_path[0]) + return -1; + + return open(cgroup_path, O_RDONLY | O_DIRECTORY); +} + +/* + * Test: allocate charging the calling process's own cgroup via a cgroupfd. + */ +static void test_alloc_cgroupfd_self(char *heap_name) +{ + int heap_fd = -1, cgroup_fd = -1, dmabuf_fd = -1, ret; + + heap_fd = dmabuf_heap_open(heap_name); + + if (!pidfd_alloc_supported(heap_fd)) { + ksft_test_result_skip("charge_fd not supported by this kernel\n"); + goto out; + } + + cgroup_fd = open_self_cgroupfd(); + if (cgroup_fd < 0) { + ksft_test_result_skip("cannot open self cgroupfd\n"); + goto out; + } + + ret = dmabuf_heap_alloc_pidfd(heap_fd, ONE_MEG, 0, cgroup_fd, &dmabuf_fd); + ksft_test_result(!ret, "Allocation with cgroupfd %d\n", ret); + if (dmabuf_fd >= 0) + close(dmabuf_fd); + close(cgroup_fd); +out: + close(heap_fd); +} + +/* + * Test: transfer a dma-buf's charge to self cgroup via DMA_BUF_IOCTL_XFER_CHARGE. + */ +static void test_xfer_charge_cgroupfd_self(char *heap_name) +{ + int heap_fd = -1, cgroup_fd = -1, dmabuf_fd = -1, ret; + struct dma_buf_transfer_charge xfer = { 0 }; + + heap_fd = dmabuf_heap_open(heap_name); + + if (!pidfd_alloc_supported(heap_fd)) { + ksft_test_result_skip("charge_fd not supported by this kernel\n"); + goto out; + } + + ret = dmabuf_heap_alloc(heap_fd, ONE_MEG, 0, &dmabuf_fd); + if (ret) { + ksft_test_result_fail("Allocation failed %d\n", ret); + goto out; + } + + cgroup_fd = open_self_cgroupfd(); + if (cgroup_fd < 0) { + ksft_test_result_skip("cannot open self cgroupfd\n"); + close(dmabuf_fd); + goto out; + } + + xfer.charge_fd = cgroup_fd; + ret = ioctl(dmabuf_fd, DMA_BUF_IOCTL_XFER_CHARGE, &xfer); + ksft_test_result(!ret, "XFER_CHARGE to self cgroupfd %d\n", ret); + + close(cgroup_fd); + close(dmabuf_fd); out: close(heap_fd); } @@ -530,7 +751,7 @@ int main(void) return KSFT_SKIP; }
- ksft_set_plan(12 * numer_of_heaps()); + ksft_set_plan(16 * numer_of_heaps());
while ((dir = readdir(d))) { if (!strncmp(dir->d_name, ".", 2)) @@ -546,6 +767,10 @@ int main(void) test_alloc_compat(dir->d_name); test_alloc_errors(dir->d_name); test_alloc_no_fd_leak_on_efault(dir->d_name); + test_alloc_pidfd_self(dir->d_name); + test_alloc_pidfd_child(dir->d_name); + test_alloc_cgroupfd_self(dir->d_name); + test_xfer_charge_cgroupfd_self(dir->d_name); } closedir(d);
diff --git a/tools/testing/selftests/dmabuf-heaps/vmtest.sh b/tools/testing/selftests/dmabuf-heaps/vmtest.sh new file mode 100755 index 000000000000..3574b71ca83e --- /dev/null +++ b/tools/testing/selftests/dmabuf-heaps/vmtest.sh @@ -0,0 +1,226 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# +# Copyright (c) 2026 Red Hat +# +# Dependencies: +# * virtme-ng +# * qemu (used by virtme-ng) + +readonly SCRIPT_DIR="$(cd -P -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)" +readonly KERNEL_CHECKOUT=$(realpath "${SCRIPT_DIR}"/../../../../) +readonly CGROUP_DIR="${KERNEL_CHECKOUT}/tools/testing/selftests/cgroup" + +source "${SCRIPT_DIR}"/../kselftest/ktap_helpers.sh + +readonly DMABUF_HEAP_TEST="${SCRIPT_DIR}"/dmabuf-heap +readonly MEMCONTROL_TEST="${CGROUP_DIR}"/test_memcontrol +readonly TMP_DIR=$(mktemp -d /tmp/dmabuf-vmtest.XXXXXXXX) + +VERBOSE=false +BUILD=false +BUILD_HOST="" +BUILD_HOST_PODMAN_CONTAINER_NAME="" +SHELL_MODE=false + +usage() { + echo + echo "$0 [OPTIONS]" + echo + echo "Options" + echo " -b: build the kernel from the current source tree and use it for the VM" + echo " -H: hostname for remote build host (used with -b)" + echo " -p: podman container name for remote build host (used with -b)" + echo " Example: -H beefyserver -p vng" + echo " -s: drop into an interactive shell in the VM instead of running tests" + echo " -v: enable verbose vng/qemu output" + echo + + exit 1 +} + +die() { + echo "$*" >&2 + exit "${KSFT_FAIL}" +} + +cleanup() { + rm -rf "${TMP_DIR}" +} + +check_deps() { + for dep in vng make; do + if [[ ! -x $(command -v "${dep}") ]]; then + echo -e "skip: dependency ${dep} not found!\n" + exit "${KSFT_SKIP}" + fi + done + + if ${SHELL_MODE}; then + return + fi + + if [[ ! -x "${DMABUF_HEAP_TEST}" ]]; then + printf "skip: %s not found!" "${DMABUF_HEAP_TEST}" + printf " Please build the kselftest dmabuf-heaps target (or use -b).\n" + exit "${KSFT_SKIP}" + fi + + if [[ ! -x "${MEMCONTROL_TEST}" ]]; then + printf "skip: %s not found!" "${MEMCONTROL_TEST}" + printf " Please build the kselftest cgroup target (or use -b).\n" + exit "${KSFT_SKIP}" + fi +} + +check_vng() { + local tested_versions=("1.36" "1.37") + local version + local ok=0 + + version="$(vng --version)" + for tv in "${tested_versions[@]}"; do + if [[ "${version}" == *"${tv}"* ]]; then + ok=1 + break + fi + done + + if [[ "${ok}" -eq 0 ]]; then + printf "warning: vng version '%s' has not been tested and may " "${version}" >&2 + printf "not function properly.\n\tThe following versions have been tested: " >&2 + echo "${tested_versions[@]}" >&2 + fi +} + +build_selftests() { + make -C "${KERNEL_CHECKOUT}" headers_install \ + INSTALL_HDR_PATH="${TMP_DIR}/usr" -j"$(nproc)" + + local khdr="-isystem ${TMP_DIR}/usr/include" + + if ! make -C "${SCRIPT_DIR}" KHDR_INCLUDES="${khdr}" -j"$(nproc)"; then + die "failed to build dmabuf-heaps selftests" + fi + + if ! make -C "${CGROUP_DIR}" KHDR_INCLUDES="${khdr}" \ + "${MEMCONTROL_TEST}" -j"$(nproc)"; then + die "failed to build cgroup/test_memcontrol selftest" + fi +} + +handle_build() { + if ! ${BUILD}; then + return + fi + + if [[ ! -d "${KERNEL_CHECKOUT}" ]]; then + echo "-b requires vmtest.sh called from the kernel source tree" >&2 + exit 1 + fi + + pushd "${KERNEL_CHECKOUT}" &>/dev/null + + if ! vng --kconfig --config "${SCRIPT_DIR}/config"; then + die "failed to generate .config for kernel source tree (${KERNEL_CHECKOUT})" + fi + + local vng_args=("-v" "--config" "${SCRIPT_DIR}/config" "--build") + + if [[ -n "${BUILD_HOST}" ]]; then + vng_args+=("--build-host" "${BUILD_HOST}") + fi + + if [[ -n "${BUILD_HOST_PODMAN_CONTAINER_NAME}" ]]; then + vng_args+=("--build-host-exec-prefix" \ + "podman exec -ti ${BUILD_HOST_PODMAN_CONTAINER_NAME}") + fi + + if ! vng "${vng_args[@]}"; then + die "failed to build kernel from source tree (${KERNEL_CHECKOUT})" + fi + + build_selftests + + popd &>/dev/null +} + +make_runner() { + # virtme-ng shares the host filesystem, so TMP_DIR is accessible + # inside the VM at the same absolute path. + cat > "${TMP_DIR}/run_tests.sh" <<-EOF + #!/bin/sh + set -u + + # Enable memory controller for child cgroups + echo +memory > /sys/fs/cgroup/cgroup.subtree_control 2>/dev/null + + PASS=0; FAIL=0; SKIP=0; N=0 + + run() { + name="$1"; shift + N=$((N+1)) + "$@"; rc=$? + if [ $rc -eq 0 ]; then echo "ok $N $name"; PASS=$((PASS+1)) + elif [ $rc -eq 4 ]; then echo "ok $N $name # SKIP"; SKIP=$((SKIP+1)) + else echo "not ok $N $name"; FAIL=$((FAIL+1)) + fi + } + + run "dmabuf-heap charge_fd ioctl" ${DMABUF_HEAP_TEST} + run "memcontrol dma-buf memcg" ${MEMCONTROL_TEST} test_memcg_dmabuf + run "memcontrol dma-buf xfer limit" \ + ${MEMCONTROL_TEST} test_memcg_dmabuf_xfer_at_limit + echo "# PASS=$PASS SKIP=$SKIP FAIL=$FAIL" + [ $FAIL -eq 0 ] + EOF + chmod +x "${TMP_DIR}/run_tests.sh" +} + +run_vm() { + local verbose_opt="" + local kernel_opt="" + + ${VERBOSE} && verbose_opt="--verbose" + + # If we are running from within the kernel source tree, use the kernel + # source tree as the kernel to boot, otherwise use the running kernel. + if [[ "$(realpath "$(pwd)")" == "${KERNEL_CHECKOUT}"* ]]; then + kernel_opt="${KERNEL_CHECKOUT}" + fi + + if ${SHELL_MODE}; then + echo "Starting interactive shell in VM. Exit to stop VM." + vng --run ${kernel_opt} ${verbose_opt} --user root --memory 512M \ + --append "dma_heap.mem_accounting=1" \ + -- bash --noprofile --norc + exit "${KSFT_PASS}" + fi + + vng --run ${kernel_opt} ${verbose_opt} --user root --memory 512M \ + --append "dma_heap.mem_accounting=1" \ + --exec "${TMP_DIR}/run_tests.sh" +} + +while getopts :hvsbH:p: o +do + case $o in + v) VERBOSE=true;; + b) BUILD=true;; + s) SHELL_MODE=true;; + H) BUILD_HOST=$OPTARG;; + p) BUILD_HOST_PODMAN_CONTAINER_NAME=$OPTARG;; + h|*) usage;; + esac +done +shift $((OPTIND-1)) + +trap cleanup EXIT + +check_deps +check_vng +handle_build +make_runner + +echo "Booting VM and running tests..." +run_vm
Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider: - [Medium] selftests/dmabuf-heaps: check_deps() runs before handle_build() in vmtest.sh - [Medium] selftests/dmabuf-heaps: fail instead of skip on EOPNOTSUPP in XFER_CHARGE test - [Medium] selftests/dmabuf-heaps: pidfd_alloc_supported() treats EOPNOTSUPP as supported - [Medium] selftests: dmabuf-heaps: no memory controller check in open_self_cgroupfd() - [Medium] selftests: cgroup: MEMCG_DMABUF_WAIT_RETRIES is shorter than 2s rstat FLUSH_TIME --
commit 764a620a8574ac6b7e6c57e6f71e4c743c373656 Author: Albert Esteve aesteve@redhat.com
selftests/dmabuf-heaps: Add dma-buf memcg accounting tests
This commit adds selftests for dma-buf memcg accounting via charge_fd in DMA_HEAP_IOCTL_ALLOC and DMA_BUF_IOCTL_XFER_CHARGE across the dmabuf-heaps and cgroup test suites, along with a vmtest.sh runner script.
diff --git a/tools/testing/selftests/cgroup/test_memcontrol.c b/tools/testing/selftests/cgroup/test_memcontrol.c index 0ed82347044ed..0b41c3c6f62c1 100644 --- a/tools/testing/selftests/cgroup/test_memcontrol.c +++ b/tools/testing/selftests/cgroup/test_memcontrol.c @@ -19,10 +19,20 @@ #include <errno.h> #include <sys/mman.h> +#include <linux/dma-buf.h> +#include <linux/dma-heap.h> +#include <signal.h> +#include <sys/ioctl.h>
+#include "../pidfd/pidfd.h" #include "kselftest.h" #include "cgroup_util.h" +#define DMA_HEAP_SYSTEM "/dev/dma_heap/system" +#define ONE_MEG (1024 * 1024)
#define MEMCG_SOCKSTAT_WAIT_RETRIES 30 +#define MEMCG_DMABUF_WAIT_RETRIES 15
[Severity: Medium] Is MEMCG_DMABUF_WAIT_RETRIES (15 retries at 100ms intervals, or 1.4 seconds before the final read) long enough to avoid races with the periodic rstat flush in test_memcg_dmabuf() and test_memcg_dmabuf_xfer_at_limit()?
In mm/memcontrol.c, reading memory.stat calls mem_cgroup_flush_stats(), which only flushes rstat synchronously when stats_updates exceeds MEMCG_CHARGE_BATCH * num_online_cpus() (64 * num_online_cpus()):
mm/memcontrol.c:mem_cgroup_flush_stats() if (atomic_long_read(&vmstats->stats_updates) > MEMCG_CHARGE_BATCH * num_online_cpus()) ...
Charging or uncharging a ONE_MEG (1 MB) dma-buf in a fresh child cgroup updates MEMCG_DMABUF by 256 pages on 4KB-page kernels (or 16 pages on 64KB-page kernels). When num_online_cpus() >= 4 (where 256 > 64 * 4 is false) or on >4KB-page kernels, reading memory.stat skips synchronous flushing and relies on the periodic deferred work stats_flush_dwork, which runs every FLUSH_TIME (2UL * HZ, or 2.0 seconds).
In test_memcg_dmabuf(), even if the first poll for ONE_MEG catches a periodic flush, that flush resets stats_updates to 0 and schedules the next stats_flush_dwork 2.0 seconds later. Immediately closing dmabuf_fd adds only 256 to stats_updates, so the subsequent 1.4-second poll for dmabuf == 0 can time out before the next 2.0-second periodic flush runs. Should MEMCG_DMABUF_WAIT_RETRIES be at least 30, matching MEMCG_SOCKSTAT_WAIT_RETRIES?
diff --git a/tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c b/tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c index 1d49df6719198..90ef14d2ecba0 100644 --- a/tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c +++ b/tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c
[ ... ]
@@ -496,6 +505,218 @@ static void test_alloc_no_fd_leak_on_efault(char *heap_name)
[ ... ]
+/*
- Probe whether the kernel honours charge_fd in DMA_HEAP_IOCTL_ALLOC.
- */
+static bool pidfd_alloc_supported(int heap_fd) +{
- int devnull_fd, dmabuf_fd = -1, ret;
- devnull_fd = open("/dev/null", O_RDONLY);
- if (devnull_fd < 0)
return false;- ret = dmabuf_heap_alloc_pidfd(heap_fd, ONE_MEG, 0, devnull_fd, &dmabuf_fd);
- if (dmabuf_fd >= 0) {
close(dmabuf_fd);dmabuf_fd = -1;- }
- close(devnull_fd);
- return ret < 0;
+}
[Severity: Medium] What happens in pidfd_alloc_supported() when the kernel is built with CONFIG_MEMCG=n?
In drivers/dma-buf/dma-heap.c, dma_heap_ioctl_allocate() passes a non-zero charge_fd to mem_cgroup_get_from_fd():
drivers/dma-buf/dma-heap.c:dma_heap_ioctl_allocate() if (heap_allocation->charge_fd) { memcg = mem_cgroup_get_from_fd(heap_allocation->charge_fd); if (IS_ERR(memcg)) return ERR_CAST(memcg); }
When CONFIG_MEMCG is disabled, the stub in include/linux/memcontrol.h returns ERR_PTR(-EOPNOTSUPP):
include/linux/memcontrol.h:mem_cgroup_get_from_fd() static inline struct mem_cgroup *mem_cgroup_get_from_fd(int fd) { return ERR_PTR(-EOPNOTSUPP); }
Because dmabuf_heap_alloc_pidfd() returns -1 with errno == EOPNOTSUPP, returning ret < 0 causes pidfd_alloc_supported() to return true when CONFIG_MEMCG=n, and test_alloc_pidfd_self(), test_alloc_pidfd_child(), test_alloc_cgroupfd_self(), and test_xfer_charge_cgroupfd_self() then proceed and report test failures instead of skipping. Should pidfd_alloc_supported() also verify that errno != EOPNOTSUPP (or errno == EINVAL)?
[ ... ]
+static int open_self_cgroupfd(void) +{
- char cgroup_path[PATH_MAX];
- char buf[PATH_MAX];
- FILE *f;
- f = fopen("/proc/self/cgroup", "r");
- if (!f)
return -1;- cgroup_path[0] = '\0';
- while (fgets(buf, sizeof(buf), f)) {
char *p = buf;if (strncmp(p, "0::", 3) == 0) {p += 3;p[strcspn(p, "\n")] = '\0';snprintf(cgroup_path, sizeof(cgroup_path),"/sys/fs/cgroup%s", p);break;}- }
- fclose(f);
- if (!cgroup_path[0])
return -1;- return open(cgroup_path, O_RDONLY | O_DIRECTORY);
+}
+/*
- Test: allocate charging the calling process's own cgroup via a cgroupfd.
- */
+static void test_alloc_cgroupfd_self(char *heap_name) +{
- int heap_fd = -1, cgroup_fd = -1, dmabuf_fd = -1, ret;
- heap_fd = dmabuf_heap_open(heap_name);
- if (!pidfd_alloc_supported(heap_fd)) {
ksft_test_result_skip("charge_fd not supported by this kernel\n");goto out;- }
- cgroup_fd = open_self_cgroupfd();
- if (cgroup_fd < 0) {
ksft_test_result_skip("cannot open self cgroupfd\n");goto out;- }
- ret = dmabuf_heap_alloc_pidfd(heap_fd, ONE_MEG, 0, cgroup_fd, &dmabuf_fd);
- ksft_test_result(!ret, "Allocation with cgroupfd %d\n", ret);
- if (dmabuf_fd >= 0)
close(dmabuf_fd);- close(cgroup_fd);
+out:
- close(heap_fd);
+}
[Severity: Medium] Can test_alloc_cgroupfd_self() and test_xfer_charge_cgroupfd_self() fail with ENOENT instead of skipping when the test binary is run outside vmtest.sh (for example via make run_tests or vmtest.sh -s) in a cgroup where the memory controller is not enabled?
Unlike the pidfd path in mem_cgroup_get_from_fd(), which uses get_mem_cgroup_from_mm() and walks up to an online ancestor, the cgroupfd path requires the memory controller CSS to be directly enabled on the target cgroup itself:
mm/memcontrol.c:mem_cgroup_get_from_fd() cgrp = cgroup_get_from_fd(fd); if (!IS_ERR(cgrp)) { rcu_read_lock(); css = cgroup_css(cgrp, &memory_cgrp_subsys); if (!css || !css_tryget_online(css)) { rcu_read_unlock(); cgroup_put(cgrp); return ERR_PTR(-ENOENT); }
When dmabuf-heap runs in a leaf cgroup (such as a standard systemd user or service scope) whose parent has not enabled +memory in cgroup.subtree_control, or when booted with cgroup_disable=memory, open_self_cgroupfd() still succeeds in opening the directory, and the ioctl fails with ENOENT. Should open_self_cgroupfd() check that memory is active in that cgroup (or should these two tests skip when the ioctl fails with errno == ENOENT)?
+/*
- Test: transfer a dma-buf's charge to self cgroup via DMA_BUF_IOCTL_XFER_CHARGE.
- */
+static void test_xfer_charge_cgroupfd_self(char *heap_name) +{
- int heap_fd = -1, cgroup_fd = -1, dmabuf_fd = -1, ret;
- struct dma_buf_transfer_charge xfer = { 0 };
- heap_fd = dmabuf_heap_open(heap_name);
- if (!pidfd_alloc_supported(heap_fd)) {
ksft_test_result_skip("charge_fd not supported by this kernel\n");goto out;- }
- ret = dmabuf_heap_alloc(heap_fd, ONE_MEG, 0, &dmabuf_fd);
- if (ret) {
ksft_test_result_fail("Allocation failed %d\n", ret);goto out;- }
- cgroup_fd = open_self_cgroupfd();
- if (cgroup_fd < 0) {
ksft_test_result_skip("cannot open self cgroupfd\n");close(dmabuf_fd);goto out;- }
- xfer.charge_fd = cgroup_fd;
- ret = ioctl(dmabuf_fd, DMA_BUF_IOCTL_XFER_CHARGE, &xfer);
- ksft_test_result(!ret, "XFER_CHARGE to self cgroupfd %d\n", ret);
[Severity: Medium] Does test_xfer_charge_cgroupfd_self() report a failure instead of skipping when main() iterates over non-system heaps in /dev/dma_heap (such as CMA heaps when CONFIG_DMABUF_HEAPS_CMA=y)?
In drivers/dma-buf/dma-heap.c, dma_heap_ioctl_allocate() validates charge_fd before calling heap->ops->allocate(), so pidfd_alloc_supported() returns true for all heaps. However, in drivers/dma-buf/dma-buf.c, dma_buf_xfer_charge() checks whether the exporter enabled memcg accounting on the buffer:
drivers/dma-buf/dma-buf.c:dma_buf_xfer_charge() if (!dmabuf->memcg_account) return -EOPNOTSUPP;
Because only system_heap sets exp_info.memcg_account = true (while exporters such as cma_heap leave it false), ioctl(dmabuf_fd, DMA_BUF_IOCTL_XFER_CHARGE, &xfer) returns -1 with errno == EOPNOTSUPP on those heaps. Should test_xfer_charge_cgroupfd_self() call ksft_test_result_skip() when ret < 0 and errno == EOPNOTSUPP?
diff --git a/tools/testing/selftests/dmabuf-heaps/vmtest.sh b/tools/testing/selftests/dmabuf-heaps/vmtest.sh new file mode 100755 index 0000000000000..3574b71ca83e8 --- /dev/null +++ b/tools/testing/selftests/dmabuf-heaps/vmtest.sh
[ ... ]
+check_deps() {
- for dep in vng make; do
if [[ ! -x $(command -v "${dep}") ]]; thenecho -e "skip: dependency ${dep} not found!\n"exit "${KSFT_SKIP}"fi- done
- if ${SHELL_MODE}; then
return- fi
- if [[ ! -x "${DMABUF_HEAP_TEST}" ]]; then
printf "skip: %s not found!" "${DMABUF_HEAP_TEST}"printf " Please build the kselftest dmabuf-heaps target (or use -b).\n"exit "${KSFT_SKIP}"- fi
- if [[ ! -x "${MEMCONTROL_TEST}" ]]; then
printf "skip: %s not found!" "${MEMCONTROL_TEST}"printf " Please build the kselftest cgroup target (or use -b).\n"exit "${KSFT_SKIP}"- fi
+}
[ ... ]
+trap cleanup EXIT
+check_deps +check_vng +handle_build +make_runner
[Severity: Medium] If vmtest.sh is invoked with -b on a clean tree where the selftest binaries have not been built yet, won't check_deps() exit with KSFT_SKIP before handle_build() has a chance to build them?
In the main flow, check_deps() runs before handle_build(), and inside check_deps() only SHELL_MODE returns early before checking for DMABUF_HEAP_TEST and MEMCONTROL_TEST. Should check_deps() also skip the binary existence checks when BUILD is true (or run after handle_build())?
op-tee@lists.trustedfirmware.org