* [PATCH v2 0/8] dma-debug cleanup and dynamic allocation
@ 2018-12-05 19:56 Robin Murphy
2018-12-05 19:56 ` [PATCH v2 1/8] dma-debug: Use pr_fmt() Robin Murphy
` (7 more replies)
0 siblings, 8 replies; 19+ messages in thread
From: Robin Murphy @ 2018-12-05 19:56 UTC (permalink / raw)
To: hch
Cc: m.szyprowski, iommu, linux-kernel, cai, salil.mehta, john.garry,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, H. Peter Anvin,
x86
Hi all,
Here's some assorted cleanup and improvements to dma-debug which grew
out of the problem that certain drivers use very large numbers of DMA
mappings, and knowing when to override "dma_debug_entries=..." and what
value to override it with can be a less-than-obvious task for users.
The main part is patches #3 and #4, wherein we make dma-debug clever
enough to allocate more entries dynamically if needed, such that the
preallocation value becomes more of a quality-of-life option than a
necessity. Patches #6 and #7 do some cruft-removal to allow patch #8
to make the allocation behaviour more efficient in general.
Patches #1, #2 and #5 are some other cleanup and handy features which
fell out of the discussion/development.
Robin.
Robin Murphy (8):
dma-debug: Use pr_fmt()
dma-debug: Expose nr_total_entries in debugfs
dma-debug: Refactor dma_debug_entry allocation
dma-debug: Dynamically expand the dma_debug_entry pool
dma-debug: Make leak-like behaviour apparent
x86/dma/amd-gart: Stop resizing dma_debug_entry pool
dma/debug: Remove dma_debug_resize_entries()
dma-debug: Batch dma_debug_entry allocation
Documentation/DMA-API.txt | 20 +-
Documentation/x86/x86_64/boot-options.txt | 5 +-
arch/x86/kernel/amd_gart_64.c | 23 ---
include/linux/dma-debug.h | 7 -
kernel/dma/debug.c | 211 ++++++++++------------
5 files changed, 107 insertions(+), 159 deletions(-)
--
2.19.1.dirty
^ permalink raw reply [flat|nested] 19+ messages in thread* [PATCH v2 1/8] dma-debug: Use pr_fmt() 2018-12-05 19:56 [PATCH v2 0/8] dma-debug cleanup and dynamic allocation Robin Murphy @ 2018-12-05 19:56 ` Robin Murphy 2018-12-05 19:56 ` [PATCH v2 2/8] dma-debug: Expose nr_total_entries in debugfs Robin Murphy ` (6 subsequent siblings) 7 siblings, 0 replies; 19+ messages in thread From: Robin Murphy @ 2018-12-05 19:56 UTC (permalink / raw) To: hch; +Cc: m.szyprowski, iommu, linux-kernel, cai, salil.mehta, john.garry Use pr_fmt() to generate the "DMA-API: " prefix consistently. This results in it being added to a couple of pr_*() messages which were missing it before, and for the err_printk() calls moves it to the actual start of the message instead of somewhere in the middle. Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Robin Murphy <robin.murphy@arm.com> --- v2: Add Christoph's review tag kernel/dma/debug.c | 74 ++++++++++++++++++++++++---------------------- 1 file changed, 38 insertions(+), 36 deletions(-) diff --git a/kernel/dma/debug.c b/kernel/dma/debug.c index 231ca4628062..91b84140e4a5 100644 --- a/kernel/dma/debug.c +++ b/kernel/dma/debug.c @@ -17,6 +17,8 @@ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */ +#define pr_fmt(fmt) "DMA-API: " fmt + #include <linux/sched/task_stack.h> #include <linux/scatterlist.h> #include <linux/dma-mapping.h> @@ -234,7 +236,7 @@ static bool driver_filter(struct device *dev) error_count += 1; \ if (driver_filter(dev) && \ (show_all_errors || show_num_errors > 0)) { \ - WARN(1, "%s %s: " format, \ + WARN(1, pr_fmt("%s %s: ") format, \ dev ? dev_driver_string(dev) : "NULL", \ dev ? dev_name(dev) : "NULL", ## arg); \ dump_entry_trace(entry); \ @@ -519,7 +521,7 @@ static void active_cacheline_inc_overlap(phys_addr_t cln) * prematurely. */ WARN_ONCE(overlap > ACTIVE_CACHELINE_MAX_OVERLAP, - "DMA-API: exceeded %d overlapping mappings of cacheline %pa\n", + pr_fmt("exceeded %d overlapping mappings of cacheline %pa\n"), ACTIVE_CACHELINE_MAX_OVERLAP, &cln); } @@ -614,7 +616,7 @@ void debug_dma_assert_idle(struct page *page) cln = to_cacheline_number(entry); err_printk(entry->dev, entry, - "DMA-API: cpu touching an active dma mapped cacheline [cln=%pa]\n", + "cpu touching an active dma mapped cacheline [cln=%pa]\n", &cln); } @@ -634,7 +636,7 @@ static void add_dma_entry(struct dma_debug_entry *entry) rc = active_cacheline_insert(entry); if (rc == -ENOMEM) { - pr_err("DMA-API: cacheline tracking ENOMEM, dma-debug disabled\n"); + pr_err("cacheline tracking ENOMEM, dma-debug disabled\n"); global_disable = true; } @@ -673,7 +675,7 @@ static struct dma_debug_entry *dma_entry_alloc(void) if (list_empty(&free_entries)) { global_disable = true; spin_unlock_irqrestore(&free_entries_lock, flags); - pr_err("DMA-API: debugging out of memory - disabling\n"); + pr_err("debugging out of memory - disabling\n"); return NULL; } @@ -777,7 +779,7 @@ static int prealloc_memory(u32 num_entries) num_free_entries = num_entries; min_free_entries = num_entries; - pr_info("DMA-API: preallocated %d debug entries\n", num_entries); + pr_info("preallocated %d debug entries\n", num_entries); return 0; @@ -850,7 +852,7 @@ static ssize_t filter_write(struct file *file, const char __user *userbuf, * switched off. */ if (current_driver_name[0]) - pr_info("DMA-API: switching off dma-debug driver filter\n"); + pr_info("switching off dma-debug driver filter\n"); current_driver_name[0] = 0; current_driver = NULL; goto out_unlock; @@ -868,7 +870,7 @@ static ssize_t filter_write(struct file *file, const char __user *userbuf, current_driver_name[i] = 0; current_driver = NULL; - pr_info("DMA-API: enable driver filter for driver [%s]\n", + pr_info("enable driver filter for driver [%s]\n", current_driver_name); out_unlock: @@ -887,7 +889,7 @@ static int dma_debug_fs_init(void) { dma_debug_dent = debugfs_create_dir("dma-api", NULL); if (!dma_debug_dent) { - pr_err("DMA-API: can not create debugfs directory\n"); + pr_err("can not create debugfs directory\n"); return -ENOMEM; } @@ -973,7 +975,7 @@ static int dma_debug_device_change(struct notifier_block *nb, unsigned long acti count = device_dma_allocations(dev, &entry); if (count == 0) break; - err_printk(dev, entry, "DMA-API: device driver has pending " + err_printk(dev, entry, "device driver has pending " "DMA allocations while released from device " "[count=%d]\n" "One of leaked entries details: " @@ -1023,14 +1025,14 @@ static int dma_debug_init(void) } if (dma_debug_fs_init() != 0) { - pr_err("DMA-API: error creating debugfs entries - disabling\n"); + pr_err("error creating debugfs entries - disabling\n"); global_disable = true; return 0; } if (prealloc_memory(nr_prealloc_entries) != 0) { - pr_err("DMA-API: debugging out of memory error - disabled\n"); + pr_err("debugging out of memory error - disabled\n"); global_disable = true; return 0; @@ -1040,7 +1042,7 @@ static int dma_debug_init(void) dma_debug_initialized = true; - pr_info("DMA-API: debugging enabled by kernel config\n"); + pr_info("debugging enabled by kernel config\n"); return 0; } core_initcall(dma_debug_init); @@ -1051,7 +1053,7 @@ static __init int dma_debug_cmdline(char *str) return -EINVAL; if (strncmp(str, "off", 3) == 0) { - pr_info("DMA-API: debugging disabled on kernel command line\n"); + pr_info("debugging disabled on kernel command line\n"); global_disable = true; } @@ -1085,11 +1087,11 @@ static void check_unmap(struct dma_debug_entry *ref) if (dma_mapping_error(ref->dev, ref->dev_addr)) { err_printk(ref->dev, NULL, - "DMA-API: device driver tries to free an " + "device driver tries to free an " "invalid DMA memory address\n"); } else { err_printk(ref->dev, NULL, - "DMA-API: device driver tries to free DMA " + "device driver tries to free DMA " "memory it has not allocated [device " "address=0x%016llx] [size=%llu bytes]\n", ref->dev_addr, ref->size); @@ -1098,7 +1100,7 @@ static void check_unmap(struct dma_debug_entry *ref) } if (ref->size != entry->size) { - err_printk(ref->dev, entry, "DMA-API: device driver frees " + err_printk(ref->dev, entry, "device driver frees " "DMA memory with different size " "[device address=0x%016llx] [map size=%llu bytes] " "[unmap size=%llu bytes]\n", @@ -1106,7 +1108,7 @@ static void check_unmap(struct dma_debug_entry *ref) } if (ref->type != entry->type) { - err_printk(ref->dev, entry, "DMA-API: device driver frees " + err_printk(ref->dev, entry, "device driver frees " "DMA memory with wrong function " "[device address=0x%016llx] [size=%llu bytes] " "[mapped as %s] [unmapped as %s]\n", @@ -1114,7 +1116,7 @@ static void check_unmap(struct dma_debug_entry *ref) type2name[entry->type], type2name[ref->type]); } else if ((entry->type == dma_debug_coherent) && (phys_addr(ref) != phys_addr(entry))) { - err_printk(ref->dev, entry, "DMA-API: device driver frees " + err_printk(ref->dev, entry, "device driver frees " "DMA memory with different CPU address " "[device address=0x%016llx] [size=%llu bytes] " "[cpu alloc address=0x%016llx] " @@ -1126,7 +1128,7 @@ static void check_unmap(struct dma_debug_entry *ref) if (ref->sg_call_ents && ref->type == dma_debug_sg && ref->sg_call_ents != entry->sg_call_ents) { - err_printk(ref->dev, entry, "DMA-API: device driver frees " + err_printk(ref->dev, entry, "device driver frees " "DMA sg list with different entry count " "[map count=%d] [unmap count=%d]\n", entry->sg_call_ents, ref->sg_call_ents); @@ -1137,7 +1139,7 @@ static void check_unmap(struct dma_debug_entry *ref) * DMA API don't handle this properly, so check for it here */ if (ref->direction != entry->direction) { - err_printk(ref->dev, entry, "DMA-API: device driver frees " + err_printk(ref->dev, entry, "device driver frees " "DMA memory with different direction " "[device address=0x%016llx] [size=%llu bytes] " "[mapped with %s] [unmapped with %s]\n", @@ -1153,7 +1155,7 @@ static void check_unmap(struct dma_debug_entry *ref) */ if (entry->map_err_type == MAP_ERR_NOT_CHECKED) { err_printk(ref->dev, entry, - "DMA-API: device driver failed to check map error" + "device driver failed to check map error" "[device address=0x%016llx] [size=%llu bytes] " "[mapped as %s]", ref->dev_addr, ref->size, @@ -1178,7 +1180,7 @@ static void check_for_stack(struct device *dev, return; addr = page_address(page) + offset; if (object_is_on_stack(addr)) - err_printk(dev, NULL, "DMA-API: device driver maps memory from stack [addr=%p]\n", addr); + err_printk(dev, NULL, "device driver maps memory from stack [addr=%p]\n", addr); } else { /* Stack is vmalloced. */ int i; @@ -1188,7 +1190,7 @@ static void check_for_stack(struct device *dev, continue; addr = (u8 *)current->stack + i * PAGE_SIZE + offset; - err_printk(dev, NULL, "DMA-API: device driver maps memory from stack [probable addr=%p]\n", addr); + err_printk(dev, NULL, "device driver maps memory from stack [probable addr=%p]\n", addr); break; } } @@ -1208,7 +1210,7 @@ static void check_for_illegal_area(struct device *dev, void *addr, unsigned long { if (overlap(addr, len, _stext, _etext) || overlap(addr, len, __start_rodata, __end_rodata)) - err_printk(dev, NULL, "DMA-API: device driver maps memory from kernel text or rodata [addr=%p] [len=%lu]\n", addr, len); + err_printk(dev, NULL, "device driver maps memory from kernel text or rodata [addr=%p] [len=%lu]\n", addr, len); } static void check_sync(struct device *dev, @@ -1224,7 +1226,7 @@ static void check_sync(struct device *dev, entry = bucket_find_contain(&bucket, ref, &flags); if (!entry) { - err_printk(dev, NULL, "DMA-API: device driver tries " + err_printk(dev, NULL, "device driver tries " "to sync DMA memory it has not allocated " "[device address=0x%016llx] [size=%llu bytes]\n", (unsigned long long)ref->dev_addr, ref->size); @@ -1232,7 +1234,7 @@ static void check_sync(struct device *dev, } if (ref->size > entry->size) { - err_printk(dev, entry, "DMA-API: device driver syncs" + err_printk(dev, entry, "device driver syncs" " DMA memory outside allocated range " "[device address=0x%016llx] " "[allocation size=%llu bytes] " @@ -1245,7 +1247,7 @@ static void check_sync(struct device *dev, goto out; if (ref->direction != entry->direction) { - err_printk(dev, entry, "DMA-API: device driver syncs " + err_printk(dev, entry, "device driver syncs " "DMA memory with different direction " "[device address=0x%016llx] [size=%llu bytes] " "[mapped with %s] [synced with %s]\n", @@ -1256,7 +1258,7 @@ static void check_sync(struct device *dev, if (to_cpu && !(entry->direction == DMA_FROM_DEVICE) && !(ref->direction == DMA_TO_DEVICE)) - err_printk(dev, entry, "DMA-API: device driver syncs " + err_printk(dev, entry, "device driver syncs " "device read-only DMA memory for cpu " "[device address=0x%016llx] [size=%llu bytes] " "[mapped with %s] [synced with %s]\n", @@ -1266,7 +1268,7 @@ static void check_sync(struct device *dev, if (!to_cpu && !(entry->direction == DMA_TO_DEVICE) && !(ref->direction == DMA_FROM_DEVICE)) - err_printk(dev, entry, "DMA-API: device driver syncs " + err_printk(dev, entry, "device driver syncs " "device write-only DMA memory to device " "[device address=0x%016llx] [size=%llu bytes] " "[mapped with %s] [synced with %s]\n", @@ -1276,7 +1278,7 @@ static void check_sync(struct device *dev, if (ref->sg_call_ents && ref->type == dma_debug_sg && ref->sg_call_ents != entry->sg_call_ents) { - err_printk(ref->dev, entry, "DMA-API: device driver syncs " + err_printk(ref->dev, entry, "device driver syncs " "DMA sg list with different entry count " "[map count=%d] [sync count=%d]\n", entry->sg_call_ents, ref->sg_call_ents); @@ -1297,7 +1299,7 @@ static void check_sg_segment(struct device *dev, struct scatterlist *sg) * whoever generated the list forgot to check them. */ if (sg->length > max_seg) - err_printk(dev, NULL, "DMA-API: mapping sg segment longer than device claims to support [len=%u] [max=%u]\n", + err_printk(dev, NULL, "mapping sg segment longer than device claims to support [len=%u] [max=%u]\n", sg->length, max_seg); /* * In some cases this could potentially be the DMA API @@ -1307,7 +1309,7 @@ static void check_sg_segment(struct device *dev, struct scatterlist *sg) start = sg_dma_address(sg); end = start + sg_dma_len(sg) - 1; if ((start ^ end) & ~boundary) - err_printk(dev, NULL, "DMA-API: mapping sg segment across boundary [start=0x%016llx] [end=0x%016llx] [boundary=0x%016llx]\n", + err_printk(dev, NULL, "mapping sg segment across boundary [start=0x%016llx] [end=0x%016llx] [boundary=0x%016llx]\n", start, end, boundary); #endif } @@ -1319,11 +1321,11 @@ void debug_dma_map_single(struct device *dev, const void *addr, return; if (!virt_addr_valid(addr)) - err_printk(dev, NULL, "DMA-API: device driver maps memory from invalid area [addr=%p] [len=%lu]\n", + err_printk(dev, NULL, "device driver maps memory from invalid area [addr=%p] [len=%lu]\n", addr, len); if (is_vmalloc_addr(addr)) - err_printk(dev, NULL, "DMA-API: device driver maps memory from vmalloc area [addr=%p] [len=%lu]\n", + err_printk(dev, NULL, "device driver maps memory from vmalloc area [addr=%p] [len=%lu]\n", addr, len); } EXPORT_SYMBOL(debug_dma_map_single); @@ -1780,7 +1782,7 @@ static int __init dma_debug_driver_setup(char *str) } if (current_driver_name[0]) - pr_info("DMA-API: enable driver filter for driver [%s]\n", + pr_info("enable driver filter for driver [%s]\n", current_driver_name); -- 2.19.1.dirty ^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v2 2/8] dma-debug: Expose nr_total_entries in debugfs 2018-12-05 19:56 [PATCH v2 0/8] dma-debug cleanup and dynamic allocation Robin Murphy 2018-12-05 19:56 ` [PATCH v2 1/8] dma-debug: Use pr_fmt() Robin Murphy @ 2018-12-05 19:56 ` Robin Murphy 2018-12-06 14:14 ` Christoph Hellwig 2018-12-05 19:56 ` [PATCH v2 3/8] dma-debug: Refactor dma_debug_entry allocation Robin Murphy ` (5 subsequent siblings) 7 siblings, 1 reply; 19+ messages in thread From: Robin Murphy @ 2018-12-05 19:56 UTC (permalink / raw) To: hch; +Cc: m.szyprowski, iommu, linux-kernel, cai, salil.mehta, john.garry Expose nr_total_entries in debugfs, so that {num,min}_free_entries become even more meaningful to users interested in current/maximum utilisation. This becomes even more relevant once nr_total_entries may change at runtime beyond just the existing AMD GART debug code. Suggested-by: John Garry <john.garry@huawei.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> --- v2: New Documentation/DMA-API.txt | 3 +++ kernel/dma/debug.c | 7 +++++++ 2 files changed, 10 insertions(+) diff --git a/Documentation/DMA-API.txt b/Documentation/DMA-API.txt index ac66ae2509a9..6bdb095393b0 100644 --- a/Documentation/DMA-API.txt +++ b/Documentation/DMA-API.txt @@ -723,6 +723,9 @@ dma-api/min_free_entries This read-only file can be read to get the dma-api/num_free_entries The current number of free dma_debug_entries in the allocator. +dma-api/nr_total_entries The total number of dma_debug_entries in the + allocator, both free and used. + dma-api/driver-filter You can write a name of a driver into this file to limit the debug output to requests from that particular driver. Write an empty string to diff --git a/kernel/dma/debug.c b/kernel/dma/debug.c index 91b84140e4a5..29486eb9d1dc 100644 --- a/kernel/dma/debug.c +++ b/kernel/dma/debug.c @@ -144,6 +144,7 @@ static struct dentry *show_all_errors_dent __read_mostly; static struct dentry *show_num_errors_dent __read_mostly; static struct dentry *num_free_entries_dent __read_mostly; static struct dentry *min_free_entries_dent __read_mostly; +static struct dentry *nr_total_entries_dent __read_mostly; static struct dentry *filter_dent __read_mostly; /* per-driver filter related state */ @@ -928,6 +929,12 @@ static int dma_debug_fs_init(void) if (!min_free_entries_dent) goto out_err; + nr_total_entries_dent = debugfs_create_u32("nr_total_entries", 0444, + dma_debug_dent, + &nr_total_entries); + if (!nr_total_entries_dent) + goto out_err; + filter_dent = debugfs_create_file("driver_filter", 0644, dma_debug_dent, NULL, &filter_fops); if (!filter_dent) -- 2.19.1.dirty ^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: [PATCH v2 2/8] dma-debug: Expose nr_total_entries in debugfs 2018-12-05 19:56 ` [PATCH v2 2/8] dma-debug: Expose nr_total_entries in debugfs Robin Murphy @ 2018-12-06 14:14 ` Christoph Hellwig 0 siblings, 0 replies; 19+ messages in thread From: Christoph Hellwig @ 2018-12-06 14:14 UTC (permalink / raw) To: Robin Murphy Cc: hch, m.szyprowski, iommu, linux-kernel, cai, salil.mehta, john.garry On Wed, Dec 05, 2018 at 07:56:51PM +0000, Robin Murphy wrote: > Expose nr_total_entries in debugfs, so that {num,min}_free_entries > become even more meaningful to users interested in current/maximum > utilisation. This becomes even more relevant once nr_total_entries > may change at runtime beyond just the existing AMD GART debug code. > > Suggested-by: John Garry <john.garry@huawei.com> > Signed-off-by: Robin Murphy <robin.murphy@arm.com> Looks fine, Reviewed-by: Christoph Hellwig <hch@lst.de> ^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH v2 3/8] dma-debug: Refactor dma_debug_entry allocation 2018-12-05 19:56 [PATCH v2 0/8] dma-debug cleanup and dynamic allocation Robin Murphy 2018-12-05 19:56 ` [PATCH v2 1/8] dma-debug: Use pr_fmt() Robin Murphy 2018-12-05 19:56 ` [PATCH v2 2/8] dma-debug: Expose nr_total_entries in debugfs Robin Murphy @ 2018-12-05 19:56 ` Robin Murphy 2018-12-06 14:23 ` Christoph Hellwig 2018-12-05 19:56 ` [PATCH v2 4/8] dma-debug: Dynamically expand the dma_debug_entry pool Robin Murphy ` (4 subsequent siblings) 7 siblings, 1 reply; 19+ messages in thread From: Robin Murphy @ 2018-12-05 19:56 UTC (permalink / raw) To: hch; +Cc: m.szyprowski, iommu, linux-kernel, cai, salil.mehta, john.garry Make the prealloc_memory() logic a little more general and robust so that it serves for runtime reallocations too. The first thing we can do with that is clean up dma_debug_resize_entries() a bit. Signed-off-by: Robin Murphy <robin.murphy@arm.com> --- v2: Give it a better name, simplify the locking mess kernel/dma/debug.c | 87 +++++++++++++++++++--------------------------- 1 file changed, 36 insertions(+), 51 deletions(-) diff --git a/kernel/dma/debug.c b/kernel/dma/debug.c index 29486eb9d1dc..1b0858d7edfd 100644 --- a/kernel/dma/debug.c +++ b/kernel/dma/debug.c @@ -646,6 +646,36 @@ static void add_dma_entry(struct dma_debug_entry *entry) */ } +static int dma_debug_add_entries(u32 num_entries, gfp_t gfp) +{ + struct dma_debug_entry *entry, *next_entry; + LIST_HEAD(tmp); + int i; + + for (i = 0; i < num_entries; ++i) { + entry = kzalloc(sizeof(*entry), gfp); + if (!entry) + goto out_err; + + list_add_tail(&entry->list, &tmp); + } + + list_splice(&tmp, &free_entries); + num_free_entries += num_entries; + nr_total_entries += num_entries; + + return 0; + +out_err: + + list_for_each_entry_safe(entry, next_entry, &tmp, list) { + list_del(&entry->list); + kfree(entry); + } + + return -ENOMEM; +} + static struct dma_debug_entry *__dma_entry_alloc(void) { struct dma_debug_entry *entry; @@ -715,28 +745,13 @@ int dma_debug_resize_entries(u32 num_entries) int i, delta, ret = 0; unsigned long flags; struct dma_debug_entry *entry; - LIST_HEAD(tmp); spin_lock_irqsave(&free_entries_lock, flags); if (nr_total_entries < num_entries) { delta = num_entries - nr_total_entries; - spin_unlock_irqrestore(&free_entries_lock, flags); - - for (i = 0; i < delta; i++) { - entry = kzalloc(sizeof(*entry), GFP_KERNEL); - if (!entry) - break; - - list_add_tail(&entry->list, &tmp); - } - - spin_lock_irqsave(&free_entries_lock, flags); - - list_splice(&tmp, &free_entries); - nr_total_entries += i; - num_free_entries += i; + ret = dma_debug_add_entries(delta, GFP_ATOMIC); } else { delta = nr_total_entries - num_entries; @@ -746,11 +761,10 @@ int dma_debug_resize_entries(u32 num_entries) } nr_total_entries -= i; + if (nr_total_entries != num_entries) + ret = -EBUSY; } - if (nr_total_entries != num_entries) - ret = 1; - spin_unlock_irqrestore(&free_entries_lock, flags); return ret; @@ -764,36 +778,6 @@ int dma_debug_resize_entries(u32 num_entries) * 2. Preallocate a given number of dma_debug_entry structs */ -static int prealloc_memory(u32 num_entries) -{ - struct dma_debug_entry *entry, *next_entry; - int i; - - for (i = 0; i < num_entries; ++i) { - entry = kzalloc(sizeof(*entry), GFP_KERNEL); - if (!entry) - goto out_err; - - list_add_tail(&entry->list, &free_entries); - } - - num_free_entries = num_entries; - min_free_entries = num_entries; - - pr_info("preallocated %d debug entries\n", num_entries); - - return 0; - -out_err: - - list_for_each_entry_safe(entry, next_entry, &free_entries, list) { - list_del(&entry->list); - kfree(entry); - } - - return -ENOMEM; -} - static ssize_t filter_read(struct file *file, char __user *user_buf, size_t count, loff_t *ppos) { @@ -1038,14 +1022,15 @@ static int dma_debug_init(void) return 0; } - if (prealloc_memory(nr_prealloc_entries) != 0) { + if (dma_debug_add_entries(nr_prealloc_entries, GFP_KERNEL) != 0) { pr_err("debugging out of memory error - disabled\n"); global_disable = true; return 0; } - nr_total_entries = num_free_entries; + min_free_entries = num_free_entries; + pr_info("preallocated %d debug entries\n", nr_total_entries); dma_debug_initialized = true; -- 2.19.1.dirty ^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: [PATCH v2 3/8] dma-debug: Refactor dma_debug_entry allocation 2018-12-05 19:56 ` [PATCH v2 3/8] dma-debug: Refactor dma_debug_entry allocation Robin Murphy @ 2018-12-06 14:23 ` Christoph Hellwig 2018-12-06 18:10 ` Robin Murphy 0 siblings, 1 reply; 19+ messages in thread From: Christoph Hellwig @ 2018-12-06 14:23 UTC (permalink / raw) To: Robin Murphy Cc: hch, m.szyprowski, iommu, linux-kernel, cai, salil.mehta, john.garry > +static int dma_debug_add_entries(u32 num_entries, gfp_t gfp) > +{ > + struct dma_debug_entry *entry, *next_entry; > + LIST_HEAD(tmp); > + int i; > + > + for (i = 0; i < num_entries; ++i) { > + entry = kzalloc(sizeof(*entry), gfp); > + if (!entry) > + goto out_err; > + > + list_add_tail(&entry->list, &tmp); > + } > + > + list_splice(&tmp, &free_entries); > + num_free_entries += num_entries; > + nr_total_entries += num_entries; The adding to a local list and splicing seems a bit pointless if we do it all under lock anyway. Either we change the locking in dma_debug_resize_entries and your upcoming automatic allocation that we only do it over the splice and counter adjustment, which would have the advantage of allowing freeing of entries in parallel to these allocations, or we could just drop the local tmp list. ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v2 3/8] dma-debug: Refactor dma_debug_entry allocation 2018-12-06 14:23 ` Christoph Hellwig @ 2018-12-06 18:10 ` Robin Murphy 2018-12-06 18:46 ` Christoph Hellwig 0 siblings, 1 reply; 19+ messages in thread From: Robin Murphy @ 2018-12-06 18:10 UTC (permalink / raw) To: Christoph Hellwig Cc: m.szyprowski, iommu, linux-kernel, cai, salil.mehta, john.garry On 06/12/2018 14:23, Christoph Hellwig wrote: >> +static int dma_debug_add_entries(u32 num_entries, gfp_t gfp) >> +{ >> + struct dma_debug_entry *entry, *next_entry; >> + LIST_HEAD(tmp); >> + int i; >> + >> + for (i = 0; i < num_entries; ++i) { >> + entry = kzalloc(sizeof(*entry), gfp); >> + if (!entry) >> + goto out_err; >> + >> + list_add_tail(&entry->list, &tmp); >> + } >> + >> + list_splice(&tmp, &free_entries); >> + num_free_entries += num_entries; >> + nr_total_entries += num_entries; > > The adding to a local list and splicing seems a bit pointless if we > do it all under lock anyway. Either we change the locking in > dma_debug_resize_entries and your upcoming automatic allocation that > we only do it over the splice and counter adjustment, which would > have the advantage of allowing freeing of entries in parallel to these > allocations, or we could just drop the local tmp list. AFAICS the tmp list wasn't about locking as much as meaning that if kzalloc() failed at any point, we can free the partial allocation and back out without disturbing free_entries at all - that still makes sense to me up until patch #8 where we embrace the "never free anything" paradigm and rip out the final traces. That said, maybe I should just drop the refactoring of dma_debug_resize_entries() now that I'm deleting it as part of the same series anyway - then I guess I squash what's left of this patch into #4 and bring forward some of the simplification from #8 to start with. Would that be more agreeable? Robin. ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v2 3/8] dma-debug: Refactor dma_debug_entry allocation 2018-12-06 18:10 ` Robin Murphy @ 2018-12-06 18:46 ` Christoph Hellwig 0 siblings, 0 replies; 19+ messages in thread From: Christoph Hellwig @ 2018-12-06 18:46 UTC (permalink / raw) To: Robin Murphy Cc: Christoph Hellwig, m.szyprowski, iommu, linux-kernel, cai, salil.mehta, john.garry On Thu, Dec 06, 2018 at 06:10:47PM +0000, Robin Murphy wrote: > AFAICS the tmp list wasn't about locking as much as meaning that if > kzalloc() failed at any point, we can free the partial allocation and back > out without disturbing free_entries at all - that still makes sense to me > up until patch #8 where we embrace the "never free anything" paradigm and > rip out the final traces. > > That said, maybe I should just drop the refactoring of > dma_debug_resize_entries() now that I'm deleting it as part of the same > series anyway - then I guess I squash what's left of this patch into #4 and > bring forward some of the simplification from #8 to start with. Would that > be more agreeable? Yes, I just noticed all this goes away toward the end anyway. We can either keep it as is, or just drop the intermediate step if that is easy enough for you. ^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH v2 4/8] dma-debug: Dynamically expand the dma_debug_entry pool 2018-12-05 19:56 [PATCH v2 0/8] dma-debug cleanup and dynamic allocation Robin Murphy ` (2 preceding siblings ...) 2018-12-05 19:56 ` [PATCH v2 3/8] dma-debug: Refactor dma_debug_entry allocation Robin Murphy @ 2018-12-05 19:56 ` Robin Murphy 2018-12-06 14:24 ` Christoph Hellwig 2018-12-05 19:56 ` [PATCH v2 5/8] dma-debug: Make leak-like behaviour apparent Robin Murphy ` (3 subsequent siblings) 7 siblings, 1 reply; 19+ messages in thread From: Robin Murphy @ 2018-12-05 19:56 UTC (permalink / raw) To: hch; +Cc: m.szyprowski, iommu, linux-kernel, cai, salil.mehta, john.garry Certain drivers such as large multi-queue network adapters can use pools of mapped DMA buffers larger than the default dma_debug_entry pool of 65536 entries, with the result that merely probing such a device can cause DMA debug to disable itself during boot unless explicitly given an appropriate "dma_debug_entries=..." option. Developers trying to debug some other driver on such a system may not be immediately aware of this, and at worst it can hide bugs if they fail to realise that dma-debug has already disabled itself unexpectedly by the time the code of interest gets to run. Even once they do realise, it can be a bit of a pain to emprirically determine a suitable number of preallocated entries to configure without massively over-allocating. There's really no need for such a static limit, though, since we can quite easily expand the pool at runtime in those rare cases that the preallocated entries are insufficient, which is arguably the least surprising and most useful behaviour. Signed-off-by: Robin Murphy <robin.murphy@arm.com> --- v2: Use GFP_ATOMIC, make retry loop redundant, update documentation Documentation/DMA-API.txt | 11 +++++------ kernel/dma/debug.c | 15 +++++++++------ 2 files changed, 14 insertions(+), 12 deletions(-) diff --git a/Documentation/DMA-API.txt b/Documentation/DMA-API.txt index 6bdb095393b0..0fcb7561af1e 100644 --- a/Documentation/DMA-API.txt +++ b/Documentation/DMA-API.txt @@ -717,8 +717,8 @@ dma-api/num_errors The number in this file shows how many dma-api/min_free_entries This read-only file can be read to get the minimum number of free dma_debug_entries the allocator has ever seen. If this value goes - down to zero the code will disable itself - because it is not longer reliable. + down to zero the code will attempt to increase + nr_total_entries to compensate. dma-api/num_free_entries The current number of free dma_debug_entries in the allocator. @@ -745,10 +745,9 @@ driver filter at boot time. The debug code will only print errors for that driver afterwards. This filter can be disabled or changed later using debugfs. When the code disables itself at runtime this is most likely because it ran -out of dma_debug_entries. These entries are preallocated at boot. The number -of preallocated entries is defined per architecture. If it is too low for you -boot with 'dma_debug_entries=<your_desired_number>' to overwrite the -architectural default. +out of dma_debug_entries and was unable to allocate more on-demand. 65536 +entries are preallocated at boot - if this is too low for you boot with +'dma_debug_entries=<your_desired_number>' to overwrite the default. :: diff --git a/kernel/dma/debug.c b/kernel/dma/debug.c index 1b0858d7edfd..b1fe8b78acc4 100644 --- a/kernel/dma/debug.c +++ b/kernel/dma/debug.c @@ -47,6 +47,8 @@ #ifndef PREALLOC_DMA_DEBUG_ENTRIES #define PREALLOC_DMA_DEBUG_ENTRIES (1 << 16) #endif +/* If the pool runs out, add this many new entries at once */ +#define DMA_DEBUG_DYNAMIC_ENTRIES 256 enum { dma_debug_single, @@ -702,12 +704,13 @@ static struct dma_debug_entry *dma_entry_alloc(void) unsigned long flags; spin_lock_irqsave(&free_entries_lock, flags); - - if (list_empty(&free_entries)) { - global_disable = true; - spin_unlock_irqrestore(&free_entries_lock, flags); - pr_err("debugging out of memory - disabling\n"); - return NULL; + if (num_free_entries == 0) { + if (dma_debug_create_entries(DMA_DEBUG_DYNAMIC_ENTRIES, GFP_ATOMIC)) { + global_disable = true; + spin_unlock_irqrestore(&free_entries_lock, flags); + pr_err("debugging out of memory - disabling\n"); + return NULL; + } } entry = __dma_entry_alloc(); -- 2.19.1.dirty ^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: [PATCH v2 4/8] dma-debug: Dynamically expand the dma_debug_entry pool 2018-12-05 19:56 ` [PATCH v2 4/8] dma-debug: Dynamically expand the dma_debug_entry pool Robin Murphy @ 2018-12-06 14:24 ` Christoph Hellwig 2018-12-06 18:31 ` Robin Murphy 0 siblings, 1 reply; 19+ messages in thread From: Christoph Hellwig @ 2018-12-06 14:24 UTC (permalink / raw) To: Robin Murphy; +Cc: hch, cai, linux-kernel, iommu > @@ -47,6 +47,8 @@ > #ifndef PREALLOC_DMA_DEBUG_ENTRIES > #define PREALLOC_DMA_DEBUG_ENTRIES (1 << 16) > #endif FYI, I think we should drop the potential arch hook with the ifndef here once we support the dynamic adjustment. > + if (dma_debug_create_entries(DMA_DEBUG_DYNAMIC_ENTRIES, GFP_ATOMIC)) { Overly long line. ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v2 4/8] dma-debug: Dynamically expand the dma_debug_entry pool 2018-12-06 14:24 ` Christoph Hellwig @ 2018-12-06 18:31 ` Robin Murphy 0 siblings, 0 replies; 19+ messages in thread From: Robin Murphy @ 2018-12-06 18:31 UTC (permalink / raw) To: Christoph Hellwig; +Cc: cai, linux-kernel, iommu On 06/12/2018 14:24, Christoph Hellwig wrote: >> @@ -47,6 +47,8 @@ >> #ifndef PREALLOC_DMA_DEBUG_ENTRIES >> #define PREALLOC_DMA_DEBUG_ENTRIES (1 << 16) >> #endif > > FYI, I think we should drop the potential arch hook with the ifndef > here once we support the dynamic adjustment. Sure - I have a vague feeling that thought did cross my mind at the time, but it didn't stick. > >> + if (dma_debug_create_entries(DMA_DEBUG_DYNAMIC_ENTRIES, GFP_ATOMIC)) { > > Overly long line. Bah, I deliberately didn't bother wrapping this since it gets shorter again later, but now I realise it also slipped through the s/create/add/ rebase so it's just busted, and due to another last-minute fix I actually ended up with an even longer line elsewhere. I'll tidy all of that up for a v3. Robin. ^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH v2 5/8] dma-debug: Make leak-like behaviour apparent 2018-12-05 19:56 [PATCH v2 0/8] dma-debug cleanup and dynamic allocation Robin Murphy ` (3 preceding siblings ...) 2018-12-05 19:56 ` [PATCH v2 4/8] dma-debug: Dynamically expand the dma_debug_entry pool Robin Murphy @ 2018-12-05 19:56 ` Robin Murphy 2018-12-06 14:24 ` Christoph Hellwig 2018-12-05 19:56 ` [PATCH v2 6/8] x86/dma/amd-gart: Stop resizing dma_debug_entry pool Robin Murphy ` (2 subsequent siblings) 7 siblings, 1 reply; 19+ messages in thread From: Robin Murphy @ 2018-12-05 19:56 UTC (permalink / raw) To: hch; +Cc: m.szyprowski, iommu, linux-kernel, cai, salil.mehta, john.garry Now that we can dynamically allocate DMA debug entries to cope with drivers maintaining excessively large numbers of live mappings, a driver which *does* actually have a bug leaking mappings (and is not unloaded) will no longer trigger the "DMA-API: debugging out of memory - disabling" message until it gets to actual kernel OOM conditions, which means it could go unnoticed for a while. To that end, let's inform the user each time the pool has grown to a multiple of its initial size, which should make it apparent that they either have a leak or might want to increase the preallocation size. Signed-off-by: Robin Murphy <robin.murphy@arm.com> --- v2: Move the suggestion of a leak from the message to the documentation Documentation/DMA-API.txt | 6 +++++- kernel/dma/debug.c | 13 +++++++++++++ 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/Documentation/DMA-API.txt b/Documentation/DMA-API.txt index 0fcb7561af1e..7a7d8a415ce8 100644 --- a/Documentation/DMA-API.txt +++ b/Documentation/DMA-API.txt @@ -747,7 +747,11 @@ driver afterwards. This filter can be disabled or changed later using debugfs. When the code disables itself at runtime this is most likely because it ran out of dma_debug_entries and was unable to allocate more on-demand. 65536 entries are preallocated at boot - if this is too low for you boot with -'dma_debug_entries=<your_desired_number>' to overwrite the default. +'dma_debug_entries=<your_desired_number>' to overwrite the default. The +code will print to the kernel log each time it has dynamically allocated +as many entries as were initially preallocated. This is to indicate that a +larger preallocation size may be appropriate, or if it happens continually +that a driver may be leaking mappings. :: diff --git a/kernel/dma/debug.c b/kernel/dma/debug.c index b1fe8b78acc4..cd9a12bf6a3a 100644 --- a/kernel/dma/debug.c +++ b/kernel/dma/debug.c @@ -693,6 +693,18 @@ static struct dma_debug_entry *__dma_entry_alloc(void) return entry; } +void __dma_entry_alloc_check_leak(void) +{ + u32 tmp = nr_total_entries % nr_prealloc_entries; + + /* Shout each time we tick over some multiple of the initial pool */ + if (tmp < DMA_DEBUG_DYNAMIC_ENTRIES) { + pr_info("dma_debug_entry pool grown to %u (%u00%%)\n", + nr_total_entries, + (nr_total_entries / nr_prealloc_entries)); + } +} + /* struct dma_entry allocator * * The next two functions implement the allocator for @@ -711,6 +723,7 @@ static struct dma_debug_entry *dma_entry_alloc(void) pr_err("debugging out of memory - disabling\n"); return NULL; } + __dma_entry_alloc_check_leak(); } entry = __dma_entry_alloc(); -- 2.19.1.dirty ^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: [PATCH v2 5/8] dma-debug: Make leak-like behaviour apparent 2018-12-05 19:56 ` [PATCH v2 5/8] dma-debug: Make leak-like behaviour apparent Robin Murphy @ 2018-12-06 14:24 ` Christoph Hellwig 0 siblings, 0 replies; 19+ messages in thread From: Christoph Hellwig @ 2018-12-06 14:24 UTC (permalink / raw) To: Robin Murphy Cc: hch, m.szyprowski, iommu, linux-kernel, cai, salil.mehta, john.garry Looks good, Reviewed-by: Christoph Hellwig <hch@lst.de> ^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH v2 6/8] x86/dma/amd-gart: Stop resizing dma_debug_entry pool 2018-12-05 19:56 [PATCH v2 0/8] dma-debug cleanup and dynamic allocation Robin Murphy ` (4 preceding siblings ...) 2018-12-05 19:56 ` [PATCH v2 5/8] dma-debug: Make leak-like behaviour apparent Robin Murphy @ 2018-12-05 19:56 ` Robin Murphy 2018-12-06 14:25 ` Christoph Hellwig 2018-12-05 19:56 ` [PATCH v2 7/8] dma/debug: Remove dma_debug_resize_entries() Robin Murphy 2018-12-05 19:56 ` [PATCH v2 8/8] dma-debug: Batch dma_debug_entry allocation Robin Murphy 7 siblings, 1 reply; 19+ messages in thread From: Robin Murphy @ 2018-12-05 19:56 UTC (permalink / raw) To: hch Cc: m.szyprowski, iommu, linux-kernel, cai, salil.mehta, john.garry, Thomas Gleixner, Ingo Molnar, Borislav Petkov, H. Peter Anvin, x86 dma-debug is now capable of adding new entries to its pool on-demand if the initial preallocation was insufficient, so the IOMMU_LEAK logic no longer needs to explicitly change the pool size. This does lose it the ability to save a couple of megabytes of RAM by reducing the pool size below its default, but it seems unlikely that that is a realistic concern these days (or indeed that anyone is actively debugging AGP drivers' DMA usage any more). Getting rid of dma_debug_resize_entries() will make room for further streamlining in the dma-debug code itself. Removing the call reveals quite a lot of cruft which has been useless for nearly a decade since commit 19c1a6f5764d ("x86 gart: reimplement IOMMU_LEAK feature by using DMA_API_DEBUG"), including the entire 'iommu=leak' parameter, which controlled nothing except whether dma_debug_resize_entries() was called or not. CC: Thomas Gleixner <tglx@linutronix.de> CC: Ingo Molnar <mingo@redhat.com> CC: Borislav Petkov <bp@alien8.de> CC: "H. Peter Anvin" <hpa@zytor.com> CC: x86@kernel.org Signed-off-by: Robin Murphy <robin.murphy@arm.com> --- v2: New Documentation/x86/x86_64/boot-options.txt | 5 +---- arch/x86/kernel/amd_gart_64.c | 23 ----------------------- 2 files changed, 1 insertion(+), 27 deletions(-) diff --git a/Documentation/x86/x86_64/boot-options.txt b/Documentation/x86/x86_64/boot-options.txt index ad6d2a80cf05..abc53886655e 100644 --- a/Documentation/x86/x86_64/boot-options.txt +++ b/Documentation/x86/x86_64/boot-options.txt @@ -209,7 +209,7 @@ IOMMU (input/output memory management unit) mapping with memory protection, etc. Kernel boot message: "PCI-DMA: Using Calgary IOMMU" - iommu=[<size>][,noagp][,off][,force][,noforce][,leak[=<nr_of_leak_pages>] + iommu=[<size>][,noagp][,off][,force][,noforce] [,memaper[=<order>]][,merge][,fullflush][,nomerge] [,noaperture][,calgary] @@ -228,9 +228,6 @@ IOMMU (input/output memory management unit) allowed Overwrite iommu off workarounds for specific chipsets. fullflush Flush IOMMU on each allocation (default). nofullflush Don't use IOMMU fullflush. - leak Turn on simple iommu leak tracing (only when - CONFIG_IOMMU_LEAK is on). Default number of leak pages - is 20. memaper[=<order>] Allocate an own aperture over RAM with size 32MB<<order. (default: order=1, i.e. 64MB) merge Do scatter-gather (SG) merging. Implies "force" diff --git a/arch/x86/kernel/amd_gart_64.c b/arch/x86/kernel/amd_gart_64.c index 3f9d1b4019bb..aa7706bf4d25 100644 --- a/arch/x86/kernel/amd_gart_64.c +++ b/arch/x86/kernel/amd_gart_64.c @@ -155,9 +155,6 @@ static void flush_gart(void) #ifdef CONFIG_IOMMU_LEAK /* Debugging aid for drivers that don't free their IOMMU tables */ -static int leak_trace; -static int iommu_leak_pages = 20; - static void dump_leak(void) { static int dump; @@ -774,16 +771,6 @@ int __init gart_iommu_init(void) if (!iommu_gart_bitmap) panic("Cannot allocate iommu bitmap\n"); -#ifdef CONFIG_IOMMU_LEAK - if (leak_trace) { - int ret; - - ret = dma_debug_resize_entries(iommu_pages); - if (ret) - pr_debug("PCI-DMA: Cannot trace all the entries\n"); - } -#endif - /* * Out of IOMMU space handling. * Reserve some invalid pages at the beginning of the GART. @@ -853,16 +840,6 @@ void __init gart_parse_options(char *p) { int arg; -#ifdef CONFIG_IOMMU_LEAK - if (!strncmp(p, "leak", 4)) { - leak_trace = 1; - p += 4; - if (*p == '=') - ++p; - if (isdigit(*p) && get_option(&p, &arg)) - iommu_leak_pages = arg; - } -#endif if (isdigit(*p) && get_option(&p, &arg)) iommu_size = arg; if (!strncmp(p, "fullflush", 9)) -- 2.19.1.dirty ^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: [PATCH v2 6/8] x86/dma/amd-gart: Stop resizing dma_debug_entry pool 2018-12-05 19:56 ` [PATCH v2 6/8] x86/dma/amd-gart: Stop resizing dma_debug_entry pool Robin Murphy @ 2018-12-06 14:25 ` Christoph Hellwig 0 siblings, 0 replies; 19+ messages in thread From: Christoph Hellwig @ 2018-12-06 14:25 UTC (permalink / raw) To: Robin Murphy Cc: hch, m.szyprowski, iommu, linux-kernel, cai, salil.mehta, john.garry, Thomas Gleixner, Ingo Molnar, Borislav Petkov, H. Peter Anvin, x86 Looks good, Reviewed-by: Christoph Hellwig <hch@lst.de> ^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH v2 7/8] dma/debug: Remove dma_debug_resize_entries() 2018-12-05 19:56 [PATCH v2 0/8] dma-debug cleanup and dynamic allocation Robin Murphy ` (5 preceding siblings ...) 2018-12-05 19:56 ` [PATCH v2 6/8] x86/dma/amd-gart: Stop resizing dma_debug_entry pool Robin Murphy @ 2018-12-05 19:56 ` Robin Murphy 2018-12-06 14:25 ` Christoph Hellwig 2018-12-05 19:56 ` [PATCH v2 8/8] dma-debug: Batch dma_debug_entry allocation Robin Murphy 7 siblings, 1 reply; 19+ messages in thread From: Robin Murphy @ 2018-12-05 19:56 UTC (permalink / raw) To: hch; +Cc: m.szyprowski, iommu, linux-kernel, cai, salil.mehta, john.garry With no callers left, we can clean up this part of dma-debug's exposed internals, making way to tweak the allocation behaviour. Signed-off-by: Robin Murphy <robin.murphy@arm.com> --- v2: New include/linux/dma-debug.h | 7 ------- kernel/dma/debug.c | 30 ------------------------------ 2 files changed, 37 deletions(-) diff --git a/include/linux/dma-debug.h b/include/linux/dma-debug.h index 30213adbb6b9..46e6131a72b6 100644 --- a/include/linux/dma-debug.h +++ b/include/linux/dma-debug.h @@ -30,8 +30,6 @@ struct bus_type; extern void dma_debug_add_bus(struct bus_type *bus); -extern int dma_debug_resize_entries(u32 num_entries); - extern void debug_dma_map_single(struct device *dev, const void *addr, unsigned long len); @@ -101,11 +99,6 @@ static inline void dma_debug_add_bus(struct bus_type *bus) { } -static inline int dma_debug_resize_entries(u32 num_entries) -{ - return 0; -} - static inline void debug_dma_map_single(struct device *dev, const void *addr, unsigned long len) { diff --git a/kernel/dma/debug.c b/kernel/dma/debug.c index cd9a12bf6a3a..2202402afe9a 100644 --- a/kernel/dma/debug.c +++ b/kernel/dma/debug.c @@ -756,36 +756,6 @@ static void dma_entry_free(struct dma_debug_entry *entry) spin_unlock_irqrestore(&free_entries_lock, flags); } -int dma_debug_resize_entries(u32 num_entries) -{ - int i, delta, ret = 0; - unsigned long flags; - struct dma_debug_entry *entry; - - spin_lock_irqsave(&free_entries_lock, flags); - - if (nr_total_entries < num_entries) { - delta = num_entries - nr_total_entries; - - ret = dma_debug_add_entries(delta, GFP_ATOMIC); - } else { - delta = nr_total_entries - num_entries; - - for (i = 0; i < delta && !list_empty(&free_entries); i++) { - entry = __dma_entry_alloc(); - kfree(entry); - } - - nr_total_entries -= i; - if (nr_total_entries != num_entries) - ret = -EBUSY; - } - - spin_unlock_irqrestore(&free_entries_lock, flags); - - return ret; -} - /* * DMA-API debugging init code * -- 2.19.1.dirty ^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: [PATCH v2 7/8] dma/debug: Remove dma_debug_resize_entries() 2018-12-05 19:56 ` [PATCH v2 7/8] dma/debug: Remove dma_debug_resize_entries() Robin Murphy @ 2018-12-06 14:25 ` Christoph Hellwig 0 siblings, 0 replies; 19+ messages in thread From: Christoph Hellwig @ 2018-12-06 14:25 UTC (permalink / raw) To: Robin Murphy Cc: hch, m.szyprowski, iommu, linux-kernel, cai, salil.mehta, john.garry On Wed, Dec 05, 2018 at 07:56:56PM +0000, Robin Murphy wrote: > With no callers left, we can clean up this part of dma-debug's exposed > internals, making way to tweak the allocation behaviour. > > Signed-off-by: Robin Murphy <robin.murphy@arm.com> Looks good, Reviewed-by: Christoph Hellwig <hch@lst.de> ^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH v2 8/8] dma-debug: Batch dma_debug_entry allocation 2018-12-05 19:56 [PATCH v2 0/8] dma-debug cleanup and dynamic allocation Robin Murphy ` (6 preceding siblings ...) 2018-12-05 19:56 ` [PATCH v2 7/8] dma/debug: Remove dma_debug_resize_entries() Robin Murphy @ 2018-12-05 19:56 ` Robin Murphy 2018-12-06 14:27 ` Christoph Hellwig 7 siblings, 1 reply; 19+ messages in thread From: Robin Murphy @ 2018-12-05 19:56 UTC (permalink / raw) To: hch; +Cc: m.szyprowski, iommu, linux-kernel, cai, salil.mehta, john.garry DMA debug entries are one of those things which aren't that useful individually - we will always want some larger quantity of them - and that we don't really need to manage the exact number of - we only care about having 'enough'. In that regard, the current behaviour of creating them one-by-one from the slab allocator means an awful lot of function call overhead and memory wasted on alignment padding. Now that we don't have to worry about freeing anything via dma_debug_resize_entries(), we can optimise the allocation behaviour by grabbing whole pages at once, which will save considerably on the aforementioned overheads, and probably offer a little more cache/TLB locality benefit for traversing the lists under normal operation. Since freeing a whole page of entries at once becomes enough of a challenge that it's not really worth complicating dma_debug_init(), we may as well tweak the preallocation behaviour so that as long as we manage to allocate *some* pages, we can leave debugging enabled on a best-effort basis rather than otherwise wasting them. Signed-off-by: Robin Murphy <robin.murphy@arm.com> --- v2: New Documentation/DMA-API.txt | 4 +++- kernel/dma/debug.c | 45 +++++++++++++++++---------------------- 2 files changed, 22 insertions(+), 27 deletions(-) diff --git a/Documentation/DMA-API.txt b/Documentation/DMA-API.txt index 7a7d8a415ce8..097c51b79330 100644 --- a/Documentation/DMA-API.txt +++ b/Documentation/DMA-API.txt @@ -747,7 +747,9 @@ driver afterwards. This filter can be disabled or changed later using debugfs. When the code disables itself at runtime this is most likely because it ran out of dma_debug_entries and was unable to allocate more on-demand. 65536 entries are preallocated at boot - if this is too low for you boot with -'dma_debug_entries=<your_desired_number>' to overwrite the default. The +'dma_debug_entries=<your_desired_number>' to overwrite the default. Note +that the code allocates entires in batches, so the exact number of +preallocated entries may be greater than the actual number requested. The code will print to the kernel log each time it has dynamically allocated as many entries as were initially preallocated. This is to indicate that a larger preallocation size may be appropriate, or if it happens continually diff --git a/kernel/dma/debug.c b/kernel/dma/debug.c index 2202402afe9a..a6a603526c8f 100644 --- a/kernel/dma/debug.c +++ b/kernel/dma/debug.c @@ -48,7 +48,7 @@ #define PREALLOC_DMA_DEBUG_ENTRIES (1 << 16) #endif /* If the pool runs out, add this many new entries at once */ -#define DMA_DEBUG_DYNAMIC_ENTRIES 256 +#define DMA_DEBUG_DYNAMIC_ENTRIES (PAGE_SIZE / sizeof(struct dma_debug_entry)) enum { dma_debug_single, @@ -648,34 +648,22 @@ static void add_dma_entry(struct dma_debug_entry *entry) */ } -static int dma_debug_add_entries(u32 num_entries, gfp_t gfp) +static int dma_debug_add_entries(gfp_t gfp) { - struct dma_debug_entry *entry, *next_entry; - LIST_HEAD(tmp); + struct dma_debug_entry *entry; int i; - for (i = 0; i < num_entries; ++i) { - entry = kzalloc(sizeof(*entry), gfp); - if (!entry) - goto out_err; + entry = (void *)get_zeroed_page(gfp); + if (!entry) + return -ENOMEM; - list_add_tail(&entry->list, &tmp); - } + for (i = 0; i < DMA_DEBUG_DYNAMIC_ENTRIES; i++) + list_add_tail(&entry[i].list, &free_entries); - list_splice(&tmp, &free_entries); - num_free_entries += num_entries; - nr_total_entries += num_entries; + num_free_entries += DMA_DEBUG_DYNAMIC_ENTRIES; + nr_total_entries += DMA_DEBUG_DYNAMIC_ENTRIES; return 0; - -out_err: - - list_for_each_entry_safe(entry, next_entry, &tmp, list) { - list_del(&entry->list); - kfree(entry); - } - - return -ENOMEM; } static struct dma_debug_entry *__dma_entry_alloc(void) @@ -717,7 +705,7 @@ static struct dma_debug_entry *dma_entry_alloc(void) spin_lock_irqsave(&free_entries_lock, flags); if (num_free_entries == 0) { - if (dma_debug_create_entries(DMA_DEBUG_DYNAMIC_ENTRIES, GFP_ATOMIC)) { + if (dma_debug_add_entries(GFP_ATOMIC)) { global_disable = true; spin_unlock_irqrestore(&free_entries_lock, flags); pr_err("debugging out of memory - disabling\n"); @@ -1008,15 +996,20 @@ static int dma_debug_init(void) return 0; } - if (dma_debug_add_entries(nr_prealloc_entries, GFP_KERNEL) != 0) { + for (i = 0; i < DIV_ROUND_UP(nr_prealloc_entries, DMA_DEBUG_DYNAMIC_ENTRIES); ++i) + dma_debug_add_entries(GFP_KERNEL); + if (num_free_entries >= nr_prealloc_entries) { + pr_info("preallocated %d debug entries\n", nr_total_entries); + } else if (num_free_entries > 0) { + pr_warn("%d debug entries requested but only %d allocated\n", + nr_prealloc_entries, nr_total_entries); + } else { pr_err("debugging out of memory error - disabled\n"); global_disable = true; return 0; } - min_free_entries = num_free_entries; - pr_info("preallocated %d debug entries\n", nr_total_entries); dma_debug_initialized = true; -- 2.19.1.dirty ^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: [PATCH v2 8/8] dma-debug: Batch dma_debug_entry allocation 2018-12-05 19:56 ` [PATCH v2 8/8] dma-debug: Batch dma_debug_entry allocation Robin Murphy @ 2018-12-06 14:27 ` Christoph Hellwig 0 siblings, 0 replies; 19+ messages in thread From: Christoph Hellwig @ 2018-12-06 14:27 UTC (permalink / raw) To: Robin Murphy Cc: hch, m.szyprowski, iommu, linux-kernel, cai, salil.mehta, john.garry This looks very nice, thanks! Reviewed-by: Christoph Hellwig <hch@lst.de> ^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2018-12-06 18:46 UTC | newest] Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2018-12-05 19:56 [PATCH v2 0/8] dma-debug cleanup and dynamic allocation Robin Murphy 2018-12-05 19:56 ` [PATCH v2 1/8] dma-debug: Use pr_fmt() Robin Murphy 2018-12-05 19:56 ` [PATCH v2 2/8] dma-debug: Expose nr_total_entries in debugfs Robin Murphy 2018-12-06 14:14 ` Christoph Hellwig 2018-12-05 19:56 ` [PATCH v2 3/8] dma-debug: Refactor dma_debug_entry allocation Robin Murphy 2018-12-06 14:23 ` Christoph Hellwig 2018-12-06 18:10 ` Robin Murphy 2018-12-06 18:46 ` Christoph Hellwig 2018-12-05 19:56 ` [PATCH v2 4/8] dma-debug: Dynamically expand the dma_debug_entry pool Robin Murphy 2018-12-06 14:24 ` Christoph Hellwig 2018-12-06 18:31 ` Robin Murphy 2018-12-05 19:56 ` [PATCH v2 5/8] dma-debug: Make leak-like behaviour apparent Robin Murphy 2018-12-06 14:24 ` Christoph Hellwig 2018-12-05 19:56 ` [PATCH v2 6/8] x86/dma/amd-gart: Stop resizing dma_debug_entry pool Robin Murphy 2018-12-06 14:25 ` Christoph Hellwig 2018-12-05 19:56 ` [PATCH v2 7/8] dma/debug: Remove dma_debug_resize_entries() Robin Murphy 2018-12-06 14:25 ` Christoph Hellwig 2018-12-05 19:56 ` [PATCH v2 8/8] dma-debug: Batch dma_debug_entry allocation Robin Murphy 2018-12-06 14:27 ` Christoph Hellwig
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).