* [PATCH 1/3] rbtree: include rcu.h because we use it
@ 2017-06-27 16:16 Sebastian Andrzej Siewior
2017-06-27 16:16 ` [PATCH 2/3] iommu/iova: don't disable preempt around this_cpu_ptr() Sebastian Andrzej Siewior
2017-06-27 16:16 ` [PATCH 3/3] iommu/vt-d: don't disable preemption while accessing deferred_flush() Sebastian Andrzej Siewior
0 siblings, 2 replies; 6+ messages in thread
From: Sebastian Andrzej Siewior @ 2017-06-27 16:16 UTC (permalink / raw)
To: tglx
Cc: linux-kernel, Sebastian Andrzej Siewior, David Howells, Andrew Morton
Since commit c1adf20052d8 ("Introduce rb_replace_node_rcu()")
rbtree_augmented.h uses RCU related data structures but does not include
them. It works as long as gets somehow included before that and fails
otherwise.
Noticed during -RT rebase.
Cc: David Howells <dhowells@redhat.com
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
include/linux/rbtree_augmented.h | 1 +
include/linux/rbtree_latch.h | 1 +
2 files changed, 2 insertions(+)
diff --git a/include/linux/rbtree_augmented.h b/include/linux/rbtree_augmented.h
index 9702b6e183bc..110e14060dc9 100644
--- a/include/linux/rbtree_augmented.h
+++ b/include/linux/rbtree_augmented.h
@@ -26,6 +26,7 @@
#include <linux/compiler.h>
#include <linux/rbtree.h>
+#include <linux/rcupdate.h>
/*
* Please note - only struct rb_augment_callbacks and the prototypes for
diff --git a/include/linux/rbtree_latch.h b/include/linux/rbtree_latch.h
index 4f3432c61d12..0a7762cf04f7 100644
--- a/include/linux/rbtree_latch.h
+++ b/include/linux/rbtree_latch.h
@@ -34,6 +34,7 @@
#include <linux/rbtree.h>
#include <linux/seqlock.h>
+#include <linux/rcupdate.h>
struct latch_tree_node {
struct rb_node node[2];
--
2.13.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 2/3] iommu/iova: don't disable preempt around this_cpu_ptr()
2017-06-27 16:16 [PATCH 1/3] rbtree: include rcu.h because we use it Sebastian Andrzej Siewior
@ 2017-06-27 16:16 ` Sebastian Andrzej Siewior
2017-06-28 9:22 ` Joerg Roedel
2017-06-27 16:16 ` [PATCH 3/3] iommu/vt-d: don't disable preemption while accessing deferred_flush() Sebastian Andrzej Siewior
1 sibling, 1 reply; 6+ messages in thread
From: Sebastian Andrzej Siewior @ 2017-06-27 16:16 UTC (permalink / raw)
To: tglx
Cc: linux-kernel, Sebastian Andrzej Siewior, Joerg Roedel, iommu,
Andrew Morton
Commit 583248e6620a ("iommu/iova: Disable preemption around use of
this_cpu_ptr()") disables preemption while accessing a per-CPU variable.
This does keep lockdep quiet. However I don't see the point why it is
bad if we get migrated after its access to another CPU.
__iova_rcache_insert() and __iova_rcache_get() immediately locks the
variable after obtaining it - before accessing its members.
_If_ we get migrated away after retrieving the address of cpu_rcache
before taking the lock then the *other* task on the same CPU will
retrieve the same address of cpu_rcache and will spin on the lock.
alloc_iova_fast() disables preemption while invoking
free_cpu_cached_iovas() on each CPU. The function itself uses
per_cpu_ptr() which does not trigger a warning (like this_cpu_ptr()
does). It _could_ make sense to use get_online_cpus() instead but the we
have a hotplug notifier for CPU down (and none for up) so we are good.
Cc: Joerg Roedel <joro@8bytes.org>
Cc: iommu@lists.linux-foundation.org
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
drivers/iommu/iova.c | 9 +++------
1 file changed, 3 insertions(+), 6 deletions(-)
diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c
index 5c88ba70e4e0..f0ff0aa04081 100644
--- a/drivers/iommu/iova.c
+++ b/drivers/iommu/iova.c
@@ -22,6 +22,7 @@
#include <linux/slab.h>
#include <linux/smp.h>
#include <linux/bitops.h>
+#include <linux/cpu.h>
static bool iova_rcache_insert(struct iova_domain *iovad,
unsigned long pfn,
@@ -398,10 +399,8 @@ alloc_iova_fast(struct iova_domain *iovad, unsigned long size,
/* Try replenishing IOVAs by flushing rcache. */
flushed_rcache = true;
- preempt_disable();
for_each_online_cpu(cpu)
free_cpu_cached_iovas(cpu, iovad);
- preempt_enable();
goto retry;
}
@@ -729,7 +728,7 @@ static bool __iova_rcache_insert(struct iova_domain *iovad,
bool can_insert = false;
unsigned long flags;
- cpu_rcache = get_cpu_ptr(rcache->cpu_rcaches);
+ cpu_rcache = raw_cpu_ptr(rcache->cpu_rcaches);
spin_lock_irqsave(&cpu_rcache->lock, flags);
if (!iova_magazine_full(cpu_rcache->loaded)) {
@@ -759,7 +758,6 @@ static bool __iova_rcache_insert(struct iova_domain *iovad,
iova_magazine_push(cpu_rcache->loaded, iova_pfn);
spin_unlock_irqrestore(&cpu_rcache->lock, flags);
- put_cpu_ptr(rcache->cpu_rcaches);
if (mag_to_free) {
iova_magazine_free_pfns(mag_to_free, iovad);
@@ -793,7 +791,7 @@ static unsigned long __iova_rcache_get(struct iova_rcache *rcache,
bool has_pfn = false;
unsigned long flags;
- cpu_rcache = get_cpu_ptr(rcache->cpu_rcaches);
+ cpu_rcache = raw_cpu_ptr(rcache->cpu_rcaches);
spin_lock_irqsave(&cpu_rcache->lock, flags);
if (!iova_magazine_empty(cpu_rcache->loaded)) {
@@ -815,7 +813,6 @@ static unsigned long __iova_rcache_get(struct iova_rcache *rcache,
iova_pfn = iova_magazine_pop(cpu_rcache->loaded, limit_pfn);
spin_unlock_irqrestore(&cpu_rcache->lock, flags);
- put_cpu_ptr(rcache->cpu_rcaches);
return iova_pfn;
}
--
2.13.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 3/3] iommu/vt-d: don't disable preemption while accessing deferred_flush()
2017-06-27 16:16 [PATCH 1/3] rbtree: include rcu.h because we use it Sebastian Andrzej Siewior
2017-06-27 16:16 ` [PATCH 2/3] iommu/iova: don't disable preempt around this_cpu_ptr() Sebastian Andrzej Siewior
@ 2017-06-27 16:16 ` Sebastian Andrzej Siewior
1 sibling, 0 replies; 6+ messages in thread
From: Sebastian Andrzej Siewior @ 2017-06-27 16:16 UTC (permalink / raw)
To: tglx
Cc: linux-kernel, Sebastian Andrzej Siewior, David Woodhouse,
Joerg Roedel, iommu, Andrew Morton
get_cpu() disables preemption and returns the current CPU number. The
CPU number is only used once while retrieving the address of the local's
CPU deferred_flush pointer.
We can instead use raw_cpu_ptr() while we remain preemptible. The worst
thing that can happen is that flush_unmaps_timeout() is invoked multiple
times: once by taskA after seeing HIGH_WATER_MARK and then preempted to
another CPU and then by taskB which saw HIGH_WATER_MARK on the same CPU
as taskA. It is also likely that ->size got from HIGH_WATER_MARK to 0
right after its read because another CPU invoked flush_unmaps_timeout()
for this CPU.
The access to flush_data is protected by a spinlock so even if we get
migrated to another CPU or preempted - the data structure is protected.
While at it, I marked deferred_flush static since I can't find a
reference to it outside of this file.
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: iommu@lists.linux-foundation.org
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
drivers/iommu/intel-iommu.c | 8 ++------
1 file changed, 2 insertions(+), 6 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 8500deda9175..1c7118d1525e 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -481,7 +481,7 @@ struct deferred_flush_data {
struct deferred_flush_table *tables;
};
-DEFINE_PER_CPU(struct deferred_flush_data, deferred_flush);
+static DEFINE_PER_CPU(struct deferred_flush_data, deferred_flush);
/* bitmap for indexing intel_iommus */
static int g_num_of_iommus;
@@ -3725,10 +3725,8 @@ static void add_unmap(struct dmar_domain *dom, unsigned long iova_pfn,
struct intel_iommu *iommu;
struct deferred_flush_entry *entry;
struct deferred_flush_data *flush_data;
- unsigned int cpuid;
- cpuid = get_cpu();
- flush_data = per_cpu_ptr(&deferred_flush, cpuid);
+ flush_data = raw_cpu_ptr(&deferred_flush);
/* Flush all CPUs' entries to avoid deferring too much. If
* this becomes a bottleneck, can just flush us, and rely on
@@ -3761,8 +3759,6 @@ static void add_unmap(struct dmar_domain *dom, unsigned long iova_pfn,
}
flush_data->size++;
spin_unlock_irqrestore(&flush_data->lock, flags);
-
- put_cpu();
}
static void intel_unmap(struct device *dev, dma_addr_t dev_addr, size_t size)
--
2.13.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH 2/3] iommu/iova: don't disable preempt around this_cpu_ptr()
2017-06-27 16:16 ` [PATCH 2/3] iommu/iova: don't disable preempt around this_cpu_ptr() Sebastian Andrzej Siewior
@ 2017-06-28 9:22 ` Joerg Roedel
2017-06-28 9:31 ` Sebastian Andrzej Siewior
0 siblings, 1 reply; 6+ messages in thread
From: Joerg Roedel @ 2017-06-28 9:22 UTC (permalink / raw)
To: Sebastian Andrzej Siewior; +Cc: tglx, linux-kernel, iommu, Andrew Morton
On Tue, Jun 27, 2017 at 06:16:47PM +0200, Sebastian Andrzej Siewior wrote:
> Commit 583248e6620a ("iommu/iova: Disable preemption around use of
> this_cpu_ptr()") disables preemption while accessing a per-CPU variable.
> This does keep lockdep quiet. However I don't see the point why it is
> bad if we get migrated after its access to another CPU.
> __iova_rcache_insert() and __iova_rcache_get() immediately locks the
> variable after obtaining it - before accessing its members.
> _If_ we get migrated away after retrieving the address of cpu_rcache
> before taking the lock then the *other* task on the same CPU will
> retrieve the same address of cpu_rcache and will spin on the lock.
>
> alloc_iova_fast() disables preemption while invoking
> free_cpu_cached_iovas() on each CPU. The function itself uses
> per_cpu_ptr() which does not trigger a warning (like this_cpu_ptr()
> does). It _could_ make sense to use get_online_cpus() instead but the we
> have a hotplug notifier for CPU down (and none for up) so we are good.
Does that really matter? The spin_lock disables irqs and thus avoids
preemption too. We also can't get rid of the irqsave lock here because
these locks are taken in the dma-api path which is used from interrupt
context.
Joerg
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 2/3] iommu/iova: don't disable preempt around this_cpu_ptr()
2017-06-28 9:22 ` Joerg Roedel
@ 2017-06-28 9:31 ` Sebastian Andrzej Siewior
2017-06-28 10:26 ` Joerg Roedel
0 siblings, 1 reply; 6+ messages in thread
From: Sebastian Andrzej Siewior @ 2017-06-28 9:31 UTC (permalink / raw)
To: Joerg Roedel; +Cc: tglx, linux-kernel, iommu, Andrew Morton
On 2017-06-28 11:22:05 [+0200], Joerg Roedel wrote:
> On Tue, Jun 27, 2017 at 06:16:47PM +0200, Sebastian Andrzej Siewior wrote:
> > Commit 583248e6620a ("iommu/iova: Disable preemption around use of
> > this_cpu_ptr()") disables preemption while accessing a per-CPU variable.
> > This does keep lockdep quiet. However I don't see the point why it is
> > bad if we get migrated after its access to another CPU.
> > __iova_rcache_insert() and __iova_rcache_get() immediately locks the
> > variable after obtaining it - before accessing its members.
> > _If_ we get migrated away after retrieving the address of cpu_rcache
> > before taking the lock then the *other* task on the same CPU will
> > retrieve the same address of cpu_rcache and will spin on the lock.
> >
> > alloc_iova_fast() disables preemption while invoking
> > free_cpu_cached_iovas() on each CPU. The function itself uses
> > per_cpu_ptr() which does not trigger a warning (like this_cpu_ptr()
> > does). It _could_ make sense to use get_online_cpus() instead but the we
> > have a hotplug notifier for CPU down (and none for up) so we are good.
>
> Does that really matter? The spin_lock disables irqs and thus avoids
> preemption too. We also can't get rid of the irqsave lock here because
> these locks are taken in the dma-api path which is used from interrupt
> context.
It really does. The spin_lock() does disable preemption but this is not
the problem. The thing is that the preempt_disable() is superfluous and
it hurts Preempt-RT (and this is how I noticed it). Also the
get_cpu_ptr() is not requited and was only added to keep lockdep quiet
(according to the history).
Everything else here can stay as-is, I am just asking for the removal of
the redundant preempt_disable() where it is not required.
> Joerg
Sebastian
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 2/3] iommu/iova: don't disable preempt around this_cpu_ptr()
2017-06-28 9:31 ` Sebastian Andrzej Siewior
@ 2017-06-28 10:26 ` Joerg Roedel
0 siblings, 0 replies; 6+ messages in thread
From: Joerg Roedel @ 2017-06-28 10:26 UTC (permalink / raw)
To: Sebastian Andrzej Siewior; +Cc: tglx, linux-kernel, iommu, Andrew Morton
On Wed, Jun 28, 2017 at 11:31:55AM +0200, Sebastian Andrzej Siewior wrote:
> It really does. The spin_lock() does disable preemption but this is not
> the problem. The thing is that the preempt_disable() is superfluous and
> it hurts Preempt-RT (and this is how I noticed it). Also the
> get_cpu_ptr() is not requited and was only added to keep lockdep quiet
> (according to the history).
> Everything else here can stay as-is, I am just asking for the removal of
> the redundant preempt_disable() where it is not required.
Okay, makes sense, I applied both patches.
Thanks,
Joerg
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2017-06-28 10:26 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-06-27 16:16 [PATCH 1/3] rbtree: include rcu.h because we use it Sebastian Andrzej Siewior
2017-06-27 16:16 ` [PATCH 2/3] iommu/iova: don't disable preempt around this_cpu_ptr() Sebastian Andrzej Siewior
2017-06-28 9:22 ` Joerg Roedel
2017-06-28 9:31 ` Sebastian Andrzej Siewior
2017-06-28 10:26 ` Joerg Roedel
2017-06-27 16:16 ` [PATCH 3/3] iommu/vt-d: don't disable preemption while accessing deferred_flush() Sebastian Andrzej Siewior
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).