[PATCH] x86/mm: Add barriers and document switch_mm()-vs-flush synchronization follow-up

[PATCH] x86/mm: Add barriers and document switch_mm()-vs-flush synchronization follow-up

[Rafael Aquini]

While backporting 71b3c126e611 ("x86/mm: Add barriers and document switch_mm()-vs-flush synchronization")
we stumbled across a possibly missing barrier at flush_tlb_page().

Following the reasoning presented while introducing the synchronization
barrier at flush_tlb_mm_range(), for the current->active_mm != mm checkpoint:

        if (current->active_mm != mm) {
                /* Synchronize with switch_mm. */
                smp_mb();

                goto out;
        }

it suggests the same barrier should be introduced for the similar
outcome at flush_tlb_page(). This patch add that mentioned missing
barrier and documents its case.

Suggested-by: Andrea Arcangeli <aarcange@redhat.com>
Signed-off-by: Rafael Aquini <aquini@redhat.com>
---
 arch/x86/mm/tlb.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c
index 4dbe656..3b4addc 100644
--- a/arch/x86/mm/tlb.c
+++ b/arch/x86/mm/tlb.c
@@ -375,6 +375,12 @@ void flush_tlb_page(struct vm_area_struct *vma, unsigned long start)
 			/* Synchronize with switch_mm. */
 			smp_mb();
 		}
+	} else {
+		/*
+		 * current->active_mm != mm
+		 * Synchronize with switch_mm.
+		 */
+		smp_mb();
 	}
 
 	if (cpumask_any_but(mm_cpumask(mm), smp_processor_id()) < nr_cpu_ids)
-- 
2.5.5

[PATCH] x86/mm: Add barriers and document switch_mm()-vs-flush synchronization follow-up

[Rafael Aquini]

While backporting 71b3c126e611 ("x86/mm: Add barriers and document switch_mm()-vs-flush synchronization")
we stumbled across a possibly missing barrier at flush_tlb_page().

Following the reasoning presented while introducing the synchronization
barrier at flush_tlb_mm_range(), for the current->active_mm != mm checkpoint:

        if (current->active_mm != mm) {
                /* Synchronize with switch_mm. */
                smp_mb();

                goto out;
        }

it suggests the same barrier should be introduced for the similar
outcome at flush_tlb_page(). This patch add that mentioned missing
barrier and documents its case.

Suggested-by: Andrea Arcangeli <aarcange@redhat.com>
Signed-off-by: Rafael Aquini <aquini@redhat.com>
---
 arch/x86/mm/tlb.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c
index 4dbe656..3b4addc 100644
--- a/arch/x86/mm/tlb.c
+++ b/arch/x86/mm/tlb.c
@@ -375,6 +375,12 @@ void flush_tlb_page(struct vm_area_struct *vma, unsigned long start)
 			/* Synchronize with switch_mm. */
 			smp_mb();
 		}
+	} else {
+		/*
+		 * current->active_mm != mm
+		 * Synchronize with switch_mm.
+		 */
+		smp_mb();
 	}
 
 	if (cpumask_any_but(mm_cpumask(mm), smp_processor_id()) < nr_cpu_ids)
-- 
2.5.5

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] x86/mm: Add barriers and document switch_mm()-vs-flush synchronization follow-up

[Nadav Amit]

Rafael Aquini <aquini@redhat.com> wrote:

> While backporting 71b3c126e611 ("x86/mm: Add barriers and document switch_mm()-vs-flush synchronization")
> we stumbled across a possibly missing barrier at flush_tlb_page().

I too noticed it and submitted a similar patch that never got a response [1].

Regards,
Nadav

[1] https://lkml.org/lkml/2016/7/15/598

Re: [PATCH] x86/mm: Add barriers and document switch_mm()-vs-flush synchronization follow-up

[Nadav Amit]

Rafael Aquini <aquini@redhat.com> wrote:

> While backporting 71b3c126e611 ("x86/mm: Add barriers and document switch_mm()-vs-flush synchronization")
> we stumbled across a possibly missing barrier at flush_tlb_page().

I too noticed it and submitted a similar patch that never got a response [1].

Regards,
Nadav

[1] https://lkml.org/lkml/2016/7/15/598

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] x86/mm: Add barriers and document switch_mm()-vs-flush synchronization follow-up

[Andy Lutomirski]

On Tue, Aug 2, 2016 at 3:27 PM, Nadav Amit <nadav.amit@gmail.com> wrote:
> Rafael Aquini <aquini@redhat.com> wrote:
>
>> While backporting 71b3c126e611 ("x86/mm: Add barriers and document switch_mm()-vs-flush synchronization")
>> we stumbled across a possibly missing barrier at flush_tlb_page().
>
> I too noticed it and submitted a similar patch that never got a response [1].
>
> Regards,
> Nadav
>
> [1] https://lkml.org/lkml/2016/7/15/598
>

Yeah, sorry, I've been busy.  I'll try to get to this soon.

-- 
Andy Lutomirski
AMA Capital Management, LLC

Re: [PATCH] x86/mm: Add barriers and document switch_mm()-vs-flush synchronization follow-up

[Andy Lutomirski]

On Tue, Aug 2, 2016 at 3:27 PM, Nadav Amit <nadav.amit@gmail.com> wrote:
> Rafael Aquini <aquini@redhat.com> wrote:
>
>> While backporting 71b3c126e611 ("x86/mm: Add barriers and document switch_mm()-vs-flush synchronization")
>> we stumbled across a possibly missing barrier at flush_tlb_page().
>
> I too noticed it and submitted a similar patch that never got a response [1].
>
> Regards,
> Nadav
>
> [1] https://lkml.org/lkml/2016/7/15/598
>

Yeah, sorry, I've been busy.  I'll try to get to this soon.

-- 
Andy Lutomirski
AMA Capital Management, LLC

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] x86/mm: Add barriers and document switch_mm()-vs-flush synchronization follow-up

[Rafael Aquini]

On Tue, Aug 02, 2016 at 03:27:06PM -0700, Nadav Amit wrote:
> Rafael Aquini <aquini@redhat.com> wrote:
> 
> > While backporting 71b3c126e611 ("x86/mm: Add barriers and document switch_mm()-vs-flush synchronization")
> > we stumbled across a possibly missing barrier at flush_tlb_page().
> 
> I too noticed it and submitted a similar patch that never got a response [1].
>

As far as I understood Andy's rationale for the original patch you need
a full memory barrier there in flush_tlb_page to get that cache-eviction
race sorted out.

Regards,
-- Rafael

Re: [PATCH] x86/mm: Add barriers and document switch_mm()-vs-flush synchronization follow-up

[Rafael Aquini]

On Tue, Aug 02, 2016 at 03:27:06PM -0700, Nadav Amit wrote:
> Rafael Aquini <aquini@redhat.com> wrote:
> 
> > While backporting 71b3c126e611 ("x86/mm: Add barriers and document switch_mm()-vs-flush synchronization")
> > we stumbled across a possibly missing barrier at flush_tlb_page().
> 
> I too noticed it and submitted a similar patch that never got a response [1].
>

As far as I understood Andy's rationale for the original patch you need
a full memory barrier there in flush_tlb_page to get that cache-eviction
race sorted out.

Regards,
-- Rafael

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] x86/mm: Add barriers and document switch_mm()-vs-flush synchronization follow-up

[Nadav Amit]

Rafael Aquini <aquini@redhat.com> wrote:

> On Tue, Aug 02, 2016 at 03:27:06PM -0700, Nadav Amit wrote:
>> Rafael Aquini <aquini@redhat.com> wrote:
>> 
>>> While backporting 71b3c126e611 ("x86/mm: Add barriers and document switch_mm()-vs-flush synchronization")
>>> we stumbled across a possibly missing barrier at flush_tlb_page().
>> 
>> I too noticed it and submitted a similar patch that never got a response [1].
> 
> As far as I understood Andy's rationale for the original patch you need
> a full memory barrier there in flush_tlb_page to get that cache-eviction
> race sorted out.

I am completely ok with your fix (except for the missing barrier in
set_tlb_ubc_flush_pending() ). However, I think mine should suffice. As far as
I saw, an atomic operation preceded every invocation of flush_tlb_page(). I
was afraid someone would send me to measure the patch performance impact so I
looked for one with the least impact.

See Intel SDM 8.2.2 "Memory Ordering in P6 and More Recent Processor Families"
for the reasoning behind smp_mb__after_atomic() . The result of an atomic
operation followed by smp_mb__after_atomic should be identical to smp_mb().

Regards,
Nadav

Re: [PATCH] x86/mm: Add barriers and document switch_mm()-vs-flush synchronization follow-up

[Nadav Amit]

Rafael Aquini <aquini@redhat.com> wrote:

> On Tue, Aug 02, 2016 at 03:27:06PM -0700, Nadav Amit wrote:
>> Rafael Aquini <aquini@redhat.com> wrote:
>> 
>>> While backporting 71b3c126e611 ("x86/mm: Add barriers and document switch_mm()-vs-flush synchronization")
>>> we stumbled across a possibly missing barrier at flush_tlb_page().
>> 
>> I too noticed it and submitted a similar patch that never got a response [1].
> 
> As far as I understood Andy's rationale for the original patch you need
> a full memory barrier there in flush_tlb_page to get that cache-eviction
> race sorted out.

I am completely ok with your fix (except for the missing barrier in
set_tlb_ubc_flush_pending() ). However, I think mine should suffice. As far as
I saw, an atomic operation preceded every invocation of flush_tlb_page(). I
was afraid someone would send me to measure the patch performance impact so I
looked for one with the least impact.

See Intel SDM 8.2.2 "Memory Ordering in P6 and More Recent Processor Families"
for the reasoning behind smp_mb__after_atomic() . The result of an atomic
operation followed by smp_mb__after_atomic should be identical to smp_mb().

Regards,
Nadav




--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>