[PATCH v2] mm: Fix slab->page _count corruption.

[PATCH v2] mm: Fix slab->page _count corruption.

[Pravin B Shelar]

On arches that do not support this_cpu_cmpxchg_double slab_lock is used
to do atomic cmpxchg() on double word which contains page->_count.
page count can be changed from get_page() or put_page() without taking
slab_lock. That corrupts page counter.

Following patch fixes it by moving page->_count out of cmpxchg_double
data. So that slub does no change it while updating slub meta-data in
struct page.

Reported-by: Amey Bhide <abhide@nicira.com>
Signed-off-by: Pravin B Shelar <pshelar@nicira.com>
---
 include/linux/mm_types.h |    8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index dad95bd..5f558dc 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -57,8 +57,16 @@ struct page {
 		};
 
 		union {
+#if defined(CONFIG_HAVE_CMPXCHG_DOUBLE) && \
+    defined(CONFIG_HAVE_ALIGNED_STRUCT_PAGE)
 			/* Used for cmpxchg_double in slub */
 			unsigned long counters;
+#else
+			/* Keep _count separate from slub cmpxchg_double data,
+			 * As rest of double word is protected by slab_lock
+			 * but _count is not. */
+			unsigned counters;
+#endif
 
 			struct {
 
-- 
1.7.10

Re: [PATCH v2] mm: Fix slab->page _count corruption.

[Eric Dumazet]

On Mon, 2012-05-14 at 15:29 -0700, Pravin B Shelar wrote:
> On arches that do not support this_cpu_cmpxchg_double slab_lock is used
> to do atomic cmpxchg() on double word which contains page->_count.
> page count can be changed from get_page() or put_page() without taking
> slab_lock. That corrupts page counter.
> 
> Following patch fixes it by moving page->_count out of cmpxchg_double
> data. So that slub does no change it while updating slub meta-data in
> struct page.

I say again : Page is owned by slub, so get_page() or put_page() is not
allowed ?

How is put_page() going to work with order-1 or order-2 allocations ?

Me very confused by these Nicira patches.

Re: [PATCH v2] mm: Fix slab->page _count corruption.

[Pravin Shelar]

On Mon, May 14, 2012 at 3:34 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> On Mon, 2012-05-14 at 15:29 -0700, Pravin B Shelar wrote:
>> On arches that do not support this_cpu_cmpxchg_double slab_lock is used
>> to do atomic cmpxchg() on double word which contains page->_count.
>> page count can be changed from get_page() or put_page() without taking
>> slab_lock. That corrupts page counter.
>>
>> Following patch fixes it by moving page->_count out of cmpxchg_double
>> data. So that slub does no change it while updating slub meta-data in
>> struct page.
>
> I say again : Page is owned by slub, so get_page() or put_page() is not
> allowed ?
>
This is already done in multiple subsystem in Linux kernel. e.g.
ocfs, xfs, etc.
So object from slab can be passed to IO using DMA. I don't think this
rule you referring to is enforced anywhere.

Thanks,
Pravin.

Re: [PATCH v2] mm: Fix slab->page _count corruption.

[Christoph Lameter]

On Tue, 15 May 2012, Eric Dumazet wrote:

> > Following patch fixes it by moving page->_count out of cmpxchg_double
> > data. So that slub does no change it while updating slub meta-data in
> > struct page.
>
> I say again : Page is owned by slub, so get_page() or put_page() is not
> allowed ?

It is allowed since slab memory can be used for DMA.

> How is put_page() going to work with order-1 or order-2 allocations ?

It is always incrementing the page count of the head page.

Re: [PATCH v2] mm: Fix slab->page _count corruption.

[Pravin Shelar]

Hi Christoph,
Can you comment on this patch. I have changed it according to your comments.

Thanks,
Pravin.

On Mon, May 14, 2012 at 3:29 PM, Pravin B Shelar <pshelar@nicira.com> wrote:
> On arches that do not support this_cpu_cmpxchg_double slab_lock is used
> to do atomic cmpxchg() on double word which contains page->_count.
> page count can be changed from get_page() or put_page() without taking
> slab_lock. That corrupts page counter.
>
> Following patch fixes it by moving page->_count out of cmpxchg_double
> data. So that slub does no change it while updating slub meta-data in
> struct page.
>
> Reported-by: Amey Bhide <abhide@nicira.com>
> Signed-off-by: Pravin B Shelar <pshelar@nicira.com>
> ---
>  include/linux/mm_types.h |    8 ++++++++
>  1 file changed, 8 insertions(+)
>
> diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
> index dad95bd..5f558dc 100644
> --- a/include/linux/mm_types.h
> +++ b/include/linux/mm_types.h
> @@ -57,8 +57,16 @@ struct page {
>                };
>
>                union {
> +#if defined(CONFIG_HAVE_CMPXCHG_DOUBLE) && \
> +    defined(CONFIG_HAVE_ALIGNED_STRUCT_PAGE)
>                        /* Used for cmpxchg_double in slub */
>                        unsigned long counters;
> +#else
> +                       /* Keep _count separate from slub cmpxchg_double data,
> +                        * As rest of double word is protected by slab_lock
> +                        * but _count is not. */
> +                       unsigned counters;
> +#endif
>
>                        struct {
>
> --
> 1.7.10
>

Re: [PATCH v2] mm: Fix slab->page _count corruption.

[Christoph Lameter]

On Wed, 16 May 2012, Pravin Shelar wrote:

> Can you comment on this patch. I have changed it according to your comments.

Looks fine to me.

Acked-by: Christoph Lameter <cl@linux.com>