All of lore.kernel.org
 help / color / mirror / Atom feed
From: Jason Gunthorpe <jgg@ziepe.ca>
To: "jianchao.wang" <jianchao.w.wang@oracle.com>
Cc: Eric Dumazet <eric.dumazet@gmail.com>,
	Tariq Toukan <tariqt@mellanox.com>,
	junxiao.bi@oracle.com, netdev@vger.kernel.org,
	linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org,
	Saeed Mahameed <saeedm@mellanox.com>
Subject: Re: [PATCH] net/mlx4_en: ensure rx_desc updating reaches HW before prod db updating
Date: Mon, 22 Jan 2018 08:47:34 -0700	[thread overview]
Message-ID: <20180122154734.GD14372@ziepe.ca> (raw)
In-Reply-To: <460fca68-f8a8-e3c4-2e60-e90dc0e2f843@oracle.com>

On Mon, Jan 22, 2018 at 10:40:53AM +0800, jianchao.wang wrote:
> Hi Eric
> 
> On 01/22/2018 12:43 AM, Eric Dumazet wrote:
> > On Sun, 2018-01-21 at 18:24 +0200, Tariq Toukan wrote:
> >>
> >> On 21/01/2018 11:31 AM, Tariq Toukan wrote:
> >>>
> >>>
> >>> On 19/01/2018 5:49 PM, Eric Dumazet wrote:
> >>>> On Fri, 2018-01-19 at 23:16 +0800, jianchao.wang wrote:
> >>>>> Hi Tariq
> >>>>>
> >>>>> Very sad that the crash was reproduced again after applied the patch.
> >>
> >> Memory barriers vary for different Archs, can you please share more 
> >> details regarding arch and repro steps?
> > 
> > Yeah, mlx4 NICs in Google fleet receive trillions of packets per
> > second, and we never noticed an issue.
> > 
> > Although we are using a slightly different driver, using order-0 pages
> > and fast pages recycling.
> > 
> > 
> The driver we use will will set the page reference count to (size of pages)/stride, the
> pages will be freed by networking stack when the reference become zero, and the order-3
> pages maybe allocated soon, this give NIC device a chance to corrupt the pages which have
>  been allocated by others, such as slab.

But it looks like the wmb() is placed when stuffing new rx descriptors
into the device - how can it prevent corruption of pages where
ownership was transfered from device to the host? That sounds more like a
rmb() is missing someplace to me...

(Granted the missing wmb() is a bug, but it may not be fully solving this
 issue??)

Jason

  reply	other threads:[~2018-01-22 15:47 UTC|newest]

Thread overview: 41+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-01-12  3:42 [PATCH] net/mlx4_en: ensure rx_desc updating reaches HW before prod db updating Jianchao Wang
2018-01-12  3:42 ` Jianchao Wang
2018-01-12 16:32 ` Jason Gunthorpe
     [not found]   ` <20180112163247.GB15974-uk2M96/98Pc@public.gmane.org>
2018-01-12 16:46     ` Eric Dumazet
2018-01-12 16:46       ` Eric Dumazet
2018-01-12 19:53       ` Saeed Mahameed
     [not found]         ` <85116e56-52b1-944d-6ee2-916ccfc3a7a6-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2018-01-12 20:16           ` Eric Dumazet
2018-01-12 20:16             ` Eric Dumazet
2018-01-12 21:01             ` Saeed Mahameed
2018-01-12 21:21               ` Eric Dumazet
     [not found]               ` <e902138a-3508-3504-51e5-46152cc2fb31-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2018-01-13 19:15                 ` Jason Gunthorpe
2018-01-13 19:15                   ` Jason Gunthorpe
2018-01-14  2:40       ` jianchao.wang
     [not found]         ` <a40e44f4-106b-1075-8f92-f7741508372c-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2018-01-14  9:47           ` Tariq Toukan
2018-01-14  9:47             ` Tariq Toukan
2018-01-15  5:50             ` jianchao.wang
     [not found]               ` <fea0aa1c-b68e-9485-3826-2dfad7824911-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2018-01-19 15:16                 ` jianchao.wang
2018-01-19 15:16                   ` jianchao.wang
     [not found]                   ` <53b1ac4d-a294-eb98-149e-65d7954243da-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2018-01-19 15:49                     ` Eric Dumazet
2018-01-19 15:49                       ` Eric Dumazet
2018-01-21  9:31                       ` Tariq Toukan
     [not found]                         ` <dfc02a48-7d2a-56da-dc4e-d90a9fcc559c-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2018-01-21 16:24                           ` Tariq Toukan
2018-01-21 16:24                             ` Tariq Toukan
2018-01-21 16:43                             ` Eric Dumazet
     [not found]                               ` <1516552998.3478.5.camel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2018-01-22  2:40                                 ` jianchao.wang
2018-01-22  2:40                                   ` jianchao.wang
2018-01-22 15:47                                   ` Jason Gunthorpe [this message]
2018-01-23  3:25                                     ` jianchao.wang
2018-01-22  2:12                             ` jianchao.wang
     [not found]                               ` <c8b0955b-a3fc-afe2-2c67-e655ca2ee6f6-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2018-01-25  3:27                                 ` jianchao.wang
2018-01-25  3:27                                   ` jianchao.wang
     [not found]                                   ` <532b4d71-e2eb-35f3-894e-1c3288e7bc3f-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2018-01-25  3:55                                     ` Eric Dumazet
2018-01-25  3:55                                       ` Eric Dumazet
     [not found]                                       ` <1516852543.3715.43.camel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2018-01-25  6:25                                         ` jianchao.wang
2018-01-25  6:25                                           ` jianchao.wang
     [not found]                                           ` <89066a75-43db-0f62-f171-70b0abaa8ea0-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2018-01-25  9:54                                             ` Tariq Toukan
2018-01-25  9:54                                               ` Tariq Toukan
     [not found]                                               ` <918db4ec-8c3c-aafa-4be6-0e00a99632e2-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2018-01-27 12:41                                                 ` jianchao.wang
2018-01-27 12:41                                                   ` jianchao.wang
     [not found]                                                   ` <d9883261-e93e-400a-757c-3a81d8b6aca1@mellanox.com>
2019-01-02  1:43                                                     ` jianchao.wang
2018-01-21 20:40                         ` Jason Gunthorpe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180122154734.GD14372@ziepe.ca \
    --to=jgg@ziepe.ca \
    --cc=eric.dumazet@gmail.com \
    --cc=jianchao.w.wang@oracle.com \
    --cc=junxiao.bi@oracle.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=saeedm@mellanox.com \
    --cc=tariqt@mellanox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.