All of lore.kernel.org
 help / color / mirror / Atom feed
From: Jason Gunthorpe <jgg@ziepe.ca>
To: Tariq Toukan <tariqt@mellanox.com>
Cc: Eric Dumazet <eric.dumazet@gmail.com>,
	"jianchao.wang" <jianchao.w.wang@oracle.com>,
	junxiao.bi@oracle.com, netdev@vger.kernel.org,
	linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org,
	Saeed Mahameed <saeedm@mellanox.com>
Subject: Re: [PATCH] net/mlx4_en: ensure rx_desc updating reaches HW before prod db updating
Date: Sun, 21 Jan 2018 13:40:13 -0700	[thread overview]
Message-ID: <20180121204013.GB14372@ziepe.ca> (raw)
In-Reply-To: <dfc02a48-7d2a-56da-dc4e-d90a9fcc559c@mellanox.com>

> Hmm, this is actually consistent with the example below [1].
> 
> AIU from the example, it seems that the dma_wmb/dma_rmb barriers are good
> for synchronizing cpu/device accesses to the "Streaming DMA mapped" buffers
> (the descriptors, went through the dma_map_page() API), but not for the
> doorbell (a coherent memory, typically allocated via dma_alloc_coherent)
> that requires using the stronger wmb() barrier.

If x86 truely requires a wmb() (aka SFENCE) here then the userspace
RDMA stuff is broken too, and that has been tested to death at this
point..

I looked into this at one point and I thought I concluded that x86 did
not require a SFENCE between a posted PCI write and writes to system
memory to guarnetee order with-respect-to the PCI device?

Well, so long as non-temporal stores and other specialty accesses are
not being used.. Is there a chance a fancy sse optimized memcpy or
memset, crypto or something is being involved here?

However, Documentation/memory-barriers.txt does seem pretty clear that
the kernel definition of wmb() makes it required here, even if it
might be overkill for x86?

Jason

      parent reply	other threads:[~2018-01-21 20:40 UTC|newest]

Thread overview: 41+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-01-12  3:42 [PATCH] net/mlx4_en: ensure rx_desc updating reaches HW before prod db updating Jianchao Wang
2018-01-12  3:42 ` Jianchao Wang
2018-01-12 16:32 ` Jason Gunthorpe
     [not found]   ` <20180112163247.GB15974-uk2M96/98Pc@public.gmane.org>
2018-01-12 16:46     ` Eric Dumazet
2018-01-12 16:46       ` Eric Dumazet
2018-01-12 19:53       ` Saeed Mahameed
     [not found]         ` <85116e56-52b1-944d-6ee2-916ccfc3a7a6-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2018-01-12 20:16           ` Eric Dumazet
2018-01-12 20:16             ` Eric Dumazet
2018-01-12 21:01             ` Saeed Mahameed
2018-01-12 21:21               ` Eric Dumazet
     [not found]               ` <e902138a-3508-3504-51e5-46152cc2fb31-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2018-01-13 19:15                 ` Jason Gunthorpe
2018-01-13 19:15                   ` Jason Gunthorpe
2018-01-14  2:40       ` jianchao.wang
     [not found]         ` <a40e44f4-106b-1075-8f92-f7741508372c-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2018-01-14  9:47           ` Tariq Toukan
2018-01-14  9:47             ` Tariq Toukan
2018-01-15  5:50             ` jianchao.wang
     [not found]               ` <fea0aa1c-b68e-9485-3826-2dfad7824911-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2018-01-19 15:16                 ` jianchao.wang
2018-01-19 15:16                   ` jianchao.wang
     [not found]                   ` <53b1ac4d-a294-eb98-149e-65d7954243da-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2018-01-19 15:49                     ` Eric Dumazet
2018-01-19 15:49                       ` Eric Dumazet
2018-01-21  9:31                       ` Tariq Toukan
     [not found]                         ` <dfc02a48-7d2a-56da-dc4e-d90a9fcc559c-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2018-01-21 16:24                           ` Tariq Toukan
2018-01-21 16:24                             ` Tariq Toukan
2018-01-21 16:43                             ` Eric Dumazet
     [not found]                               ` <1516552998.3478.5.camel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2018-01-22  2:40                                 ` jianchao.wang
2018-01-22  2:40                                   ` jianchao.wang
2018-01-22 15:47                                   ` Jason Gunthorpe
2018-01-23  3:25                                     ` jianchao.wang
2018-01-22  2:12                             ` jianchao.wang
     [not found]                               ` <c8b0955b-a3fc-afe2-2c67-e655ca2ee6f6-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2018-01-25  3:27                                 ` jianchao.wang
2018-01-25  3:27                                   ` jianchao.wang
     [not found]                                   ` <532b4d71-e2eb-35f3-894e-1c3288e7bc3f-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2018-01-25  3:55                                     ` Eric Dumazet
2018-01-25  3:55                                       ` Eric Dumazet
     [not found]                                       ` <1516852543.3715.43.camel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2018-01-25  6:25                                         ` jianchao.wang
2018-01-25  6:25                                           ` jianchao.wang
     [not found]                                           ` <89066a75-43db-0f62-f171-70b0abaa8ea0-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2018-01-25  9:54                                             ` Tariq Toukan
2018-01-25  9:54                                               ` Tariq Toukan
     [not found]                                               ` <918db4ec-8c3c-aafa-4be6-0e00a99632e2-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2018-01-27 12:41                                                 ` jianchao.wang
2018-01-27 12:41                                                   ` jianchao.wang
     [not found]                                                   ` <d9883261-e93e-400a-757c-3a81d8b6aca1@mellanox.com>
2019-01-02  1:43                                                     ` jianchao.wang
2018-01-21 20:40                         ` Jason Gunthorpe [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180121204013.GB14372@ziepe.ca \
    --to=jgg@ziepe.ca \
    --cc=eric.dumazet@gmail.com \
    --cc=jianchao.w.wang@oracle.com \
    --cc=junxiao.bi@oracle.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=saeedm@mellanox.com \
    --cc=tariqt@mellanox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.