From: David Ahern <dsahern@kernel.org> To: Mina Almasry <almasrymina@google.com> Cc: linux-doc@vger.kernel.org, "Kaiyuan Zhang" <kaiyuanz@google.com>, dri-devel@lists.freedesktop.org, "Eric Dumazet" <edumazet@google.com>, linux-kselftest@vger.kernel.org, "Shuah Khan" <shuah@kernel.org>, "Sumit Semwal" <sumit.semwal@linaro.org>, linux-arch@vger.kernel.org, "Willem de Bruijn" <willemdebruijn.kernel@gmail.com>, "Jeroen de Borst" <jeroendb@google.com>, "Jonathan Corbet" <corbet@lwn.net>, "Jakub Kicinski" <kuba@kernel.org>, "Paolo Abeni" <pabeni@redhat.com>, linux-media@vger.kernel.org, "Jesper Dangaard Brouer" <hawk@kernel.org>, "Arnd Bergmann" <arnd@arndb.de>, "Shailend Chand" <shailend@google.com>, "Shakeel Butt" <shakeelb@google.com>, "Harshitha Ramamurthy" <hramamurthy@google.com>, "Willem de Bruijn" <willemb@google.com>, netdev@vger.kernel.org, "Ilias Apalodimas" <ilias.apalodimas@linaro.org>, linux-kernel@vger.kernel.org, "Christian König" <christian.koenig@amd.com>, "Yunsheng Lin" <linyunsheng@huawei.com>, "Praveen Kaligineedi" <pkaligineedi@google.com>, bpf@vger.kernel.org, "David S. Miller" <davem@davemloft.net> Subject: Re: [net-next v1 06/16] netdev: support binding dma-buf to netdevice Date: Sat, 9 Dec 2023 16:29:09 -0700 [thread overview] Message-ID: <279a2999-3c0a-4839-aa2e-602864197410@kernel.org> (raw) In-Reply-To: <CAHS8izPsQ2XoJy-vYWkn051Yc=D_kSprtQcG4mmPutf1G3+-aw@mail.gmail.com> On 12/8/23 12:22 PM, Mina Almasry wrote: > On Fri, Dec 8, 2023 at 9:48 AM David Ahern <dsahern@kernel.org> wrote: >> >> On 12/7/23 5:52 PM, Mina Almasry wrote: > ... >>> + >>> + xa_for_each(&binding->bound_rxq_list, xa_idx, rxq) { >>> + if (rxq->binding == binding) { >>> + /* We hold the rtnl_lock while binding/unbinding >>> + * dma-buf, so we can't race with another thread that >>> + * is also modifying this value. However, the driver >>> + * may read this config while it's creating its >>> + * rx-queues. WRITE_ONCE() here to match the >>> + * READ_ONCE() in the driver. >>> + */ >>> + WRITE_ONCE(rxq->binding, NULL); >>> + >>> + rxq_idx = get_netdev_rx_queue_index(rxq); >>> + >>> + netdev_restart_rx_queue(binding->dev, rxq_idx); >> >> Blindly restarting a queue when a dmabuf is heavy handed. If the dmabuf >> has no outstanding references (ie., no references in the RxQ), then no >> restart is needed. >> > > I think I need to stop the queue while binding to a dmabuf for the > sake of concurrency, no? I.e. the softirq thread may be delivering a > packet, and in parallel a separate thread holds rtnl_lock and tries to > bind the dma-buf. At that point the page_pool recreation will race > with the driver doing page_pool_alloc_page(). I don't think I can > insert a lock to handle this into the rx fast path, no? I think it depends on the details of how entries are added and removed from the pool. I am behind on the pp details at this point, so I do need to do some homework. > > Also, this sounds like it requires (lots of) more changes. The > page_pool + driver need to report how many pending references there > are (with locking so we don't race with incoming packets), and have > them reported via an ndo so that we can skip restarting the queue. > Implementing the changes in to a huge issue but handling the > concurrency may be a genuine blocker. Not sure it's worth the upside > of not restarting the single rx queue? It has to do with the usability of this overall solution. As I mentioned most ML use cases can (and will want to) use many memory allocations for receiving packets - e.g., allocations per message and receiving multiple messages per socket connection. > >>> + } >>> + } >>> + >>> + xa_erase(&netdev_dmabuf_bindings, binding->id); >>> + >>> + netdev_dmabuf_binding_put(binding); >>> +} >>> + >>> +int netdev_bind_dmabuf_to_queue(struct net_device *dev, u32 rxq_idx, >>> + struct netdev_dmabuf_binding *binding) >>> +{ >>> + struct netdev_rx_queue *rxq; >>> + u32 xa_idx; >>> + int err; >>> + >>> + rxq = __netif_get_rx_queue(dev, rxq_idx); >>> + >>> + if (rxq->binding) >>> + return -EEXIST; >>> + >>> + err = xa_alloc(&binding->bound_rxq_list, &xa_idx, rxq, xa_limit_32b, >>> + GFP_KERNEL); >>> + if (err) >>> + return err; >>> + >>> + /* We hold the rtnl_lock while binding/unbinding dma-buf, so we can't >>> + * race with another thread that is also modifying this value. However, >>> + * the driver may read this config while it's creating its * rx-queues. >>> + * WRITE_ONCE() here to match the READ_ONCE() in the driver. >>> + */ >>> + WRITE_ONCE(rxq->binding, binding); >>> + >>> + err = netdev_restart_rx_queue(dev, rxq_idx); >> >> Similarly, here binding a dmabuf to a queue. I was expecting the dmabuf >> binding to add entries to the page pool for the queue. > > To be honest, I think maybe there's a slight disconnect between how > you think the page_pool works, and my primitive understanding of how > it works. Today, I see a 1:1 mapping between rx-queue and page_pool in > the code. I don't see 1:many or many:1 mappings. I am not referring to 1:N or N:1 for page pool and queues. I am referring to entries within a single page pool for a single Rx queue. > > In theory mapping 1 rx-queue to n page_pools is trivial: the driver > can call page_pool_create() multiple times to generate n queues and > decide for incoming packets which one to use. > > However, mapping n rx-queues to 1 page_pool seems like a can of worms. > I see code in the page_pool that looks to me (and Willem) like it's > safe only because the page_pool is used from the same napi context. > with a n rx-queueue: 1 page_pool mapping, that is no longer true, no? > There is a tail end of issues to resolve to be able to map 1 page_pool > to n queues as I understand and even if resolved I'm not sure the > maintainers are interested in taking the code. > > So, per my humble understanding there is no such thing as "add entries > to the page pool for the (specific) queue", the page_pool is always > used by 1 queue. > > Note that even though this limitation exists, we still support binding > 1 dma-buf to multiple queues, because multiple page pools can use the > same netdev_dmabuf_binding. I should add that to the docs. > >> If the pool was >> previously empty, then maybe the queue needs to be "started" in the >> sense of creating with h/w or just pushing buffers into the queue and >> moving the pidx. >> >> > > I don't think it's enough to add buffers to the page_pool, no? The > existing buffers in the page_pool (host mem) must be purged. I think > maybe the queue needs to be stopped as well so that we don't race with > incoming packets and end up with skbs with devmem and non-devmem frags > (unless you're thinking it becomes a requirement to support that, I > think things are complicated as-is and it's a good simplification). > When we already purge the existing buffers & restart the queue, it's > little effort to migrate this to become in line with Jakub's queue-api > that he also wants to use for per-queue configuration & ndo_stop/open. >
WARNING: multiple messages have this Message-ID (diff)
From: David Ahern <dsahern@kernel.org> To: Mina Almasry <almasrymina@google.com> Cc: "Shailend Chand" <shailend@google.com>, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-arch@vger.kernel.org, linux-kselftest@vger.kernel.org, bpf@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, "David S. Miller" <davem@davemloft.net>, "Eric Dumazet" <edumazet@google.com>, "Jakub Kicinski" <kuba@kernel.org>, "Paolo Abeni" <pabeni@redhat.com>, "Jonathan Corbet" <corbet@lwn.net>, "Jeroen de Borst" <jeroendb@google.com>, "Praveen Kaligineedi" <pkaligineedi@google.com>, "Jesper Dangaard Brouer" <hawk@kernel.org>, "Ilias Apalodimas" <ilias.apalodimas@linaro.org>, "Arnd Bergmann" <arnd@arndb.de>, "Willem de Bruijn" <willemdebruijn.kernel@gmail.com>, "Shuah Khan" <shuah@kernel.org>, "Sumit Semwal" <sumit.semwal@linaro.org>, "Christian König" <christian.koenig@amd.com>, "Yunsheng Lin" <linyunsheng@huawei.com>, "Harshitha Ramamurthy" <hramamurthy@google.com>, "Shakeel Butt" <shakeelb@google.com>, "Willem de Bruijn" <willemb@google.com>, "Kaiyuan Zhang" <kaiyuanz@google.com> Subject: Re: [net-next v1 06/16] netdev: support binding dma-buf to netdevice Date: Sat, 9 Dec 2023 16:29:09 -0700 [thread overview] Message-ID: <279a2999-3c0a-4839-aa2e-602864197410@kernel.org> (raw) In-Reply-To: <CAHS8izPsQ2XoJy-vYWkn051Yc=D_kSprtQcG4mmPutf1G3+-aw@mail.gmail.com> On 12/8/23 12:22 PM, Mina Almasry wrote: > On Fri, Dec 8, 2023 at 9:48 AM David Ahern <dsahern@kernel.org> wrote: >> >> On 12/7/23 5:52 PM, Mina Almasry wrote: > ... >>> + >>> + xa_for_each(&binding->bound_rxq_list, xa_idx, rxq) { >>> + if (rxq->binding == binding) { >>> + /* We hold the rtnl_lock while binding/unbinding >>> + * dma-buf, so we can't race with another thread that >>> + * is also modifying this value. However, the driver >>> + * may read this config while it's creating its >>> + * rx-queues. WRITE_ONCE() here to match the >>> + * READ_ONCE() in the driver. >>> + */ >>> + WRITE_ONCE(rxq->binding, NULL); >>> + >>> + rxq_idx = get_netdev_rx_queue_index(rxq); >>> + >>> + netdev_restart_rx_queue(binding->dev, rxq_idx); >> >> Blindly restarting a queue when a dmabuf is heavy handed. If the dmabuf >> has no outstanding references (ie., no references in the RxQ), then no >> restart is needed. >> > > I think I need to stop the queue while binding to a dmabuf for the > sake of concurrency, no? I.e. the softirq thread may be delivering a > packet, and in parallel a separate thread holds rtnl_lock and tries to > bind the dma-buf. At that point the page_pool recreation will race > with the driver doing page_pool_alloc_page(). I don't think I can > insert a lock to handle this into the rx fast path, no? I think it depends on the details of how entries are added and removed from the pool. I am behind on the pp details at this point, so I do need to do some homework. > > Also, this sounds like it requires (lots of) more changes. The > page_pool + driver need to report how many pending references there > are (with locking so we don't race with incoming packets), and have > them reported via an ndo so that we can skip restarting the queue. > Implementing the changes in to a huge issue but handling the > concurrency may be a genuine blocker. Not sure it's worth the upside > of not restarting the single rx queue? It has to do with the usability of this overall solution. As I mentioned most ML use cases can (and will want to) use many memory allocations for receiving packets - e.g., allocations per message and receiving multiple messages per socket connection. > >>> + } >>> + } >>> + >>> + xa_erase(&netdev_dmabuf_bindings, binding->id); >>> + >>> + netdev_dmabuf_binding_put(binding); >>> +} >>> + >>> +int netdev_bind_dmabuf_to_queue(struct net_device *dev, u32 rxq_idx, >>> + struct netdev_dmabuf_binding *binding) >>> +{ >>> + struct netdev_rx_queue *rxq; >>> + u32 xa_idx; >>> + int err; >>> + >>> + rxq = __netif_get_rx_queue(dev, rxq_idx); >>> + >>> + if (rxq->binding) >>> + return -EEXIST; >>> + >>> + err = xa_alloc(&binding->bound_rxq_list, &xa_idx, rxq, xa_limit_32b, >>> + GFP_KERNEL); >>> + if (err) >>> + return err; >>> + >>> + /* We hold the rtnl_lock while binding/unbinding dma-buf, so we can't >>> + * race with another thread that is also modifying this value. However, >>> + * the driver may read this config while it's creating its * rx-queues. >>> + * WRITE_ONCE() here to match the READ_ONCE() in the driver. >>> + */ >>> + WRITE_ONCE(rxq->binding, binding); >>> + >>> + err = netdev_restart_rx_queue(dev, rxq_idx); >> >> Similarly, here binding a dmabuf to a queue. I was expecting the dmabuf >> binding to add entries to the page pool for the queue. > > To be honest, I think maybe there's a slight disconnect between how > you think the page_pool works, and my primitive understanding of how > it works. Today, I see a 1:1 mapping between rx-queue and page_pool in > the code. I don't see 1:many or many:1 mappings. I am not referring to 1:N or N:1 for page pool and queues. I am referring to entries within a single page pool for a single Rx queue. > > In theory mapping 1 rx-queue to n page_pools is trivial: the driver > can call page_pool_create() multiple times to generate n queues and > decide for incoming packets which one to use. > > However, mapping n rx-queues to 1 page_pool seems like a can of worms. > I see code in the page_pool that looks to me (and Willem) like it's > safe only because the page_pool is used from the same napi context. > with a n rx-queueue: 1 page_pool mapping, that is no longer true, no? > There is a tail end of issues to resolve to be able to map 1 page_pool > to n queues as I understand and even if resolved I'm not sure the > maintainers are interested in taking the code. > > So, per my humble understanding there is no such thing as "add entries > to the page pool for the (specific) queue", the page_pool is always > used by 1 queue. > > Note that even though this limitation exists, we still support binding > 1 dma-buf to multiple queues, because multiple page pools can use the > same netdev_dmabuf_binding. I should add that to the docs. > >> If the pool was >> previously empty, then maybe the queue needs to be "started" in the >> sense of creating with h/w or just pushing buffers into the queue and >> moving the pidx. >> >> > > I don't think it's enough to add buffers to the page_pool, no? The > existing buffers in the page_pool (host mem) must be purged. I think > maybe the queue needs to be stopped as well so that we don't race with > incoming packets and end up with skbs with devmem and non-devmem frags > (unless you're thinking it becomes a requirement to support that, I > think things are complicated as-is and it's a good simplification). > When we already purge the existing buffers & restart the queue, it's > little effort to migrate this to become in line with Jakub's queue-api > that he also wants to use for per-queue configuration & ndo_stop/open. >
next prev parent reply other threads:[~2023-12-09 23:29 UTC|newest] Thread overview: 145+ messages / expand[flat|nested] mbox.gz Atom feed top 2023-12-08 0:52 [net-next v1 00/16] Device Memory TCP Mina Almasry 2023-12-08 0:52 ` Mina Almasry 2023-12-08 0:52 ` [net-next v1 01/16] net: page_pool: factor out releasing DMA from releasing the page Mina Almasry 2023-12-08 0:52 ` Mina Almasry 2023-12-10 3:49 ` Shakeel Butt 2023-12-10 3:49 ` Shakeel Butt 2023-12-12 8:11 ` Ilias Apalodimas 2023-12-12 8:11 ` Ilias Apalodimas 2023-12-08 0:52 ` [net-next v1 02/16] net: page_pool: create hooks for custom page providers Mina Almasry 2023-12-08 0:52 ` Mina Almasry 2023-12-12 8:07 ` Ilias Apalodimas 2023-12-12 8:07 ` Ilias Apalodimas 2023-12-12 14:47 ` Mina Almasry 2023-12-12 14:47 ` Mina Almasry 2023-12-08 0:52 ` [net-next v1 03/16] queue_api: define queue api Mina Almasry 2023-12-08 0:52 ` Mina Almasry 2023-12-14 1:15 ` Jakub Kicinski 2023-12-14 1:15 ` Jakub Kicinski 2023-12-08 0:52 ` [net-next v1 04/16] gve: implement " Mina Almasry 2023-12-08 0:52 ` Mina Almasry 2024-03-05 11:45 ` Arnd Bergmann 2023-12-08 0:52 ` [net-next v1 05/16] net: netdev netlink api to bind dma-buf to a net device Mina Almasry 2023-12-08 0:52 ` Mina Almasry 2023-12-14 1:17 ` Jakub Kicinski 2023-12-14 1:17 ` Jakub Kicinski 2023-12-08 0:52 ` [net-next v1 06/16] netdev: support binding dma-buf to netdevice Mina Almasry 2023-12-08 0:52 ` Mina Almasry 2023-12-08 15:40 ` kernel test robot 2023-12-08 15:40 ` kernel test robot 2023-12-08 16:02 ` kernel test robot 2023-12-08 16:02 ` kernel test robot 2023-12-08 17:48 ` David Ahern 2023-12-08 17:48 ` David Ahern 2023-12-08 19:22 ` Mina Almasry 2023-12-08 19:22 ` Mina Almasry 2023-12-08 20:32 ` Mina Almasry 2023-12-08 20:32 ` Mina Almasry 2023-12-09 23:29 ` David Ahern [this message] 2023-12-09 23:29 ` David Ahern 2023-12-11 2:19 ` Mina Almasry 2023-12-11 2:19 ` Mina Almasry 2023-12-08 0:52 ` [net-next v1 07/16] netdev: netdevice devmem allocator Mina Almasry 2023-12-08 0:52 ` Mina Almasry 2023-12-08 17:56 ` David Ahern 2023-12-08 17:56 ` David Ahern 2023-12-08 19:27 ` Mina Almasry 2023-12-08 19:27 ` Mina Almasry 2023-12-08 0:52 ` [net-next v1 08/16] memory-provider: dmabuf devmem memory provider Mina Almasry 2023-12-08 0:52 ` Mina Almasry 2023-12-08 22:48 ` Pavel Begunkov 2023-12-08 22:48 ` Pavel Begunkov 2023-12-08 23:25 ` Mina Almasry 2023-12-08 23:25 ` Mina Almasry 2023-12-10 3:03 ` Pavel Begunkov 2023-12-10 3:03 ` Pavel Begunkov 2023-12-11 2:30 ` Mina Almasry 2023-12-11 2:30 ` Mina Almasry 2023-12-11 20:35 ` Pavel Begunkov 2023-12-11 20:35 ` Pavel Begunkov 2023-12-14 20:03 ` Mina Almasry 2023-12-14 20:03 ` Mina Almasry 2023-12-19 23:55 ` Pavel Begunkov 2023-12-19 23:55 ` Pavel Begunkov 2023-12-08 23:05 ` Pavel Begunkov 2023-12-08 23:05 ` Pavel Begunkov 2023-12-12 12:25 ` Jason Gunthorpe 2023-12-12 12:25 ` Jason Gunthorpe 2023-12-12 13:07 ` Christoph Hellwig 2023-12-12 14:26 ` Mina Almasry 2023-12-12 14:26 ` Mina Almasry 2023-12-12 14:39 ` Jason Gunthorpe 2023-12-12 14:39 ` Jason Gunthorpe 2023-12-12 14:58 ` Mina Almasry 2023-12-12 14:58 ` Mina Almasry 2023-12-12 15:08 ` Jason Gunthorpe 2023-12-12 15:08 ` Jason Gunthorpe 2023-12-13 1:09 ` Mina Almasry 2023-12-13 1:09 ` Mina Almasry 2023-12-13 2:19 ` David Ahern 2023-12-13 2:19 ` David Ahern 2023-12-13 7:49 ` Yinjun Zhang 2023-12-13 7:49 ` Yinjun Zhang 2023-12-08 0:52 ` [net-next v1 09/16] page_pool: device memory support Mina Almasry 2023-12-08 0:52 ` Mina Almasry 2023-12-08 9:30 ` Yunsheng Lin 2023-12-08 9:30 ` Yunsheng Lin 2023-12-08 16:05 ` Mina Almasry 2023-12-08 16:05 ` Mina Almasry 2023-12-11 2:04 ` Yunsheng Lin 2023-12-11 2:04 ` Yunsheng Lin 2023-12-11 2:26 ` Mina Almasry 2023-12-11 2:26 ` Mina Almasry 2023-12-11 4:04 ` Mina Almasry 2023-12-11 4:04 ` Mina Almasry 2023-12-11 11:51 ` Yunsheng Lin 2023-12-11 11:51 ` Yunsheng Lin 2023-12-11 18:14 ` Mina Almasry 2023-12-11 18:14 ` Mina Almasry 2023-12-12 11:17 ` Yunsheng Lin 2023-12-12 11:17 ` Yunsheng Lin 2023-12-12 14:28 ` Mina Almasry 2023-12-12 14:28 ` Mina Almasry 2023-12-13 11:48 ` Yunsheng Lin 2023-12-13 11:48 ` Yunsheng Lin 2023-12-13 7:52 ` Mina Almasry 2023-12-13 7:52 ` Mina Almasry 2023-12-08 0:52 ` [net-next v1 10/16] page_pool: don't release iov on elevanted refcount Mina Almasry 2023-12-08 0:52 ` Mina Almasry 2023-12-08 0:52 ` [net-next v1 11/16] net: support non paged skb frags Mina Almasry 2023-12-08 0:52 ` Mina Almasry 2023-12-08 0:52 ` [net-next v1 12/16] net: add support for skbs with unreadable frags Mina Almasry 2023-12-08 0:52 ` Mina Almasry 2023-12-08 0:52 ` [net-next v1 13/16] tcp: RX path for devmem TCP Mina Almasry 2023-12-08 0:52 ` Mina Almasry 2023-12-08 15:40 ` kernel test robot 2023-12-08 15:40 ` kernel test robot 2023-12-08 17:55 ` David Ahern 2023-12-08 17:55 ` David Ahern 2023-12-08 19:23 ` Mina Almasry 2023-12-08 19:23 ` Mina Almasry 2023-12-08 0:52 ` [net-next v1 14/16] net: add SO_DEVMEM_DONTNEED setsockopt to release RX frags Mina Almasry 2023-12-08 0:52 ` Mina Almasry 2023-12-12 19:08 ` Simon Horman 2023-12-12 19:08 ` Simon Horman 2023-12-08 0:52 ` [net-next v1 15/16] net: add devmem TCP documentation Mina Almasry 2023-12-08 0:52 ` Mina Almasry 2023-12-12 19:14 ` Simon Horman 2023-12-12 19:14 ` Simon Horman 2023-12-08 0:52 ` [net-next v1 16/16] selftests: add ncdevmem, netcat for devmem TCP Mina Almasry 2023-12-08 0:52 ` Mina Almasry 2023-12-08 1:47 ` [net-next v1 00/16] Device Memory TCP Mina Almasry 2023-12-08 1:47 ` Mina Almasry 2023-12-08 17:57 ` David Ahern 2023-12-08 17:57 ` David Ahern 2023-12-08 19:31 ` Mina Almasry 2023-12-08 19:31 ` Mina Almasry 2023-12-10 3:48 ` Shakeel Butt 2023-12-10 3:48 ` Shakeel Butt 2023-12-12 5:58 ` Christoph Hellwig 2023-12-14 6:20 ` patchwork-bot+netdevbpf 2023-12-14 6:20 ` patchwork-bot+netdevbpf 2023-12-14 6:48 ` Christoph Hellwig 2023-12-14 6:51 ` Mina Almasry 2023-12-14 6:51 ` Mina Almasry 2023-12-14 6:59 ` Christoph Hellwig
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=279a2999-3c0a-4839-aa2e-602864197410@kernel.org \ --to=dsahern@kernel.org \ --cc=almasrymina@google.com \ --cc=arnd@arndb.de \ --cc=bpf@vger.kernel.org \ --cc=christian.koenig@amd.com \ --cc=corbet@lwn.net \ --cc=davem@davemloft.net \ --cc=dri-devel@lists.freedesktop.org \ --cc=edumazet@google.com \ --cc=hawk@kernel.org \ --cc=hramamurthy@google.com \ --cc=ilias.apalodimas@linaro.org \ --cc=jeroendb@google.com \ --cc=kaiyuanz@google.com \ --cc=kuba@kernel.org \ --cc=linux-arch@vger.kernel.org \ --cc=linux-doc@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-kselftest@vger.kernel.org \ --cc=linux-media@vger.kernel.org \ --cc=linyunsheng@huawei.com \ --cc=netdev@vger.kernel.org \ --cc=pabeni@redhat.com \ --cc=pkaligineedi@google.com \ --cc=shailend@google.com \ --cc=shakeelb@google.com \ --cc=shuah@kernel.org \ --cc=sumit.semwal@linaro.org \ --cc=willemb@google.com \ --cc=willemdebruijn.kernel@gmail.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.