From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88EFEC32751 for ; Wed, 7 Aug 2019 16:06:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5F8B421871 for ; Wed, 7 Aug 2019 16:06:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="rHZRSP4M" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388894AbfHGQGZ (ORCPT ); Wed, 7 Aug 2019 12:06:25 -0400 Received: from mail-ot1-f66.google.com ([209.85.210.66]:45306 "EHLO mail-ot1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387626AbfHGQGY (ORCPT ); Wed, 7 Aug 2019 12:06:24 -0400 Received: by mail-ot1-f66.google.com with SMTP id x21so13099080otq.12; Wed, 07 Aug 2019 09:06:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=AOfNJoVis4tu5FW/x4K5fd3Kb4BI9lC9c+PcHuGbSJI=; b=rHZRSP4Mj7U/HkytF9Zva56bn+5Sg4odmEri/N7n+ITRLCyiUj2eMmCJlHy4XlLYsc 7gJIN/GF3DAsUHa4BH7V0W2QaOPem6U3rgj0srz/w+BOfWwZ+8s7urdEe+Z3nXeAP9o6 6+oCplvH6V5cloglseO3mDXC06ioD25CV2idliDu8whCh8Py2B8HSijwVJ4l/U3wzT/e 2PI/Sf1UElctkc8X8QP9CZGa6RbSuvH81uctJMqTFtQhvbRtItLQ5e4wLnXl8N3jSmuI E0FWZ/m5t803vlXqBH1l4W/l02psddI0zSf8Vq/2EzBEDcA8/sbkzk7IYRug/6obhO5Q MI3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=AOfNJoVis4tu5FW/x4K5fd3Kb4BI9lC9c+PcHuGbSJI=; b=f8r64OV/fk/WlgkfqMqSEgiUXYv3muVCAOZJrjj8H1j1lKbOG6nSlVnOoGe6fp1LD4 TCacV1ixcT6qwXfynZByFjErAGeO1pxPXk28YKHUT9UOXHsRZu/08G1U2wuplTbsLYlH SMtLlM6jEzs3I6mh1nvieO2i1ItKNOyZihAb9nPWMHYH0AjUemAMdrB0bNcCOFjl30sa 6GMhSgkYnOOeYmIjBOUtnpMtjjwFfWRYFSTTQza7+p4hQYpRPgepweyRpWh/bY4W37aT QPmWEfu9T9sm3/dciLpQHVFQFzrDQ7Hz3oRDmKR9i/XmOydaqHHEb4V0jbnReSrMdNBr zCAQ== X-Gm-Message-State: APjAAAU8Y8ae2gFUKJ1k1X/zo7JCWe9Ss0VSPmHbhnbisNhvf0kgEq4i NJFDuhyVeRfgNp4O16/7wfEg6ssQIP1H9/q1SLc= X-Google-Smtp-Source: APXvYqwVvjHIkuTBZZW6ETURsPt3jc50m2JDUoKCPRZ5XBlEz5RskqQ9NfuypJ8JlV4IerpOHeGb3SA1EexymMe2UPE= X-Received: by 2002:a6b:dd18:: with SMTP id f24mr9312803ioc.97.1565193983972; Wed, 07 Aug 2019 09:06:23 -0700 (PDT) MIME-Version: 1.0 References: <20190807024917.27682-1-firo.yang@suse.com> <85aaefdf-d454-1823-5840-d9e2f71ffb19@oracle.com> <20190807083831.GA6811@linux-6qg8> <20190807160853.00001d71@gmail.com> In-Reply-To: <20190807160853.00001d71@gmail.com> From: Alexander Duyck Date: Wed, 7 Aug 2019 09:06:12 -0700 Message-ID: Subject: Re: [Intel-wired-lan] [PATCH v2 1/1] ixgbe: sync the first fragment unconditionally To: Maciej Fijalkowski Cc: Firo Yang , Netdev , LKML , intel-wired-lan , Jacob Wen , "davem@davemloft.net" , Alexander Duyck Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Aug 7, 2019 at 7:09 AM Maciej Fijalkowski wrote: > > On Wed, 7 Aug 2019 08:38:43 +0000 > Firo Yang wrote: > > > The 08/07/2019 15:56, Jacob Wen wrote: > > > I think the description is not correct. Consider using something like below. > > Thank you for comments. > > > > > > > > In Xen environment, due to memory fragmentation ixgbe may allocate a 'DMA' > > > buffer with pages that are not physically contiguous. > > Actually, I didn't look into the reason why ixgbe got a DMA buffer which > > was mapped to Xen-swiotlb area. > > I think that neither of these descriptions are telling us what was truly > broken. They lack what Alexander wrote on v1 thread of this patch. > > ixgbe_dma_sync_frag is called only on case when the current descriptor has EOP > bit set, skb was already allocated and you'll be adding a current buffer as a > frag. DMA unmapping for the first frag was intentionally skipped and we will be > unmapping it here, in ixgbe_dma_sync_frag. As Alex said, we're using the > DMA_ATTR_SKIP_CPU_SYNC attribute which obliges us to perform a sync manually > and it was missing. > > So IMHO the commit description should include descriptions from both xen and > ixgbe worlds, the v2 lacks info about ixgbe case. > > BTW Alex, what was the initial reason for holding off with unmapping the first > buffer? Asking because IIRC the i40e and ice behave a bit different here. We > don't look there for EOP at all when building/constructing skb and not delaying > the unmap of non-eop buffers. > > Thanks, > Maciej The reason why we have to hold off on unmapping the first buffer is because in the case of Receive Side Coalescing (RSC), also known as Large Receive Offload (LRO), the header of the packet is updated for each additional frame that is added. As such you can end up with the device writing data, header, data, header, data, header where each data write would update a new descriptor, but the header will only ever update the first descriptor in the chain. As such if we unmapped it earlier it would result in an IOMMU fault because the device would be rewriting the header after it had been unmapped. The devices supported by the ixgbe driver are the only ones that have RSC/LRO support. As such this behavior is present for ixgbe, but not for other Intel NIC drivers including igb, igbvf, ixgbevf, i40e, etc. - Alex