From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2A4DCC433EF for ; Thu, 24 Mar 2022 16:52:43 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id BCC1F60BB1; Thu, 24 Mar 2022 16:52:42 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id E3H02YXHa6jz; Thu, 24 Mar 2022 16:52:42 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 8764060B6A; Thu, 24 Mar 2022 16:52:41 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 68B5DC0012; Thu, 24 Mar 2022 16:52:41 +0000 (UTC) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 4E8BDC000B for ; Thu, 24 Mar 2022 16:52:40 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 2ECC884472 for ; Thu, 24 Mar 2022 16:52:40 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mkpbn5fsFlnW for ; Thu, 24 Mar 2022 16:52:39 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp1.osuosl.org (Postfix) with ESMTP id 1479384470 for ; Thu, 24 Mar 2022 16:52:38 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 39B06D6E; Thu, 24 Mar 2022 09:52:38 -0700 (PDT) Received: from [10.57.41.19] (unknown [10.57.41.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 6D3993F73B; Thu, 24 Mar 2022 09:52:35 -0700 (PDT) Message-ID: Date: Thu, 24 Mar 2022 16:52:31 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:91.0) Gecko/20100101 Thunderbird/91.7.0 Subject: Re: [REGRESSION] Recent swiotlb DMA_FROM_DEVICE fixes break ath9k-based AP Content-Language: en-GB To: Christoph Hellwig , Maxime Bizon References: <1812355.tdWV9SEqCh@natalenko.name> <20220324055732.GB12078@lst.de> <4386660.LvFx2qVVIh@natalenko.name> <81ffc753-72aa-6327-b87b-3f11915f2549@arm.com> <878rsza0ih.fsf@toke.dk> <4be26f5d8725cdb016c6fdd9d05cfeb69cdd9e09.camel@freebox.fr> <20220324163132.GB26098@lst.de> From: Robin Murphy In-Reply-To: <20220324163132.GB26098@lst.de> Cc: =?UTF-8?Q?Toke_H=c3=b8iland-J=c3=b8rgensen?= , Netdev , Kalle Valo , linux-wireless , Oleksandr Natalenko , stable , Linux Kernel Mailing List , Halil Pasic , iommu , Olha Cherevyk , Greg Kroah-Hartman , Jakub Kicinski , Paolo Abeni , Linus Torvalds , "David S. Miller" X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" On 2022-03-24 16:31, Christoph Hellwig wrote: > On Thu, Mar 24, 2022 at 05:29:12PM +0100, Maxime Bizon wrote: >>> I'm looking into this; but in the interest of a speedy resolution of >>> the regression I would be in favour of merging that partial revert >>> and reinstating it if/when we identify (and fix) any bugs in ath9k :) >> >> This looks fishy: >> >> ath9k/recv.c >> >> /* We will now give hardware our shiny new allocated skb */ >> new_buf_addr = dma_map_single(sc->dev, requeue_skb->data, >> common->rx_bufsize, dma_type); >> if (unlikely(dma_mapping_error(sc->dev, new_buf_addr))) { >> dev_kfree_skb_any(requeue_skb); >> goto requeue_drop_frag; >> } >> >> /* Unmap the frame */ >> dma_unmap_single(sc->dev, bf->bf_buf_addr, >> common->rx_bufsize, dma_type); >> >> bf->bf_mpdu = requeue_skb; >> bf->bf_buf_addr = new_buf_addr; > > Creating a new mapping for the same buffer before unmapping the > previous one does looks rather bogus. But it does not fit the > pattern where revering the sync_single changes make the driver > work again. OK, you made me look :) Now that it's obvious what to look for, I can only conclude that during the stanza in ath_edma_get_buffers(), the device is still writing to the buffer while ownership has been transferred to the CPU, and whatever got written while ath9k_hw_process_rxdesc_edma() was running then gets wiped out by the subsequent sync_for_device, which currently resets the SWIOTLB slot to the state that sync_for_cpu copied out. By the letter of the DMA API that's not allowed, but on the other hand I'm not sure if we even have a good idiom for "I can't tell if the device has finished with this buffer or not unless I look at it" :/ Robin. _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F154C4332F for ; Thu, 24 Mar 2022 16:54:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352038AbiCXQ4N (ORCPT ); Thu, 24 Mar 2022 12:56:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38808 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352043AbiCXQzq (ORCPT ); Thu, 24 Mar 2022 12:55:46 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id AAB55B7156; Thu, 24 Mar 2022 09:52:38 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 39B06D6E; Thu, 24 Mar 2022 09:52:38 -0700 (PDT) Received: from [10.57.41.19] (unknown [10.57.41.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 6D3993F73B; Thu, 24 Mar 2022 09:52:35 -0700 (PDT) Message-ID: Date: Thu, 24 Mar 2022 16:52:31 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:91.0) Gecko/20100101 Thunderbird/91.7.0 Subject: Re: [REGRESSION] Recent swiotlb DMA_FROM_DEVICE fixes break ath9k-based AP Content-Language: en-GB To: Christoph Hellwig , Maxime Bizon Cc: =?UTF-8?Q?Toke_H=c3=b8iland-J=c3=b8rgensen?= , Oleksandr Natalenko , Linus Torvalds , Halil Pasic , Marek Szyprowski , Kalle Valo , "David S. Miller" , Jakub Kicinski , Paolo Abeni , Olha Cherevyk , iommu , linux-wireless , Netdev , Linux Kernel Mailing List , Greg Kroah-Hartman , stable References: <1812355.tdWV9SEqCh@natalenko.name> <20220324055732.GB12078@lst.de> <4386660.LvFx2qVVIh@natalenko.name> <81ffc753-72aa-6327-b87b-3f11915f2549@arm.com> <878rsza0ih.fsf@toke.dk> <4be26f5d8725cdb016c6fdd9d05cfeb69cdd9e09.camel@freebox.fr> <20220324163132.GB26098@lst.de> From: Robin Murphy In-Reply-To: <20220324163132.GB26098@lst.de> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org On 2022-03-24 16:31, Christoph Hellwig wrote: > On Thu, Mar 24, 2022 at 05:29:12PM +0100, Maxime Bizon wrote: >>> I'm looking into this; but in the interest of a speedy resolution of >>> the regression I would be in favour of merging that partial revert >>> and reinstating it if/when we identify (and fix) any bugs in ath9k :) >> >> This looks fishy: >> >> ath9k/recv.c >> >> /* We will now give hardware our shiny new allocated skb */ >> new_buf_addr = dma_map_single(sc->dev, requeue_skb->data, >> common->rx_bufsize, dma_type); >> if (unlikely(dma_mapping_error(sc->dev, new_buf_addr))) { >> dev_kfree_skb_any(requeue_skb); >> goto requeue_drop_frag; >> } >> >> /* Unmap the frame */ >> dma_unmap_single(sc->dev, bf->bf_buf_addr, >> common->rx_bufsize, dma_type); >> >> bf->bf_mpdu = requeue_skb; >> bf->bf_buf_addr = new_buf_addr; > > Creating a new mapping for the same buffer before unmapping the > previous one does looks rather bogus. But it does not fit the > pattern where revering the sync_single changes make the driver > work again. OK, you made me look :) Now that it's obvious what to look for, I can only conclude that during the stanza in ath_edma_get_buffers(), the device is still writing to the buffer while ownership has been transferred to the CPU, and whatever got written while ath9k_hw_process_rxdesc_edma() was running then gets wiped out by the subsequent sync_for_device, which currently resets the SWIOTLB slot to the state that sync_for_cpu copied out. By the letter of the DMA API that's not allowed, but on the other hand I'm not sure if we even have a good idiom for "I can't tell if the device has finished with this buffer or not unless I look at it" :/ Robin.