From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_2 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2428C4363A for ; Tue, 20 Oct 2020 22:06:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8F0072225C for ; Tue, 20 Oct 2020 22:06:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2410331AbgJTWG2 (ORCPT ); Tue, 20 Oct 2020 18:06:28 -0400 Received: from kernel.crashing.org ([76.164.61.194]:43502 "EHLO kernel.crashing.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2391762AbgJTWG2 (ORCPT ); Tue, 20 Oct 2020 18:06:28 -0400 Received: from localhost (gate.crashing.org [63.228.1.57]) (authenticated bits=0) by kernel.crashing.org (8.14.7/8.14.7) with ESMTP id 09KM5Zhv006771 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 20 Oct 2020 17:05:44 -0500 Message-ID: <7b44608bed9eccad80457cbfdfcca9043aae56f2.camel@kernel.crashing.org> Subject: Re: [PATCH] net: ftgmac100: Fix missing TX-poll issue From: Benjamin Herrenschmidt To: David Laight , "'Dylan Hung'" , Jakub Kicinski , Joel Stanley Cc: "David S . Miller" , "netdev@vger.kernel.org" , Linux Kernel Mailing List , Po-Yu Chuang , linux-aspeed , OpenBMC Maillist , BMC-SW Date: Wed, 21 Oct 2020 09:05:34 +1100 In-Reply-To: References: <20201019073908.32262-1-dylan_hung@aspeedtech.com> <20201019120040.3152ea0b@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5-0ubuntu0.18.04.2 Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 2020-10-20 at 13:15 +0000, David Laight wrote: > That rather depends where the data is 'stuck'. > > An old sparc cpu would flush the cpu store buffer before the read. > But a modern x86 cpu will satisfy the read from the store buffer > for cached data. > > If the write is 'posted' on a PCI(e) bus then the read can't overtake it. > But that is a memory access so shouldn't be to a PCI(e) address. > > Shouldn't dma_wb() actually force your 'cpu to dram' queue be flushed? > In which case you need one after writing the ring descriptor and > before the poke of the mac engine. > > The barrier before the descriptor write only needs to guarantee > ordering of the writes - it can probably be a lighter barrier? > > It might be that your dma_wmb() needs to do a write+read of > an uncached DRAM location in order to empty the cpu to dram queue. This is a specific bug with how a specific IP block is hooked up in those SOCs, I wouldn't bloat the global dma_wmb for that. The read back in the driver with appropriate comment should be enough. Cheers, Ben. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_2 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93E7FC4363A for ; Tue, 20 Oct 2020 22:07:58 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9EFEE22256 for ; Tue, 20 Oct 2020 22:07:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9EFEE22256 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.crashing.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=openbmc-bounces+openbmc=archiver.kernel.org@lists.ozlabs.org Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 4CG76L6L15zDqdj for ; Wed, 21 Oct 2020 09:07:54 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; spf=permerror (SPF Permanent Error: Unknown mechanism found: ip:192.40.192.88/32) smtp.mailfrom=kernel.crashing.org (client-ip=76.164.61.194; helo=kernel.crashing.org; envelope-from=benh@kernel.crashing.org; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.crashing.org Received: from kernel.crashing.org (kernel.crashing.org [76.164.61.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4CG74d6xQ2zDqc7; Wed, 21 Oct 2020 09:06:23 +1100 (AEDT) Received: from localhost (gate.crashing.org [63.228.1.57]) (authenticated bits=0) by kernel.crashing.org (8.14.7/8.14.7) with ESMTP id 09KM5Zhv006771 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 20 Oct 2020 17:05:44 -0500 Message-ID: <7b44608bed9eccad80457cbfdfcca9043aae56f2.camel@kernel.crashing.org> Subject: Re: [PATCH] net: ftgmac100: Fix missing TX-poll issue From: Benjamin Herrenschmidt To: David Laight , "'Dylan Hung'" , Jakub Kicinski , Joel Stanley Date: Wed, 21 Oct 2020 09:05:34 +1100 In-Reply-To: References: <20201019073908.32262-1-dylan_hung@aspeedtech.com> <20201019120040.3152ea0b@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5-0ubuntu0.18.04.2 Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-BeenThere: openbmc@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development list for OpenBMC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: BMC-SW , Po-Yu Chuang , linux-aspeed , "netdev@vger.kernel.org" , OpenBMC Maillist , Linux Kernel Mailing List , "David S . Miller" Errors-To: openbmc-bounces+openbmc=archiver.kernel.org@lists.ozlabs.org Sender: "openbmc" On Tue, 2020-10-20 at 13:15 +0000, David Laight wrote: > That rather depends where the data is 'stuck'. > > An old sparc cpu would flush the cpu store buffer before the read. > But a modern x86 cpu will satisfy the read from the store buffer > for cached data. > > If the write is 'posted' on a PCI(e) bus then the read can't overtake it. > But that is a memory access so shouldn't be to a PCI(e) address. > > Shouldn't dma_wb() actually force your 'cpu to dram' queue be flushed? > In which case you need one after writing the ring descriptor and > before the poke of the mac engine. > > The barrier before the descriptor write only needs to guarantee > ordering of the writes - it can probably be a lighter barrier? > > It might be that your dma_wmb() needs to do a write+read of > an uncached DRAM location in order to empty the cpu to dram queue. This is a specific bug with how a specific IP block is hooked up in those SOCs, I wouldn't bloat the global dma_wmb for that. The read back in the driver with appropriate comment should be enough. Cheers, Ben.