From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sinan Kaya Subject: Re: [PATCH 7/7] ixgbevf: eliminate duplicate barriers on weakly-ordered archs Date: Thu, 15 Mar 2018 12:21:17 -0400 Message-ID: References: <1520997629-17361-1-git-send-email-okaya@codeaurora.org> <1520997629-17361-7-git-send-email-okaya@codeaurora.org> <12150aa0-77ba-878e-31f4-d4f8d6a28ccb@codeaurora.org> <2a4f4dec64b7462ae64152f6c2df9754@codeaurora.org> <53bf7dfe-32ee-1861-e6ea-81f667590a43@codeaurora.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: Content-Language: en-US Sender: linux-kernel-owner@vger.kernel.org To: Alexander Duyck Cc: Timur Tabi , Netdev , sulrich@codeaurora.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Jeff Kirsher , intel-wired-lan , LKML List-Id: linux-arm-msm@vger.kernel.org On 3/15/2018 10:32 AM, Alexander Duyck wrote: > We tend to do something like: > update tx_buffer_info > update tx_desc > wmb() > point first tx_buffer_info next_to_watch value at last tx_desc > update next_to_use > notify device via writel > > We do it this way because we have to synchronize between the Tx > cleanup path and the hardware so we basically lump the two barriers > together. instead of invoking both a smp_wmb and a wmb. Now that I > look at the pseudocode though I wonder if we shouldn't move the > next_to_use update before the wmb, but that might be material for > another patch. Anyway, in the Tx cleanup path we should have an > smp_rmb() after we read the next_to_watch values so that we avoid > reading any of the other fields in the buffer_info if either the field > is NULL or the descriptor pointed to has not been written back. How do you feel about keeping wmb() very close to writel_relaxed() like this? update tx_buffer_info update tx_desc point first tx_buffer_info next_to_watch value at last tx_desc update next_to_use wmb() notify device via writel_relaxed() I'm afraid that if the order of wmb() and writel() is not very obvious or hidden in multiple functions, somebody can introduce a very nasty bug in the future. We also have to think about code maintenance. -- Sinan Kaya Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm Technologies, Inc. Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project. From mboxrd@z Thu Jan 1 00:00:00 1970 From: okaya@codeaurora.org (Sinan Kaya) Date: Thu, 15 Mar 2018 12:21:17 -0400 Subject: [PATCH 7/7] ixgbevf: eliminate duplicate barriers on weakly-ordered archs In-Reply-To: References: <1520997629-17361-1-git-send-email-okaya@codeaurora.org> <1520997629-17361-7-git-send-email-okaya@codeaurora.org> <12150aa0-77ba-878e-31f4-d4f8d6a28ccb@codeaurora.org> <2a4f4dec64b7462ae64152f6c2df9754@codeaurora.org> <53bf7dfe-32ee-1861-e6ea-81f667590a43@codeaurora.org> Message-ID: To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org On 3/15/2018 10:32 AM, Alexander Duyck wrote: > We tend to do something like: > update tx_buffer_info > update tx_desc > wmb() > point first tx_buffer_info next_to_watch value at last tx_desc > update next_to_use > notify device via writel > > We do it this way because we have to synchronize between the Tx > cleanup path and the hardware so we basically lump the two barriers > together. instead of invoking both a smp_wmb and a wmb. Now that I > look at the pseudocode though I wonder if we shouldn't move the > next_to_use update before the wmb, but that might be material for > another patch. Anyway, in the Tx cleanup path we should have an > smp_rmb() after we read the next_to_watch values so that we avoid > reading any of the other fields in the buffer_info if either the field > is NULL or the descriptor pointed to has not been written back. How do you feel about keeping wmb() very close to writel_relaxed() like this? update tx_buffer_info update tx_desc point first tx_buffer_info next_to_watch value at last tx_desc update next_to_use wmb() notify device via writel_relaxed() I'm afraid that if the order of wmb() and writel() is not very obvious or hidden in multiple functions, somebody can introduce a very nasty bug in the future. We also have to think about code maintenance. -- Sinan Kaya Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm Technologies, Inc. Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project. From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sinan Kaya Date: Thu, 15 Mar 2018 12:21:17 -0400 Subject: [Intel-wired-lan] [PATCH 7/7] ixgbevf: eliminate duplicate barriers on weakly-ordered archs In-Reply-To: References: <1520997629-17361-1-git-send-email-okaya@codeaurora.org> <1520997629-17361-7-git-send-email-okaya@codeaurora.org> <12150aa0-77ba-878e-31f4-d4f8d6a28ccb@codeaurora.org> <2a4f4dec64b7462ae64152f6c2df9754@codeaurora.org> <53bf7dfe-32ee-1861-e6ea-81f667590a43@codeaurora.org> Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: intel-wired-lan@osuosl.org List-ID: On 3/15/2018 10:32 AM, Alexander Duyck wrote: > We tend to do something like: > update tx_buffer_info > update tx_desc > wmb() > point first tx_buffer_info next_to_watch value at last tx_desc > update next_to_use > notify device via writel > > We do it this way because we have to synchronize between the Tx > cleanup path and the hardware so we basically lump the two barriers > together. instead of invoking both a smp_wmb and a wmb. Now that I > look at the pseudocode though I wonder if we shouldn't move the > next_to_use update before the wmb, but that might be material for > another patch. Anyway, in the Tx cleanup path we should have an > smp_rmb() after we read the next_to_watch values so that we avoid > reading any of the other fields in the buffer_info if either the field > is NULL or the descriptor pointed to has not been written back. How do you feel about keeping wmb() very close to writel_relaxed() like this? update tx_buffer_info update tx_desc point first tx_buffer_info next_to_watch value at last tx_desc update next_to_use wmb() notify device via writel_relaxed() I'm afraid that if the order of wmb() and writel() is not very obvious or hidden in multiple functions, somebody can introduce a very nasty bug in the future. We also have to think about code maintenance. -- Sinan Kaya Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm Technologies, Inc. Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.