From mboxrd@z Thu Jan 1 00:00:00 1970 From: Tariq Toukan Subject: Re: [PATCH] net/mlx4_en: ensure rx_desc updating reaches HW before prod db updating Date: Sun, 14 Jan 2018 11:47:13 +0200 Message-ID: References: <1515728542-3060-1-git-send-email-jianchao.w.wang@oracle.com> <20180112163247.GB15974@ziepe.ca> <1515775567.131759.42.camel@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: Content-Language: en-US Sender: linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: "jianchao.wang" , Eric Dumazet , Jason Gunthorpe Cc: tariqt-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org, junxiao.bi-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org, netdev-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Saeed Mahameed List-Id: linux-rdma@vger.kernel.org Thanks Jianchao for your patch. And Thank you guys for your reviews, much appreciated. I was off-work on Friday and Saturday. On 14/01/2018 4:40 AM, jianchao.wang wrote: > Dear all > > Thanks for the kindly response and reviewing. That's really appreciated. > > On 01/13/2018 12:46 AM, Eric Dumazet wrote: >>> Does this need to be dma_wmb(), and should it be in >>> mlx4_en_update_rx_prod_db ? >>> >> +1 on dma_wmb() >> >> On what architecture bug was observed ? > This issue was observed on x86-64. > And I will send a new patch, in which replace wmb() with dma_wmb(), to customer > to confirm. +1 on dma_wmb, let us know once customer confirms. Please place it within mlx4_en_update_rx_prod_db as suggested. All other calls to mlx4_en_update_rx_prod_db are in control/slow path so I prefer being on the safe side, and care less about bulking the barrier. Thanks, Tariq -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html From mboxrd@z Thu Jan 1 00:00:00 1970 Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751505AbeANJr1 (ORCPT + 1 other); Sun, 14 Jan 2018 04:47:27 -0500 Received: from mail-ve1eur01on0069.outbound.protection.outlook.com ([104.47.1.69]:39776 "EHLO EUR01-VE1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1750915AbeANJrY (ORCPT ); Sun, 14 Jan 2018 04:47:24 -0500 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=tariqt@mellanox.com; Subject: Re: [PATCH] net/mlx4_en: ensure rx_desc updating reaches HW before prod db updating To: "jianchao.wang" , Eric Dumazet , Jason Gunthorpe Cc: tariqt@mellanox.com, junxiao.bi@oracle.com, netdev@vger.kernel.org, linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org, Saeed Mahameed References: <1515728542-3060-1-git-send-email-jianchao.w.wang@oracle.com> <20180112163247.GB15974@ziepe.ca> <1515775567.131759.42.camel@gmail.com> From: Tariq Toukan Message-ID: Date: Sun, 14 Jan 2018 11:47:13 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Originating-IP: [193.47.165.251] X-ClientProxiedBy: HE1PR0202CA0005.eurprd02.prod.outlook.com (2603:10a6:3:8c::15) To VI1PR05MB1149.eurprd05.prod.outlook.com (2a01:111:e400:582f::19) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: dd5de3f4-0b7c-4db9-2437-08d55b33ce92 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(7020095)(4652020)(48565401081)(5600026)(4604075)(2017052603307)(7153060)(7193020);SRVR:VI1PR05MB1149; X-Microsoft-Exchange-Diagnostics: 1;VI1PR05MB1149;3:nDW4LLMu0Z8pfBcYTPmO+PMbyQ+r2eqK0TNz41Conns/VoVBRWPhMwsfBt4vblhH7X3HWY/nuSR9YJ8mZfQy4kn+AXkZhoEZ1qp2YIThSmIBl2g0qfNFp+FhshEe8OjxEUuXREVREtOfFPNT+zfq3YTacRsZ4lSeyYgcmjbCiGTeV+mxrKmDBfY++xOKgZNHpj/8TXYefsgB5WZDFHvxNqKeeCR4ptNxFodrvGezAy+Ls+Ws8RBU+HZODv1PsaUf;25:BC6WOIHdjUkacPSaz5JqYon2kZORX00Oe6q0jl0gCyh+nlJCBBLEkSQ3PdNCmY0hQC/G/QW1+KlinCNobyv2r7cDlVKdXdydBehChd4z37aAPlVsz58BraIr7C/a3aGbTORJ2unnhFgxuWKtA285ed/jzs6IkVpYXsbUIJIPnYT21ibEYgbVmv/aRLvZukXZe3TqUA+m3TImkgipgtNPS1K5P1c83EDGKdmCaX9t8g/9tRA/bg03Ptik3Fp7nxKq0oe/CLcNJyInBbPiTY4Wp+aFspcL/Mz+XYSz03s7CoaJUEcG/ADTkEApzxQ8LY2Tcw6Snnm3FlG3Hn+wtX+bBw==;31:4aRJvYmwTbCWgGA8hAmGby+DUw0TIoOX+KvROOoXgoR6B8jVAs/WIviOSBGqhj/V/j5N/ZKbkeUp37B5rD+v8ZQexFnvonPisXd7kZ7HeY9m9/5h0EGw3F0BN+1jXE4qtDpxSkHbdn6DcodzwCRIJZ584SkfqF2jEieuR0nKfZqKgB/XbfGnzh6LyhUNqgkuXVvbEVpRphOH5sWGVRdWyXnf7gE89hg1GPOB7V5U0ys= X-MS-TrafficTypeDiagnostic: VI1PR05MB1149: X-Microsoft-Exchange-Diagnostics: 1;VI1PR05MB1149;20:R9+Aw4HbkgMRIXi82Fy9aa+LzoSbxSg8mjm6zOopgR07NESqgzuLBglSXHXtsx5cGSpl1Kp2ubH8DNQ9m3h8GrVMnexkE8Y2A0xr8Aji2bWT4RYV6IR7vNw0zace250O/9sxmR9KKJJmclrUAKJO2If1ZPVLRbtAiGNyWSxPdWUGgrgIwExz8ABuMlEOGoIizTWIttY5G0K0uQ5hjw1Q0QDeXwPKkVoFHc0FeU8/tRBlC+bbyVW7GLfbpRr5a8K13RvNQvidn48upKl/yf5uT2gyvIfh/8eW2WF/7VLaWnsYePWyPBHYLXE9Ha6eyRzG2HoJm91wQ6DuLN48gPQWtrviXTgyZppxUyPl0cANanz4IfZhap+t848hkIfHei+t6PcpkeOgEMjikV2Cc7PL9a3wOzPLINNNjcZ2/JqvQQ7AhUtYnPtasHeR9XbCVP6cjMJ8nuga+fp4I3FWknpZBKo4kej7JDuPJZNewDrlYEu6B9SFjN4oBneYp6c4Mmso;4:Ef40LOlGlwj2bbu+BwW+KYE9iz0o6rNGSka5PyV7ORP05j/6uzFHqUYW4X27FID4QJ55iLyO30dTn/y6ibM1c+NdNG0I4p661wMb+90aqcCWY4FMm21fGYWLjsTzkTDf8zXDxfYN1brYDFwa0KMUXhQHyM1foFSuGQjjZINOciXEMfLNfjThANDZO/9BnmxjbUlgLTv23zbY6+6D/Sg1OCZmnUplHMavy5FyxyTVtNd7hF19Im7muunXcTG8GCfIkU5tyODLZ1yigEoxRHrS6IMAq7JerJdZSt0ddVzCk57uZFzmkn1ZEGiptdY32mht X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(788757137089); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040470)(2401047)(8121501046)(5005006)(10201501046)(3231023)(944501161)(3002001)(93006095)(93001095)(6055026)(6041268)(20161123560045)(20161123564045)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(6072148)(201708071742011);SRVR:VI1PR05MB1149;BCL:0;PCL:0;RULEID:(100000803101)(100110400095);SRVR:VI1PR05MB1149; X-Forefront-PRVS: 05529C6FDB X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(6049001)(39380400002)(396003)(376002)(346002)(39860400002)(366004)(199004)(189003)(24454002)(51914003)(81166006)(8676002)(68736007)(3846002)(6116002)(66066001)(65806001)(478600001)(47776003)(65956001)(8936002)(81156014)(2906002)(7736002)(305945005)(31696002)(105586002)(106356001)(50466002)(64126003)(77096006)(36756003)(230700001)(229853002)(93886005)(59450400001)(2486003)(23676004)(52116002)(83506002)(52146003)(39060400002)(316002)(110136005)(58126008)(107886003)(67846002)(386003)(6246003)(2950100002)(4326008)(53546011)(16576012)(97736004)(6666003)(53936002)(16526018)(6486002)(5660300001)(76176011)(31686004)(65826007)(86362001)(25786009);DIR:OUT;SFP:1101;SCL:1;SRVR:VI1PR05MB1149;H:[10.8.0.245];FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtWSTFQUjA1TUIxMTQ5OzIzOlZtZVBjMlJjaG5wNXUvVXhNSVhkTmR2T21u?= =?utf-8?B?Nzc0bCtDUnJhak5CS3phZ0ZjTGFzeW10aTlVa0tBT3dzVTFjSkt4VTJXUS9l?= =?utf-8?B?NjNESUxaeUg4dXl0b3BKek82N3JOYnMzeThxZ3E1M0szRnJSMGx3TzkzdWZk?= =?utf-8?B?OGFuNnErN3doQ1YzRE5TN1NUOXJ1dzhaU01WOGJyNlVEb3ZMYzlnT1Q4UmlV?= =?utf-8?B?RlNaUXZyaGZXRmMzSmJObFdkOTJDQXQrYktuUTluR0xUelVDS1VMWXJrekFn?= =?utf-8?B?cFlYY21xeHpoMWtSUzNCNVJJbnE1YjNyNkl5MlYzdzdRT2xIbWZDL1FRT3hz?= =?utf-8?B?V1BDOElmWS9Uc1JOenRILzZUQ1JGMjhqN1lCMDg0aFpoRlRQcHNxZnVZOXJn?= =?utf-8?B?bWxkdm05dmNsNURQT25sQWJ2S2tJaG9wWWZ6ZWJqcGRYcCt6a2huejJ3U3l5?= =?utf-8?B?TGV3dFhqVmJ1clJ0Q2I1ZmRWQnFvOEttdVJmK2k1MGFvaVRDRHdYVjRkVTRB?= =?utf-8?B?SjU3Sml2dnlYRk1SWmJMWmpENElobXJLWGNpNkM1eGtBNlNQZ3ZMczJVWm80?= =?utf-8?B?dWNtYmJqVzJOZDZQVU1SOElkay9Qb3lZN2RiQ2ZXeUEvWWRLMmlkd2VrVFla?= =?utf-8?B?UGo5MnhBblppQWxkLyswemthZ0dMaURPSEx0bktDMEgrMnR2V01HYm9RMStE?= =?utf-8?B?TzhuU2Q4OE1DZ05RVFFhVElTVmIvbGZycDNueVk0YXdIUjNQNDJxMS82dldM?= =?utf-8?B?NHhSY2NUbDdGL1hLNTlnMElmOHA1ZnpJNk5rREhkZUZRUURmYXRCajdXQWxJ?= =?utf-8?B?QklwaXJBT1J3d1NNNHFLMDZ2a2lSeGJCWGgyc3VsYTQ4dHpNNFlGNE5UTnVM?= =?utf-8?B?TTZwajUzV0FDcC9tMXJQSWdOb0prZ0twbXFjOTVKYSt1Q1BFUEJJR3hJYlFz?= =?utf-8?B?MDNvZllkVFdTUlU3cmxManVJVngvcjhvWEhFUjJoUUZwdCs3bUx1a3NPVmNp?= =?utf-8?B?TlpwMmE4WStDVWVaeHZzQ3pGcnpRMndIbDBScG14cWZXSjJ5VUN6Ym5jV2Z0?= =?utf-8?B?cUl1RmFScXlBWEh5anhaUy9zVlBhbENlWXhtSHRtdkRkME1nWXB0eW92WWVI?= =?utf-8?B?QXNwcmxLVWQ1cDV0RW5zU0t5enczd2pOSFlFS1E1M1MreS8vdEFwZ3VOWHQz?= =?utf-8?B?dDRSYmFtVTlJanVqNnVJNUNtSTlvdHZjZ0I0RWEwaHE2czJmdElyYUs5SzVi?= =?utf-8?B?M3dRcCtYc1NBNEFPV2V1Ui9ybCtTc2RWNVZGR1liWGpGRnBVVFhGalhrcnFt?= =?utf-8?B?LzFqNWNYeUt1dmdjR1JjbXlkWkJ0QnQ4MFNWbHd2Q2lhL0VtdmZUajI2bTNK?= =?utf-8?B?U2lOQ0VTd3c0N0Vnc0lXZGRPbUdvcE5samd0b2prV20yRC85SXhVOXpqZWZF?= =?utf-8?B?NmE5TDc3VDBTWUMwOGE2dnp2M1hHNDVoalV4cml2MmZMRkdRa0loeG45Zk8v?= =?utf-8?B?OWd5SzZ3dHpLV0d4NTNXM3Rvb3M2ZHVVMStRczBFT3lXc0pZZDNZZU1pTU11?= =?utf-8?B?N0xqY1N0b1loNzRxV3dDL1hwR3c3UE1nalUxVWdjRmg0ZUQxWSt4U2dHcDZn?= =?utf-8?B?UUlKaVRBbTNDVm1UUC9CSm5qOGt0RWhSWFB2RUVSOFVUeVVVajNaL2tzQnJE?= =?utf-8?B?eUlLTmdWZmtIMmtBbXhQS2NXc1VJZFA5U1E4Rk52TVVmOHR1cUhBclNwMFFk?= =?utf-8?B?VVNjSmNSbW9sZnNGR0tHRllpZXpkNzMxaDloZUQ3RnRzYzdQL2E0RHdsT3JW?= =?utf-8?B?ekIreFpoRVl0c3hsdmtpeVlIVENCOGpZcnlkNk51RnhSMThOM2gzRkkrV3l6?= =?utf-8?B?TFpqOFJ1MGtHYy92cTYrZ1RScWc2dXVmT1JFNjhvZWdJdEVzV1E5T25vUGdJ?= =?utf-8?B?MWhSdk9KZlJKekdRMERock5sSzJHQy9pVmVwbmEyQkJpRklPU05QZEZ6MEZk?= =?utf-8?B?RjhjNDBZNEFhWXJ5RldpSnNZcG5UQkNFYmZ1QT09?= X-Microsoft-Exchange-Diagnostics: 1;VI1PR05MB1149;6:5K9OdsfTzzya8LcoF5EvgUsEm69/l6DdEpxBbTinyN4cYgMFQqy1JvHjjyNGoN1gierfZBadLROcdGb9ndipxo4DlDvnz6AZ+6UOVHXtQkYDlCxYUWRHSdAxSYX1GHrMcZbSsA93UaZHPyeAm5uGFr4DWO4rd0Zpc4epVyNAp6f0IAOl6g5bzXchJ070VGYcn/xO9kD8kYuOhW+jC/MPNsYqc80q86VgOymEHNzCzJhLc3p98al6erxCyaYI/SsvO1EMOrZvOIWS3Z3NezB02rL0cZlznzvpoCfVcSNUVfrMYTUmYxl4p8eDCgIqyvqgJctKMNWMRcHkJptJ2eW2ztBw8Y4oA7H8muGJt0wmN4A=;5:OWj61Um3NRkl09Govv+VpTuHnqHrThLiwlG+QtVTXrVwp17zxrLC6dhKemNszW2/1L2QTCsQ5BWBBBAj1qm87BpukOLZygu0tLcr+VlCfBVTFttqCY/e3G64r1IvWm+W5oF2cmNvcpKInFYWKM5LJ4zqPZRQAbzbw8PTHwouIwM=;24:wn7GexCwcZV8b4ZG2eT8pEIFNjmH665RjsW0IEPjTs2bHs3PBj+PCegO3s3OIBcPy0m/qHmVCwRa6MAIYKBD3jrsb/sNJsTifLBLC3EaANA=;7:D1++pEfe/HD3JquXsI6PiFPnJKGPnpQ36qB8m+TGZdC+gfg3TQYJO2AAMiXKur6aMxCnl/4nFTx0hXp2oPQ45Rf1OPKMBLF0Dq6pE9qgqoNZJ4JDZw16ztu2ULhAhKHMLjv60N2NGJk724ACDVunOkyP+phMtASCwR769CiNxDuEiDpWULDPZP7jc2EWuZ6tW31pYnU4malScNOuC4AKTSofK35LoxW4ICOGE3PoQJo+UDg7Zj5B7tjxgLOCr2RC SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Jan 2018 09:47:19.4312 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: dd5de3f4-0b7c-4db9-2437-08d55b33ce92 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR05MB1149 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Return-Path: Thanks Jianchao for your patch. And Thank you guys for your reviews, much appreciated. I was off-work on Friday and Saturday. On 14/01/2018 4:40 AM, jianchao.wang wrote: > Dear all > > Thanks for the kindly response and reviewing. That's really appreciated. > > On 01/13/2018 12:46 AM, Eric Dumazet wrote: >>> Does this need to be dma_wmb(), and should it be in >>> mlx4_en_update_rx_prod_db ? >>> >> +1 on dma_wmb() >> >> On what architecture bug was observed ? > This issue was observed on x86-64. > And I will send a new patch, in which replace wmb() with dma_wmb(), to customer > to confirm. +1 on dma_wmb, let us know once customer confirms. Please place it within mlx4_en_update_rx_prod_db as suggested. All other calls to mlx4_en_update_rx_prod_db are in control/slow path so I prefer being on the safe side, and care less about bulking the barrier. Thanks, Tariq