From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.windriver.com (mail.windriver.com [147.11.1.11]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 9530C211D59B2 for ; Sun, 10 Mar 2019 17:45:47 -0700 (PDT) From: "Liu, Yongxin" Subject: RE: [PATCH RT] nvdimm: make lane acquirement RT aware Date: Mon, 11 Mar 2019 00:44:58 +0000 Message-ID: <597B109EC20B76429F71A8A97770610D12A5643B@ALA-MBD.corp.ad.wrs.com> References: <20190306095709.23138-1-yongxin.liu@windriver.com> <20190307143344.ytsnbmot5tjzjhip@linutronix.de> <597B109EC20B76429F71A8A97770610D12A52669@ALA-MBD.corp.ad.wrs.com> <20190308094131.ge4wbsvz4p6xikdf@linutronix.de> In-Reply-To: <20190308094131.ge4wbsvz4p6xikdf@linutronix.de> Content-Language: en-US MIME-Version: 1.0 List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" To: Sebastian Andrzej Siewior Cc: "linux-rt-users@vger.kernel.org" , "linux-nvdimm@lists.01.org" , "linux-kernel@vger.kernel.org" , "rostedt@goodmis.org" , "Gortmaker, Paul , tglx@linutronix.de" List-ID: > -----Original Message----- > From: linux-kernel-owner@vger.kernel.org [mailto:linux-kernel- > owner@vger.kernel.org] On Behalf Of Sebastian Andrzej Siewior > Sent: Friday, March 8, 2019 17:42 > To: Liu, Yongxin > Cc: linux-kernel@vger.kernel.org; linux-rt-users@vger.kernel.org; > tglx@linutronix.de; rostedt@goodmis.org; dan.j.williams@intel.com; > pagupta@redhat.com; Gortmaker, Paul; linux-nvdimm@lists.01.org > Subject: Re: [PATCH RT] nvdimm: make lane acquirement RT aware > > On 2019-03-08 00:07:41 [+0000], Liu, Yongxin wrote: > > The lane is critical resource which needs to be protected. One CPU can > use only one > > lane. If CPU number is greater than the number of total lane, the lane > can be shared > > among CPUs. > > > > In non-RT kernel, get_cpu() disable preemption by calling > preempt_disable() first. > > Only one thread on the same CPU can get the lane. > > > > In RT kernel, if we only use raw_smp_processor_id(), this doesn't > protect the lane. > > Thus two threads on the same CPU can get the same lane at the same time. > > > > In this patch, two-level lock can avoid race condition for the lane. > > but you still have the ndl_lock->lock which protects the resource. So in > the unlikely (but possible event) that you switch CPUs after obtaining > the CPU number you block on the lock. No harm is done, right? The resource "lane" can be acquired recursively, so "ndl_lock->lock" is a conditional lock. ndl_count->count is per CPU. ndl_lock->lock is per lane. Here is an example: Thread A on CPU 5 --> nd_region_acquire_lane --> lane# 5 --> get "ndl_lock->lock" --> nd_region_acquire_lane --> lane# 5 --> bypass "ndl_lock->lock" due to "ndl_count->count++". Thread B on CPU 5 --> nd_region_acquire_lane --> lane# 5 --> bypass "ndl_lock->lock" ("ndl_count->count" was changed by Thread A) If we use raw_smp_processor_id(), no matter which CPU the thread was migrated to, if there is another thread running on the old CPU, there will be race condition due to per CPU variable "ndl_count->count". Thanks, Yongxin > > Sebastian _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11BBEC43381 for ; Mon, 11 Mar 2019 00:45:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id DBA2A20652 for ; Mon, 11 Mar 2019 00:45:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727128AbfCKApu (ORCPT ); Sun, 10 Mar 2019 20:45:50 -0400 Received: from mail.windriver.com ([147.11.1.11]:44571 "EHLO mail.windriver.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726879AbfCKApt (ORCPT ); Sun, 10 Mar 2019 20:45:49 -0400 Received: from ALA-HCA.corp.ad.wrs.com ([147.11.189.40]) by mail.windriver.com (8.15.2/8.15.1) with ESMTPS id x2B0j0ib009225 (version=TLSv1 cipher=AES128-SHA bits=128 verify=FAIL); Sun, 10 Mar 2019 17:45:00 -0700 (PDT) Received: from ALA-MBD.corp.ad.wrs.com ([169.254.3.68]) by ALA-HCA.corp.ad.wrs.com ([147.11.189.40]) with mapi id 14.03.0439.000; Sun, 10 Mar 2019 17:44:59 -0700 From: "Liu, Yongxin" To: Sebastian Andrzej Siewior CC: "linux-kernel@vger.kernel.org" , "linux-rt-users@vger.kernel.org" , "tglx@linutronix.de" , "rostedt@goodmis.org" , "dan.j.williams@intel.com" , "pagupta@redhat.com" , "Gortmaker, Paul" , "linux-nvdimm@lists.01.org" Subject: RE: [PATCH RT] nvdimm: make lane acquirement RT aware Thread-Topic: [PATCH RT] nvdimm: make lane acquirement RT aware Thread-Index: AQHU1AQaa42orSwEx0mmBzBs0WZNkaYAw5wAgAAVVLCAAStcgIADmlcQ Date: Mon, 11 Mar 2019 00:44:58 +0000 Message-ID: <597B109EC20B76429F71A8A97770610D12A5643B@ALA-MBD.corp.ad.wrs.com> References: <20190306095709.23138-1-yongxin.liu@windriver.com> <20190307143344.ytsnbmot5tjzjhip@linutronix.de> <597B109EC20B76429F71A8A97770610D12A52669@ALA-MBD.corp.ad.wrs.com> <20190308094131.ge4wbsvz4p6xikdf@linutronix.de> In-Reply-To: <20190308094131.ge4wbsvz4p6xikdf@linutronix.de> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [128.224.162.171] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org DQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IGxpbnV4LWtlcm5lbC1vd25l ckB2Z2VyLmtlcm5lbC5vcmcgW21haWx0bzpsaW51eC1rZXJuZWwtDQo+IG93bmVyQHZnZXIua2Vy bmVsLm9yZ10gT24gQmVoYWxmIE9mIFNlYmFzdGlhbiBBbmRyemVqIFNpZXdpb3INCj4gU2VudDog RnJpZGF5LCBNYXJjaCA4LCAyMDE5IDE3OjQyDQo+IFRvOiBMaXUsIFlvbmd4aW4NCj4gQ2M6IGxp bnV4LWtlcm5lbEB2Z2VyLmtlcm5lbC5vcmc7IGxpbnV4LXJ0LXVzZXJzQHZnZXIua2VybmVsLm9y ZzsNCj4gdGdseEBsaW51dHJvbml4LmRlOyByb3N0ZWR0QGdvb2RtaXMub3JnOyBkYW4uai53aWxs aWFtc0BpbnRlbC5jb207DQo+IHBhZ3VwdGFAcmVkaGF0LmNvbTsgR29ydG1ha2VyLCBQYXVsOyBs aW51eC1udmRpbW1AbGlzdHMuMDEub3JnDQo+IFN1YmplY3Q6IFJlOiBbUEFUQ0ggUlRdIG52ZGlt bTogbWFrZSBsYW5lIGFjcXVpcmVtZW50IFJUIGF3YXJlDQo+IA0KPiBPbiAyMDE5LTAzLTA4IDAw OjA3OjQxIFsrMDAwMF0sIExpdSwgWW9uZ3hpbiB3cm90ZToNCj4gPiBUaGUgbGFuZSBpcyBjcml0 aWNhbCByZXNvdXJjZSB3aGljaCBuZWVkcyB0byBiZSBwcm90ZWN0ZWQuIE9uZSBDUFUgY2FuDQo+ IHVzZSBvbmx5IG9uZQ0KPiA+IGxhbmUuIElmIENQVSBudW1iZXIgaXMgZ3JlYXRlciB0aGFuIHRo ZSBudW1iZXIgb2YgdG90YWwgbGFuZSwgdGhlIGxhbmUNCj4gY2FuIGJlIHNoYXJlZA0KPiA+IGFt b25nIENQVXMuDQo+ID4NCj4gPiBJbiBub24tUlQga2VybmVsLCBnZXRfY3B1KCkgZGlzYWJsZSBw cmVlbXB0aW9uIGJ5IGNhbGxpbmcNCj4gcHJlZW1wdF9kaXNhYmxlKCkgZmlyc3QuDQo+ID4gT25s eSBvbmUgdGhyZWFkIG9uIHRoZSBzYW1lIENQVSBjYW4gZ2V0IHRoZSBsYW5lLg0KPiA+DQo+ID4g SW4gUlQga2VybmVsLCBpZiB3ZSBvbmx5IHVzZSByYXdfc21wX3Byb2Nlc3Nvcl9pZCgpLCB0aGlz IGRvZXNuJ3QNCj4gcHJvdGVjdCB0aGUgbGFuZS4NCj4gPiBUaHVzIHR3byB0aHJlYWRzIG9uIHRo ZSBzYW1lIENQVSBjYW4gZ2V0IHRoZSBzYW1lIGxhbmUgYXQgdGhlIHNhbWUgdGltZS4NCj4gPg0K PiA+IEluIHRoaXMgcGF0Y2gsIHR3by1sZXZlbCBsb2NrIGNhbiBhdm9pZCByYWNlIGNvbmRpdGlv biBmb3IgdGhlIGxhbmUuDQo+IA0KPiBidXQgeW91IHN0aWxsIGhhdmUgdGhlIG5kbF9sb2NrLT5s b2NrIHdoaWNoIHByb3RlY3RzIHRoZSByZXNvdXJjZS4gU28gaW4NCj4gdGhlIHVubGlrZWx5IChi dXQgcG9zc2libGUgZXZlbnQpIHRoYXQgeW91IHN3aXRjaCBDUFVzIGFmdGVyIG9idGFpbmluZw0K PiB0aGUgQ1BVIG51bWJlciB5b3UgYmxvY2sgb24gdGhlIGxvY2suIE5vIGhhcm0gaXMgZG9uZSwg cmlnaHQ/DQoNClRoZSByZXNvdXJjZSAibGFuZSIgY2FuIGJlIGFjcXVpcmVkIHJlY3Vyc2l2ZWx5 LCBzbyAibmRsX2xvY2stPmxvY2siIGlzIGEgY29uZGl0aW9uYWwgbG9jay4NCg0KbmRsX2NvdW50 LT5jb3VudCBpcyBwZXIgQ1BVLg0KbmRsX2xvY2stPmxvY2sgaXMgcGVyIGxhbmUuDQoNCkhlcmUg aXMgYW4gZXhhbXBsZToNClRocmVhZCBBICBvbiBDUFUgNSAtLT4gbmRfcmVnaW9uX2FjcXVpcmVf bGFuZSAtLT4gbGFuZSMgNSAtLT4gZ2V0ICJuZGxfbG9jay0+bG9jayINCi0tPiBuZF9yZWdpb25f YWNxdWlyZV9sYW5lIC0tPiBsYW5lIyA1IC0tPiBieXBhc3MgIm5kbF9sb2NrLT5sb2NrIiBkdWUg dG8gIm5kbF9jb3VudC0+Y291bnQrKyIuDQoNClRocmVhZCBCIG9uIENQVSA1IC0tPiBuZF9yZWdp b25fYWNxdWlyZV9sYW5lIC0tPiBsYW5lIyA1IC0tPiBieXBhc3MgIm5kbF9sb2NrLT5sb2NrIiAo Im5kbF9jb3VudC0+Y291bnQiDQp3YXMgY2hhbmdlZCBieSBUaHJlYWQgQSkNCg0KSWYgd2UgdXNl IHJhd19zbXBfcHJvY2Vzc29yX2lkKCksIG5vIG1hdHRlciB3aGljaCBDUFUgdGhlIHRocmVhZCB3 YXMgbWlncmF0ZWQgdG8sIA0KaWYgdGhlcmUgaXMgYW5vdGhlciB0aHJlYWQgcnVubmluZyBvbiB0 aGUgb2xkIENQVSwgdGhlcmUgd2lsbCBiZSByYWNlIGNvbmRpdGlvbiANCmR1ZSB0byBwZXIgQ1BV IHZhcmlhYmxlICJuZGxfY291bnQtPmNvdW50Ii4NCg0KDQpUaGFua3MsDQpZb25neGluDQoNCj4g DQo+IFNlYmFzdGlhbg0K