From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: From: Bart Van Assche To: "peterz@infradead.org" CC: "linux-kernel@vger.kernel.org" , "linux-block@vger.kernel.org" , "kernel-team@fb.com" , "oleg@redhat.com" , "hch@lst.de" , "axboe@kernel.dk" , "jianchao.w.wang@oracle.com" , "osandov@fb.com" , "tj@kernel.org" Subject: Re: [PATCH 2/6] blk-mq: replace timeout synchronization with a RCU and generation based scheme Date: Thu, 14 Dec 2017 21:42:48 +0000 Message-ID: <1513287766.2475.73.camel@wdc.com> References: <20171212190134.535941-1-tj@kernel.org> <20171212190134.535941-3-tj@kernel.org> <1513277469.2475.43.camel@wdc.com> <20171214202042.GG3326@worktop> In-Reply-To: <20171214202042.GG3326@worktop> Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 List-ID: T24gVGh1LCAyMDE3LTEyLTE0IGF0IDIxOjIwICswMTAwLCBQZXRlciBaaWpsc3RyYSB3cm90ZToN Cj4gT24gVGh1LCBEZWMgMTQsIDIwMTcgYXQgMDY6NTE6MTFQTSArMDAwMCwgQmFydCBWYW4gQXNz Y2hlIHdyb3RlOg0KPiA+IE9uIFR1ZSwgMjAxNy0xMi0xMiBhdCAxMTowMSAtMDgwMCwgVGVqdW4g SGVvIHdyb3RlOg0KPiA+ID4gKwl3cml0ZV9zZXFjb3VudF9iZWdpbigmcnEtPmdzdGF0ZV9zZXEp Ow0KPiA+ID4gKwlibGtfbXFfcnFfdXBkYXRlX3N0YXRlKHJxLCBNUV9SUV9JTl9GTElHSFQpOw0K PiA+ID4gKwlibGtfYWRkX3RpbWVyKHJxKTsNCj4gPiA+ICsJd3JpdGVfc2VxY291bnRfZW5kKCZy cS0+Z3N0YXRlX3NlcSk7DQo+ID4gDQo+ID4gTXkgdW5kZXJzdGFuZGluZyBpcyB0aGF0IGJvdGgg d3JpdGVfc2VxY291bnRfYmVnaW4oKSBhbmQgd3JpdGVfc2VxY291bnRfZW5kKCkNCj4gPiB0cmln Z2VyIGEgd3JpdGUgbWVtb3J5IGJhcnJpZXIuIElzIGEgc2VxY291bnQgcmVhbGx5IGZhc3RlciB0 aGFuIGEgc3BpbmxvY2s/DQo+IA0KPiBZZXMgbG90cywgbm8gYXRvbWljIG9wZXJhdGlvbnMgYW5k IG5vIHdhaXRpbmcuDQo+IA0KPiBUaGUgb25seSBjb25zdHJhaW50IGZvciB3cml0ZV9zZXFsb2Nr IGlzIHRoYXQgdGhlcmUgbXVzdCBub3QgYmUgYW55DQo+IGNvbmN1cnJlbmN5Lg0KPiANCj4gQnV0 IG5vdyB0aGF0IEkgbG9vayBhdCB0aGlzIGFnYWluLCBUSiwgd2h5IGNhbid0IHRoZSBiZWxvdyBo YXBwZW4/DQo+IA0KPiAJd3JpdGVfc2VxbG9ja19iZWdpbigpOw0KPiAJYmxrX21xX3JxX3VwZGF0 ZV9zdGF0ZShycSwgSU5fRkxJR0hUKTsNCj4gCWJsa19hZGRfdGltZXIocnEpOw0KPiAJPHRpbWVy LWlycT4NCj4gCQlyZWFkX3NlcWNvdW50X2JlZ2luKCkNCj4gCQkJd2hpbGUgKHNlcSAmIDEpDQo+ IAkJCQljcHVyZWxheCgpOw0KPiAJCS8vIGxpZmUtbG9jaw0KPiAJPC90aW1lci1pcnE+DQo+IAl3 cml0ZV9zZXFsb2NrX2VuZCgpOw0KDQpIZWxsbyBQZXRlciwNCg0KU29tZSB0aW1lIGFnbyB0aGUg YmxvY2sgbGF5ZXIgd2FzIGNoYW5nZWQgdG8gaGFuZGxlIHRpbWVvdXRzIGluIHRocmVhZCBjb250 ZXh0DQppbnN0ZWFkIG9mIGludGVycnVwdCBjb250ZXh0LiBTZWUgYWxzbyBjb21taXQgMjg3OTIy ZWIwYjE4ICgiYmxvY2s6IGRlZmVyDQp0aW1lb3V0cyB0byBhIHdvcmtxdWV1ZSIpLg0KDQpCYXJ0 Lg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753248AbdLNVmy (ORCPT ); Thu, 14 Dec 2017 16:42:54 -0500 Received: from esa2.hgst.iphmx.com ([68.232.143.124]:31833 "EHLO esa2.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752154AbdLNVmw (ORCPT ); Thu, 14 Dec 2017 16:42:52 -0500 X-IronPort-AV: E=Sophos;i="5.45,401,1508774400"; d="scan'208";a="161344703" From: Bart Van Assche To: "peterz@infradead.org" CC: "linux-kernel@vger.kernel.org" , "linux-block@vger.kernel.org" , "kernel-team@fb.com" , "oleg@redhat.com" , "hch@lst.de" , "axboe@kernel.dk" , "jianchao.w.wang@oracle.com" , "osandov@fb.com" , "tj@kernel.org" Subject: Re: [PATCH 2/6] blk-mq: replace timeout synchronization with a RCU and generation based scheme Thread-Topic: [PATCH 2/6] blk-mq: replace timeout synchronization with a RCU and generation based scheme Thread-Index: AQHTc3vSEJieVXIS9EOPhdZv8hd1dKNDMgqAgAAZBQCAABbuAA== Date: Thu, 14 Dec 2017 21:42:48 +0000 Message-ID: <1513287766.2475.73.camel@wdc.com> References: <20171212190134.535941-1-tj@kernel.org> <20171212190134.535941-3-tj@kernel.org> <1513277469.2475.43.camel@wdc.com> <20171214202042.GG3326@worktop> In-Reply-To: <20171214202042.GG3326@worktop> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [199.255.44.171] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;BY1PR0401MB1530;20:MYX46/bx8wofPE05qI95WR+u7fTOHeTcBG11GbsIrDqNo6W1J7zkebP4E5omyaCdf8ZB1KrD/2+miPUOgwx/HvUNaOBIh/GqhEcoSJNlgtRRtzGlTb/DjdyBRgw0gM4EHeL5YNU9Zvn7tOnY6wWVEpuvDK/317jXEzZnhCwtTN0= x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-correlation-id: 27138866-f98a-488d-18d2-08d5433b9e77 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(5600026)(4604075)(4534020)(4602075)(4627115)(201703031133081)(201702281549075)(48565401081)(2017052603307)(7153051);SRVR:BY1PR0401MB1530; x-ms-traffictypediagnostic: BY1PR0401MB1530: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Bart.VanAssche@wdc.com; wdcipoutbound: EOP-TRUE x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(788757137089); x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(6040450)(2401047)(5005006)(8121501046)(10201501046)(3002001)(93006095)(93001095)(3231023)(6055026)(6041248)(20161123562025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123558100)(20161123555025)(20161123564025)(20161123560025)(6072148)(201708071742011);SRVR:BY1PR0401MB1530;BCL:0;PCL:0;RULEID:(100000803101)(100110400095);SRVR:BY1PR0401MB1530; x-forefront-prvs: 05214FD68E x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(396003)(366004)(346002)(376002)(39860400002)(377424004)(24454002)(199004)(189003)(53936002)(478600001)(14454004)(575784001)(54906003)(72206003)(2351001)(2900100001)(86362001)(3660700001)(6246003)(3280700002)(106356001)(105586002)(316002)(93886005)(5250100002)(25786009)(103116003)(99286004)(6512007)(97736004)(7416002)(66066001)(1730700003)(5660300001)(8676002)(81156014)(81166006)(36756003)(7736002)(4001150100001)(59450400001)(229853002)(5640700003)(6916009)(76176011)(6436002)(6486002)(6506007)(305945005)(68736007)(102836003)(6116002)(2501003)(3846002)(8936002)(4326008)(2906002)(2950100002);DIR:OUT;SFP:1102;SCL:1;SRVR:BY1PR0401MB1530;H:BY1PR0401MB1532.namprd04.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="utf-8" Content-ID: <24C8802F26D2FF4396AE9D1F7E40E03A@namprd04.prod.outlook.com> MIME-Version: 1.0 X-OriginatorOrg: wdc.com X-MS-Exchange-CrossTenant-Network-Message-Id: 27138866-f98a-488d-18d2-08d5433b9e77 X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Dec 2017 21:42:48.2436 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b61c8803-16f3-4c35-9b17-6f65f441df86 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY1PR0401MB1530 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by nfs id vBELh0We013445 On Thu, 2017-12-14 at 21:20 +0100, Peter Zijlstra wrote: > On Thu, Dec 14, 2017 at 06:51:11PM +0000, Bart Van Assche wrote: > > On Tue, 2017-12-12 at 11:01 -0800, Tejun Heo wrote: > > > + write_seqcount_begin(&rq->gstate_seq); > > > + blk_mq_rq_update_state(rq, MQ_RQ_IN_FLIGHT); > > > + blk_add_timer(rq); > > > + write_seqcount_end(&rq->gstate_seq); > > > > My understanding is that both write_seqcount_begin() and write_seqcount_end() > > trigger a write memory barrier. Is a seqcount really faster than a spinlock? > > Yes lots, no atomic operations and no waiting. > > The only constraint for write_seqlock is that there must not be any > concurrency. > > But now that I look at this again, TJ, why can't the below happen? > > write_seqlock_begin(); > blk_mq_rq_update_state(rq, IN_FLIGHT); > blk_add_timer(rq); > > read_seqcount_begin() > while (seq & 1) > cpurelax(); > // life-lock > > write_seqlock_end(); Hello Peter, Some time ago the block layer was changed to handle timeouts in thread context instead of interrupt context. See also commit 287922eb0b18 ("block: defer timeouts to a workqueue"). Bart.