From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: From: Bart Van Assche To: "hch@lst.de" CC: "keith.busch@intel.com" , "linux-nvme@lists.infradead.org" , "linux-block@vger.kernel.org" , "axboe@kernel.dk" , "sagi@grimberg.me" Subject: Re: [PATCH 10/10] nvme: implement multipath access to nvme subsystems Date: Thu, 24 Aug 2017 20:17:32 +0000 Message-ID: <1503605798.2899.43.camel@wdc.com> References: <20170823175815.3646-1-hch@lst.de> <20170823175815.3646-11-hch@lst.de> <1503512512.2484.16.camel@wdc.com> <20170824085954.GD19504@lst.de> In-Reply-To: <20170824085954.GD19504@lst.de> Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 List-ID: T24gVGh1LCAyMDE3LTA4LTI0IGF0IDEwOjU5ICswMjAwLCBoY2hAbHN0LmRlIHdyb3RlOg0KPiBP biBXZWQsIEF1ZyAyMywgMjAxNyBhdCAwNjoyMTo1NVBNICswMDAwLCBCYXJ0IFZhbiBBc3NjaGUg d3JvdGU6DQo+ID4gU2luY2UgZ2VuZXJpY19tYWtlX3JlcXVlc3RfZmFzdCgpIHJldHVybnMgQkxL X1NUU19BR0FJTiBmb3IgYSBkeWluZyBwYXRoOg0KPiA+IGNhbiB0aGUgc2FtZSBraW5kIG9mIHNv ZnQgbG9ja3VwcyBvY2N1ciB3aXRoIHRoZSBOVk1lIG11bHRpcGF0aGluZyBjb2RlIGFzDQo+ID4g d2l0aCB0aGUgY3VycmVudCB1cHN0cmVhbSBkZXZpY2UgbWFwcGVyIG11bHRpcGF0aGluZyBjb2Rl PyBTZWUgZS5nLg0KPiA+ICJbUEFUQ0ggMy83XSBkbS1tcGF0aDogRG8gbm90IGxvY2sgdXAgYSBD UFUgd2l0aCByZXF1ZXVpbmcgYWN0aXZpdHkiDQo+ID4gKGh0dHBzOi8vd3d3LnJlZGhhdC5jb20v YXJjaGl2ZXMvZG0tZGV2ZWwvMjAxNy1BdWd1c3QvbXNnMDAxMjQuaHRtbCkuDQo+IA0KPiBJIHN1 c3BlY3QgdGhlIGNvZGUgaXMgbm90IGdvaW5nIHRvIGhpdCBpdCBiZWNhdXNlIHdlIGNoZWNrIHRo ZSBjb250cm9sbGVyDQo+IHN0YXRlIGJlZm9yZSB0cnlpbmcgdG8gcXVldWUgSS9PIG9uIHRoZSBs b3dlciBxdWV1ZS4gIEJ1dCBpZiB5b3UgcG9pbnQNCj4gbWUgdG8gYSBnb29kIHJlcHJvZHVjZXIg dGVzdCBjYXNlIEknZCBsaWtlIHRvIGNoZWNrLg0KDQpGb3IgTlZNZSBvdmVyIFJETUEsIGhvdyBh Ym91dCB0aGUgc2ltdWxhdGVfbmV0d29ya19mYWlsdXJlX2xvb3AoKSBmdW5jdGlvbiBpbg0KaHR0 cHM6Ly9naXRodWIuY29tL2J2YW5hc3NjaGUvc3JwLXRlc3QvYmxvYi9tYXN0ZXIvbGliL2Z1bmN0 aW9ucz8gSXQgc2ltdWxhdGVzDQphIG5ldHdvcmsgZmFpbHVyZSBieSB3cml0aW5nIGludG8gdGhl IHJlc2V0X2NvbnRyb2xsZXIgc3lzZnMgYXR0cmlidXRlLg0KDQo+IEFsc28gZG9lcyB0aGUgInNp bmdsZSBxdWV1ZSIgY2FzZSBpbiB5b3VyIG1haWwgcmVmZXIgdG8gdGhlIG9sZA0KPiByZXF1ZXN0 IGNvZGU/ICBudm1lIG9ubHkgdXNlcyBibGstbXEgc28gaXQgd291bGQgbm90IGhpdCB0aGF0Lg0K PiBCdXQgZWl0aGVyIHdheSBJIHRoaW5rIGdldF9yZXF1ZXN0IHNob3VsZCBiZSBmaXhlZCB0byBy ZXR1cm4NCj4gQkxLX1NUU19JT0VSUiBpZiB0aGUgcXVldWUgaXMgZHlpbmcgaW5zdGVhZCBvZiBC TEtfU1RTX0FHQUlOLg0KDQpUaGUgZGVzY3JpcHRpb24gaW4gdGhlIHBhdGNoIEkgcmVmZXJyZWQg dG8gaW5kZWVkIHJlZmVycyB0byB0aGUgb2xkIHJlcXVlc3QNCmNvZGUgaW4gdGhlIGJsb2NrIGxh eWVyLiBXaGVuIEkgcHJlcGFyZWQgdGhhdCBwYXRjaCBJIGhhZCBhbmFseXplZCB0aGUNCmJlaGF2 aW9yIG9mIHRoZSBvbGQgcmVxdWVzdCBjb2RlIG9ubHkuDQoNCkJhcnQu From mboxrd@z Thu Jan 1 00:00:00 1970 From: Bart.VanAssche@wdc.com (Bart Van Assche) Date: Thu, 24 Aug 2017 20:17:32 +0000 Subject: [PATCH 10/10] nvme: implement multipath access to nvme subsystems In-Reply-To: <20170824085954.GD19504@lst.de> References: <20170823175815.3646-1-hch@lst.de> <20170823175815.3646-11-hch@lst.de> <1503512512.2484.16.camel@wdc.com> <20170824085954.GD19504@lst.de> Message-ID: <1503605798.2899.43.camel@wdc.com> On Thu, 2017-08-24@10:59 +0200, hch@lst.de wrote: > On Wed, Aug 23, 2017@06:21:55PM +0000, Bart Van Assche wrote: > > Since generic_make_request_fast() returns BLK_STS_AGAIN for a dying path: > > can the same kind of soft lockups occur with the NVMe multipathing code as > > with the current upstream device mapper multipathing code? See e.g. > > "[PATCH 3/7] dm-mpath: Do not lock up a CPU with requeuing activity" > > (https://www.redhat.com/archives/dm-devel/2017-August/msg00124.html). > > I suspect the code is not going to hit it because we check the controller > state before trying to queue I/O on the lower queue. But if you point > me to a good reproducer test case I'd like to check. For NVMe over RDMA, how about the simulate_network_failure_loop() function in https://github.com/bvanassche/srp-test/blob/master/lib/functions? It simulates a network failure by writing into the reset_controller sysfs attribute. > Also does the "single queue" case in your mail refer to the old > request code? nvme only uses blk-mq so it would not hit that. > But either way I think get_request should be fixed to return > BLK_STS_IOERR if the queue is dying instead of BLK_STS_AGAIN. The description in the patch I referred to indeed refers to the old request code in the block layer. When I prepared that patch I had analyzed the behavior of the old request code only. Bart.