From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: From: Bart Van Assche To: "snitzer@redhat.com" , "axboe@kernel.dk" , "loberman@redhat.com" CC: "dm-devel@redhat.com" , "linux-kernel@vger.kernel.org" , "hch@infradead.org" , "linux-block@vger.kernel.org" , "osandov@fb.com" , "ming.lei@redhat.com" Subject: Re: [RFC PATCH] blk-mq: fixup RESTART when queue becomes idle Date: Thu, 18 Jan 2018 22:55:03 +0000 Message-ID: <1516316102.2676.97.camel@wdc.com> References: <20180118170353.GB19734@redhat.com> <1516296056.2676.23.camel@wdc.com> <20180118183039.GA20121@redhat.com> <1516301278.2676.35.camel@wdc.com> <20180118204856.GA31679@redhat.com> <1516309128.2676.38.camel@wdc.com> <20180118212327.GB31679@redhat.com> <1516311554.2676.50.camel@wdc.com> <20180118220132.GA20860@redhat.com> <1516313920.24506.4.camel@redhat.com> <1516314239.2676.79.camel@wdc.com> <1516314954.24506.6.camel@redhat.com> In-Reply-To: Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 List-ID: T24gVGh1LCAyMDE4LTAxLTE4IGF0IDE1OjM5IC0wNzAwLCBKZW5zIEF4Ym9lIHdyb3RlOg0KPiBX aGVuIHlvdSBkbyBoYXZlIGEgc29saWQgdGVzdCBjYXNlLCBwbGVhc2UgcGxlYXNlIHN1Ym1pdCBh IGJsa3Rlc3RzDQo+IHRlc3QgY2FzZSBmb3IgaXQhIFRoaXMgbmVlZHMgdG8gYmUgc29tZXRoaW5n IHdlIGNhbiByZWd1bGFybHkgaW4NCj4gdGVzdGluZy4NCg0KSGVsbG8gSmVucywNCg0KVGhhdCBz b3VuZHMgbGlrZSBhIGdvb2QgaWRlYSB0byBtZS4gQlRXLCBJIHRoaW5rIHRoZSByZWFzb24gd2h5 IHNvIGZhciBJDQpjYW4gcmVwcm9kdWNlIHRoZXNlIHF1ZXVlIHN0YWxscyBlYXNpZXIgdGhhbiBv dGhlcnMgaXMgYmVjYXVzZSBJIG1vZGlmaWVkIHRoZQ0KU1JQIGluaXRpYXRvciB0byBtYWtlIGl0 IGVhc3kgdG8gY2F1c2UgdGhlIC5nZXRfYnVkZ2V0KCkgY2FsbCB0byBzdWNjZWVkIGFuZA0KdGhl IHNjc2lfcXVldWVfcnEoKSB0byByZXR1cm4gQkxLX1NUU19CVVNZLiBBIHBvc3NpYmxlIG5leHQg c3RlcCBpcyB0byBhcHBseQ0KYSBzaW1pbGFyIGNoYW5nZSB0byB0aGUgc2NzaV9kZWJ1ZyBkcml2 ZXIuIFRoZSBjaGFuZ2UgSSBtYWRlIGlzIGFzIGZvbGxvd3M6DQoNCmRpZmYgLS1naXQgYS9kcml2 ZXJzL2luZmluaWJhbmQvdWxwL3NycC9pYl9zcnAuYyBiL2RyaXZlcnMvaW5maW5pYmFuZC91bHAv c3JwL2liX3NycC5jDQppbmRleCAwYzg4N2ViZmJjNjQuLjdmM2M0YTE5NzQyNSAxMDA2NDQNCi0t LSBhL2RyaXZlcnMvaW5maW5pYmFuZC91bHAvc3JwL2liX3NycC5jDQorKysgYi9kcml2ZXJzL2lu ZmluaWJhbmQvdWxwL3NycC9pYl9zcnAuYw0KQEAgLTMxMzUsNiArMzEzNSwxNiBAQCBzdGF0aWMg aW50IHNycF9yZXNldF9ob3N0KHN0cnVjdCBzY3NpX2NtbmQgKnNjbW5kKQ0KIAlyZXR1cm4gc3Jw X3JlY29ubmVjdF9ycG9ydCh0YXJnZXQtPnJwb3J0KSA9PSAwID8gU1VDQ0VTUyA6IEZBSUxFRDsN CiB9DQogDQorc3RhdGljIGludCBzcnBfdGFyZ2V0X2FsbG9jKHN0cnVjdCBzY3NpX3RhcmdldCAq c3RhcmdldCkNCit7DQorCXN0cnVjdCBTY3NpX0hvc3QgKnNob3N0ID0gZGV2X3RvX3Nob3N0KHN0 YXJnZXQtPmRldi5wYXJlbnQpOw0KKwlzdHJ1Y3Qgc3JwX3RhcmdldF9wb3J0ICp0YXJnZXQgPSBo b3N0X3RvX3RhcmdldChzaG9zdCk7DQorDQorCWlmICh0YXJnZXQtPnRhcmdldF9jYW5fcXVldWUp DQorCQlzdGFyZ2V0LT5jYW5fcXVldWUgPSB0YXJnZXQtPnRhcmdldF9jYW5fcXVldWU7DQorCXJl dHVybiAwOw0KK30NCisNCiBzdGF0aWMgaW50IHNycF9zbGF2ZV9hbGxvYyhzdHJ1Y3Qgc2NzaV9k ZXZpY2UgKnNkZXYpDQogew0KIAlzdHJ1Y3QgU2NzaV9Ib3N0ICpzaG9zdCA9IHNkZXYtPmhvc3Q7 DQpAQCAtMzM0OCw2ICszMzU4LDcgQEAgc3RhdGljIHN0cnVjdCBzY3NpX2hvc3RfdGVtcGxhdGUg c3JwX3RlbXBsYXRlID0gew0KIAkubW9kdWxlCQkJCT0gVEhJU19NT0RVTEUsDQogCS5uYW1lCQkJ CT0gIkluZmluaUJhbmQgU1JQIGluaXRpYXRvciIsDQogCS5wcm9jX25hbWUJCQk9IERSVl9OQU1F LA0KKwkudGFyZ2V0X2FsbG9jCQkJPSBzcnBfdGFyZ2V0X2FsbG9jLA0KIAkuc2xhdmVfYWxsb2MJ CQk9IHNycF9zbGF2ZV9hbGxvYywNCiAJLnNsYXZlX2NvbmZpZ3VyZQkJPSBzcnBfc2xhdmVfY29u ZmlndXJlLA0KIAkuaW5mbwkJCQk9IHNycF90YXJnZXRfaW5mbywNCkBAIC0zNTE1LDYgKzM1MjYs NyBAQCBlbnVtIHsNCiAJU1JQX09QVF9RVUVVRV9TSVpFCT0gMSA8PCAxNCwNCiAJU1JQX09QVF9J UF9TUkMJCT0gMSA8PCAxNSwNCiAJU1JQX09QVF9JUF9ERVNUCQk9IDEgPDwgMTYsDQorCVNSUF9P UFRfVEFSR0VUX0NBTl9RVUVVRT0gMSA8PCAxNywNCiB9Ow0KIA0KIHN0YXRpYyB1bnNpZ25lZCBp bnQgc3JwX29wdF9tYW5kYXRvcnlbXSA9IHsNCkBAIC0zNTM2LDYgKzM1NDgsNyBAQCBzdGF0aWMg Y29uc3QgbWF0Y2hfdGFibGVfdCBzcnBfb3B0X3Rva2VucyA9IHsNCiAJeyBTUlBfT1BUX1NFUlZJ Q0VfSUQsCQkic2VydmljZV9pZD0lcyIJCX0sDQogCXsgU1JQX09QVF9NQVhfU0VDVCwJCSJtYXhf c2VjdD0lZCIgCQl9LA0KIAl7IFNSUF9PUFRfTUFYX0NNRF9QRVJfTFVOLAkibWF4X2NtZF9wZXJf bHVuPSVkIiAJfSwNCisJeyBTUlBfT1BUX1RBUkdFVF9DQU5fUVVFVUUsCSJ0YXJnZXRfY2FuX3F1 ZXVlPSVkIgl9LA0KIAl7IFNSUF9PUFRfSU9fQ0xBU1MsCQkiaW9fY2xhc3M9JXgiCQl9LA0KIAl7 IFNSUF9PUFRfSU5JVElBVE9SX0VYVCwJImluaXRpYXRvcl9leHQ9JXMiCX0sDQogCXsgU1JQX09Q VF9DTURfU0dfRU5UUklFUywJImNtZF9zZ19lbnRyaWVzPSV1Igl9LA0KQEAgLTM3MjQsNiArMzcz NywxNSBAQCBzdGF0aWMgaW50IHNycF9wYXJzZV9vcHRpb25zKHN0cnVjdCBuZXQgKm5ldCwgY29u c3QgY2hhciAqYnVmLA0KIAkJCXRhcmdldC0+c2NzaV9ob3N0LT5jbWRfcGVyX2x1biA9IHRva2Vu Ow0KIAkJCWJyZWFrOw0KIA0KKwkJY2FzZSBTUlBfT1BUX1RBUkdFVF9DQU5fUVVFVUU6DQorCQkJ aWYgKG1hdGNoX2ludChhcmdzLCAmdG9rZW4pIHx8IHRva2VuIDwgMSkgew0KKwkJCQlwcl93YXJu KCJiYWQgbWF4IHRhcmdldF9jYW5fcXVldWUgcGFyYW1ldGVyICclcydcbiIsDQorCQkJCQlwKTsN CisJCQkJZ290byBvdXQ7DQorCQkJfQ0KKwkJCXRhcmdldC0+dGFyZ2V0X2Nhbl9xdWV1ZSA9IHRv a2VuOw0KKwkJCWJyZWFrOw0KKw0KIAkJY2FzZSBTUlBfT1BUX0lPX0NMQVNTOg0KIAkJCWlmICht YXRjaF9oZXgoYXJncywgJnRva2VuKSkgew0KIAkJCQlwcl93YXJuKCJiYWQgSU8gY2xhc3MgcGFy YW1ldGVyICclcydcbiIsIHApOw0KZGlmZiAtLWdpdCBhL2RyaXZlcnMvaW5maW5pYmFuZC91bHAv c3JwL2liX3NycC5oIGIvZHJpdmVycy9pbmZpbmliYW5kL3VscC9zcnAvaWJfc3JwLmgNCmluZGV4 IGQ2NmM5MDU3ZDVlYS4uNzAzMzRmYTNkZThlIDEwMDY0NA0KLS0tIGEvZHJpdmVycy9pbmZpbmli YW5kL3VscC9zcnAvaWJfc3JwLmgNCisrKyBiL2RyaXZlcnMvaW5maW5pYmFuZC91bHAvc3JwL2li X3NycC5oDQpAQCAtMjE2LDYgKzIxNiw3IEBAIHN0cnVjdCBzcnBfdGFyZ2V0X3BvcnQgew0KIAlj aGFyCQkJdGFyZ2V0X25hbWVbMzJdOw0KIAl1bnNpZ25lZCBpbnQJCXNjc2lfaWQ7DQogCXVuc2ln bmVkIGludAkJc2dfdGFibGVzaXplOw0KKwl1bnNpZ25lZCBpbnQJCXRhcmdldF9jYW5fcXVldWU7 DQogCWludAkJCW1yX3Bvb2xfc2l6ZTsNCiAJaW50CQkJbXJfcGVyX2NtZDsNCiAJaW50CQkJcXVl dWVfc2l6ZTsNCg0KQmFydC4= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932526AbeARWzR (ORCPT ); Thu, 18 Jan 2018 17:55:17 -0500 Received: from esa6.hgst.iphmx.com ([216.71.154.45]:20883 "EHLO esa6.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932131AbeARWzH (ORCPT ); Thu, 18 Jan 2018 17:55:07 -0500 X-IronPort-AV: E=Sophos;i="5.46,378,1511798400"; d="scan'208";a="69903315" From: Bart Van Assche To: "snitzer@redhat.com" , "axboe@kernel.dk" , "loberman@redhat.com" CC: "dm-devel@redhat.com" , "linux-kernel@vger.kernel.org" , "hch@infradead.org" , "linux-block@vger.kernel.org" , "osandov@fb.com" , "ming.lei@redhat.com" Subject: Re: [RFC PATCH] blk-mq: fixup RESTART when queue becomes idle Thread-Topic: [RFC PATCH] blk-mq: fixup RESTART when queue becomes idle Thread-Index: AQHTkKfyH9yE3b9g50uHm4ZmGS9HzaN6M0UAgAABfYCAAANUAIAAAOmAgAAEcAA= Date: Thu, 18 Jan 2018 22:55:03 +0000 Message-ID: <1516316102.2676.97.camel@wdc.com> References: <20180118170353.GB19734@redhat.com> <1516296056.2676.23.camel@wdc.com> <20180118183039.GA20121@redhat.com> <1516301278.2676.35.camel@wdc.com> <20180118204856.GA31679@redhat.com> <1516309128.2676.38.camel@wdc.com> <20180118212327.GB31679@redhat.com> <1516311554.2676.50.camel@wdc.com> <20180118220132.GA20860@redhat.com> <1516313920.24506.4.camel@redhat.com> <1516314239.2676.79.camel@wdc.com> <1516314954.24506.6.camel@redhat.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Bart.VanAssche@wdc.com; x-originating-ip: [199.255.44.172] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;CY1PR0401MB0907;7:BA5JEN/piStJyjNqYluIJ17pLzoEWpXFhOoUAiLUFomJNtMWtNPp2tLvuSHzfZkZfBwJFP5Ni7cNSHKPw4N5mYNc9D+rdgWMnGs2/pJVP8mpvwFV52+iklQm2OIbQeZf80e2DnhS0M12XEW51xAlmxzOnhREBPVYBkW/H99QyCD9xqH2Vqj0lP5iiCC9JlNmRTm8S72XInmT/yePdcxXDg45Pmqk3LBnv7oxJsHVb4fhpzXqp7LplzKTUB5bnGfp;20:ez4mygkSutWrcMg/ocgGqivviYU8/WJNwX1lQn6J595cO546KV8+4qInFOh50wFxiNi3bKidHidDr+7/5IXUq8bepxku/X56tsTqNOJlzFP/XWzcljU2xPX/BddFaWJjKRaYHrsibREAT31fLqxhLZQdQpGcZmSmLqmcMlZkf8s= x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-correlation-id: a66d1257-12e0-4bf7-005d-08d55ec68340 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(7020095)(4652020)(5600026)(4604075)(3008032)(48565401081)(2017052603307)(7153060)(7193020);SRVR:CY1PR0401MB0907; x-ms-traffictypediagnostic: CY1PR0401MB0907: wdcipoutbound: EOP-TRUE x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(6040470)(2401047)(5005006)(8121501046)(3002001)(93006095)(93001095)(3231023)(944501161)(10201501046)(6055026)(6041268)(20161123558120)(20161123564045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123562045)(6072148)(201708071742011);SRVR:CY1PR0401MB0907;BCL:0;PCL:0;RULEID:(100000803101)(100110400095);SRVR:CY1PR0401MB0907; x-forefront-prvs: 05568D1FF7 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(396003)(376002)(366004)(39860400002)(39380400002)(346002)(377424004)(199004)(189003)(51234002)(72206003)(102836004)(6506007)(229853002)(106356001)(14454004)(478600001)(4326008)(76176011)(25786009)(3280700002)(3660700001)(105586002)(97736004)(99286004)(86362001)(103116003)(54906003)(316002)(110136005)(26005)(6246003)(2201001)(93886005)(77096007)(5660300001)(6116002)(2906002)(3846002)(81156014)(8676002)(305945005)(7736002)(81166006)(68736007)(36756003)(6486002)(6512007)(66066001)(6436002)(2950100002)(8936002)(2501003)(2900100001)(53936002);DIR:OUT;SFP:1102;SCL:1;SRVR:CY1PR0401MB0907;H:CY1PR0401MB1536.namprd04.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; x-microsoft-antispam-message-info: u0SdK3pstl53h6SfARziy/wI1yiBHDWNYfy9jt41iCrA0lv/NTrx6VHUZZaKn4kGvpIH0BSSadvJ5uF1CuJyNw== spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="utf-8" Content-ID: <5EBDAE80C4F8E049AC4112F30A50ACA7@namprd04.prod.outlook.com> MIME-Version: 1.0 X-OriginatorOrg: wdc.com X-MS-Exchange-CrossTenant-Network-Message-Id: a66d1257-12e0-4bf7-005d-08d55ec68340 X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Jan 2018 22:55:04.0088 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b61c8803-16f3-4c35-9b17-6f65f441df86 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0401MB0907 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by mail.home.local id w0IMtM95018986 On Thu, 2018-01-18 at 15:39 -0700, Jens Axboe wrote: > When you do have a solid test case, please please submit a blktests > test case for it! This needs to be something we can regularly in > testing. Hello Jens, That sounds like a good idea to me. BTW, I think the reason why so far I can reproduce these queue stalls easier than others is because I modified the SRP initiator to make it easy to cause the .get_budget() call to succeed and the scsi_queue_rq() to return BLK_STS_BUSY. A possible next step is to apply a similar change to the scsi_debug driver. The change I made is as follows: diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c index 0c887ebfbc64..7f3c4a197425 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.c +++ b/drivers/infiniband/ulp/srp/ib_srp.c @@ -3135,6 +3135,16 @@ static int srp_reset_host(struct scsi_cmnd *scmnd) return srp_reconnect_rport(target->rport) == 0 ? SUCCESS : FAILED; } +static int srp_target_alloc(struct scsi_target *starget) +{ + struct Scsi_Host *shost = dev_to_shost(starget->dev.parent); + struct srp_target_port *target = host_to_target(shost); + + if (target->target_can_queue) + starget->can_queue = target->target_can_queue; + return 0; +} + static int srp_slave_alloc(struct scsi_device *sdev) { struct Scsi_Host *shost = sdev->host; @@ -3348,6 +3358,7 @@ static struct scsi_host_template srp_template = { .module = THIS_MODULE, .name = "InfiniBand SRP initiator", .proc_name = DRV_NAME, + .target_alloc = srp_target_alloc, .slave_alloc = srp_slave_alloc, .slave_configure = srp_slave_configure, .info = srp_target_info, @@ -3515,6 +3526,7 @@ enum { SRP_OPT_QUEUE_SIZE = 1 << 14, SRP_OPT_IP_SRC = 1 << 15, SRP_OPT_IP_DEST = 1 << 16, + SRP_OPT_TARGET_CAN_QUEUE= 1 << 17, }; static unsigned int srp_opt_mandatory[] = { @@ -3536,6 +3548,7 @@ static const match_table_t srp_opt_tokens = { { SRP_OPT_SERVICE_ID, "service_id=%s" }, { SRP_OPT_MAX_SECT, "max_sect=%d" }, { SRP_OPT_MAX_CMD_PER_LUN, "max_cmd_per_lun=%d" }, + { SRP_OPT_TARGET_CAN_QUEUE, "target_can_queue=%d" }, { SRP_OPT_IO_CLASS, "io_class=%x" }, { SRP_OPT_INITIATOR_EXT, "initiator_ext=%s" }, { SRP_OPT_CMD_SG_ENTRIES, "cmd_sg_entries=%u" }, @@ -3724,6 +3737,15 @@ static int srp_parse_options(struct net *net, const char *buf, target->scsi_host->cmd_per_lun = token; break; + case SRP_OPT_TARGET_CAN_QUEUE: + if (match_int(args, &token) || token < 1) { + pr_warn("bad max target_can_queue parameter '%s'\n", + p); + goto out; + } + target->target_can_queue = token; + break; + case SRP_OPT_IO_CLASS: if (match_hex(args, &token)) { pr_warn("bad IO class parameter '%s'\n", p); diff --git a/drivers/infiniband/ulp/srp/ib_srp.h b/drivers/infiniband/ulp/srp/ib_srp.h index d66c9057d5ea..70334fa3de8e 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.h +++ b/drivers/infiniband/ulp/srp/ib_srp.h @@ -216,6 +216,7 @@ struct srp_target_port { char target_name[32]; unsigned int scsi_id; unsigned int sg_tablesize; + unsigned int target_can_queue; int mr_pool_size; int mr_per_cmd; int queue_size; Bart.