From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752373AbdAMPX1 (ORCPT ); Fri, 13 Jan 2017 10:23:27 -0500 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:58873 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752032AbdAMPXX (ORCPT ); Fri, 13 Jan 2017 10:23:23 -0500 Subject: Re: [PATCHSET v6] blk-mq scheduling framework To: Hannes Reinecke , , References: <1484170803-9311-1-git-send-email-axboe@fb.com> <5850a4ad-74a1-8d30-e865-fbd35679e84f@suse.de> CC: , From: Jens Axboe Message-ID: <2e968ed9-7269-488b-4c38-b318ad99ed3f@fb.com> Date: Fri, 13 Jan 2017 08:23:07 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <5850a4ad-74a1-8d30-e865-fbd35679e84f@suse.de> Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 7bit X-Originating-IP: [216.160.245.98] X-ClientProxiedBy: BN6PR06CA0003.namprd06.prod.outlook.com (10.175.123.141) To MWHPR15MB1199.namprd15.prod.outlook.com (10.175.2.141) X-MS-Office365-Filtering-Correlation-Id: c0e16665-fdc1-4aae-7d42-08d43bc817fb X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:MWHPR15MB1199; X-Microsoft-Exchange-Diagnostics: 1;MWHPR15MB1199;3:XeMGQ4ctBQhuwgn+jVhOLDZWvxZbU9aSuC1FW/ALyUdLW1G90N43bWjqoRipQHRo8tSLUkm0GL4yEeHzSIlPZ0Uy7x4lBkt0Co1v3ZIN5TjvOdQzW8VEtqui15Aiaq2sa5A9ePnIQNzxlT81BhzNqR6+pNCWa/FsQ3EU1dJcF+h0RLQdckTH3nWbQ57EJum5vTEfgzfSAqZ07SkFMPQLg3Us+BXsJTFmdQwBeVgEMed7A2yquItRefjWzEm4DTpuPO5VX5QiomBOWnySM2Me4w== X-Microsoft-Exchange-Diagnostics: 1;MWHPR15MB1199;25:PFikY6lsiH0LLTNVOwa0+8bcRLPT/usuaEhntPKJXYpwRKQ4UAJHvivDlB1Ayq70UTaW1WOirdPNYU5/KAg0ZLrtf/zyDfug39kD3ZnX+gQPNNGpays1vkDgB6YYkDXSJIiXrUElQ/ZLulLGCkk4APYquyU3Yxe+JCH4PQAqBRBRHamXEw6UVlvrCx2O6nP3XUHbGQV+1ybpl7kbcq1CY3UHmGMWl439SUiN1ac3cndhJh9/5hK8Vw1sm1TucdwyP38GbQm6HRxXJypM8KXKYDJX+eJ08qRBFiBO8r1NtA2w2reiJv4/MnovJqlz4KCTI7isSYN557eR6RO0B/FLtBsSdm81DRXqgc4DxKfSdgwGWfcjCf91WEUxgWyzF1arDHa390jWcY0sowokmwVL/2Kf/3OaTZnKHrC0PVN/gp7FPA3oPibURea6252BAKXGiRqmxQHx0S+Cn6eudiE3DOn2FSPkTOQTZwpfzQV+IQjhbg9pMeWvDBymQu6CMslkJYFNB8kua2NBtKCZsIWVTB6Js0qZAADgZIq8eF5Y3k4UZbu6F/UXAQhJoXw8K6qla76Bi8Nirffv744VAncnBeC7du8pvaExQE1374nV6pubZKhFq/LLMvcfaI/O1LSNOnxbbRQGifUS6LqZTVZ0dGAt89PcGhyuaA33ZvfuJJbP/1WgjOXyncoB6mBcfLbaaYBnGKxGO+tqi0xm+5iYVkryydYfobgaspXovvIMJDL9Nia40C7ItrkOh+9yOOXSw1htSTAz+5ilHPahyBLFipHZFQ7THzqe2fy5tl5cjpNeVJyncNjG2Fw9L66TWDUcg5aLC4bdMnQUSW+MpN6CU06TT91o0UWhK8OScGp1PTw0PovmQTRgSjeDan+e0ULe X-Microsoft-Exchange-Diagnostics: 1;MWHPR15MB1199;31:4ob1Xp9lz1+aAHRcNEGQkV1nNeXpzeryj/eNqaELMcBQ0wxBDLymmfDRQGIl/WgxrY+ENiURZ9g3M9Z6T4sFj+QyZEjnz3uzi27aeTzqCidwaHNsySA4+8Dm002m87SV6wSvSX9FHT/ru4/x7k1ReugY17c4J8TJTRnR8FD54KGvwrz/FzSy6z9GtJpPUQkRc4/QussEcwdloye54MYWHPQ3glNAPgnblnWT5iXzJOkFbm6dG/a7ema6DvC/oODkgQaVBKPLKHOu2dGqseWhpQ==;20:MTREJF0BossIGaqoZM2pNBjACj2onqmvTX/bEUvGpUjDweCKL+BpJ2qvT46iX9bscPB5NEFFB3ClbvVbwaYR5ONgOM+w08mfnwFHTGCpILTEgQFEiUUdigbpHpxDwahFEJ5HY7d9iYi4IcMnobAdtTF4nFFdObbQWpSUtk508f/nCjvVWh26HW4sF/tleMlMFY3FUN+KIaJR9G2v6IsreOjglinESsbR/SYO7iLI/2oDwhn2OtLO/RKosphqnV7p2+YKWCb6N3Ox81ZwhpTzOUM1b7HNqhbyN/FAPGgmR9KpcEuhF3fiddIIaJVg4YBF0IDw7bURTgW8QS4zx44wt9SNPDFLxSO1szagpxRslL1igviiwDkgBusAH7MybwUj4+RXKqH5ejgT6XbiRqU4i1vfg2DlFP7c9TvqxheDCvSZINaMzQJNYOhSSBPTb9TSss06EBluWEZGEXNPs1QVQV+uD0IY/a7dqCeN6IbnpK/MYtHj+Xit371gk2Puo/Zn X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040375)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6041248)(20161123562025)(20161123555025)(20161123564025)(20161123560025)(6072148);SRVR:MWHPR15MB1199;BCL:0;PCL:0;RULEID:;SRVR:MWHPR15MB1199; X-Microsoft-Exchange-Diagnostics: 1;MWHPR15MB1199;4:fg/dfJTY+VjngQO7Q9/VCigHvn8wC6H1re1l4kHjf3qVXgPx5sBscg6nlypuhghwxb8r/6nemOtzNqPaIYXPVFpONA6YCzhhSlKE96dnRblzs5NL5njFyl4nRkPY8CmZ4oau2RnfqUPpxTd7zZw7vI6O5OVp8zHCPwCIceKVfjN5DhvQLZs0IqrU5GHsAHDFLVsCulA7ZpcZsqZNdJnP5YTj2DVSEWVBwRJouiBUDgVI5MmTx1XxmWTznlyCeJ0jsHmCSDZZ3OkGVAQ7J9vhIyCLZFr3aNaX9r0WXWX5Yffd0UD8w3Pfb98/oJyty7qaOr++cxn1h+hxDEyMe4iHifMxW7n3EOHfjS21mVTcAArt2j60b3POW5tKQovm0LO5/J9+iS27Z1CStygouPhQCRtcUO6JERrS3YBEpSjxsgh3YgItdKVEAVp29/B9GExLp6DvstN5t6PYKe4NZAS1dM97nhmBY0sN+OopbbX+y75BtQoFAnsETZ3Ehd8VlhSd30wCphlSgN6iYF5bfyMzBvGRBmjv5FmzdOcIz0Mq4XJo2Y0JfyKSZEkQqI3+zbOsdTHmyonNbiB1Z7JPLRq0rw== X-Forefront-PRVS: 018632C080 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6049001)(6009001)(7916002)(39410400002)(39830400002)(39450400003)(377454003)(51234002)(189002)(24454002)(199003)(105586002)(106356001)(25786008)(54906002)(36756003)(2201001)(305945005)(6486002)(64126003)(117156001)(2950100002)(229853002)(86362001)(6666003)(33646002)(38730400001)(92566002)(77096006)(8656002)(42186005)(90366009)(68736007)(50466002)(7736002)(5890100001)(31696002)(47776003)(4326007)(66066001)(65826007)(81166006)(81156014)(2906002)(8676002)(50986999)(65806001)(6306002)(5660300001)(5001770100001)(189998001)(65956001)(76176999)(23746002)(97736004)(54356999)(6116002)(83506001)(3846002)(230700001)(101416001)(4001350100001)(31686004)(122876001);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR15MB1199;H:[192.168.1.129];FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1;MWHPR15MB1199;23:v28LKCNNGZl699/GqKK1xLxRnpgV/Wq1gWGMr?= =?Windows-1252?Q?atPUpk2tqm7CLPM3yeiSuxiCAI4JVwPlbUkdYcqZNzGqDdwa9R2O25rQ?= =?Windows-1252?Q?z9kN4xrg9r6JP739ao/369tdAz/vuOV3AsOlm5ZzlVv8EplsTBi+oVS3?= =?Windows-1252?Q?o+oCaTekC+hprB+JnvwDqkY0p8urwzr8UGsirxq7POmQX/ow5ZKD61Lc?= =?Windows-1252?Q?2Sk7ntwVYnmtenhhW/pplRIEya/hSg56eKi/7JmYToe3hdbtZlePm610?= =?Windows-1252?Q?YZmSKePWUvYAM/Y1kQw9QLxfoVhagK8MQ24TwtOsJAVrBTG868fLCDAk?= =?Windows-1252?Q?hBVGnbP0S8xd0J7yx/FB1bLUzV+pxrySguY4jZjp3PCCurvSpFzrV2e5?= =?Windows-1252?Q?R/pK5TMwpfdJfCZd73YiAtXXYXzvAD3VeCJCXJaDgjOeK7/HP3Jq0nyh?= =?Windows-1252?Q?le45BmsnCObo0cb4ssTMz18QLdhwJ6yZX7UzzK0ceWzVULOI1+5OM2F9?= =?Windows-1252?Q?fcUbhPMB75c9cw5X4LpMMzuUQglQKeH/7DaM6G1XIhCIvhYygN3+1FK8?= =?Windows-1252?Q?fU8W7UnfaGwEmvE1+XU+kmwRDJARCvrQ0+EH8r/IKDyLDEGEbR/obzn/?= =?Windows-1252?Q?GTH9/9si2MiWAmRYv5vO3/GxoBI1j+g+UZzYg0CKiXhuoYqvjUynto5D?= =?Windows-1252?Q?neZxEUx1F42Fl3M+ttYm6LvAl9gDVZhskacuNxliYleKb4aQqEnRVwFR?= =?Windows-1252?Q?zaJjv5M6Af9hzYE7rWlLROLEgx0bwQBSXx7KssQHZ47FxtpFCfSSXN6H?= =?Windows-1252?Q?SG8vUxJLa/eUkwQh7AWvdOzoeroN6IU7SpqOv4Msu69Fp2047QbbG66N?= =?Windows-1252?Q?+ieM4LSsE4wb6vP6j7u1i+GusGpxTUmtTYB9VCgnj6hcHHBL5mb+D2S0?= =?Windows-1252?Q?33L5w8k/caEC1UxMqvr4ZqLDkF4VEiAJ15tBKmQD2p7YZm+e16tuPppE?= =?Windows-1252?Q?0840XutIrfFGXkPdIdyqHkOOlQJCpSgA0hQRzyG2RUl0eUuq6ZJyn3oD?= =?Windows-1252?Q?WjzLOH3dY5Vawnszw4jowkRQ50bKYVIkIwpW5ZjeeHoK2V74LxrgiJpQ?= =?Windows-1252?Q?beUwc7jp5LiIJBSNlS2NhE3ETcZpw6Ywg1qNFVyYPTQf3TYgeg6ws3Fz?= =?Windows-1252?Q?43CPySptgDnc3Hldb78s/7tnBg5JsQRt9nvlY8YvV8s7HTYOsziihwnc?= =?Windows-1252?Q?SyuIzN6jhjdJvtEjMLcapi+AqYWP+CpKmp5b0LYG+Iqs+WBQNM2eux2v?= =?Windows-1252?Q?R5dU2BUqyl7fp92Zp8+zd/6uhXh+1v3L7hP+7uXy/WyrWOfu/Feo63It?= =?Windows-1252?Q?smu/3nRkGqwOY39yDtb8xE/DENz19wMKaKO6RJwYH1Y87ztVhUL86OAf?= =?Windows-1252?Q?w1YybaQIWVro10zkxQ64lfj6ugIjYZmnWCsLrMVcHmcdf8WIsiTc/ysc?= =?Windows-1252?Q?yYFz0bYClrE5SJjTewVt5S92JJqGVfBsO6YLrY8cFCMd8Of0N/AUChPF?= =?Windows-1252?Q?fdD+L/BPKJlFBGY4BLnK0LhyMkPlEC6KGwacgPUIjdixhbRGFi5Q61cU?= =?Windows-1252?Q?DieCswaRSA9k/2O0VPV4yLxb9vRT/JrN91mH0brHvW6goc+CR8dqvWt0?= =?Windows-1252?Q?gkO5Jj3mQ=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1;MWHPR15MB1199;6:9mX/RcKiixzNXG7rMRwz8NFSruGn8meoEU/0OU597W89Y0FMh5osubFK+sQ54Kwwhe4NVJ0wiPG0VYjWo4S3Js12FtlwNTSynCCytrYOVCSxSinJWQaeyBD/kavQ84TvWLLJLYw6WXtFxznO4M0tFfrnyablk9VxMJQmTY61RF7r5wUHJXd/GduSmuuVAJjdg0yY8b1qChE/VQYLWNhOfOqItaQz9wEMfVWeoQBhAPaFRYyHrG2Eu+F6x5cBNkfEvgBOK1INzT9TRWqMTW1gN6izCvT0Yu/527QlWotKdsyJKT/YfDpxhX6MUOEP3hfd8pkcLdK37QNnD9DFilpQX9aJNqdBFa0P3qbd+t4RZ3zmhGA2kzShwlhf9wYRB+UtzvQfdGvCWWviu/gAyzrcs7uBZY8hD48D7cHLix+Azyo=;5:q2YkxbgFMH5lx2plsZmDCQNjINeHxEG/Du4b8sJ0wtOf2XtumZgYfAxsQtuVPczWJlP8Yus24GqEfZV8lh+WqNmmMCGVeyDwzQrIO2xuCDWo2mo+ASva8BHetuSKcRvX+p+pI5oQ4HHFtBmRW9yGOA==;24:Lgq3ORaBYptCnPdSCSZhMfEvozuqs1r7ieKjEltmm/cNiAnogtFG8thzvsRurTRw1VJjmTBD7irB2gu/qSLcOhvZQsR+DJKYFmQjsMi+Cl8= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;MWHPR15MB1199;7:Hl9YGRJ72v4JDtSYoUazDIfQZXJnIrLz6FhcDCqQx3uw2wovl4U7/+ybCFbKHj6kiS72wQ/l4qGZM9TnfMgzV47iW5PNbuwzGX9TQXSRVT0K5EFlZ8h1zXbjeutRx/byu8B6QTe05ZpyHbJghPGvgFYkeRLPcV3NOtFSBl0o7ZRklCJKCT6wJ5S81vayC1jIfkyCcWipY2BwMqVRY5yaEy6QZmNhCTIgStrhIo3xxttEAlqX2gVhYqpFzAOVHUQKbSkgP2PNgXVVGfuVhZI+bHnMJNAcfPTjehpT2rJLlyMe7cZnzrJNKTQQvOvtk5Wqhmd/Mfe1dBDRm2cNuSNNi/p1WlLmYIPYZblwLzwcNjTCTeo+2yZJaafHPxhhJzh2ninA2j6AXbUwgUqeCRpGryqVsg4FYXQRQL740zRRctcLmAw8Hj4seK9a2cuvF6q7cKO5FgOy4Kig2LhZK9FOBQ==;20:c2l6rPyM0UFSl9cdT43TIw4bxjzNOnhEPj5WByd7CAZs7lmFK4tBquya3CIJWhI9CjjxWfuVgYFIxz7ASr124uTfQeDPjYHp3VK31T3MdZXkGFF9ZubAW9BRATOAEsuvX8+qRVkFmHJIfBN71SMObJFOv5JLCB73Gk6wqSrJiUY= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jan 2017 15:23:13.7549 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR15MB1199 X-OriginatorOrg: fb.com X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-01-13_09:,, signatures=0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 01/13/2017 04:04 AM, Hannes Reinecke wrote: > On 01/13/2017 09:15 AM, Hannes Reinecke wrote: >> On 01/11/2017 10:39 PM, Jens Axboe wrote: >>> Another year, another posting of this patchset. The previous posting >>> was here: >>> >>> https://www.spinics.net/lists/kernel/msg2406106.html >>> >>> (yes, I've skipped v5, it was fixes on top of v4, not the rework). >>> >>> I've reworked bits of this to get rid of the shadow requests, thanks >>> to Bart for the inspiration. The missing piece, for me, was the fact >>> that we have the tags->rqs[] indirection array already. I've done this >>> somewhat differently, though, by having the internal scheduler tag >>> map be allocated/torn down when an IO scheduler is attached or >>> detached. This also means that when we run without a scheduler, we >>> don't have to do double tag allocations, it'll work like before. >>> >>> The patchset applies on top of 4.10-rc3, or can be pulled here: >>> >>> git://git.kernel.dk/linux-block blk-mq-sched.6 >>> >> Well ... something's wrong here on my machine: >> >> [ 39.886886] ------------[ cut here ]------------ >> [ 39.886895] WARNING: CPU: 9 PID: 62 at block/blk-mq.c:342 >> __blk_mq_finish_request+0x124/0x140 >> [ 39.886895] Modules linked in: sd_mod ahci uhci_hcd ehci_pci >> mpt3sas(+) libahci ehci_hcd serio_raw crc32c_intel raid_class drm libata >> usbcore hpsa usb_common scsi_transport_sas sg dm_multipath dm_mod >> scsi_dh_rdac scsi_dh_emc scsi_dh_alua autofs4 >> [ 39.886910] CPU: 9 PID: 62 Comm: kworker/u130:0 Not tainted >> 4.10.0-rc3+ #528 >> [ 39.886911] Hardware name: HP ProLiant ML350p Gen8, BIOS P72 09/08/2013 >> [ 39.886917] Workqueue: events_unbound async_run_entry_fn >> [ 39.886918] Call Trace: >> [ 39.886923] dump_stack+0x85/0xc9 >> [ 39.886927] __warn+0xd1/0xf0 >> [ 39.886928] warn_slowpath_null+0x1d/0x20 >> [ 39.886930] __blk_mq_finish_request+0x124/0x140 >> [ 39.886932] blk_mq_finish_request+0x55/0x60 >> [ 39.886934] blk_mq_sched_put_request+0x78/0x80 >> [ 39.886936] blk_mq_free_request+0xe/0x10 >> [ 39.886938] blk_put_request+0x25/0x60 >> [ 39.886944] __scsi_execute.isra.24+0x104/0x160 >> [ 39.886946] scsi_execute_req_flags+0x94/0x100 >> [ 39.886948] scsi_report_opcode+0xab/0x100 >> >> checking ... >> > Ah. > Seems like the elevator switch races with device setup: > > 1188.490326] ------------[ cut here ]------------ > [ 1188.490334] WARNING: CPU: 9 PID: 30155 at block/blk-mq.c:342 > __blk_mq_finish_request+0x172/0x180 > [ 1188.490335] Modules linked in: mpt3sas(+) raid_class rpcsec_gss_krb5 > auth_rpcgss nfsv4 nfs lockd grace fscache ebtable_filt > er ebtables ip6table_filter ip6_tables iptable_filter ip_tables x_tables > af_packet br_netfilter bridge stp llc iscsi_ibft iscs > i_boot_sysfs sb_edac edac_core x86_pkg_temp_thermal intel_powerclamp > coretemp kvm_intel kvm irqbypass crct10dif_pclmul crc32_p > clmul tg3 ixgbe ghash_clmulni_intel pcbc ptp aesni_intel pps_core > aes_x86_64 ipmi_ssif hpilo hpwdt mdio libphy pcc_cpufreq cry > pto_simd glue_helper iTCO_wdt iTCO_vendor_support acpi_cpufreq tpm_tis > ipmi_si ipmi_devintf cryptd lpc_ich pcspkr ioatdma tpm_ > tis_core thermal wmi shpchp dca ipmi_msghandler tpm fjes button sunrpc > btrfs xor sr_mod raid6_pq cdrom ehci_pci mgag200 i2c_al > go_bit drm_kms_helper syscopyarea sysfillrect uhci_hcd > [ 1188.490399] sysimgblt fb_sys_fops sd_mod ahci ehci_hcd ttm libahci > crc32c_intel serio_raw drm libata usbcore usb_common hp > sa scsi_transport_sas sg dm_multipath dm_mod scsi_dh_rdac scsi_dh_emc > scsi_dh_alua autofs4 > [ 1188.490411] CPU: 9 PID: 30155 Comm: kworker/u130:6 Not tainted > 4.10.0-rc3+ #535 > [ 1188.490411] Hardware name: HP ProLiant ML350p Gen8, BIOS P72 09/08/2013 > [ 1188.490425] Workqueue: events_unbound async_run_entry_fn > [ 1188.490427] Call Trace: > [ 1188.490433] dump_stack+0x85/0xc9 > [ 1188.490436] __warn+0xd1/0xf0 > [ 1188.490438] warn_slowpath_null+0x1d/0x20 > [ 1188.490440] __blk_mq_finish_request+0x172/0x180 > [ 1188.490442] blk_mq_finish_request+0x55/0x60 > [ 1188.490443] blk_mq_sched_put_request+0x78/0x80 > [ 1188.490445] blk_mq_free_request+0xe/0x10 > [ 1188.490448] blk_put_request+0x25/0x60 > [ 1188.490453] __scsi_execute.isra.24+0x104/0x160 > [ 1188.490455] scsi_execute_req_flags+0x94/0x100 > [ 1188.490457] scsi_report_opcode+0xab/0x100 > [ 1188.490461] sd_revalidate_disk+0xaef/0x1450 [sd_mod] > [ 1188.490464] sd_probe_async+0xd1/0x1d0 [sd_mod] > [ 1188.490466] async_run_entry_fn+0x37/0x150 > [ 1188.490470] process_one_work+0x1d0/0x660 > [ 1188.490472] ? process_one_work+0x151/0x660 > [ 1188.490474] worker_thread+0x12b/0x4a0 > [ 1188.490475] kthread+0x10c/0x140 > [ 1188.490477] ? process_one_work+0x660/0x660 > [ 1188.490478] ? kthread_create_on_node+0x40/0x40 > [ 1188.490483] ret_from_fork+0x2a/0x40 > [ 1188.490484] ---[ end trace d5e3a32ac269fc2a ]--- > [ 1188.490485] rq (487/52) rqs (-1/-1) > [ 1188.523518] sd 7:0:0:0: [sdb] Attached SCSI disk > [ 1188.540954] elevator: switch to deadline failed Where did this last line come from? Do you have a udev rule or something that attempts to switch to deadline, which will fail with scsi-mq? Trying to get closer to reproducing this, no luck just setting the async scan option, mine still boots fine. -- Jens Axboe