From: Orlando Chamberlain <redecorating@protonmail.com>
To: Keith Busch <kbusch@kernel.org>
Cc: "linux-nvme@lists.infradead.org" <linux-nvme@lists.infradead.org>,
"regressions@lists.linux.dev" <regressions@lists.linux.dev>,
"gargaditya08@live.com" <gargaditya08@live.com>,
"sagi@grimberg.me" <sagi@grimberg.me>,
"hare@suse.de" <hare@suse.de>,
"dwagner@suse.de" <dwagner@suse.de>, "hch@lst.de" <hch@lst.de>
Subject: Re: [REGRESSION] nvme: code command_id with a genctr for use-after-free validation crashes apple T2 SSD
Date: Sat, 25 Sep 2021 23:40:19 +0000 [thread overview]
Message-ID: <fa9de055-c3b8-20d3-41e0-12e43d0c336a@protonmail.com> (raw)
In-Reply-To: <20210925171618.GA116968@dhcp-10-100-145-180.wdc.com>
On 26/9/21 03:16, Keith Busch wrote:
> On Sat, Sep 25, 2021 at 01:10:42PM +0000, Orlando Chamberlain wrote:
>> Commit e7006de6c238 causes the SSD controller on Apple T2 computers to crash
>> and prevents linux from booting.
>>
>> This commit implemented a counter that is stored within the NVMe command_id,
>> however this counter makes the command_id higher than normal, causing a panic
>> on the T2 security chip that functions as the SSD controller, which then
>> causes the system to power off after a few seconds.
>
> Ah, yet another spec non-complainat quirk from these controllers.
Apple does like to think different.
>> This is the entry in lspci -nn for the ssd:
>>
>> 04:00.0 Mass storage controller [0180]: Apple Inc. ANS2 NVMe Controller [106b:2005] (rev 01)
>>
>> This commit was included in 5.14.6 and backported to 5.10.67, but does not
>> occur in 5.14.5 and 5.10.66. I am on a MacBookPro16,1, the crash has been
>> reproduced on a MacBookPro16,2 as well.
>
> Is the PCI VID:DID the same from in your lspci output for all affected
> macbooks?
Yes, they all have 106b:2005
>> I've tried to modify the genctr so that it is in the other side of the
>> command_id (which I thought might make the command_id's lower) with the patch
>> below, but it did not prevent the crash.
>
> That might mean the h/w is using the command id as an index into
> internal structures. That is not spec compliant, so it sounds like
> we'll need to introduce another quirk for the macs.
>
I've managed to get it to boot by commenting out the counter increment, which might work
as a quirk:
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1027,7 +1027,7 @@ blk_status_t nvme_setup_cmd(struct nvme_ns *ns, struct request *req)
return BLK_STS_IOERR;
}
- nvme_req(req)->genctr++;
+ //nvme_req(req)->genctr++;
cmd->common.command_id = nvme_cid(req);
trace_nvme_setup_cmd(req, cmd);
return ret;
_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme
WARNING: multiple messages have this Message-ID (diff)
From: Orlando Chamberlain <redecorating@protonmail.com>
To: Keith Busch <kbusch@kernel.org>
Cc: "linux-nvme@lists.infradead.org" <linux-nvme@lists.infradead.org>,
"regressions@lists.linux.dev" <regressions@lists.linux.dev>,
"gargaditya08@live.com" <gargaditya08@live.com>,
"sagi@grimberg.me" <sagi@grimberg.me>,
"hare@suse.de" <hare@suse.de>,
"dwagner@suse.de" <dwagner@suse.de>, "hch@lst.de" <hch@lst.de>
Subject: Re: [REGRESSION] nvme: code command_id with a genctr for use-after-free validation crashes apple T2 SSD
Date: Sat, 25 Sep 2021 23:40:19 +0000 [thread overview]
Message-ID: <fa9de055-c3b8-20d3-41e0-12e43d0c336a@protonmail.com> (raw)
In-Reply-To: <20210925171618.GA116968@dhcp-10-100-145-180.wdc.com>
On 26/9/21 03:16, Keith Busch wrote:
> On Sat, Sep 25, 2021 at 01:10:42PM +0000, Orlando Chamberlain wrote:
>> Commit e7006de6c238 causes the SSD controller on Apple T2 computers to crash
>> and prevents linux from booting.
>>
>> This commit implemented a counter that is stored within the NVMe command_id,
>> however this counter makes the command_id higher than normal, causing a panic
>> on the T2 security chip that functions as the SSD controller, which then
>> causes the system to power off after a few seconds.
>
> Ah, yet another spec non-complainat quirk from these controllers.
Apple does like to think different.
>> This is the entry in lspci -nn for the ssd:
>>
>> 04:00.0 Mass storage controller [0180]: Apple Inc. ANS2 NVMe Controller [106b:2005] (rev 01)
>>
>> This commit was included in 5.14.6 and backported to 5.10.67, but does not
>> occur in 5.14.5 and 5.10.66. I am on a MacBookPro16,1, the crash has been
>> reproduced on a MacBookPro16,2 as well.
>
> Is the PCI VID:DID the same from in your lspci output for all affected
> macbooks?
Yes, they all have 106b:2005
>> I've tried to modify the genctr so that it is in the other side of the
>> command_id (which I thought might make the command_id's lower) with the patch
>> below, but it did not prevent the crash.
>
> That might mean the h/w is using the command id as an index into
> internal structures. That is not spec compliant, so it sounds like
> we'll need to introduce another quirk for the macs.
>
I've managed to get it to boot by commenting out the counter increment, which might work
as a quirk:
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1027,7 +1027,7 @@ blk_status_t nvme_setup_cmd(struct nvme_ns *ns, struct request *req)
return BLK_STS_IOERR;
}
- nvme_req(req)->genctr++;
+ //nvme_req(req)->genctr++;
cmd->common.command_id = nvme_cid(req);
trace_nvme_setup_cmd(req, cmd);
return ret;
next prev parent reply other threads:[~2021-09-25 23:40 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-25 13:10 [REGRESSION] nvme: code command_id with a genctr for use-after-free validation crashes apple T2 SSD Orlando Chamberlain
2021-09-25 13:10 ` Orlando Chamberlain
2021-09-25 17:16 ` Keith Busch
2021-09-25 17:16 ` Keith Busch
2021-09-25 23:40 ` Orlando Chamberlain [this message]
2021-09-25 23:40 ` Orlando Chamberlain
2021-09-26 2:08 ` Keith Busch
2021-09-26 2:08 ` Keith Busch
2021-09-26 3:53 ` Orlando Chamberlain
2021-09-26 3:53 ` Orlando Chamberlain
2021-09-26 4:35 ` Orlando Chamberlain
2021-09-26 4:35 ` Orlando Chamberlain
[not found] ` <PNZPR01MB4415801C6084E8CFD068A84AB8A69@PNZPR01MB4415.INDPRD01.PROD.OUTLOOK.COM>
2021-09-26 8:44 ` Sagi Grimberg
2021-09-26 8:44 ` Sagi Grimberg
2021-09-27 4:22 ` Orlando Chamberlain
2021-09-27 4:22 ` Orlando Chamberlain
2021-09-27 4:51 ` Aditya Garg
2021-09-27 4:51 ` Aditya Garg
2021-09-27 6:05 ` Sven Peter
2021-09-27 6:05 ` Sven Peter
2021-09-27 15:02 ` Keith Busch
2021-09-27 15:02 ` Keith Busch
2021-09-26 5:59 ` Thorsten Leemhuis
2021-10-04 9:17 ` Thorsten Leemhuis
2021-10-04 9:27 ` Greg KH
2021-10-04 10:11 ` Thorsten Leemhuis
2021-10-04 11:36 ` Greg KH
2021-10-05 5:50 ` Thorsten Leemhuis
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=fa9de055-c3b8-20d3-41e0-12e43d0c336a@protonmail.com \
--to=redecorating@protonmail.com \
--cc=dwagner@suse.de \
--cc=gargaditya08@live.com \
--cc=hare@suse.de \
--cc=hch@lst.de \
--cc=kbusch@kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=regressions@lists.linux.dev \
--cc=sagi@grimberg.me \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.