Re: Regression in suspend-to-ram (TPM related) with 5.1-rc1

Re: Regression in suspend-to-ram (TPM related) with 5.1-rc1

[Paul Zimmerman]

On Mon, Mar 18, 2019 at 2:39 PM Paul Zimmerman <pauldzim@gmail.com> wrote:

< snip >

> It takes about 45 minutes to do a full kernel rebuild on this machine,
> so doing a full bisection would be a little painful.

Actually, that was because I had the Linux disk attached via USB. I have it
connected internally now, and a full kernel rebuild only takes about 8 minutes.
So I can do a bisection if required.

-- Paul

Re: Regression in suspend-to-ram (TPM related) with 5.1-rc1

[Paul Zimmerman]

Hmm, looks like my original email didn't make it to the linux-integrity
list, maybe the two attachments were too big. You can read it on the
linux-kernel list here:
https://marc.info/?l=linux-kernel&m=155294522323580

On Mon, Mar 18, 2019 at 2:39 PM Paul Zimmerman <pauldzim@gmail.com> wrote:
>
> Hello,
>
> I'm seeing suspend-to-ram fail consistently in 5.1-rc1. Dmesg shows a
> failure in the TPM subsystem. This is on an HP Elitebook 640 G1 running
> Linux Mint.
>
> [   43.110604] wlo1: deauthenticating from 58:8b:f3:44:8f:5c by local choice (Reason: 3=DEAUTH_LEAVING)
> [   53.179672] PM: suspend entry (deep)
> [   53.179674] PM: Syncing filesystems ... done.
> [   53.190349] Freezing user space processes ... (elapsed 0.001 seconds) done.
> [   53.192107] OOM killer disabled.
> [   53.192107] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done.
> [   53.193147] printk: Suspending console(s) (use no_console_suspend to debug)
> [   53.209184] sd 2:0:0:0: [sdb] Synchronizing SCSI cache
> [   53.213137] sd 0:0:0:0: [sda] Synchronizing SCSI cache
> [   53.214632] sd 0:0:0:0: [sda] Stopping disk
> [   53.241598] tpm tpm0: tpm_try_transmit: send(): error -5
> [   53.241600] tpm tpm0: Error (-5) sending savestate before suspend
> [   53.241606] PM: __pnp_bus_suspend(): tpm_pm_suspend+0x0/0x90 returns -5
> [   53.241609] PM: dpm_run_callback(): pnp_bus_suspend+0x0/0x20 returns -5
> [   53.241610] PM: Device 00:06 failed to suspend: error -5
>
> Full dmesg and config attached. Anything I can do to help debug this?
> It takes about 45 minutes to do a full kernel rebuild on this machine,
> so doing a full bisection would be a little painful.
>
> -- Paul

Re: Regression in suspend-to-ram (TPM related) with 5.1-rc1 (BISECTED)

[Paul Zimmerman]

So I bisected this down to:

# first bad commit: [a3fbfae82b4cb3ff9928e29f34c64d0507cad874] tpm:
take TPM chip power gating out of tpm_transmit()

but this doesn't revert cleanly on Linus' HEAD. Anyone have an idea what
could be wrong here?

Thanks,
-- Paul

On Mon, Mar 18, 2019 at 6:08 PM Paul Zimmerman <pauldzim@gmail.com> wrote:
>
> Hmm, looks like my original email didn't make it to the linux-integrity
> list, maybe the two attachments were too big. You can read it on the
> linux-kernel list here:
> https://marc.info/?l=linux-kernel&m=155294522323580
>
> On Mon, Mar 18, 2019 at 2:39 PM Paul Zimmerman <pauldzim@gmail.com> wrote:
> >
> > Hello,
> >
> > I'm seeing suspend-to-ram fail consistently in 5.1-rc1. Dmesg shows a
> > failure in the TPM subsystem. This is on an HP Elitebook 640 G1 running
> > Linux Mint.
> >
> > [   43.110604] wlo1: deauthenticating from 58:8b:f3:44:8f:5c by local choice (Reason: 3=DEAUTH_LEAVING)
> > [   53.179672] PM: suspend entry (deep)
> > [   53.179674] PM: Syncing filesystems ... done.
> > [   53.190349] Freezing user space processes ... (elapsed 0.001 seconds) done.
> > [   53.192107] OOM killer disabled.
> > [   53.192107] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done.
> > [   53.193147] printk: Suspending console(s) (use no_console_suspend to debug)
> > [   53.209184] sd 2:0:0:0: [sdb] Synchronizing SCSI cache
> > [   53.213137] sd 0:0:0:0: [sda] Synchronizing SCSI cache
> > [   53.214632] sd 0:0:0:0: [sda] Stopping disk
> > [   53.241598] tpm tpm0: tpm_try_transmit: send(): error -5
> > [   53.241600] tpm tpm0: Error (-5) sending savestate before suspend
> > [   53.241606] PM: __pnp_bus_suspend(): tpm_pm_suspend+0x0/0x90 returns -5
> > [   53.241609] PM: dpm_run_callback(): pnp_bus_suspend+0x0/0x20 returns -5
> > [   53.241610] PM: Device 00:06 failed to suspend: error -5
> >
> > Full dmesg and config attached. Anything I can do to help debug this?
> > It takes about 45 minutes to do a full kernel rebuild on this machine,
> > so doing a full bisection would be a little painful.
> >
> > -- Paul

Re: Regression in suspend-to-ram (TPM related) with 5.1-rc1 (BISECTED)

[Jarkko Sakkinen]

On Tue, Mar 19, 2019 at 04:03:37PM -0700, Paul Zimmerman wrote:
> So I bisected this down to:
> 
> # first bad commit: [a3fbfae82b4cb3ff9928e29f34c64d0507cad874] tpm:
> take TPM chip power gating out of tpm_transmit()
> 
> but this doesn't revert cleanly on Linus' HEAD. Anyone have an idea what
> could be wrong here?

Sorry I've been in flu for the early week. I spotted the bug
immediately.  When I did these patches I did not have TPM 1.x at my
hand. I used fTPM 2.0 and dTPM 2.0. Stefan did TPM 1.x testing but I
probably forgot to ask him to try out suspend.

Anyway, the bug is obvious and I'll send you a patch to try out.
Thanks a lot for bisecting this!

/Jarkko

Re: Regression in suspend-to-ram (TPM related) with 5.1-rc1 (BISECTED)

[Martin Kepplinger]

[-- Attachment #1: Type: text/plain, Size: 885 bytes --]

On 21.03.19 14:41, Jarkko Sakkinen wrote:
> On Tue, Mar 19, 2019 at 04:03:37PM -0700, Paul Zimmerman wrote:
>> So I bisected this down to:
>>
>> # first bad commit: [a3fbfae82b4cb3ff9928e29f34c64d0507cad874] tpm:
>> take TPM chip power gating out of tpm_transmit()
>>
>> but this doesn't revert cleanly on Linus' HEAD. Anyone have an idea what
>> could be wrong here?
> 
> Sorry I've been in flu for the early week. I spotted the bug
> immediately.  When I did these patches I did not have TPM 1.x at my
> hand. I used fTPM 2.0 and dTPM 2.0. Stefan did TPM 1.x testing but I
> probably forgot to ask him to try out suspend.
> 
> Anyway, the bug is obvious and I'll send you a patch to try out.
> Thanks a lot for bisecting this!
> 
> /Jarkko
> 


Hi,

Any news on this? It seems not to be fixed in -rc4. I'd happily test a
patch too.

thanks a lot,

                            martin

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 3616 bytes --]

Re: Regression in suspend-to-ram (TPM related) with 5.1-rc1 (BISECTED)

[Paul Zimmerman]

On Sun, Apr 7, 2019 at 10:53 PM Martin Kepplinger
<martin.kepplinger@ginzinger.com> wrote:
>
> On 21.03.19 14:41, Jarkko Sakkinen wrote:
> > On Tue, Mar 19, 2019 at 04:03:37PM -0700, Paul Zimmerman wrote:
> >> So I bisected this down to:
> >>
> >> # first bad commit: [a3fbfae82b4cb3ff9928e29f34c64d0507cad874] tpm:
> >> take TPM chip power gating out of tpm_transmit()
> >>
> >> but this doesn't revert cleanly on Linus' HEAD. Anyone have an idea what
> >> could be wrong here?
> >
> > Sorry I've been in flu for the early week. I spotted the bug
> > immediately.  When I did these patches I did not have TPM 1.x at my
> > hand. I used fTPM 2.0 and dTPM 2.0. Stefan did TPM 1.x testing but I
> > probably forgot to ask him to try out suspend.
> >
> > Anyway, the bug is obvious and I'll send you a patch to try out.
> > Thanks a lot for bisecting this!
> >
> > /Jarkko
>
> Hi,
>
> Any news on this? It seems not to be fixed in -rc4. I'd happily test a
> patch too.
>
Hi Martin,
Patch is here: https://patchwork.kernel.org/patch/10865495/
Don't know what the status of it is though.

-- Paul

Re: Regression in suspend-to-ram (TPM related) with 5.1-rc1 (BISECTED)

[Martin Kepplinger]

[-- Attachment #1: Type: text/plain, Size: 1223 bytes --]

On 08.04.19 08:59, Paul Zimmerman wrote:
> On Sun, Apr 7, 2019 at 10:53 PM Martin Kepplinger
> <martin.kepplinger@ginzinger.com> wrote:
>>
>> On 21.03.19 14:41, Jarkko Sakkinen wrote:
>>> On Tue, Mar 19, 2019 at 04:03:37PM -0700, Paul Zimmerman wrote:
>>>> So I bisected this down to:
>>>>
>>>> # first bad commit: [a3fbfae82b4cb3ff9928e29f34c64d0507cad874] tpm:
>>>> take TPM chip power gating out of tpm_transmit()
>>>>
>>>> but this doesn't revert cleanly on Linus' HEAD. Anyone have an idea what
>>>> could be wrong here?
>>>
>>> Sorry I've been in flu for the early week. I spotted the bug
>>> immediately.  When I did these patches I did not have TPM 1.x at my
>>> hand. I used fTPM 2.0 and dTPM 2.0. Stefan did TPM 1.x testing but I
>>> probably forgot to ask him to try out suspend.
>>>
>>> Anyway, the bug is obvious and I'll send you a patch to try out.
>>> Thanks a lot for bisecting this!
>>>
>>> /Jarkko
>>
>> Hi,
>>
>> Any news on this? It seems not to be fixed in -rc4. I'd happily test a
>> patch too.
>>
> Hi Martin,
> Patch is here: https://patchwork.kernel.org/patch/10865495/
> Don't know what the status of it is though.
> 
> -- Paul
> 

thanks. lgtm.

Tested-by: Martin Kepplinger <martink@posteo.de>

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 3616 bytes --]

Re: Regression in suspend-to-ram (TPM related) with 5.1-rc1 (BISECTED)

[Jarkko Sakkinen]

On Mon, Apr 08, 2019 at 07:53:34AM +0200, Martin Kepplinger wrote:
> On 21.03.19 14:41, Jarkko Sakkinen wrote:
> > On Tue, Mar 19, 2019 at 04:03:37PM -0700, Paul Zimmerman wrote:
> >> So I bisected this down to:
> >>
> >> # first bad commit: [a3fbfae82b4cb3ff9928e29f34c64d0507cad874] tpm:
> >> take TPM chip power gating out of tpm_transmit()
> >>
> >> but this doesn't revert cleanly on Linus' HEAD. Anyone have an idea what
> >> could be wrong here?
> > 
> > Sorry I've been in flu for the early week. I spotted the bug
> > immediately.  When I did these patches I did not have TPM 1.x at my
> > hand. I used fTPM 2.0 and dTPM 2.0. Stefan did TPM 1.x testing but I
> > probably forgot to ask him to try out suspend.
> > 
> > Anyway, the bug is obvious and I'll send you a patch to try out.
> > Thanks a lot for bisecting this!
> > 
> > /Jarkko
> > 
> 
> 
> Hi,
> 
> Any news on this? It seems not to be fixed in -rc4. I'd happily test a
> patch too.
> 
> thanks a lot,
> 
>                             martin

I sent a PR to James last week with the fix. Probably has not landed
yet.

https://lore.kernel.org/linux-integrity/20190329115544.GA27351@linux.intel.com/

/Jarkko

Re: Regression in suspend-to-ram (TPM related) with 5.1-rc1 (BISECTED)

[Jarkko Sakkinen]

On Mon, Apr 08, 2019 at 11:04:22AM +0200, Martin Kepplinger wrote:
> On 08.04.19 08:59, Paul Zimmerman wrote:
> > On Sun, Apr 7, 2019 at 10:53 PM Martin Kepplinger
> > <martin.kepplinger@ginzinger.com> wrote:
> >>
> >> On 21.03.19 14:41, Jarkko Sakkinen wrote:
> >>> On Tue, Mar 19, 2019 at 04:03:37PM -0700, Paul Zimmerman wrote:
> >>>> So I bisected this down to:
> >>>>
> >>>> # first bad commit: [a3fbfae82b4cb3ff9928e29f34c64d0507cad874] tpm:
> >>>> take TPM chip power gating out of tpm_transmit()
> >>>>
> >>>> but this doesn't revert cleanly on Linus' HEAD. Anyone have an idea what
> >>>> could be wrong here?
> >>>
> >>> Sorry I've been in flu for the early week. I spotted the bug
> >>> immediately.  When I did these patches I did not have TPM 1.x at my
> >>> hand. I used fTPM 2.0 and dTPM 2.0. Stefan did TPM 1.x testing but I
> >>> probably forgot to ask him to try out suspend.
> >>>
> >>> Anyway, the bug is obvious and I'll send you a patch to try out.
> >>> Thanks a lot for bisecting this!
> >>>
> >>> /Jarkko
> >>
> >> Hi,
> >>
> >> Any news on this? It seems not to be fixed in -rc4. I'd happily test a
> >> patch too.
> >>
> > Hi Martin,
> > Patch is here: https://patchwork.kernel.org/patch/10865495/
> > Don't know what the status of it is though.
> > 
> > -- Paul
> > 
> 
> thanks. lgtm.
> 
> Tested-by: Martin Kepplinger <martink@posteo.de>

Thank you for testing this. Unfortunately it is already pulled so
cannot add new tags.

/Jarkko