Re: next/master bisection: baseline.login on asus-C523NA-A20057-coral

Re: next/master bisection: baseline.login on asus-C523NA-A20057-coral

[Guillaume Tucker]

+kernelci

On 29/03/2022 19:44, Guillaume Tucker wrote:
> On 28/03/2022 13:54, Hans de Goede wrote:
>> Hi,
>>
>> On 3/24/22 23:19, Mark Brown wrote:
>>> On Thu, Mar 24, 2022 at 09:34:30PM +0100, Hans de Goede wrote:
>>>
>>>> Mark, if one of use writes a test patch, can you get that Asus machine to boot a
>>>> kernel build from next + the test patch ?
>>>
>>> I can't directly unfortunately as the board is in Collabora's lab but
>>> Guillaume (who's already CCed) ought to be able to, and I can generally
>>> prod and try to get that done.
>>
>> Ok, Guillaume, can you try a kernel with commit 5949965ec9340cfc0e65f7d8a576b660b26e2535
>> ("x86/PCI: Preserve host bridge windows completely covered by E820") + the 
>> attached patch added on top a try on the asus-C523NA-A20057-coral machine please
>> and see if that makes it boot again ?
> 
> Sorry I've been busy with a conference.  Sure, will put that
> through KernelCI tomorrow and let you know the outcome.

Well the issue seems to have been fixed on mainline, unless it's
intermittent.  In any case, next-20220404 is booting fine:

  https://linux.kernelci.org/test/plan/id/624aed811a5acd09adae071e/

Last time it was seen to fail was next-20220330:

  https://linux.kernelci.org/test/plan/id/62442f68e30d6f89a4ae06b7/


Ironically, the KernelCI staging linux-next job with the patches
mentioned in your previous email applied is now failing:

  https://staging.kernelci.org/test/plan/id/624b2d3b923f532dc305f4c7/

The kernel branch being used is:

  https://github.com/kernelci/linux/commits/staging-next


I haven't checked the logs or investigated any further, this is
just a quick summary based on the boot test results.

Please let us know if we should drop these patches or try
anything else.  I'll be on holiday for the rest of the week but
others can pick things up if needed.

Thanks,
Guillaume

Re: next/master bisection: baseline.login on asus-C523NA-A20057-coral

[Guillaume Tucker]

On 04/04/2022 20:44, Guillaume Tucker wrote:
> Well the issue seems to have been fixed on mainline

Sorry, I meant linux-next.  It is also booting on mainline but I
don't think the regression ever went further than linux-next:

  https://linux.kernelci.org/test/plan/id/6246654e60a1cb470cae0680/

Guillaume

Re: next/master bisection: baseline.login on asus-C523NA-A20057-coral

[Hans de Goede]

Hi All,

On 4/4/22 21:44, Guillaume Tucker wrote:
> +kernelci
> 
> On 29/03/2022 19:44, Guillaume Tucker wrote:
>> On 28/03/2022 13:54, Hans de Goede wrote:
>>> Hi,
>>>
>>> On 3/24/22 23:19, Mark Brown wrote:
>>>> On Thu, Mar 24, 2022 at 09:34:30PM +0100, Hans de Goede wrote:
>>>>
>>>>> Mark, if one of use writes a test patch, can you get that Asus machine to boot a
>>>>> kernel build from next + the test patch ?
>>>>
>>>> I can't directly unfortunately as the board is in Collabora's lab but
>>>> Guillaume (who's already CCed) ought to be able to, and I can generally
>>>> prod and try to get that done.
>>>
>>> Ok, Guillaume, can you try a kernel with commit 5949965ec9340cfc0e65f7d8a576b660b26e2535
>>> ("x86/PCI: Preserve host bridge windows completely covered by E820") + the 
>>> attached patch added on top a try on the asus-C523NA-A20057-coral machine please
>>> and see if that makes it boot again ?
>>
>> Sorry I've been busy with a conference.  Sure, will put that
>> through KernelCI tomorrow and let you know the outcome.
> 
> Well the issue seems to have been fixed on mainline, unless it's
> intermittent.  In any case, next-20220404 is booting fine:
> 
>   https://linux.kernelci.org/test/plan/id/624aed811a5acd09adae071e/
> 
> Last time it was seen to fail was next-20220330:
> 
>   https://linux.kernelci.org/test/plan/id/62442f68e30d6f89a4ae06b7/
> 
> 
> Ironically, the KernelCI staging linux-next job with the patches
> mentioned in your previous email applied is now failing:
> 
>   https://staging.kernelci.org/test/plan/id/624b2d3b923f532dc305f4c7/
> 
> The kernel branch being used is:
> 
>   https://github.com/kernelci/linux/commits/staging-next
> 
> 
> I haven't checked the logs or investigated any further, this is
> just a quick summary based on the boot test results.
> 
> Please let us know if we should drop these patches or try
> anything else.  I'll be on holiday for the rest of the week but
> others can pick things up if needed.

The reason why next and mainline are building now is because
the patch the bisect pointed out never made it into mainline
and Bjorn has dropped it from -next.

I fully expect that -next with 

https://lore.kernel.org/linux-acpi/20220304035110.988712-4-helgaas@kernel.org/

or mainline with the entire series from that link applied will
still not boot.

But we do need that last patch to fix various issues on
other boards.

See my previous reply in this thread:
https://lore.kernel.org/linux-pci/76c5de03-a3a4-8444-d7f6-496fa119d830@redhat.com/

for some further analysis of what I think is happening here.

As mentioned there I believe this can be fixed by merging
back-to-back resources into a single resource before calling
remove_e820_regions() but I could not find a good example / helper
code to do the merging of the resources.

If someone can give me some pointers wrt this I can try to
come up with something and then provide a set of patches
for testing on the asus-C523NA-A20057-coral .

Regards,

Hans

Re: next/master bisection: baseline.login on asus-C523NA-A20057-coral

[Bjorn Helgaas]

On Mon, Apr 04, 2022 at 08:44:41PM +0100, Guillaume Tucker wrote:
> On 29/03/2022 19:44, Guillaume Tucker wrote:
> > On 28/03/2022 13:54, Hans de Goede wrote:
> >> On 3/24/22 23:19, Mark Brown wrote:
> >>> On Thu, Mar 24, 2022 at 09:34:30PM +0100, Hans de Goede wrote:
> >> Ok, Guillaume, can you try a kernel with commit 5949965ec9340cfc0e65f7d8a576b660b26e2535
> >> ("x86/PCI: Preserve host bridge windows completely covered by E820") + the 
> >> attached patch added on top a try on the asus-C523NA-A20057-coral machine please
> >> and see if that makes it boot again ?
> > 
> > Sorry I've been busy with a conference.  Sure, will put that
> > through KernelCI tomorrow and let you know the outcome.
> 
> Well the issue seems to have been fixed on mainline, unless it's
> intermittent.  In any case, next-20220404 is booting fine:
> 
>   https://linux.kernelci.org/test/plan/id/624aed811a5acd09adae071e/
> 
> Last time it was seen to fail was next-20220330:
> 
>   https://linux.kernelci.org/test/plan/id/62442f68e30d6f89a4ae06b7/

This is because I dropped 5949965ec934 ("x86/PCI: Preserve host bridge
windows completely covered by E820") from the PCI tree starting with
next-20220401 because it causes the regression.  So I expect
next-20220404 to boot fine (next-20220401 should boot fine as well; I
don't know whether that was tested).

The gory details:

  20220330 should fail; it includes:
    5949965ec934 ("x86/PCI: Preserve host bridge windows completely covered by E820")
    d13f73e9108a ("x86/PCI: Log host bridge window clipping for E820 regions")
    9c253994c5ba ("x86/PCI: Eliminate remove_e820_regions() common subexpressions")
    ffb217a13a2e ("Linux 5.17-rc7")

  20220331 should fail; it includes:
    18146f25ac66 ("PCI: hv: Remove unused hv_set_msi_entry_from_desc()")
    5949965ec934 ("x86/PCI: Preserve host bridge windows completely covered by E820")
    d13f73e9108a ("x86/PCI: Log host bridge window clipping for E820 regions")
    9c253994c5ba ("x86/PCI: Eliminate remove_e820_regions() common subexpressions")
    ffb217a13a2e ("Linux 5.17-rc7")

  20220401 should boot; it includes:
    1c6cec4ab487 ("x86/PCI: Log host bridge window clipping for E820 regions")
    b2922e67d233 ("x86/PCI: Eliminate remove_e820_regions() common subexpressions")
    22ef7ee3eeb2 ("PCI: hv: Remove unused hv_set_msi_entry_from_desc()")
    148a65047695 ("Merge tag 'pci-v5.18-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci")

  20220404 should boot; it includes:
    22ef7ee3eeb2 ("PCI: hv: Remove unused hv_set_msi_entry_from_desc()")
    148a65047695 ("Merge tag 'pci-v5.18-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci")

> Ironically, the KernelCI staging linux-next job with the patches
> mentioned in your previous email applied is now failing:
> 
>   https://staging.kernelci.org/test/plan/id/624b2d3b923f532dc305f4c7/

This says we tested commit 1aceacc82d3f, which I guess is the
staging-next-20220404.1 tag at https://github.com/kernelci/linux.git.
It took me a while to find the commit history, but
https://github.com/kernelci/linux/commits/1aceacc82d3f says this
includes:

  0a0c05a90278 x86/PCI: Limit "e820 entry fully covers window" check to non ISA MMIO
  b5fd57109d22 x86/PCI: Preserve host bridge windows completely covered by E820

So the proposed fix (0a0c05a90278) apparently didn't work.

Bjorn

Re: next/master bisection: baseline.login on asus-C523NA-A20057-coral

[Denys Fedoryshchenko]

[-- Attachment #1: Type: text/plain, Size: 1116 bytes --]

On Wed, 2022-04-06 at 21:11 +0100, Guillaume Tucker wrote:
> +kernelci +Michał +Denys
> 
> On 06/04/2022 20:37, Mark Brown wrote:
> > On Wed, Apr 06, 2022 at 01:59:31PM -0500, Bjorn Helgaas wrote:
> > > On Tue, Apr 05, 2022 at 06:53:17PM -0500, Bjorn Helgaas wrote:
> > 
> > > > Is there any way to get the contents of:
> > 
> > > >   /sys/firmware/acpi/tables/DSDT
> > > >   /sys/firmware/acpi/tables/SSDT*
> > 
> > > > from these Chromebooks?
> > 
> > > Is there hope for this, or should I look for another way to get
> > > this
> > > information?
> > 
> > I believe Guillaume is out of office this week.  Copying in Guenter
> > as
> > well since he's on the ChromeOS team in case he can help or knows
> > someone who can.
> 
> Someone with access to the Collabora LAVA lab can also send a
> custom job to try and get this information.
> 
> I'm back to work on Monday.
> 
> Thanks,
> Guillaume
Hi All

Device-type: asus-C523NA-A20057-coral
Thats all i got, it is only one SSDT table.

Please let me know if anything else needed.



Best regards,
Denys Fedoryshchenko


[-- Attachment #2: acpi.tgz --]
[-- Type: application/x-compressed-tar, Size: 7150 bytes --]

Re: next/master bisection: baseline.login on asus-C523NA-A20057-coral

[Guillaume Tucker]

+kernelci +Michał +Denys

On 06/04/2022 20:37, Mark Brown wrote:
> On Wed, Apr 06, 2022 at 01:59:31PM -0500, Bjorn Helgaas wrote:
>> On Tue, Apr 05, 2022 at 06:53:17PM -0500, Bjorn Helgaas wrote:
> 
>>> Is there any way to get the contents of:
> 
>>>   /sys/firmware/acpi/tables/DSDT
>>>   /sys/firmware/acpi/tables/SSDT*
> 
>>> from these Chromebooks?
> 
>> Is there hope for this, or should I look for another way to get this
>> information?
> 
> I believe Guillaume is out of office this week.  Copying in Guenter as
> well since he's on the ChromeOS team in case he can help or knows
> someone who can.

Someone with access to the Collabora LAVA lab can also send a
custom job to try and get this information.

I'm back to work on Monday.

Thanks,
Guillaume