Re: [5.17 regression] "x86/PCI: Ignore E820 reservations for bridge windows on newer systems" breaks suspend/resume

[Hans de Goede <hdegoede@redhat.com>]

Hi,

On 2/14/22 14:42, Mika Westerberg wrote:
> Hi Hans,
> 
> On Mon, Feb 14, 2022 at 01:42:29PM +0100, Hans de Goede wrote:
>> Hi,
>>
>> On 2/10/22 07:39, Mika Westerberg wrote:
>>> Hi Hans,
>>>
>>> On Wed, Feb 09, 2022 at 05:08:13PM +0100, Hans de Goede wrote:
>>>> As mentioned in my email from 10 seconds ago I think a better simpler
>>>> fix would be to just do:
>>>>
>>>> diff --git a/arch/x86/kernel/resource.c b/arch/x86/kernel/resource.c
>>>> index 9b9fb7882c20..18656f823764 100644
>>>> --- a/arch/x86/kernel/resource.c
>>>> +++ b/arch/x86/kernel/resource.c
>>>> @@ -28,6 +28,10 @@ static void remove_e820_regions(struct resource *avail)
>>>>  	int i;
>>>>  	struct e820_entry *entry;
>>>>  
>>>> +	/* Only remove E820 reservations on classic BIOS boot */
>>>> +	if (efi_enabled(EFI_MEMMAP))
>>>> +		return;
>>>> +
>>>>  	for (i = 0; i < e820_table->nr_entries; i++) {
>>>>  		entry = &e820_table->entries[i];
>>>>  
>>>>
>>>> I'm curious what you think of that?
>>>
>>> I'm not an expert in this e820 stuff but this one looks really simple
>>> and makes sense to me. So definitely should go with it assuming there
>>> are no objections from the x86 maintainers.
>>
>> Unfortunately with this suspend/resume is still broken on the ThinkPad
>> X1 carbon gen 2 of the reporter reporting the regression. The reporter
>> has been kind enough to also test in EFI mode (at my request) and then
>> the problem is back again with this patch. So just differentiating
>> between EFI / non EFI mode is not an option.
> 
> Thanks for the update! Too bad that it did not solve the regression, though :(
> 
>> FYI, here is what I believe is the root-cause of the issue on the ThinkPad X1 carbon gen 2:
>>
>> The E820 reservations table has the following in both BIOS and EFI boot modes:
>>
>> [    0.000000] BIOS-e820: [mem 0x00000000dceff000-0x00000000dfa0ffff] reserved
>>
>> Which has a small overlap with:
>>
>> [    0.884684] pci_bus 0000:00: root bus resource [mem 0xdfa00000-0xfebfffff window]
>>
>> This leads to the following difference in assignments of PCI resources when honoring E820 reservations
>>
>> [    0.966573] pci 0000:00:1c.0: BAR 14: assigned [mem 0xdfb00000-0xdfcfffff]
>> [    0.966698] pci_bus 0000:02: resource 1 [mem 0xdfb00000-0xdfcfffff]
>>
>> vs the following when ignoring E820 reservations:
>>
>> [    0.966850] pci 0000:00:1c.0: BAR 14: assigned [mem 0xdfa00000-0xdfbfffff]
>> [    0.966973] pci_bus 0000:02: resource 1 [mem 0xdfa00000-0xdfbfffff]
>>
>> And the overlap of 0xdfa00000-0xdfa0ffff from the e820 reservations seems to be what is causing the suspend/resume issue.
> 
> Any idea what is using that range?

No, no clue I'm afraid.

>> ###
>>
>> As already somewhat discussed, I'll go and prepare this solution instead:
>>
>> 1. Add E820_TYPE_MMIO to enum e820_type and modify the 2 places which check for
>>    type == reserved to treat this as reserved too, so as to not have any
>>    functional changes there
>>
>> 2. Modify the code building e820 tables from the EFI memmap to use
>>    E820_TYPE_MMIO for MMIO EFI memmap entries.
>>
>> 3. Modify arch/x86/kernel/resource.c: remove_e820_regions() to skip
>>    e820 table entries with a type of E820_TYPE_MMIO,
>>    this would actually be a functional change and should fix the
>>    issues we are trying to fix.
> 
> Given the above regression, I can't think of a better way to solve this.

Ack, note I'm still waiting for efi=debug output from the X1 carbon gen 2,
so I hope that what seems to be the conflicting range is not also marked
as EFI_MEMORY_MAPPED_IO. Otherwise things will get a bit more complicated (*)

Regards,

Hans

*) On the systems where the EFI_MEMORY_MAPPED_IO memmap entries are causing
issues they fully overlap the PCI bridge window, so we can use that as an
extra check if necessary.




>