Setting "orig_video_isVGA" when handing off Linux framebuffer

Setting "orig_video_isVGA" when handing off Linux framebuffer

[Benjamin Moody]

Dear kexec developers,

(Please let me know if there is a better place to discuss this issue.)

In setup_linux_vesafb (arch/i386/x86-linux-setup.c), we find the
following:

	struct fb_fix_screeninfo fix;
	struct fb_var_screeninfo var;
	int fd;

	fd = open("/dev/fb0", O_RDONLY);
	if (-1 == fd)
		return -1;

	if (-1 == ioctl(fd, FBIOGET_FSCREENINFO, &fix))
		goto out;
	if (-1 == ioctl(fd, FBIOGET_VSCREENINFO, &var))
		goto out;
	if (0 == strcmp(fix.id, "VESA VGA")) {
		/* VIDEO_TYPE_VLFB */
		real_mode->orig_video_isVGA = 0x23;
	} else if (0 == strcmp(fix.id, "EFI VGA")) {
		/* VIDEO_TYPE_EFI */
		real_mode->orig_video_isVGA = 0x70;
	} else if (arch_options.reuse_video_type) {
		int err;
		off_t offset = offsetof(typeof(*real_mode), orig_video_isVGA);

		/* blindly try old boot time video type */
		err = get_bootparam(&real_mode->orig_video_isVGA, offset, 1);
		if (err)
			goto out;
	} else {
		real_mode->orig_video_isVGA = 0;
		close(fd);
		return 0;
	}

What is the rationale for this logic?  What does the
'orig_video_isVGA' parameter actually mean, what is the difference
between VIDEO_TYPE_EFI and VIDEO_TYPE_VLFB, and why does the kernel
care about it?

I'm asking because in the context of Heads
(https://github.com/osresearch/heads), the framebuffer has been
configured by Linux (not by BIOS or UEFI), and indeed there *is* no
BIOS or UEFI, only coreboot.  Thus, 'fix.id' is not "VESA VGA" or "EFI
VGA", but rather "inteldrmfb" or "i915drmfb" or something like that,
and 'get_bootparam' provides nothing useful.  As a result,
'orig_video_isVGA' ends up being set to zero, and consequently the
booted kernel either displays nothing or displays garbage.

I realize that not all possible framebuffer formats can be described
by 'struct x86_linux_param_header', and I assume that's the reason for
checking 'fix.id'.  However, in this case, the framebuffer *is* a
standard linear format, and if we set 'orig_video_isVGA' to either
0x23 or 0x70, it appears to work correctly.

From my naive perspective, it seems that, rather than checking the
value of 'fix.id', kexec ought to be looking at 'fix.type',
'fix.visual', and perhaps other fields, to determine whether the
framebuffer format is compatible.  However, it's not apparent whether
'orig_video_isVGA' should be set to 0x23 or 0x70, since the two seem
to behave identically.

I'm a little unsure about the purpose of the "blindly try old boot
time video type" option but it seems like that's not really relevant
here in any case; the original bootloader has nothing to do with the
current framebuffer.

Benjamin

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: Setting "orig_video_isVGA" when handing off Linux framebuffer

[Benjamin Moody]

Hi,

In regard to how kexec hands off the framebuffer to the newly-booted
kernel:

Commit 060eee589dd1 (2018-01-28) added the "blindly try old boot time
video type" behavior, without doing any checking to see if the
framebuffer is compatible with the stated format.

Commit fb5a8792e6e4 (2019-03-05) made this behavior conditional on the
--reuse-video-type option.  The commit message observes that:

    Currently kernel hanging is inspected on some hyper-v VMs after this
    commit, because hyperv_fb will mimic EFI (or VESA) VGA on first boot
    up, but after the real driver is loaded, it will switch to new mode
    and no longer compatible with EFI/VESA VGA. Keep setting
    orig_video_isVGA to EFI/VESA VGA flag will get wrong driver loaded
    and try to manipulate the framebuffer in a wrong way.

It's clear to me that various bad things *might* happen if kexec
pretends that the framebuffer is "VESA-compatible" or "EFI-compatible"
when in fact it isn't.

Yet, in many cases, the Linux framebuffer is VESA/EFI-compatible, at
least to the extent that blindly setting orig_video_isVGA = 0x23 or
0x70 results in a usable display.  So I have to wonder, in the
situation mentioned above:

 - was the framebuffer not in a compatible format to begin with?

 - was the framebuffer address not correctly reported by the existing
   kernel driver?

 - did the original bootloader give wrong information and somehow that
   broke the newly booted kernel?

Kairui, can you please clarify what sort of kernel hangs you were
seeing and what specific hardware and drivers you were using?

Benjamin Moody

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: Setting "orig_video_isVGA" when handing off Linux framebuffer

[Kairui Song]

 Hi Benjamin,

Sorry for the late reply, I missed your email in my inbox.

On Wed, May 5, 2021 at 7:10 AM Benjamin Moody <benjamin.moody@gmail.com> wrote:
>
> Hi,
>
> In regard to how kexec hands off the framebuffer to the newly-booted
> kernel:
>
> Commit 060eee589dd1 (2018-01-28) added the "blindly try old boot time
> video type" behavior, without doing any checking to see if the
> framebuffer is compatible with the stated format.
>
> Commit fb5a8792e6e4 (2019-03-05) made this behavior conditional on the
> --reuse-video-type option.  The commit message observes that:
>
>     Currently kernel hanging is inspected on some hyper-v VMs after this
>     commit, because hyperv_fb will mimic EFI (or VESA) VGA on first boot
>     up, but after the real driver is loaded, it will switch to new mode
>     and no longer compatible with EFI/VESA VGA. Keep setting
>     orig_video_isVGA to EFI/VESA VGA flag will get wrong driver loaded
>     and try to manipulate the framebuffer in a wrong way.
>
> It's clear to me that various bad things *might* happen if kexec
> pretends that the framebuffer is "VESA-compatible" or "EFI-compatible"
> when in fact it isn't.
>
> Yet, in many cases, the Linux framebuffer is VESA/EFI-compatible, at
> least to the extent that blindly setting orig_video_isVGA = 0x23 or
> 0x70 results in a usable display.  So I have to wonder, in the
> situation mentioned above:
>
>  - was the framebuffer not in a compatible format to begin with?
>
>  - was the framebuffer address not correctly reported by the existing
>    kernel driver?
>
>  - did the original bootloader give wrong information and somehow that
>    broke the newly booted kernel?
>
> Kairui, can you please clarify what sort of kernel hangs you were
> seeing and what specific hardware and drivers you were using?
>

For the commit fb5a8792e6e4, the problem is only observed with
hyperv_fb, and it's a HyperV VM.
The framebuffer was VESA compatible when the machine just booted, but
after hyperv_fb driver is loaded, it will ask the hypervisor to
relocate the framebuffer in a new location and in a new format.

In a later kernel commit 3cb73bc3fa2a3cb80b88aa63b48409939e0d996b, it
fixed the kernel side issue that after the relocation, the framebuffer
address is not updated in boot_params. It was not updated before this
kernel commit. Before that, the old boot_params will contain an
invalid address and cause failures in the new booted kernel.

And I also remember blindly setting orig_video_isVGA will cause
strange errors on some random graphic cards. If we can't make sure
it's really VGA, this field better left zero, so kernel won't use it
as a VGA framebuffer.

For your case, you mentioned "'fix.id' is not "VESA VGA" or "EFI VGA",
but rather "inteldrmfb" or "i915drmfb"",  'fix.id' can change after
boot, I'm not familiar with heads or coreboot, but I guess the first
kernel you booted have intel drm drivers loaded? Maybe you can try
either don't load intel drm driver in first kernel (so the framebuffer
is always being used in a VESA/EFI compatible way), or ensure same
driver is loaded in the new booted kernel (this way the driver will
reinitialize the framebuffer anyway, even if it's not set in
boot_params).

> Benjamin Moody
>


-- 
Best Regards,
Kairui Song


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: Setting "orig_video_isVGA" when handing off Linux framebuffer

[Benjamin Moody]

Thanks, this is helpful!

> For the commit fb5a8792e6e4, the problem is only observed with
> hyperv_fb, and it's a HyperV VM.
> The framebuffer was VESA compatible when the machine just booted, but
> after hyperv_fb driver is loaded, it will ask the hypervisor to
> relocate the framebuffer in a new location and in a new format.
> 
> In a later kernel commit 3cb73bc3fa2a3cb80b88aa63b48409939e0d996b, it
> fixed the kernel side issue that after the relocation, the framebuffer
> address is not updated in boot_params. It was not updated before this
> kernel commit. Before that, the old boot_params will contain an
> invalid address and cause failures in the new booted kernel.

This doesn't seem like a HyperV issue, it seems like a general problem
with assuming the boot-time screen parameters are valid after some
driver has reconfigured the display.

> And I also remember blindly setting orig_video_isVGA will cause
> strange errors on some random graphic cards. If we can't make sure
> it's really VGA, this field better left zero, so kernel won't use it
> as a VGA framebuffer.
> 
> For your case, you mentioned "'fix.id' is not "VESA VGA" or "EFI VGA",
> but rather "inteldrmfb" or "i915drmfb"",  'fix.id' can change after
> boot, I'm not familiar with heads or coreboot, but I guess the first
> kernel you booted have intel drm drivers loaded?

Exactly.

> Maybe you can try
> either don't load intel drm driver in first kernel (so the framebuffer
> is always being used in a VESA/EFI compatible way),

Good idea, but in this case there isn't any BIOS or EFI firmware.  If I
didn't load the DRM driver then there would be no display at all.

> or ensure same
> driver is loaded in the new booted kernel (this way the driver will
> reinitialize the framebuffer anyway, even if it's not set in
> boot_params).

Indeed, but this is only possible if you have control over the
kernel/OS being booted.

The example that set this off is the Debian installer: the installer
doesn't include video drivers, it only uses the framebuffer provided by
the bootloader.

But more broadly, when you're booting a new kernel, it is best to have
some way of displaying stuff on screen, if for no other reason than to
tell you why it is unable to load the DRM driver.

From my understanding and brief perusal of the code, if the bootloader
specifies an "EFI framebuffer", then Linux simply treats this as a big
reserved block of memory where pixels can be drawn.  It doesn't support
changing the resolution or format, and doesn't support pseudocolor.  If
I understand correctly, EFI itself doesn't permit changing the
framebuffer configuration "after boot time".

So it seems to me that:

 - if fix.smem_start != 0
 - and fix.smem_len != 0
 - and fix.type == FB_TYPE_PACKED_PIXELS
 - and fix.visual == FB_VISUAL_TRUECOLOR

then it ought to be safe to treat this as an "EFI framebuffer" and
allow the booted kernel to draw on it.  This works, as far as I can
see, with all the drivers I've tested (i915, radeon, cirrus, uvesafb).

Looking at the hyperv_fb driver, however, it appears to be setting
smem_start and smem_len (indicating that the framebuffer is located in
contiguous "physical" memory) even when that is incorrect.  Perhaps
that is the cause of the problems you saw.

I notice that framebuffer_info (kexec/arch/i386/kexec-multiboot-x86.c)
and multiboot2_make_mbi (kexec/arch/i386/kexec-mb2-x86.c) already do
just what I'm suggesting.  So I expect that, if you tried to boot a
Multiboot kernel that supports video handoff, you would see the same
problems as before.

It would be a good sanity check if kexec could somehow test whether
smem_start is actually a valid physical memory address.  Is there any
way to do that in userspace?

Benjamin

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec