[RFC] vgaarb: Add a module parm to use the first VGA device for boot

[RFC] vgaarb: Add a module parm to use the first VGA device for boot

[Cal Peake]

Hi,

I have a dual GPU system with the GPU in the first CPU PCIe slot for the 
host and the GPU in the second CPU PCIe slot being passed through to a 
virtual machine.

Currently, the VGA arbiter enumerates through all VGA devices in the order 
that they appear in on the bus with the last one ultimately getting the 
assignment for boot device.

This is a problem because once the VFIO PCI driver claims the second GPU, 
all display output is lost -- just empty black screens on both cards.

Setting VGA_ARB_MAX_GPUS to 1 helps a little: the X server will initialize 
on the first GPU like it should, but there's no VTs to be found (i.e. 
Alt-F1/F2/etc), just more black screens.

Below is an attempt at a fix by creating a kernel param to have the VGA 
arbiter use the first VGA device it finds for the boot device and then 
subsequently ignore the rest for this task.

-- 
Cal Peake

--- ./drivers/pci/vgaarb.c	2022-07-02 10:44:56.000000000 -0400
+++ ./drivers/pci/vgaarb.c	2022-07-02 22:55:24.291822548 -0400
@@ -35,6 +35,10 @@
 
 #include <linux/vgaarb.h>
 
+static bool use_first;
+module_param(use_first, bool, 0);
+MODULE_PARM_DESC(force, "Set to one to use the first VGA device for boot");
+
 static void vga_arbiter_notify_clients(void);
 /*
  * We keep a list of all vga devices in the system to speed
@@ -815,7 +819,8 @@ static bool vga_arbiter_add_pci_device(s
 		bus = bus->parent;
 	}
 
-	if (vga_is_boot_device(vgadev)) {
+	if (vga_is_boot_device(vgadev) &&
+	    (!use_first || (use_first && !vga_default_device()))) {
 		vgaarb_info(&pdev->dev, "setting as boot VGA device%s\n",
 			    vga_default_device() ?
 			    " (overriding previous)" : "");

Re: [RFC] vgaarb: Add a module parm to use the first VGA device for boot

[Randy Dunlap]

Hi--

On 7/2/22 21:22, Cal Peake wrote:
> Hi,
> 
> I have a dual GPU system with the GPU in the first CPU PCIe slot for the 
> host and the GPU in the second CPU PCIe slot being passed through to a 
> virtual machine.
> 
> Currently, the VGA arbiter enumerates through all VGA devices in the order 
> that they appear in on the bus with the last one ultimately getting the 
> assignment for boot device.
> 
> This is a problem because once the VFIO PCI driver claims the second GPU, 
> all display output is lost -- just empty black screens on both cards.
> 
> Setting VGA_ARB_MAX_GPUS to 1 helps a little: the X server will initialize 
> on the first GPU like it should, but there's no VTs to be found (i.e. 
> Alt-F1/F2/etc), just more black screens.
> 
> Below is an attempt at a fix by creating a kernel param to have the VGA 
> arbiter use the first VGA device it finds for the boot device and then 
> subsequently ignore the rest for this task.

So that's
  vgaarb.use_first=1
right?

Seems worthy of some documentation in
Documentation/admin-guide/kernel-parameters.txt, please.

-- 
~Randy

Re: [RFC] vgaarb: Add a module parm to use the first VGA device for boot

[Cal Peake]

On Sat, 2 Jul 2022, Randy Dunlap wrote:

> So that's
>   vgaarb.use_first=1
> right?

Yep

> Seems worthy of some documentation in
> Documentation/admin-guide/kernel-parameters.txt, please.

Absolutely! This was just an RFC to see if others thought this was the 
right approach to take.

I can have a new pair of patches out later today.

Thanks, Randy.

-- 
Cal Peake