Re: [PATCH RESEND 1/2] PCI: Extend isolated function probing to s390

[Bjorn Helgaas <helgaas@kernel.org>]

On Mon, Apr 04, 2022 at 11:53:45AM +0200, Niklas Schnelle wrote:
> Like the jailhouse hypervisor s390's PCI architecture allows passing
> isolated PCI functions to an OS instance. As of now this is was not
> utilized even with multi-function support as the s390 PCI code makes
> sure that only virtual PCI busses including a function with devfn 0 are
> presented to the PCI subsystem. A subsequent change will remove this
> restriction.
> 
> Allow probing such functions by replacing the existing check for
> jailhouse_paravirt() with a new hypervisor_isolated_pci_functions()
> helper.
> 
> Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>

I'm OK with the idea of generalizing this Jailhouse test, but I wonder
if this check should be in pci_scan_slot() rather than in
pci_scan_child_bus_extend().

I think the idea is that pci_scan_slot() should find all the functions
of a device (a.k.a. "slot"), so it's a little weird to have a loop
calling pci_scan_single_device() for each function in both places.

Currently we never call pcie_aspm_init_link_state() for these
Jailhouse or s390 functions.  Maybe that's OK (and I think
pci_scan_slot() is the wrong place to initialize ASPM anyway) but if
we could move the Jailhouse/s390 checking to pci_scan_slot(), it would
at least remove the inconsistency.

I'm thinking something along the lines of the patch below.  I'm sure
Jan considered this originally, so maybe there's some reason this
won't work.

Bjorn

> ---
>  drivers/pci/probe.c        | 4 ++--
>  include/linux/hypervisor.h | 9 +++++++++
>  2 files changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
> index 17a969942d37..e8fd89a1f984 100644
> --- a/drivers/pci/probe.c
> +++ b/drivers/pci/probe.c
> @@ -2869,11 +2869,11 @@ static unsigned int pci_scan_child_bus_extend(struct pci_bus *bus,
>  		nr_devs = pci_scan_slot(bus, devfn);
>  
>  		/*
> -		 * The Jailhouse hypervisor may pass individual functions of a
> +		 * Some hypervisors may pass individual functions of a
>  		 * multi-function device to a guest without passing function 0.
>  		 * Look for them as well.
>  		 */
> -		if (jailhouse_paravirt() && nr_devs == 0) {
> +		if (hypervisor_isolated_pci_functions() && nr_devs == 0) {
>  			for (fn = 1; fn < 8; fn++) {
>  				dev = pci_scan_single_device(bus, devfn + fn);
>  				if (dev)
> diff --git a/include/linux/hypervisor.h b/include/linux/hypervisor.h
> index fc08b433c856..52abd459f9a3 100644
> --- a/include/linux/hypervisor.h
> +++ b/include/linux/hypervisor.h
> @@ -32,4 +32,13 @@ static inline bool jailhouse_paravirt(void)
>  
>  #endif /* !CONFIG_X86 */
>  
> +static inline bool hypervisor_isolated_pci_functions(void)
> +{
> +	if (IS_ENABLED(CONFIG_S390))
> +		return true;
> +	else
> +		return jailhouse_paravirt();
> +}
> +
> +
>  #endif /* __LINUX_HYPEVISOR_H */
> -- 
> 2.32.0
> 


diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 17a969942d37..83e4885e0698 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -2650,9 +2650,9 @@ int pci_scan_slot(struct pci_bus *bus, int devfn)
 		return 0; /* Already scanned the entire slot */
 
 	dev = pci_scan_single_device(bus, devfn);
-	if (!dev)
+	if (!dev && !jailhouse_paravirt())
 		return 0;
-	if (!pci_dev_is_added(dev))
+	if (dev && !pci_dev_is_added(dev))
 		nr++;
 
 	for (fn = next_fn(bus, dev, 0); fn > 0; fn = next_fn(bus, dev, fn)) {
@@ -2858,30 +2858,16 @@ static unsigned int pci_scan_child_bus_extend(struct pci_bus *bus,
 {
 	unsigned int used_buses, normal_bridges = 0, hotplug_bridges = 0;
 	unsigned int start = bus->busn_res.start;
-	unsigned int devfn, fn, cmax, max = start;
+	unsigned int devfn, cmax, max = start;
 	struct pci_dev *dev;
 	int nr_devs;
 
 	dev_dbg(&bus->dev, "scanning bus\n");
 
 	/* Go find them, Rover! */
-	for (devfn = 0; devfn < 256; devfn += 8) {
+	for (devfn = 0; devfn < 256; devfn += 8)
 		nr_devs = pci_scan_slot(bus, devfn);
 
-		/*
-		 * The Jailhouse hypervisor may pass individual functions of a
-		 * multi-function device to a guest without passing function 0.
-		 * Look for them as well.
-		 */
-		if (jailhouse_paravirt() && nr_devs == 0) {
-			for (fn = 1; fn < 8; fn++) {
-				dev = pci_scan_single_device(bus, devfn + fn);
-				if (dev)
-					dev->multifunction = 1;
-			}
-		}
-	}
-
 	/* Reserve buses for SR-IOV capability */
 	used_buses = pci_iov_bus_range(bus);
 	max += used_buses;