Help with Network SR-IOV (PF still in reset state. Is the PF interface up?)

Help with Network SR-IOV (PF still in reset state. Is the PF interface up?)

[Bhasker C V]

[-- Attachment #1: Type: text/plain, Size: 2261 bytes --]

I run  xen-4.14.3-pre from debian

I created 6 vfs for my igb card
06:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network
Connection (rev 01)
I got these
07:10.0 Ethernet controller: Intel Corporation I350 Ethernet Controller
Virtual Function (rev 01)
07:10.1 Ethernet controller: Intel Corporation I350 Ethernet Controller
Virtual Function (rev 01)
07:10.4 Ethernet controller: Intel Corporation I350 Ethernet Controller
Virtual Function (rev 01)
07:10.5 Ethernet controller: Intel Corporation I350 Ethernet Controller
Virtual Function (rev 01)
07:11.0 Ethernet controller: Intel Corporation I350 Ethernet Controller
Virtual Function (rev 01)
07:11.1 Ethernet controller: Intel Corporation I350 Ethernet Controller
Virtual Function (rev 01)

bcv@bcvibmb1:~$ sudo xl pci-assignable-list
0000:07:10.1

However the guest is not able to see the link up.
bcv@utils:~$ sudo ethtool  enp0s0f1



Settings for enp0s0f1:

        Supported ports: [ ]
...

        Link detected: no

I see this error on the guest

[    6.134890] igbvf 0000:00:00.1: Xen PCI mapped GSI0 to IRQ19

[    6.149577] igbvf 0000:00:00.1: PF still in reset state. Is the PF
interface up?

[    6.149595] igbvf 0000:00:00.1: Assigning random MAC address.



[    6.151552] igbvf 0000:00:00.1: PF still resetting

[    6.153807] igbvf 0000:00:00.1: Intel(R) I350 Virtual Function

[    6.153822] igbvf 0000:00:00.1: Address: 32:91:70:c5:62:bd


and thiis on the host:
[  390.776779] xen-blkback: backend/vbd/3/51713: using 1 queues, protocol 1
(x86_64-abi) persistent grants


[  395.849996] pciback 0000:07:10.1: xen-pciback: Driver tried to write to
a read-only configuration space field at offset 0x1d6, size 2. This may be
harmless, but if you have problems with your device:

[  395.849996] 1) see permissive attribute in sysfs



[  395.849996] 2) report problems to the xen-devel mailing list along with
details of your device obtained from lspci.


[  396.103595] pciback 0000:07:10.1: enabling device (0000 -> 0002)




However if I pass through 07:10.4 it works fine. The host still reports
this  above error but all works well in the guest.


Please could someone help ? I have been using  KVM these years and just
moved to xen and hence new to xen.


Regards
Bhasker C V

[-- Attachment #2: Type: text/html, Size: 5588 bytes --]

Re: Help with Network SR-IOV (PF still in reset state. Is the PF interface up?)

[Jan Beulich]

(please don't cross-post; removing xen-users)

On 04.09.2021 17:12, Bhasker C V wrote:
> I run  xen-4.14.3-pre from debian
> 
> I created 6 vfs for my igb card
> 06:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network
> Connection (rev 01)
> I got these
> 07:10.0 Ethernet controller: Intel Corporation I350 Ethernet Controller
> Virtual Function (rev 01)
> 07:10.1 Ethernet controller: Intel Corporation I350 Ethernet Controller
> Virtual Function (rev 01)
> 07:10.4 Ethernet controller: Intel Corporation I350 Ethernet Controller
> Virtual Function (rev 01)
> 07:10.5 Ethernet controller: Intel Corporation I350 Ethernet Controller
> Virtual Function (rev 01)
> 07:11.0 Ethernet controller: Intel Corporation I350 Ethernet Controller
> Virtual Function (rev 01)
> 07:11.1 Ethernet controller: Intel Corporation I350 Ethernet Controller
> Virtual Function (rev 01)
> 
> bcv@bcvibmb1:~$ sudo xl pci-assignable-list
> 0000:07:10.1
> 
> However the guest is not able to see the link up.
> bcv@utils:~$ sudo ethtool  enp0s0f1
> 
> 
> 
> Settings for enp0s0f1:
> 
>         Supported ports: [ ]
> ...
> 
>         Link detected: no
> 
> I see this error on the guest
> 
> [    6.134890] igbvf 0000:00:00.1: Xen PCI mapped GSI0 to IRQ19
> 
> [    6.149577] igbvf 0000:00:00.1: PF still in reset state. Is the PF
> interface up?
> 
> [    6.149595] igbvf 0000:00:00.1: Assigning random MAC address.
> 
> 
> 
> [    6.151552] igbvf 0000:00:00.1: PF still resetting
> 
> [    6.153807] igbvf 0000:00:00.1: Intel(R) I350 Virtual Function
> 
> [    6.153822] igbvf 0000:00:00.1: Address: 32:91:70:c5:62:bd

All of this is kernel functionality, which driver maintainers may be
in a better position to help with. At the first glance the Xen
version in use may be of no relevance here.

> and thiis on the host:
> [  390.776779] xen-blkback: backend/vbd/3/51713: using 1 queues, protocol 1
> (x86_64-abi) persistent grants
> 
> 
> [  395.849996] pciback 0000:07:10.1: xen-pciback: Driver tried to write to
> a read-only configuration space field at offset 0x1d6, size 2. This may be
> harmless, but if you have problems with your device:
> 
> [  395.849996] 1) see permissive attribute in sysfs
> 
> 
> 
> [  395.849996] 2) report problems to the xen-devel mailing list along with
> details of your device obtained from lspci.
> 
> 
> [  396.103595] pciback 0000:07:10.1: enabling device (0000 -> 0002)
> 
> 
> 
> 
> However if I pass through 07:10.4 it works fine. The host still reports
> this  above error but all works well in the guest.

"This above error" being which one? "PF still resetting" or "Driver tried
to write ..."?

> Please could someone help ? I have been using  KVM these years and just
> moved to xen and hence new to xen.

Well, if one of the VFs works and the other doesn't, then a close
comparison of the various logs (hypervisor, kernels, tool stack) may
already provide an indication of the problem. Above you only supply
individual lines from logs, which isn't going to be sufficient to derive
anything.

Jan