* pci passthrough issue introduced between 4.14.1 and 4.15.0
@ 2021-06-01 7:36 AL13N
2021-06-01 10:08 ` Jan Beulich
0 siblings, 1 reply; 15+ messages in thread
From: AL13N @ 2021-06-01 7:36 UTC (permalink / raw)
To: Xen-devel
Not 100% it's a bug or something i did wrong, but,
with xl create i start a PV with 3 pci passthroughs
after wards, xl pci-list shows all 3 nicely
looking at the domU boot logs, pcifront is only creating one pci device
and lspci in the guest shows only 1 pci entry
in at least 4.14.1 it still works.
looking deeper, if you start with only 1 entry or 0 and you do
pci-attach for the other pci entries, all of this works fine.
this is the pci section in question:
pci=['0000:04:00.0,permissive=1',
'0000:00:1a.0,permissive=1,rdm_policy=relaxed,seize=1',
'0000:00:1d.0,permissive=1,rdm_policy=relaxed,seize=1']
this works fine in 4.12.1 and 4.14.1, but fails in 4.15.0, however
booting with pci=['0000:04:00.0,permissive=1'] and then doing
[]# xl pci-attach <domain>
'0000:00:1a.0,permissive=1,rdm_policy=relaxed,seize=1'
[]# xl pci-attach <domain>
'0000:00:1d.0,permissive=1,rdm_policy=relaxed,seize=1'
also works,
Regards,
Maarten (alien on Libera and OFTC)
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: pci passthrough issue introduced between 4.14.1 and 4.15.0
2021-06-01 7:36 pci passthrough issue introduced between 4.14.1 and 4.15.0 AL13N
@ 2021-06-01 10:08 ` Jan Beulich
2021-06-01 14:06 ` AL13N
0 siblings, 1 reply; 15+ messages in thread
From: Jan Beulich @ 2021-06-01 10:08 UTC (permalink / raw)
To: AL13N; +Cc: Xen-devel
On 01.06.2021 09:36, AL13N wrote:
> Not 100% it's a bug or something i did wrong, but,
>
> with xl create i start a PV with 3 pci passthroughs
>
> after wards, xl pci-list shows all 3 nicely
>
> looking at the domU boot logs, pcifront is only creating one pci device
> and lspci in the guest shows only 1 pci entry
>
> in at least 4.14.1 it still works.
This reminds me of my report at
https://lists.xen.org/archives/html/xen-devel/2021-03/msg00956.html
Meanwhile the proposed pciback change has gone in upstream:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/xen/xen-pciback?id=c81d3d24602540f65256f98831d0a25599ea6b87
I wasn't, however, aware that this may have been an issue going
from 4.14.1 to 4.15.0, i.e. something that was presumably (as
George also has just said) a regression in the tools. Or else I
probably wouldn't have suggested taking care of this in Linux.
Nevertheless you may want to give that change a try.
Jan
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: pci passthrough issue introduced between 4.14.1 and 4.15.0
2021-06-01 10:08 ` Jan Beulich
@ 2021-06-01 14:06 ` AL13N
2021-06-01 14:28 ` AL13N
2021-06-01 14:33 ` Jan Beulich
0 siblings, 2 replies; 15+ messages in thread
From: AL13N @ 2021-06-01 14:06 UTC (permalink / raw)
To: Xen-devel; +Cc: Jan Beulich
Jan Beulich schreef op 2021-06-01 12:08:
> On 01.06.2021 09:36, AL13N wrote:
>> Not 100% it's a bug or something i did wrong, but,
>>
>> with xl create i start a PV with 3 pci passthroughs
>>
>> after wards, xl pci-list shows all 3 nicely
>>
>> looking at the domU boot logs, pcifront is only creating one pci
>> device
>> and lspci in the guest shows only 1 pci entry
>>
>> in at least 4.14.1 it still works.
>
> This reminds me of my report at
> https://lists.xen.org/archives/html/xen-devel/2021-03/msg00956.html
>
> Meanwhile the proposed pciback change has gone in upstream:
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/xen/xen-pciback?id=c81d3d24602540f65256f98831d0a25599ea6b87
>
> I wasn't, however, aware that this may have been an issue going
> from 4.14.1 to 4.15.0, i.e. something that was presumably (as
> George also has just said) a regression in the tools. Or else I
> probably wouldn't have suggested taking care of this in Linux.
> Nevertheless you may want to give that change a try.
Well, both tests have only different tools en hypervisor, no kernel was
changed between both tests, neither in dom0 or domU , so, it might not
be pciback?
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: pci passthrough issue introduced between 4.14.1 and 4.15.0
2021-06-01 14:06 ` AL13N
@ 2021-06-01 14:28 ` AL13N
2021-06-01 14:33 ` Jan Beulich
1 sibling, 0 replies; 15+ messages in thread
From: AL13N @ 2021-06-01 14:28 UTC (permalink / raw)
To: Xen-devel; +Cc: Jan Beulich
AL13N schreef op 2021-06-01 16:06:
> Jan Beulich schreef op 2021-06-01 12:08:
>> On 01.06.2021 09:36, AL13N wrote:
>>> Not 100% it's a bug or something i did wrong, but,
>>>
>>> with xl create i start a PV with 3 pci passthroughs
>>>
>>> after wards, xl pci-list shows all 3 nicely
>>>
>>> looking at the domU boot logs, pcifront is only creating one pci
>>> device
>>> and lspci in the guest shows only 1 pci entry
>>>
>>> in at least 4.14.1 it still works.
>>
>> This reminds me of my report at
>> https://lists.xen.org/archives/html/xen-devel/2021-03/msg00956.html
>>
>> Meanwhile the proposed pciback change has gone in upstream:
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/xen/xen-pciback?id=c81d3d24602540f65256f98831d0a25599ea6b87
>>
>> I wasn't, however, aware that this may have been an issue going
>> from 4.14.1 to 4.15.0, i.e. something that was presumably (as
>> George also has just said) a regression in the tools. Or else I
>> probably wouldn't have suggested taking care of this in Linux.
>> Nevertheless you may want to give that change a try.
>
> Well, both tests have only different tools en hypervisor, no kernel
> was changed between both tests, neither in dom0 or domU , so, it might
> not be pciback?
forgot to mention, dom0 is 5.7.19 and domU is 5.10.27 for all tests
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: pci passthrough issue introduced between 4.14.1 and 4.15.0
2021-06-01 14:06 ` AL13N
2021-06-01 14:28 ` AL13N
@ 2021-06-01 14:33 ` Jan Beulich
2021-06-01 14:44 ` AL13N
1 sibling, 1 reply; 15+ messages in thread
From: Jan Beulich @ 2021-06-01 14:33 UTC (permalink / raw)
To: AL13N; +Cc: Xen-devel
On 01.06.2021 16:06, AL13N wrote:
> Jan Beulich schreef op 2021-06-01 12:08:
>> On 01.06.2021 09:36, AL13N wrote:
>>> Not 100% it's a bug or something i did wrong, but,
>>>
>>> with xl create i start a PV with 3 pci passthroughs
>>>
>>> after wards, xl pci-list shows all 3 nicely
>>>
>>> looking at the domU boot logs, pcifront is only creating one pci
>>> device
>>> and lspci in the guest shows only 1 pci entry
>>>
>>> in at least 4.14.1 it still works.
>>
>> This reminds me of my report at
>> https://lists.xen.org/archives/html/xen-devel/2021-03/msg00956.html
>>
>> Meanwhile the proposed pciback change has gone in upstream:
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/xen/xen-pciback?id=c81d3d24602540f65256f98831d0a25599ea6b87
>>
>> I wasn't, however, aware that this may have been an issue going
>> from 4.14.1 to 4.15.0, i.e. something that was presumably (as
>> George also has just said) a regression in the tools. Or else I
>> probably wouldn't have suggested taking care of this in Linux.
>> Nevertheless you may want to give that change a try.
>
> Well, both tests have only different tools en hypervisor, no kernel was
> changed between both tests, neither in dom0 or domU , so, it might not
> be pciback?
Well, if the problem was introduced in the tools and this having been
the reason for me running into the same or a similar issue, the patch
may still address the issue, even if - in case it's a regression in
the tools - it would have been better to also address the issue there.
As said, when analyzing the issue I didn't have indications of changed
tool stack behavior, i.e. I assumed the problem would have always been
there.
Jan
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: pci passthrough issue introduced between 4.14.1 and 4.15.0
2021-06-01 14:33 ` Jan Beulich
@ 2021-06-01 14:44 ` AL13N
2021-06-01 14:48 ` AL13N
2021-06-01 14:53 ` Jan Beulich
0 siblings, 2 replies; 15+ messages in thread
From: AL13N @ 2021-06-01 14:44 UTC (permalink / raw)
To: Jan Beulich; +Cc: Xen-devel
Jan Beulich schreef op 2021-06-01 16:33:
> On 01.06.2021 16:06, AL13N wrote:
>> Jan Beulich schreef op 2021-06-01 12:08:
>>> On 01.06.2021 09:36, AL13N wrote:
>>>> Not 100% it's a bug or something i did wrong, but,
>>>>
>>>> with xl create i start a PV with 3 pci passthroughs
>>>>
>>>> after wards, xl pci-list shows all 3 nicely
>>>>
>>>> looking at the domU boot logs, pcifront is only creating one pci
>>>> device
>>>> and lspci in the guest shows only 1 pci entry
>>>>
>>>> in at least 4.14.1 it still works.
>>>
>>> This reminds me of my report at
>>> https://lists.xen.org/archives/html/xen-devel/2021-03/msg00956.html
>>>
>>> Meanwhile the proposed pciback change has gone in upstream:
>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/xen/xen-pciback?id=c81d3d24602540f65256f98831d0a25599ea6b87
>>>
>>> I wasn't, however, aware that this may have been an issue going
>>> from 4.14.1 to 4.15.0, i.e. something that was presumably (as
>>> George also has just said) a regression in the tools. Or else I
>>> probably wouldn't have suggested taking care of this in Linux.
>>> Nevertheless you may want to give that change a try.
>>
>> Well, both tests have only different tools en hypervisor, no kernel
>> was
>> changed between both tests, neither in dom0 or domU , so, it might not
>> be pciback?
>
> Well, if the problem was introduced in the tools and this having been
> the reason for me running into the same or a similar issue, the patch
> may still address the issue, even if - in case it's a regression in
> the tools - it would have been better to also address the issue there.
> As said, when analyzing the issue I didn't have indications of changed
> tool stack behavior, i.e. I assumed the problem would have always been
> there.
Yeah after rereading the thread, i got this impression.
though after looking at a quick grep:
[alien@localhost xen]$ git log RELEASE-4.14.1..RELEASE-4.15.0 --oneline
--decorate -- tools/xl/ | grep -i pci
bdc0799fe2 libxlu: introduce xlu_pci_parse_spec_string()
96ed6ff297 libxlu: introduce xlu_pci_parse_spec_string()
929f231140 libxl: introduce 'libxl_pci_bdf' in the idl...
c00da82355 libxl: add libxl_device_pci_assignable_list_free()...
7499b22ba1 libxl: make sure callers of libxl_device_pci_list() free the
list after use
6c2590967f xl: s/pcidev/pci where possible
it doesn't seem like one of these? (well, i've not familiar with any of
the xen code)
This mailing list is the correct place for the toolstack too? right?
Regards,
Maarten
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: pci passthrough issue introduced between 4.14.1 and 4.15.0
2021-06-01 14:44 ` AL13N
@ 2021-06-01 14:48 ` AL13N
2021-06-01 15:50 ` AL13N
2021-06-01 14:53 ` Jan Beulich
1 sibling, 1 reply; 15+ messages in thread
From: AL13N @ 2021-06-01 14:48 UTC (permalink / raw)
To: Xen-devel; +Cc: Jan Beulich
AL13N schreef op 2021-06-01 16:44:
> Jan Beulich schreef op 2021-06-01 16:33:
>> On 01.06.2021 16:06, AL13N wrote:
>>> Jan Beulich schreef op 2021-06-01 12:08:
>>>> On 01.06.2021 09:36, AL13N wrote:
>>>>> Not 100% it's a bug or something i did wrong, but,
>>>>>
>>>>> with xl create i start a PV with 3 pci passthroughs
>>>>>
>>>>> after wards, xl pci-list shows all 3 nicely
>>>>>
>>>>> looking at the domU boot logs, pcifront is only creating one pci
>>>>> device
>>>>> and lspci in the guest shows only 1 pci entry
>>>>>
>>>>> in at least 4.14.1 it still works.
>>>>
>>>> This reminds me of my report at
>>>> https://lists.xen.org/archives/html/xen-devel/2021-03/msg00956.html
>>>>
>>>> Meanwhile the proposed pciback change has gone in upstream:
>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/xen/xen-pciback?id=c81d3d24602540f65256f98831d0a25599ea6b87
>>>>
>>>> I wasn't, however, aware that this may have been an issue going
>>>> from 4.14.1 to 4.15.0, i.e. something that was presumably (as
>>>> George also has just said) a regression in the tools. Or else I
>>>> probably wouldn't have suggested taking care of this in Linux.
>>>> Nevertheless you may want to give that change a try.
>>>
>>> Well, both tests have only different tools en hypervisor, no kernel
>>> was
>>> changed between both tests, neither in dom0 or domU , so, it might
>>> not
>>> be pciback?
>>
>> Well, if the problem was introduced in the tools and this having been
>> the reason for me running into the same or a similar issue, the patch
>> may still address the issue, even if - in case it's a regression in
>> the tools - it would have been better to also address the issue there.
>> As said, when analyzing the issue I didn't have indications of changed
>> tool stack behavior, i.e. I assumed the problem would have always been
>> there.
>
> Yeah after rereading the thread, i got this impression.
>
> though after looking at a quick grep:
>
> [alien@localhost xen]$ git log RELEASE-4.14.1..RELEASE-4.15.0
> --oneline --decorate -- tools/xl/ | grep -i pci
> bdc0799fe2 libxlu: introduce xlu_pci_parse_spec_string()
> 96ed6ff297 libxlu: introduce xlu_pci_parse_spec_string()
> 929f231140 libxl: introduce 'libxl_pci_bdf' in the idl...
> c00da82355 libxl: add libxl_device_pci_assignable_list_free()...
> 7499b22ba1 libxl: make sure callers of libxl_device_pci_list() free
> the list after use
> 6c2590967f xl: s/pcidev/pci where possible
>
>
> it doesn't seem like one of these? (well, i've not familiar with any
> of the xen code)
>
> This mailing list is the correct place for the toolstack too? right?
i forgot the libs....
[alien@localhost xen]$ git log RELEASE-4.14.1..RELEASE-4.15.0 --oneline
--decorate -- tools/libs/light/ | grep -i pci
9cd5bbf536 libxl / libxlu: support 'xl pci-attach/detach' by name
57bff091f4 libxl: add 'name' field to 'libxl_device_pci' in the IDL...
d473d74af3 libxl: stop setting 'vdevfn' in pci_struct_fill()
8bf0fab142 libxl / libxlu: support 'xl pci-attach/detach' by name
5ab684cb3e libxl: introduce
libxl_pci_bdf_assignable_add/remove/list/list_free(), ...
66c2fbc6e8 libxl: convert internal functions in libxl_pci.c...
929f231140 libxl: introduce 'libxl_pci_bdf' in the idl...
413fd4e4e9 libxl: use COMPARE_PCI() macro is_pci_in_array()...
c00da82355 libxl: add libxl_device_pci_assignable_list_free()...
7499b22ba1 libxl: make sure callers of libxl_device_pci_list() free the
list after use
f8cfb85719 libxl: remove get_all_assigned_devices() from libxl_pci.c
4951b9ea80 libxl: remove unnecessary check from libxl__device_pci_add()
fe91a3aadc libxl: generalise 'driver_path' xenstore access functions in
libxl_pci.c
b5429d65e1 libxl: stop using aodev->device_config in
libxl__device_pci_add()...
a825ab3a6b libxl: remove extraneous arguments to do_pci_remove() in
libxl_pci.c
33e1c5a5a8 libxl: s/detatched/detached in libxl_pci.c
d8cba539f2 libxl: add/recover 'rdm_policy' to/from PCI backend in
xenstore
0fdb48ffe7 libxl: Make sure devices added by pci-attach are reflected in
the config
fce69998ed libxl: make libxl__device_list() work correctly for
LIBXL__DEVICE_KIND_PCI...
e43780f15f libxl: s/pcidev/pci and remove DEFINE_DEVICE_TYPE_STRUCT_X
what is this "f8cfb85719 libxl: remove get_all_assigned_devices() from
libxl_pci.c" doesn't it seems sus?
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: pci passthrough issue introduced between 4.14.1 and 4.15.0
2021-06-01 14:48 ` AL13N
@ 2021-06-01 15:50 ` AL13N
0 siblings, 0 replies; 15+ messages in thread
From: AL13N @ 2021-06-01 15:50 UTC (permalink / raw)
To: Xen-devel; +Cc: Jan Beulich
AL13N schreef op 2021-06-01 16:48:
> AL13N schreef op 2021-06-01 16:44:
>> Jan Beulich schreef op 2021-06-01 16:33:
>>> On 01.06.2021 16:06, AL13N wrote:
>>>> Jan Beulich schreef op 2021-06-01 12:08:
>>>>> On 01.06.2021 09:36, AL13N wrote:
>>>>>> Not 100% it's a bug or something i did wrong, but,
>>>>>>
>>>>>> with xl create i start a PV with 3 pci passthroughs
>>>>>>
>>>>>> after wards, xl pci-list shows all 3 nicely
>>>>>>
>>>>>> looking at the domU boot logs, pcifront is only creating one pci
>>>>>> device
>>>>>> and lspci in the guest shows only 1 pci entry
>>>>>>
>>>>>> in at least 4.14.1 it still works.
>>>>>
>>>>> This reminds me of my report at
>>>>> https://lists.xen.org/archives/html/xen-devel/2021-03/msg00956.html
>>>>>
>>>>> Meanwhile the proposed pciback change has gone in upstream:
>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/xen/xen-pciback?id=c81d3d24602540f65256f98831d0a25599ea6b87
>>>>>
>>>>> I wasn't, however, aware that this may have been an issue going
>>>>> from 4.14.1 to 4.15.0, i.e. something that was presumably (as
>>>>> George also has just said) a regression in the tools. Or else I
>>>>> probably wouldn't have suggested taking care of this in Linux.
>>>>> Nevertheless you may want to give that change a try.
>>>>
>>>> Well, both tests have only different tools en hypervisor, no kernel
>>>> was
>>>> changed between both tests, neither in dom0 or domU , so, it might
>>>> not
>>>> be pciback?
>>>
>>> Well, if the problem was introduced in the tools and this having been
>>> the reason for me running into the same or a similar issue, the patch
>>> may still address the issue, even if - in case it's a regression in
>>> the tools - it would have been better to also address the issue
>>> there.
>>> As said, when analyzing the issue I didn't have indications of
>>> changed
>>> tool stack behavior, i.e. I assumed the problem would have always
>>> been
>>> there.
>>
>> Yeah after rereading the thread, i got this impression.
>>
>> though after looking at a quick grep:
>>
>> [alien@localhost xen]$ git log RELEASE-4.14.1..RELEASE-4.15.0
>> --oneline --decorate -- tools/xl/ | grep -i pci
>> bdc0799fe2 libxlu: introduce xlu_pci_parse_spec_string()
>> 96ed6ff297 libxlu: introduce xlu_pci_parse_spec_string()
>> 929f231140 libxl: introduce 'libxl_pci_bdf' in the idl...
>> c00da82355 libxl: add libxl_device_pci_assignable_list_free()...
>> 7499b22ba1 libxl: make sure callers of libxl_device_pci_list() free
>> the list after use
>> 6c2590967f xl: s/pcidev/pci where possible
>>
>>
>> it doesn't seem like one of these? (well, i've not familiar with any
>> of the xen code)
>>
>> This mailing list is the correct place for the toolstack too? right?
>
> i forgot the libs....
>
> [alien@localhost xen]$ git log RELEASE-4.14.1..RELEASE-4.15.0
> --oneline --decorate -- tools/libs/light/ | grep -i pci
> 9cd5bbf536 libxl / libxlu: support 'xl pci-attach/detach' by name
> 57bff091f4 libxl: add 'name' field to 'libxl_device_pci' in the IDL...
> d473d74af3 libxl: stop setting 'vdevfn' in pci_struct_fill()
> 8bf0fab142 libxl / libxlu: support 'xl pci-attach/detach' by name
> 5ab684cb3e libxl: introduce
> libxl_pci_bdf_assignable_add/remove/list/list_free(), ...
> 66c2fbc6e8 libxl: convert internal functions in libxl_pci.c...
> 929f231140 libxl: introduce 'libxl_pci_bdf' in the idl...
> 413fd4e4e9 libxl: use COMPARE_PCI() macro is_pci_in_array()...
> c00da82355 libxl: add libxl_device_pci_assignable_list_free()...
> 7499b22ba1 libxl: make sure callers of libxl_device_pci_list() free
> the list after use
> f8cfb85719 libxl: remove get_all_assigned_devices() from libxl_pci.c
> 4951b9ea80 libxl: remove unnecessary check from libxl__device_pci_add()
> fe91a3aadc libxl: generalise 'driver_path' xenstore access functions
> in libxl_pci.c
> b5429d65e1 libxl: stop using aodev->device_config in
> libxl__device_pci_add()...
> a825ab3a6b libxl: remove extraneous arguments to do_pci_remove() in
> libxl_pci.c
> 33e1c5a5a8 libxl: s/detatched/detached in libxl_pci.c
> d8cba539f2 libxl: add/recover 'rdm_policy' to/from PCI backend in
> xenstore
> 0fdb48ffe7 libxl: Make sure devices added by pci-attach are reflected
> in the config
> fce69998ed libxl: make libxl__device_list() work correctly for
> LIBXL__DEVICE_KIND_PCI...
> e43780f15f libxl: s/pcidev/pci and remove DEFINE_DEVICE_TYPE_STRUCT_X
>
>
> what is this "f8cfb85719 libxl: remove get_all_assigned_devices() from
> libxl_pci.c" doesn't it seems sus?
or maybe "7499b22ba1 libxl: make sure callers of libxl_device_pci_list()
free the list after use" , if between adding pci, it also does the list
and then maybe it gets freed and thus stops after the first one?
sounds like a longshot, tbh...
anyone have an idea which it might be? or should i look in other places?
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: pci passthrough issue introduced between 4.14.1 and 4.15.0
2021-06-01 14:44 ` AL13N
2021-06-01 14:48 ` AL13N
@ 2021-06-01 14:53 ` Jan Beulich
2021-06-03 16:01 ` AL13N
1 sibling, 1 reply; 15+ messages in thread
From: Jan Beulich @ 2021-06-01 14:53 UTC (permalink / raw)
To: AL13N; +Cc: Xen-devel
On 01.06.2021 16:44, AL13N wrote:
> This mailing list is the correct place for the toolstack too? right?
Yes.
Jan
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: pci passthrough issue introduced between 4.14.1 and 4.15.0
2021-06-01 14:53 ` Jan Beulich
@ 2021-06-03 16:01 ` AL13N
2021-06-04 6:56 ` Jan Beulich
0 siblings, 1 reply; 15+ messages in thread
From: AL13N @ 2021-06-03 16:01 UTC (permalink / raw)
To: Xen-devel; +Cc: Jan Beulich
Jan Beulich schreef op 2021-06-01 16:53:
> On 01.06.2021 16:44, AL13N wrote:
>> This mailing list is the correct place for the toolstack too? right?
>
> Yes.
So, what's the plan to fix this? is the plan to fix the toolstack? or
put your patch in kernel to kinda workaround it?
Is there a way to make a regression test or unit test or something?
Does anyone have an idea on what patch would cause the regression? that
patch that I pointed out, could it be that one, or should i look at a
specific file/line? I can't really just test all of the patches and/or
combinations. I'm not really at home with the xen code; and my single
xen server is production, so i can really only test in weekends...
AL13N
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: pci passthrough issue introduced between 4.14.1 and 4.15.0
2021-06-03 16:01 ` AL13N
@ 2021-06-04 6:56 ` Jan Beulich
2021-06-04 7:10 ` Juergen Gross
0 siblings, 1 reply; 15+ messages in thread
From: Jan Beulich @ 2021-06-04 6:56 UTC (permalink / raw)
To: AL13N; +Cc: Xen-devel
On 03.06.2021 18:01, AL13N wrote:
> Jan Beulich schreef op 2021-06-01 16:53:
>> On 01.06.2021 16:44, AL13N wrote:
>>> This mailing list is the correct place for the toolstack too? right?
>>
>> Yes.
>
> So, what's the plan to fix this? is the plan to fix the toolstack? or
> put your patch in kernel to kinda workaround it?
The patch has already been put in the kernel, as said. It would be good
to know whether it actually has helped your case as well.
> Is there a way to make a regression test or unit test or something?
Would be nice, but may be a little difficult to arrange for in, say,
osstest.
> Does anyone have an idea on what patch would cause the regression?
Not me, but I'm also not a tool stack maintainer (nor expert in any
way).
Jan
> that
> patch that I pointed out, could it be that one, or should i look at a
> specific file/line? I can't really just test all of the patches and/or
> combinations. I'm not really at home with the xen code; and my single
> xen server is production, so i can really only test in weekends...
>
> AL13N
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: pci passthrough issue introduced between 4.14.1 and 4.15.0
2021-06-04 6:56 ` Jan Beulich
@ 2021-06-04 7:10 ` Juergen Gross
2021-06-04 8:37 ` AL13N
0 siblings, 1 reply; 15+ messages in thread
From: Juergen Gross @ 2021-06-04 7:10 UTC (permalink / raw)
To: Jan Beulich, AL13N; +Cc: Xen-devel, Durrant, Paul
[-- Attachment #1.1.1: Type: text/plain, Size: 1028 bytes --]
On 04.06.21 08:56, Jan Beulich wrote:
> On 03.06.2021 18:01, AL13N wrote:
>> Jan Beulich schreef op 2021-06-01 16:53:
>>> On 01.06.2021 16:44, AL13N wrote:
>>>> This mailing list is the correct place for the toolstack too? right?
>>>
>>> Yes.
>>
>> So, what's the plan to fix this? is the plan to fix the toolstack? or
>> put your patch in kernel to kinda workaround it?
>
> The patch has already been put in the kernel, as said. It would be good
> to know whether it actually has helped your case as well.
>
>> Is there a way to make a regression test or unit test or something?
>
> Would be nice, but may be a little difficult to arrange for in, say,
> osstest.
>
>> Does anyone have an idea on what patch would cause the regression?
>
> Not me, but I'm also not a tool stack maintainer (nor expert in any
> way).
There has been a large series by Paul Durrant [1] making heavy
modifications in this area.
Juergen
[1]: https://lists.xen.org/archives/html/xen-devel/2020-11/msg01680.html
[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3135 bytes --]
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: pci passthrough issue introduced between 4.14.1 and 4.15.0
2021-06-04 7:10 ` Juergen Gross
@ 2021-06-04 8:37 ` AL13N
0 siblings, 0 replies; 15+ messages in thread
From: AL13N @ 2021-06-04 8:37 UTC (permalink / raw)
To: Xen-devel; +Cc: Jan Beulich, Durrant, Paul
Juergen Gross schreef op 2021-06-04 09:10:
> On 04.06.21 08:56, Jan Beulich wrote:
>> On 03.06.2021 18:01, AL13N wrote:
>>> Jan Beulich schreef op 2021-06-01 16:53:
>>>> On 01.06.2021 16:44, AL13N wrote:
>>>>> This mailing list is the correct place for the toolstack too?
>>>>> right?
>>>>
>>>> Yes.
>>>
>>> So, what's the plan to fix this? is the plan to fix the toolstack? or
>>> put your patch in kernel to kinda workaround it?
>>
>> The patch has already been put in the kernel, as said. It would be
>> good
>> to know whether it actually has helped your case as well.
>>
>>> Is there a way to make a regression test or unit test or something?
>>
>> Would be nice, but may be a little difficult to arrange for in, say,
>> osstest.
>>
>>> Does anyone have an idea on what patch would cause the regression?
>>
>> Not me, but I'm also not a tool stack maintainer (nor expert in any
>> way).
>
> There has been a large series by Paul Durrant [1] making heavy
> modifications in this area.
>
> Juergen
>
> [1]:
> https://lists.xen.org/archives/html/xen-devel/2020-11/msg01680.html
Hmm after a quick look-through, nothing stands out to me; except maybe
if the pci list gets freed after the first add, it would prevent the
adding of the other pci devices.
Could anyone explain/point me to the place where the toolstack adds pci
devices from the xl create vs xl pci-add?
I'm circling back to the logs of xl create having 3 log entries "Adding
new pci device to xenstore", but only one "Creating pci backend". While
that is normal of course, it does give out 2 possibilities i can see for
only having 1 device:
I'm looking at this function:
https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=tools/libs/light/libxl_pci.c;h=1a1c2630803b5b3b4e07f093b688e0fd5780e745;hb=e25aa9939ae0cd8317605be3d5c5611b76bc4ab4#l134
.
possibility 1:
xs transaction at line 209 does not get called, which i presume would
add the device to xs.
possibility 2:
xs transaction does get called, but by that time, the other end already
has finished and thus does not look at the other devices in xs?
since xl pci-list actually does show all 3, and i see no errors, i would
presume that for possibility 1, it can only really be line 201, but
since this is a xl create, i'm assuming "starting" is true in this case,
which means no lock and that line does not get called? (there is this
weird thing where a transaction is committed and then aborted though?).
so i guess possibility 1 is no go?
but possibility 2 would mean that unless there's another layer, the
pcifront on the domU side would be faster than this function being
called 3 times... which seems odd (unless this all gets done before domU
is even started, which does not seem likely)
Of course this is all an amateur pov, i have no expertise with any of
the xen code at all...
Well, I hope someone can take a look at this and/or help me out, please.
Maarten
^ permalink raw reply [flat|nested] 15+ messages in thread
* pci passthrough issue introduced between 4.14.1 and 4.15.0
@ 2021-06-01 7:34 AL13N
2021-06-01 10:03 ` George Dunlap
0 siblings, 1 reply; 15+ messages in thread
From: AL13N @ 2021-06-01 7:34 UTC (permalink / raw)
To: xen-devel
Not 100% it's a bug or something i did wrong, but,
with xl create i start a PV with 3 pci passthroughs
after wards, xl pci-list shows all 3 nicely
looking at the domU boot logs, pcifront is only creating one pci device
and lspci in the guest shows only 1 pci entry
in at least 4.14.1 it still works.
looking deeper, if you start with only 1 entry or 0 and you do
pci-attach for the other pci entries, all of this works fine.
this is the pci section in question:
pci=['0000:04:00.0,permissive=1',
'0000:00:1a.0,permissive=1,rdm_policy=relaxed,seize=1',
'0000:00:1d.0,permissive=1,rdm_policy=relaxed,seize=1']
this works fine in 4.12.1 and 4.14.1, but fails in 4.15.0, however
booting with pci=['0000:04:00.0,permissive=1'] and then doing
[]# xl pci-attach <domain>
'0000:00:1a.0,permissive=1,rdm_policy=relaxed,seize=1'
[]# xl pci-attach <domain>
'0000:00:1d.0,permissive=1,rdm_policy=relaxed,seize=1'
also works,
Regards,
Maarten (alien on Libera and OFTC)
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: pci passthrough issue introduced between 4.14.1 and 4.15.0
2021-06-01 7:34 AL13N
@ 2021-06-01 10:03 ` George Dunlap
0 siblings, 0 replies; 15+ messages in thread
From: George Dunlap @ 2021-06-01 10:03 UTC (permalink / raw)
To: AL13N, Ian Jackson, Anthony Perard; +Cc: xen-devel
[-- Attachment #1: Type: text/plain, Size: 1087 bytes --]
On Tue, Jun 1, 2021 at 8:34 AM AL13N <alien@rmail.be> wrote:
> Not 100% it's a bug or something i did wrong, but,
>
> with xl create i start a PV with 3 pci passthroughs
>
> after wards, xl pci-list shows all 3 nicely
>
> looking at the domU boot logs, pcifront is only creating one pci device
> and lspci in the guest shows only 1 pci entry
>
> in at least 4.14.1 it still works.
>
> looking deeper, if you start with only 1 entry or 0 and you do
> pci-attach for the other pci entries, all of this works fine.
>
> this is the pci section in question:
>
> pci=['0000:04:00.0,permissive=1',
> '0000:00:1a.0,permissive=1,rdm_policy=relaxed,seize=1',
> '0000:00:1d.0,permissive=1,rdm_policy=relaxed,seize=1']
>
> this works fine in 4.12.1 and 4.14.1, but fails in 4.15.0, however
>
> booting with pci=['0000:04:00.0,permissive=1'] and then doing
>
> []# xl pci-attach <domain>
> '0000:00:1a.0,permissive=1,rdm_policy=relaxed,seize=1'
> []# xl pci-attach <domain>
> '0000:00:1d.0,permissive=1,rdm_policy=relaxed,seize=1'
>
> also works,
>
Sounds like it might be a tools issue?
-George
[-- Attachment #2: Type: text/html, Size: 1621 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2021-06-04 8:37 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-01 7:36 pci passthrough issue introduced between 4.14.1 and 4.15.0 AL13N
2021-06-01 10:08 ` Jan Beulich
2021-06-01 14:06 ` AL13N
2021-06-01 14:28 ` AL13N
2021-06-01 14:33 ` Jan Beulich
2021-06-01 14:44 ` AL13N
2021-06-01 14:48 ` AL13N
2021-06-01 15:50 ` AL13N
2021-06-01 14:53 ` Jan Beulich
2021-06-03 16:01 ` AL13N
2021-06-04 6:56 ` Jan Beulich
2021-06-04 7:10 ` Juergen Gross
2021-06-04 8:37 ` AL13N
-- strict thread matches above, loose matches on Subject: below --
2021-06-01 7:34 AL13N
2021-06-01 10:03 ` George Dunlap
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).