xl fails to work with some command

xl fails to work with some command

[Zhang, Yang Z]

Hi
As we will use xl in xen 4.1. I had try use xl to do something. And I find some command is fails.
For config file, when give "disk=" too args, it will show segment fault if we use xl create to start guest. Is there any change for xl's config file?

Following command can not work:
xl migrate:  when I use xl migrate domain localhost to do migrate, it shows error:
                        Saving to migration stream new xl format (info 0x0/0x0/15122)
						libxl: warning: libxl_dom.c:374:core_suspend Failed to get the suspend evtchn port
						Loading new save file incoming migration stream (new xl fmt info 0x0/0x0/15122)
						 Savefile contains xl domain config
						xc: Saving memory: iter 0 (last sent 0 skipped 0): 98334/1048576    9%xc: error: Failed to allocate memory for batch.!: Internal error
						libxl: error: libxl_device.c:438:libxl_wait_for_device_model Device Model not ready
						xl: fatal error: xl_cmdimpl.c:1466, rc=-1: libxl_confirm_device_model_startup(&ctx, dm_starting)
						xc: error: Error when writing to state file (4c) (errno 32) (32 = Broken pipe): Internal error
						libxl: error: libxl_utils.c:328:libxl_write_exactly failed to write qemu signature from saved-state file: Broken pipe
						migration sender: libxl_domain_suspend failed (rc=32)
						libxl: info: libxl_exec.c:69:libxl_report_child_exitstatus migration target process [7098] exited with error status 1
						Migration failed, resuming at sender.

xl pci-list-assign:  It cannot show the hidden device. With xm, it can work.
xl mem-set:  use xl mem-set to set guest's memory have no effect
xl trigger:  xl have no s3resume option and some other option cannot work
xl vnc:   The arg of dom_id for xl vnc cannot work. If there have two guest, when use xl vnc guest2 try to connect guest2, it always connect to guest1.
xl vcpu-set:  Both xm vcpu-set and xl vcpu-set can not work.

I used changeset 21951 for my test scenario.

best regards
yang

RE: xl fails to work with some command

[Zhang, Yang Z]

Hi all
I have do further investigation with xl last week. And here is the latest status with xl.
I found there are some bugs with xl:
1.	xl only can take one parameter with "disk=" in config file, or else it will show "segment fault" when run "xl create". 

2.	libxl don't have the memory balloon support, so it will fail when guest need to balloon memory from dom0. So our test case "XEN_4G_guest_64_g32e" will fail sometimes.

3.	xenu cannot work with xl. When boot xenu, guest will wait at XENBUS: waiting for devices to initializes. And all the xenu cases fail

4.	SAVE/RESTORE with xl cannot work. After restore, guest's network will fail. And all the S/R and live migration cases fail.

5.	All windows UP guest cannot start. It always will crash. And all the windows UP cases fail.

6.  vcpu-set is not supported by xl. So some RAS case which use xl vcpu-set to set guest's cpu set will fails. case "vcpu_hot_add_64_g32e" fails.

All the data collection under changeset 21903 on the stoakley.

Following it the xvs report by xl:
               Summary Test Report of Last Session
=====================================================================
                            Total   Pass    Fail    NoResult   Crash
=====================================================================
ras_ept_vpid                  3       2       1         0        0
control_panel_ept_vpid         17      8       9         0        0
gtest_ept_vpid                18      13      5         0        0
=====================================================================
ras_ept_vpid                  3       2       1         0        0
 :vcpu_hot_remove_64_g32e    1       1       0         0        0
 :cpu_online_offline_64_g       1       1       0         0        0
 :vcpu_hot_add_64_g32e       1       0       1         0        0
control_panel_ept_vpid         17      8       9         0        0
 :XEN_1500M_guest_64_g32e   1       1       0         0        0
 :XEN_4G_guest_64_g32e      1       1       1         0        0
 :XEN_256M_xenu_64_gPAE    1       0       1         0        0
 :XEN_four_vmx_xenu_seq_6   1       0       1         0        0
 :XEN_vmx_vcpu_pin_64_g32   1       1       0         0        0
 :XEN_linux_win_64_g32e      1       1       1         0        0
 :XEN_SR_Continuity_64_g3    1       0       1         0        0
 :XEN_vmx_2vcpu_64_g32e    1       1       0         0        0
 :XEN_1500M_guest_64_gPAE  1       1       0         0        0
 :XEN_four_dguest_co_64_g   1       1       1         0        0
 :XEN_4G_guest_64_gPAE     1       1       1         0        0
 :XEN_two_winxp_64_g32e    1       0       1         0        0
 :XEN_four_sguest_seq_64_   1       1       0         0        0
 :XEN_256M_guest_64_gPAE  1       1       0         0        0
 :XEN_256M_guest_64_g32e  1       1       0         0        0
 :XEN_SR_SMP_64_g32e      1       0       1         0        0
 :XEN_four_sguest_seq_64_   1       1       0         0        0
gtest_ept_vpid              18      13      5         0        0
 :reboot_xp_64_g32e        1       0       1         0        0
 :boot_up_vista_64_g32e     1       0       1         0        0
 :boot_up_acpi_xp_64_g32e   1       0       1         0        0
 :boot_smp_win7_ent_64_g3  1       1       0         0        0
 :boot_smp_acpi_xp_64_g32  1       1       0         0        0
 :boot_up_acpi_64_g32e     1       1       0         0        0
 :boot_base_kernel_64_g32   1       1       0         0        0
 :boot_up_win2008_64_g32e  1       0       1         0        0
 :kb_nightly_64_g32e        1       1       0         0        0
 :boot_up_acpi_win2k3_64_  1       0       1         0        0
 :boot_smp_vista_64_g32e   1       1       0         0        0
 :ltp_nightly_64_g32e       1       1       0         0        0
 :boot_fc9_64_g32e         1       1       0         0        0
 :boot_smp_win2008_64_g32 1       1       0         0        0
 :boot_smp_win7_ent_debug 1       1       0         0        0
 :boot_smp_acpi_win2k3_64  1       1       0         0        0
 :boot_rhel5u1_64_g32e     1       1       0         0        0
 :reboot_fc6_64_g32e       1       1       0         0        0
=====================================================================
Total                       38      23      15        0        0

best regards
yang


> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com
> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Zhang, Yang Z
> Sent: Thursday, August 19, 2010 1:51 PM
> To: Ian Campbell; xen-devel@lists.xensource.com
> Subject: [Xen-devel] xl fails to work with some command
> 
> Hi
> As we will use xl in xen 4.1. I had try use xl to do something. And I find some
> command is fails.
> For config file, when give "disk=" too args, it will show segment fault if we use
> xl create to start guest. Is there any change for xl's config file?
> 
> Following command can not work:
> xl migrate:  when I use xl migrate domain localhost to do migrate, it shows
> error:
>                         Saving to migration stream new xl format (info
> 0x0/0x0/15122)
> 						libxl: warning: libxl_dom.c:374:core_suspend
> Failed to get the suspend evtchn port
> 						Loading new save file incoming migration stream
> (new xl fmt info 0x0/0x0/15122)
> 						 Savefile contains xl domain config
> 						xc: Saving memory: iter 0 (last sent 0 skipped 0):
> 98334/1048576    9%xc: error: Failed to allocate memory for batch.!:
> Internal error
> 						libxl: error:
> libxl_device.c:438:libxl_wait_for_device_model Device Model not ready
> 						xl: fatal error: xl_cmdimpl.c:1466, rc=-1:
> libxl_confirm_device_model_startup(&ctx, dm_starting)
> 						xc: error: Error when writing to state file (4c)
> (errno 32) (32 = Broken pipe): Internal error
> 						libxl: error: libxl_utils.c:328:libxl_write_exactly
> failed to write qemu signature from saved-state file: Broken pipe
> 						migration sender: libxl_domain_suspend failed
> (rc=32)
> 						libxl: info:
> libxl_exec.c:69:libxl_report_child_exitstatus migration target process [7098]
> exited with error status 1
> 						Migration failed, resuming at sender.
> 
> xl pci-list-assign:  It cannot show the hidden device. With xm, it can work.
> xl mem-set:  use xl mem-set to set guest's memory have no effect
> xl trigger:  xl have no s3resume option and some other option cannot work
> xl vnc:   The arg of dom_id for xl vnc cannot work. If there have two guest,
> when use xl vnc guest2 try to connect guest2, it always connect to guest1.
> xl vcpu-set:  Both xm vcpu-set and xl vcpu-set can not work.
> 
> I used changeset 21951 for my test scenario.
> 
> best regards
> yang
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

RE: xl fails to work with some command

[Stefano Stabellini]

On Mon, 23 Aug 2010, Zhang, Yang Z wrote:
> Hi all
> I have do further investigation with xl last week. And here is the latest status with xl.
> I found there are some bugs with xl:
> 1.	xl only can take one parameter with "disk=" in config file, or else it will show "segment fault" when run "xl create". 
this is a known issue and people are working on it, a patch has already
been submitted


> 
> 2.	libxl don't have the memory balloon support, so it will fail when guest need to balloon memory from dom0. So our test case "XEN_4G_guest_64_g32e" will fail sometimes.

this is a known limitation, currently is recommended to add
dom0_mem=752M to the xen command line options


> 
> 3.	xenu cannot work with xl. When boot xenu, guest will wait at XENBUS: waiting for devices to initializes. And all the xenu cases fail
> 

I guess xenu is a linux pv guest in this context.
I test this use case faily often so I am surprised you found bugs;
it should work AFAIK.


> 4.	SAVE/RESTORE with xl cannot work. After restore, guest's network will fail. And all the S/R and live migration cases fail.
>

this might actually be a bug that has been introduced recently


> 5.	All windows UP guest cannot start. It always will crash. And all the windows UP cases fail.
> 

I don't know about Windows gusts but I test Linux PV on HVM guests
every day and they work fine.


> 6.  vcpu-set is not supported by xl. So some RAS case which use xl vcpu-set to set guest's cpu set will fails. case "vcpu_hot_add_64_g32e" fails.
> 

actually vcpu-set is supported by xl, have you found any bugs using it?

RE: xl fails to work with some command

[Zhang, Yang Z]

> -----Original Message-----
> From: Stefano Stabellini [mailto:stefano.stabellini@eu.citrix.com]
> Sent: Monday, August 23, 2010 6:13 PM
> To: Zhang, Yang Z
> Cc: Ian Campbell; xen-devel@lists.xensource.com; Stefano Stabellini
> Subject: RE: xl fails to work with some command
> 
> On Mon, 23 Aug 2010, Zhang, Yang Z wrote:
> > 3.	xenu cannot work with xl. When boot xenu, guest will wait at XENBUS:
> waiting for devices to initializes. And all the xenu cases fail
> >
> 
> I guess xenu is a linux pv guest in this context.
> I test this use case faily often so I am surprised you found bugs;
> it should work AFAIK.
I have double check this case with xl and xm. For xm, it work well. But for xl, it still show XENBUS:
waiting for devices to initializes.

> > 5.	All windows UP guest cannot start. It always will crash. And all the
> windows UP cases fail.
> >
> 
> I don't know about Windows gusts but I test Linux PV on HVM guests
> every day and they work fine.
Maybe you can have a try with windows UP guest.

> 
> > 6.  vcpu-set is not supported by xl. So some RAS case which use xl vcpu-set
> to set guest's cpu set will fails. case "vcpu_hot_add_64_g32e" fails.
> >
> 
> actually vcpu-set is supported by xl, have you found any bugs using it?
For pv guest, it work well. But for HVM guest, xl miss some function about cpu hotplug which was added recently.

xl seems only support FLR and didn't support Second Bus Reset and D0/D3. Will you add them in xl?


best regards
yang

Re: RE: xl fails to work with some command

[Gianni Tedesco]

On Mon, 2010-08-23 at 14:32 +0100, Zhang, Yang Z wrote:
> > -----Original Message-----
> > From: Stefano Stabellini [mailto:stefano.stabellini@eu.citrix.com]
> > Sent: Monday, August 23, 2010 6:13 PM
> > To: Zhang, Yang Z
> > Cc: Ian Campbell; xen-devel@lists.xensource.com; Stefano Stabellini
> > Subject: RE: xl fails to work with some command
> > 
> > On Mon, 23 Aug 2010, Zhang, Yang Z wrote:
> > > 3.	xenu cannot work with xl. When boot xenu, guest will wait at XENBUS:
> > waiting for devices to initializes. And all the xenu cases fail
> > >
> > 
> > I guess xenu is a linux pv guest in this context.
> > I test this use case faily often so I am surprised you found bugs;
> > it should work AFAIK.
> I have double check this case with xl and xm. For xm, it work well. But for xl, it still show XENBUS:
> waiting for devices to initializes.
> 
> > > 5.	All windows UP guest cannot start. It always will crash. And all the
> > windows UP cases fail.
> > >
> > 
> > I don't know about Windows gusts but I test Linux PV on HVM guests
> > every day and they work fine.
> Maybe you can have a try with windows UP guest.
> 
> > 
> > > 6.  vcpu-set is not supported by xl. So some RAS case which use xl vcpu-set
> > to set guest's cpu set will fails. case "vcpu_hot_add_64_g32e" fails.
> > >
> > 
> > actually vcpu-set is supported by xl, have you found any bugs using
> it?
> For pv guest, it work well. But for HVM guest, xl miss some function
> about cpu hotplug which was added recently.
> 
> xl seems only support FLR and didn't support Second Bus Reset and
> D0/D3. Will you add them in xl?

It's on my list of things to do but a lot of other work needs doing with
the PCI code first.

RE: RE: xl fails to work with some command

[Zhang, Yang Z]

[-- Attachment #1: Type: text/plain, Size: 624 bytes --]

> -----Original Message-----
> From: Gianni Tedesco [mailto:gianni.tedesco@citrix.com]
> Sent: Tuesday, August 24, 2010 9:21 PM
> To: Zhang, Yang Z
> Cc: Stefano Stabellini; Ian Campbell; xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] RE: xl fails to work with some command
> 
> > xl seems only support FLR and didn't support Second Bus Reset and
> > D0/D3. Will you add them in xl?
> 
> It's on my list of things to do but a lot of other work needs doing with
> the PCI code first.

Weidong, any comments?

And for vt-d feature, I think FLR check is an important component. 

best regards
yang

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: RE: xl fails to work with some command

[Weidong Han]

Zhang, Yang Z wrote:
>> -----Original Message-----
>> From: Gianni Tedesco [mailto:gianni.tedesco@citrix.com]
>> Sent: Tuesday, August 24, 2010 9:21 PM
>> To: Zhang, Yang Z
>> Cc: Stefano Stabellini; Ian Campbell; xen-devel@lists.xensource.com
>> Subject: Re: [Xen-devel] RE: xl fails to work with some command
>>
>>     
>>> xl seems only support FLR and didn't support Second Bus Reset and
>>> D0/D3. Will you add them in xl?
>>>       
>> It's on my list of things to do but a lot of other work needs doing with
>> the PCI code first.
>>     
>
> Weidong, any comments?
>
> And for vt-d feature, I think FLR check is an important component. 
>
>   

Yes, FLR is required for device assignment. I'm ok if it's already in 
Tedesco's todo list.

Regards,
Weidong

RE: xl fails to work with some command

[Stefano Stabellini]

On Mon, 23 Aug 2010, Zhang, Yang Z wrote:
> > 
> > > 6.  vcpu-set is not supported by xl. So some RAS case which use xl vcpu-set
> > to set guest's cpu set will fails. case "vcpu_hot_add_64_g32e" fails.
> > >
> > 
> > actually vcpu-set is supported by xl, have you found any bugs using it?
> For pv guest, it work well. But for HVM guest, xl miss some function about cpu hotplug which was added recently.
> 
> xl seems only support FLR and didn't support Second Bus Reset and D0/D3. Will you add them in xl?
 
xl uses the sysfs "reset" node provided by the pci subsystem of the
Linux kernel, that should support Secondary Bus Reset and D0/D3 already.

Re: RE: xl fails to work with some command

[Weidong Han]

Stefano Stabellini wrote:
> On Mon, 23 Aug 2010, Zhang, Yang Z wrote:
>   
>>>> 6.  vcpu-set is not supported by xl. So some RAS case which use xl vcpu-set
>>>>         
>>> to set guest's cpu set will fails. case "vcpu_hot_add_64_g32e" fails.
>>>       
>>> actually vcpu-set is supported by xl, have you found any bugs using it?
>>>       
>> For pv guest, it work well. But for HVM guest, xl miss some function about cpu hotplug which was added recently.
>>
>> xl seems only support FLR and didn't support Second Bus Reset and D0/D3. Will you add them in xl?
>>     
>  
> xl uses the sysfs "reset" node provided by the pci subsystem of the
> Linux kernel, that should support Secondary Bus Reset and D0/D3 already.
>   
Do you mean xl uses Linux kernel PCI API (i.e. pci_reset_function) to 
reset PCI device?

Regards,
Weidong
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>   

Re: RE: xl fails to work with some command

[Stefano Stabellini]

On Thu, 26 Aug 2010, Weidong Han wrote:
> Stefano Stabellini wrote:
> > On Mon, 23 Aug 2010, Zhang, Yang Z wrote:
> >   
> >>>> 6.  vcpu-set is not supported by xl. So some RAS case which use xl vcpu-set
> >>>>         
> >>> to set guest's cpu set will fails. case "vcpu_hot_add_64_g32e" fails.
> >>>       
> >>> actually vcpu-set is supported by xl, have you found any bugs using it?
> >>>       
> >> For pv guest, it work well. But for HVM guest, xl miss some function about cpu hotplug which was added recently.
> >>
> >> xl seems only support FLR and didn't support Second Bus Reset and D0/D3. Will you add them in xl?
> >>     
> >  
> > xl uses the sysfs "reset" node provided by the pci subsystem of the
> > Linux kernel, that should support Secondary Bus Reset and D0/D3 already.
> >   
> Do you mean xl uses Linux kernel PCI API (i.e. pci_reset_function) to 
> reset PCI device?

Yes, give a look at libxl_device_pci_reset: first libxl tries to use the
pciback sysfn node "do_flr", but if it is not present it will use the
standard linux sysfs "reset" node.
Please note that on pvops kernels the do_flr node is not present (but it
is present on both XCP and XCI kernels) while the reset node points to
pci_reset_function.

Re: RE: xl fails to work with some command

[Konrad Rzeszutek Wilk]

On Thu, Aug 26, 2010 at 01:17:08PM +0100, Stefano Stabellini wrote:
> On Thu, 26 Aug 2010, Weidong Han wrote:
> > Stefano Stabellini wrote:
> > > On Mon, 23 Aug 2010, Zhang, Yang Z wrote:
> > >   
> > >>>> 6.  vcpu-set is not supported by xl. So some RAS case which use xl vcpu-set
> > >>>>         
> > >>> to set guest's cpu set will fails. case "vcpu_hot_add_64_g32e" fails.
> > >>>       
> > >>> actually vcpu-set is supported by xl, have you found any bugs using it?
> > >>>       
> > >> For pv guest, it work well. But for HVM guest, xl miss some function about cpu hotplug which was added recently.
> > >>
> > >> xl seems only support FLR and didn't support Second Bus Reset and D0/D3. Will you add them in xl?
> > >>     
> > >  
> > > xl uses the sysfs "reset" node provided by the pci subsystem of the
> > > Linux kernel, that should support Secondary Bus Reset and D0/D3 already.
> > >   
> > Do you mean xl uses Linux kernel PCI API (i.e. pci_reset_function) to 
> > reset PCI device?
> 
> Yes, give a look at libxl_device_pci_reset: first libxl tries to use the
> pciback sysfn node "do_flr", but if it is not present it will use the
> standard linux sysfs "reset" node.
> Please note that on pvops kernels the do_flr node is not present (but it
> is present on both XCP and XCI kernels) while the reset node points to
> pci_reset_function.

<hint>I am all up for receiving the patches from XCP/XCI for this</hint>

Re: RE: xl fails to work with some command

[Stefano Stabellini]

On Thu, 26 Aug 2010, Konrad Rzeszutek Wilk wrote:
> On Thu, Aug 26, 2010 at 01:17:08PM +0100, Stefano Stabellini wrote:
> > On Thu, 26 Aug 2010, Weidong Han wrote:
> > > Stefano Stabellini wrote:
> > > > On Mon, 23 Aug 2010, Zhang, Yang Z wrote:
> > > >   
> > > >>>> 6.  vcpu-set is not supported by xl. So some RAS case which use xl vcpu-set
> > > >>>>         
> > > >>> to set guest's cpu set will fails. case "vcpu_hot_add_64_g32e" fails.
> > > >>>       
> > > >>> actually vcpu-set is supported by xl, have you found any bugs using it?
> > > >>>       
> > > >> For pv guest, it work well. But for HVM guest, xl miss some function about cpu hotplug which was added recently.
> > > >>
> > > >> xl seems only support FLR and didn't support Second Bus Reset and D0/D3. Will you add them in xl?
> > > >>     
> > > >  
> > > > xl uses the sysfs "reset" node provided by the pci subsystem of the
> > > > Linux kernel, that should support Secondary Bus Reset and D0/D3 already.
> > > >   
> > > Do you mean xl uses Linux kernel PCI API (i.e. pci_reset_function) to 
> > > reset PCI device?
> > 
> > Yes, give a look at libxl_device_pci_reset: first libxl tries to use the
> > pciback sysfn node "do_flr", but if it is not present it will use the
> > standard linux sysfs "reset" node.
> > Please note that on pvops kernels the do_flr node is not present (but it
> > is present on both XCP and XCI kernels) while the reset node points to
> > pci_reset_function.
> 
> <hint>I am all up for receiving the patches from XCP/XCI for this</hint>
> 

I think that using the standard linux reset mechanism is a better way to
handle this as oppose to have our own do_flr method.

Re: RE: xl fails to work with some command

[Weidong Han]

Stefano Stabellini wrote:
> On Thu, 26 Aug 2010, Konrad Rzeszutek Wilk wrote:
>   
>> On Thu, Aug 26, 2010 at 01:17:08PM +0100, Stefano Stabellini wrote:
>>     
>>> On Thu, 26 Aug 2010, Weidong Han wrote:
>>>       
>>>> Stefano Stabellini wrote:
>>>>         
>>>>> On Mon, 23 Aug 2010, Zhang, Yang Z wrote:
>>>>>   
>>>>>           
>>>>>>>> 6.  vcpu-set is not supported by xl. So some RAS case which use xl vcpu-set
>>>>>>>>         
>>>>>>>>                 
>>>>>>> to set guest's cpu set will fails. case "vcpu_hot_add_64_g32e" fails.
>>>>>>>       
>>>>>>> actually vcpu-set is supported by xl, have you found any bugs using it?
>>>>>>>       
>>>>>>>               
>>>>>> For pv guest, it work well. But for HVM guest, xl miss some function about cpu hotplug which was added recently.
>>>>>>
>>>>>> xl seems only support FLR and didn't support Second Bus Reset and D0/D3. Will you add them in xl?
>>>>>>     
>>>>>>             
>>>>>  
>>>>> xl uses the sysfs "reset" node provided by the pci subsystem of the
>>>>> Linux kernel, that should support Secondary Bus Reset and D0/D3 already.
>>>>>   
>>>>>           
>>>> Do you mean xl uses Linux kernel PCI API (i.e. pci_reset_function) to 
>>>> reset PCI device?
>>>>         
>>> Yes, give a look at libxl_device_pci_reset: first libxl tries to use the
>>> pciback sysfn node "do_flr", but if it is not present it will use the
>>> standard linux sysfs "reset" node.
>>> Please note that on pvops kernels the do_flr node is not present (but it
>>> is present on both XCP and XCI kernels) while the reset node points to
>>> pci_reset_function.
>>>       
>> <hint>I am all up for receiving the patches from XCP/XCI for this</hint>
>>
>>     
>
> I think that using the standard linux reset mechanism is a better way to
> handle this as oppose to have our own do_flr method.
>   
Pls note that there are some specific FLR functions for graphics 
pass-through. They should be ported to xl.

Regards,
Weidong

Re: RE: xl fails to work with some command

[Stefano Stabellini]

On Fri, 27 Aug 2010, Weidong Han wrote:
> >>> Yes, give a look at libxl_device_pci_reset: first libxl tries to use the
> >>> pciback sysfn node "do_flr", but if it is not present it will use the
> >>> standard linux sysfs "reset" node.
> >>> Please note that on pvops kernels the do_flr node is not present (but it
> >>> is present on both XCP and XCI kernels) while the reset node points to
> >>> pci_reset_function.
> >>>       
> >> <hint>I am all up for receiving the patches from XCP/XCI for this</hint>
> >>
> >>     
> >
> > I think that using the standard linux reset mechanism is a better way to
> > handle this as oppose to have our own do_flr method.
> >   
> Pls note that there are some specific FLR functions for graphics 
> pass-through. They should be ported to xl.
> 
 
As stated above, xl doesn't know how to FLR a device, the dom0 kernel
provides a way to do it.
XCP kernels have pciback do_flr that has all the HW specific FLR
functions you are referring to AFAIK, but the pvops kernel, that uses
the standard reset node, does not.

Re: RE: xl fails to work with some command

[Weidong Han]

Stefano Stabellini wrote:
> On Fri, 27 Aug 2010, Weidong Han wrote:
>   
>>>>> Yes, give a look at libxl_device_pci_reset: first libxl tries to use the
>>>>> pciback sysfn node "do_flr", but if it is not present it will use the
>>>>> standard linux sysfs "reset" node.
>>>>> Please note that on pvops kernels the do_flr node is not present (but it
>>>>> is present on both XCP and XCI kernels) while the reset node points to
>>>>> pci_reset_function.
>>>>>       
>>>>>           
>>>> <hint>I am all up for receiving the patches from XCP/XCI for this</hint>
>>>>
>>>>     
>>>>         
>>> I think that using the standard linux reset mechanism is a better way to
>>> handle this as oppose to have our own do_flr method.
>>>   
>>>       
>> Pls note that there are some specific FLR functions for graphics 
>> pass-through. They should be ported to xl.
>>
>>     
>  
> As stated above, xl doesn't know how to FLR a device, the dom0 kernel
> provides a way to do it.
> XCP kernels have pciback do_flr that has all the HW specific FLR
> functions you are referring to AFAIK, but the pvops kernel, that uses
> the standard reset node, does not.
>
>   
So it should implement specific FLRs in pciback like XCP/XCI.

Regards,
Weidong

Re: RE: xl fails to work with some command

[Stefano Stabellini]

On Tue, 31 Aug 2010, Weidong Han wrote:
> > As stated above, xl doesn't know how to FLR a device, the dom0 kernel
> > provides a way to do it.
> > XCP kernels have pciback do_flr that has all the HW specific FLR
> > functions you are referring to AFAIK, but the pvops kernel, that uses
> > the standard reset node, does not.
> >
> >   
> So it should implement specific FLRs in pciback like XCP/XCI.
> 
 
Why should we reimplement pci FLR in pciback when the kernel has
already a way to do it?
I am not seeing pciback FLR going upstream easily.
Besides if some devices need specific FLRs, shouldn't the generic kernel
FLR function know about it?

Re: RE: xl fails to work with some command

[Ian Campbell]

On Tue, 2010-08-31 at 10:40 +0100, Stefano Stabellini wrote:
> On Tue, 31 Aug 2010, Weidong Han wrote:
> > > As stated above, xl doesn't know how to FLR a device, the dom0 kernel
> > > provides a way to do it.
> > > XCP kernels have pciback do_flr that has all the HW specific FLR
> > > functions you are referring to AFAIK, but the pvops kernel, that uses
> > > the standard reset node, does not.
> > >
> > >   
> > So it should implement specific FLRs in pciback like XCP/XCI.
> > 
>  
> Why should we reimplement pci FLR in pciback when the kernel has
> already a way to do it?
> I am not seeing pciback FLR going upstream easily.
> Besides if some devices need specific FLRs, shouldn't the generic kernel
> FLR function know about it?

Yes, absolutely, the correct way to fix this is to add the necessary
fixes/quirks to the upstream FLR infrastructure so that everybody
benefits, not to add stuff to pciback where a) only Xen benefits and b)
we have to maintain it forever because it cannot be upstreamed because
the correct fix is to fix it in the upstream LFR infrastructure ;-).

Ian.

Re: RE: xl fails to work with some command

[Weidong Han]

Ian Campbell wrote:
> On Tue, 2010-08-31 at 10:40 +0100, Stefano Stabellini wrote:
>   
>> On Tue, 31 Aug 2010, Weidong Han wrote:
>>     
>>>> As stated above, xl doesn't know how to FLR a device, the dom0 kernel
>>>> provides a way to do it.
>>>> XCP kernels have pciback do_flr that has all the HW specific FLR
>>>> functions you are referring to AFAIK, but the pvops kernel, that uses
>>>> the standard reset node, does not.
>>>>
>>>>   
>>>>         
>>> So it should implement specific FLRs in pciback like XCP/XCI.
>>>
>>>       
>>  
>> Why should we reimplement pci FLR in pciback when the kernel has
>> already a way to do it?
>> I am not seeing pciback FLR going upstream easily.
>> Besides if some devices need specific FLRs, shouldn't the generic kernel
>> FLR function know about it?
>>     
>
> Yes, absolutely, the correct way to fix this is to add the necessary
> fixes/quirks to the upstream FLR infrastructure so that everybody
> benefits, not to add stuff to pciback where a) only Xen benefits and b)
> we have to maintain it forever because it cannot be upstreamed because
> the correct fix is to fix it in the upstream LFR infrastructure ;-).
>
> Ian.
>
>
> I totally agree with you. It's the best to push the specific FLRs to Linux FLR infrastructure. I'm afraid it will need many efforts. But it's the right direction anyway.
Regards,
Weidong
>