[PATCH 0/2] passthrough: make management of PCI D-states by guest optional

[PATCH 0/2] passthrough: make management of PCI D-states by guest optional

[Kouya Shimura]

Hi,

Using D3hot state of PCI devices in xen is not mature yet.
I met domain destruction/creation troubles in some PCI cards since the
PCI config registers can't be read with D3hot state. xend.log is attached.

This patch set makes the management of PCI D-states by guest optional.

The default is "pci_power_mgmt=0" which disables the guest OS from
managing D-states because it would be better to avoid the trouble than
advantage of low power consumption.

Thanks,
Kouya

>From xend.log:

[2009-03-04 09:49:24 4356] ERROR (XendDomainInfo:2584) XendDomainInfo.destroy: domain destruction failed.
Traceback (most recent call last):
  File "/usr/lib64/python/xen/xend/XendDomainInfo.py", line 2578, in destroy
    do_FLR(self.domid)
  File "/usr/lib64/python/xen/xend/XendDomainInfo.py", line 314, in do_FLR
    raise VmError("pci: failed to locate device and "+
VmError: pci: failed to locate device and parse it's resources - ord() expected a character, but string of length 0 found
...[snip]...
[2009-03-04 10:31:24 4343] ERROR (XendDomainInfo:2445) XendDomainInfo.initDomain: exception occurred
Traceback (most recent call last):
  File "/usr/lib64/python/xen/xend/XendDomainInfo.py", line 2437, in _initDomain
    self._createDevices()
  File "/usr/lib64/python/xen/xend/XendDomainInfo.py", line 2049, in _createDevices
    devid = self._createDevice(devclass, config)
  File "/usr/lib64/python/xen/xend/XendDomainInfo.py", line 2016, in _createDevice
    return self.getDeviceController(deviceClass).createDevice(devConfig)
  File "/usr/lib64/python/xen/xend/server/DevController.py", line 67, in createDevice
    self.setupDevice(config)
  File "/usr/lib64/python/xen/xend/server/pciif.py", line 399, in setupDevice
    raise VmError("pci: failed to locate device and "+
VmError: pci: failed to locate device and parse it's resources - ord() expected a character, but string of length 0 found

[PATCH 1/2] tool: make management of PCI D-states by guest optional

[Kouya Shimura]

[-- Attachment #1: message body text --]
[-- Type: text/plain, Size: 312 bytes --]

D3hot state in some PCI devices causes the failure of domain
creation/destruction.

The default is "pci_power_mgmt=0" which disables the guest OS from
managing D-states because it would be better to avoid the trouble than
advantage of low power consumption.

Signed-off-by: Kouya Shimura <kouya@jp.fujitsu.com>


[-- Attachment #2: pci_pm_option_tool.patch --]
[-- Type: text/x-patch, Size: 4725 bytes --]

diff -r cff29d694a89 tools/examples/xmexample.hvm
--- a/tools/examples/xmexample.hvm	Thu Mar 05 17:50:05 2009 +0000
+++ b/tools/examples/xmexample.hvm	Fri Mar 06 13:25:54 2009 +0900
@@ -308,6 +308,8 @@ serial='pty'
 #                   available options are:
 #                   - msitranslate=0|1
 #                      per-device overriden of pci_msitranslate, see below
+#                   - power_mgmt=0|1
+#                      per-device overriden of pci_power_mgmt, see below
 #
 #pci=[ '07:00.0', '07:00.1' ]
 
@@ -321,6 +323,13 @@ serial='pty'
 # turned off.
 # 
 #pci_msitranslate=1
+
+#   PCI Power Management:
+#
+#   If it's set, the guest OS will be able to program D0-D3hot states of the
+# PCI device for the purpose of low power consumption.
+# 
+#pci_power_mgmt=0
 
 #-----------------------------------------------------------------------------
 #   Configure PVSCSI devices:
diff -r cff29d694a89 tools/python/xen/xend/XendConfig.py
--- a/tools/python/xen/xend/XendConfig.py	Thu Mar 05 17:50:05 2009 +0000
+++ b/tools/python/xen/xend/XendConfig.py	Fri Mar 06 13:25:54 2009 +0900
@@ -169,6 +169,7 @@ XENAPI_PLATFORM_CFG_TYPES = {
     'hap': int,
     'xen_extended_power_mgmt': int,
     'pci_msitranslate': int,
+    'pci_power_mgmt': int,
 }
 
 # Xen API console 'other_config' keys.
diff -r cff29d694a89 tools/python/xen/xend/server/pciif.py
--- a/tools/python/xen/xend/server/pciif.py	Thu Mar 05 17:50:05 2009 +0000
+++ b/tools/python/xen/xend/server/pciif.py	Fri Mar 06 13:25:54 2009 +0900
@@ -97,6 +97,8 @@ class PciController(DevController):
         back['uuid'] = config.get('uuid','')
         if 'pci_msitranslate' in self.vm.info['platform']:
             back['msitranslate']=str(self.vm.info['platform']['pci_msitranslate'])
+        if 'pci_power_mgmt' in self.vm.info['platform']:
+            back['power_mgmt']=str(self.vm.info['platform']['pci_power_mgmt'])
 
         return (0, back, {})
 
diff -r cff29d694a89 tools/python/xen/xm/create.py
--- a/tools/python/xen/xm/create.py	Thu Mar 05 17:50:05 2009 +0000
+++ b/tools/python/xen/xm/create.py	Fri Mar 06 13:25:54 2009 +0900
@@ -322,14 +322,16 @@ gopts.var('disk', val='phy:DEV,VDEV,MODE
           backend driver domain to use for the disk.
           The option may be repeated to add more than one disk.""")
 
-gopts.var('pci', val='BUS:DEV.FUNC[,msitranslate=0|1]',
+gopts.var('pci', val='BUS:DEV.FUNC[,msitranslate=0|1][,power_mgmt=0|1]',
           fn=append_value, default=[],
           use="""Add a PCI device to a domain, using given params (in hex).
           For example 'pci=c0:02.1'.
           If msitranslate is set, MSI-INTx translation is enabled if possible.
           Guest that doesn't support MSI will get IO-APIC type IRQs
           translated from physical MSI, HVM only. Default is 1.
-          The option may be repeated to add more than one pci device.""")
+          The option may be repeated to add more than one pci device.
+          If power_mgmt is set, the guest OS will be able to program the power
+          states D0-D3hot of the device, HVM only. Default=0.""")
 
 gopts.var('vscsi', val='PDEV,VDEV[,DOM]',
           fn=append_value, default=[],
@@ -605,6 +607,10 @@ gopts.var('pci_msitranslate', val='TRANS
           use="""Global PCI MSI-INTx translation flag (0=disable;
           1=enable.""")
 
+gopts.var('pci_power_mgmt', val='POWERMGMT',
+          fn=set_int, default=0,
+          use="""Global PCI Power Management flag (0=disable;1=enable).""")
+
 def err(msg):
     """Print an error to stderr and exit.
     """
@@ -691,7 +697,7 @@ def configure_pci(config_devs, vals):
         d = comma_sep_kv_to_dict(opts)
 
         def f(k):
-            if k not in ['msitranslate']:
+            if k not in ['msitranslate', 'power_mgmt']:
                 err('Invalid pci option: ' + k)
 
             config_pci_opts.append([k, d[k]])
@@ -913,7 +919,7 @@ def configure_hvm(config_image, vals):
              'acpi', 'apic', 'usb', 'usbdevice', 'keymap', 'pci', 'hpet',
              'guest_os_type', 'hap', 'opengl', 'cpuid', 'cpuid_check',
              'viridian', 'xen_extended_power_mgmt', 'pci_msitranslate',
-             'vpt_align' ]
+             'vpt_align', 'pci_power_mgmt' ]
 
     for a in args:
         if a in vals.__dict__ and vals.__dict__[a] is not None:
diff -r cff29d694a89 tools/python/xen/xm/xenapi_create.py
--- a/tools/python/xen/xm/xenapi_create.py	Thu Mar 05 17:50:05 2009 +0000
+++ b/tools/python/xen/xm/xenapi_create.py	Fri Mar 06 13:25:54 2009 +0900
@@ -1047,6 +1047,7 @@ class sxp2xml:
             'guest_os_type',
             'hap',
             'pci_msitranslate',
+            'pci_power_mgmt',
         ]
 
         platform_configs = []

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

[PATCH 2/2] ioemu: make management of PCI D-states by guest optional

[Kouya Shimura]

[-- Attachment #1: message body text --]
[-- Type: text/plain, Size: 331 bytes --]

passthrough: make management of PCI D-states by guest optional

Commit 8c771eb6294afc5b3754a9e3de51568d4e5986c2 enables the guest OS
to program D0-D3hot states of the assigned device, however,
D3hot state in some PCI devices causes the failure of domain 
creation/destruction.

Signed-off-by: Kouya Shimura <kouya@jp.fujitsu.com>


[-- Attachment #2: pci_pm_option_ioemu.patch --]
[-- Type: text/x-patch, Size: 7780 bytes --]

diff --git a/hw/pass-through.c b/hw/pass-through.c
index 4a86309..522eb73 100644
--- a/hw/pass-through.c
+++ b/hw/pass-through.c
@@ -105,6 +105,9 @@ static int pt_long_reg_read(struct pt_dev *ptdev,
 static int pt_bar_reg_read(struct pt_dev *ptdev,
     struct pt_reg_tbl *cfg_entry,
     uint32_t *value, uint32_t valid_mask);
+static int pt_pmcsr_reg_read(struct pt_dev *ptdev,
+    struct pt_reg_tbl *cfg_entry,
+    uint16_t *value, uint16_t valid_mask);
 static int pt_byte_reg_write(struct pt_dev *ptdev,
     struct pt_reg_tbl *cfg_entry,
     uint8_t *value, uint8_t dev_value, uint8_t valid_mask);
@@ -407,7 +410,7 @@ static struct pt_reg_info_tbl pt_emu_reg_pm_tbl[] = {
         .ro_mask    = 0xE1FC,
         .emu_mask   = 0x8100,
         .init       = pt_pmcsr_reg_init,
-        .u.w.read   = pt_word_reg_read,
+        .u.w.read   = pt_pmcsr_reg_read,
         .u.w.write  = pt_pmcsr_reg_write,
         .u.w.restore  = pt_pmcsr_reg_restore,
     },
@@ -2341,6 +2344,9 @@ static uint32_t pt_pmc_reg_init(struct pt_dev *ptdev,
 {
     PCIDevice *d = &ptdev->dev;
 
+    if (!ptdev->power_mgmt)
+        return reg->init_val;
+
     /* set Power Management Capabilities register */
     ptdev->pm_state->pmc_field = *(uint16_t *)(d->config + real_offset);
 
@@ -2354,6 +2360,9 @@ static uint32_t pt_pmcsr_reg_init(struct pt_dev *ptdev,
     PCIDevice *d = &ptdev->dev;
     uint16_t cap_ver  = 0;
 
+    if (!ptdev->power_mgmt)
+        return reg->init_val;
+
     /* check PCI Power Management support version */
     cap_ver = ptdev->pm_state->pmc_field & PCI_PM_CAP_VER_MASK;
 
@@ -2553,6 +2562,9 @@ static uint8_t pt_reg_grp_size_init(struct pt_dev *ptdev,
 static uint8_t pt_pm_size_init(struct pt_dev *ptdev,
         struct pt_reg_grp_info_tbl *grp_reg, uint32_t base_offset)
 {
+    if (!ptdev->power_mgmt)
+        return grp_reg->grp_size;
+
     ptdev->pm_state = qemu_mallocz(sizeof(struct pt_pm_info));
     if (!ptdev->pm_state)
     {
@@ -2806,6 +2818,25 @@ static int pt_bar_reg_read(struct pt_dev *ptdev,
    return 0;
 }
 
+
+/* read Power Management Control/Status register */
+static int pt_pmcsr_reg_read(struct pt_dev *ptdev,
+        struct pt_reg_tbl *cfg_entry,
+        uint16_t *value, uint16_t valid_mask)
+{
+    struct pt_reg_info_tbl *reg = cfg_entry->reg;
+    uint16_t valid_emu_mask = reg->emu_mask;
+
+    if (!ptdev->power_mgmt)
+        valid_emu_mask |= PCI_PM_CTRL_STATE_MASK;
+
+    valid_emu_mask = valid_emu_mask & valid_mask ;
+    *value = PT_MERGE_VALUE(*value, cfg_entry->data, ~valid_emu_mask);
+
+    return 0;
+}
+
+
 /* write byte size emulate register */
 static int pt_byte_reg_write(struct pt_dev *ptdev,
         struct pt_reg_tbl *cfg_entry,
@@ -3082,6 +3113,24 @@ static int pt_pmcsr_reg_write(struct pt_dev *ptdev,
     struct pt_pm_info *pm_state = ptdev->pm_state;
     uint16_t read_val = 0;
 
+    if (!ptdev->power_mgmt) {
+        uint16_t emu_mask = 
+            PCI_PM_CTRL_PME_STATUS | PCI_PM_CTRL_DATA_SCALE_MASK |
+            PCI_PM_CTRL_PME_ENABLE |
+            PCI_PM_CTRL_DATA_SEL_MASK | PCI_PM_CTRL_STATE_MASK;
+        uint16_t ro_mask = PCI_PM_CTRL_DATA_SCALE_MASK;
+
+        /* modify emulate register */
+        writable_mask = emu_mask & ~ro_mask & valid_mask;
+
+        cfg_entry->data = PT_MERGE_VALUE(*value, cfg_entry->data, writable_mask);
+        /* create value for writing to I/O device register */
+        throughable_mask = ~emu_mask & valid_mask;
+        *value = PT_MERGE_VALUE(*value, dev_value, throughable_mask);
+
+        return 0;
+    }
+
     /* modify emulate register */
     writable_mask = reg->emu_mask & ~reg->ro_mask & valid_mask;
     cfg_entry->data = PT_MERGE_VALUE(*value, cfg_entry->data, writable_mask);
@@ -3564,7 +3613,7 @@ struct pt_dev * register_real_device(PCIBus *e_bus,
     struct pci_config_cf8 machine_bdf;
     int free_pci_slot = -1;
     char *key, *val;
-    int msi_translate;
+    int msi_translate, power_mgmt;
 
     PT_LOG("Assigning real physical device %02x:%02x.%x ...\n",
         r_bus, r_dev, r_func);
@@ -3597,6 +3646,7 @@ struct pt_dev * register_real_device(PCIBus *e_bus,
     }
 
     msi_translate = direct_pci_msitranslate;
+    power_mgmt = direct_pci_power_mgmt;
     while (opt) {
         if (get_next_keyval(&opt, &key, &val)) {
             PT_LOG("Error: unrecognized PCI assignment option \"%s\"\n", opt);
@@ -3618,6 +3668,21 @@ struct pt_dev * register_real_device(PCIBus *e_bus,
             else
                 PT_LOG("Error: unrecognized value for msitranslate=\n");
         }
+        else if (strcmp(key, "power_mgmt") == 0)
+        {
+            if (strcmp(val, "0") == 0)
+            {
+                PT_LOG("Disable power management\n");
+                power_mgmt = 0;
+            }
+            else if (strcmp(val, "1") == 0)
+            {
+                PT_LOG("Enable power management\n");
+                power_mgmt = 1;
+            }
+            else
+                PT_LOG("Error: unrecognized value for power_mgmt=\n");
+        }
         else
             PT_LOG("Error: unrecognized PCI assignment option \"%s=%s\"\n", key, val);
 
@@ -3639,6 +3704,7 @@ struct pt_dev * register_real_device(PCIBus *e_bus,
 
     assigned_device->pci_dev = pci_dev;
     assigned_device->msi_trans_cap = msi_translate;
+    assigned_device->power_mgmt = power_mgmt;
 
     /* Assign device */
     machine_bdf.reg = 0;
diff --git a/hw/pass-through.h b/hw/pass-through.h
index e86d311..b7b5a79 100644
--- a/hw/pass-through.h
+++ b/hw/pass-through.h
@@ -217,6 +217,7 @@ struct pt_dev {
     /* Physical MSI to guest INTx translation when possible */
     int msi_trans_cap;
     int msi_trans_en;
+    int power_mgmt;
     struct pt_pm_info *pm_state;                /* PM virtualization */
 };
 
diff --git a/hw/pci.h b/hw/pci.h
index 2800499..10fa601 100644
--- a/hw/pci.h
+++ b/hw/pci.h
@@ -65,6 +65,7 @@ struct PCIDevice {
 
 extern char direct_pci_str[];
 extern int direct_pci_msitranslate;
+extern int direct_pci_power_mgmt;
 
 PCIDevice *pci_register_device(PCIBus *bus, const char *name,
                                int instance_size, int devfn,
diff --git a/xenstore.c b/xenstore.c
index 928e950..4ee6ceb 100644
--- a/xenstore.c
+++ b/xenstore.c
@@ -303,8 +303,10 @@ const char *xenstore_get_guest_uuid(void)
 
 #define DIRECT_PCI_STR_LEN 512
 #define PT_PCI_MSITRANSLATE_DEFAULT 1
+#define PT_PCI_POWER_MANAGEMENT_DEFAULT 0
 char direct_pci_str[DIRECT_PCI_STR_LEN];
 int direct_pci_msitranslate;
+int direct_pci_power_mgmt;
 void xenstore_parse_domain_config(int hvm_domid)
 {
     char **e_danger = NULL;
@@ -603,15 +605,26 @@ void xenstore_parse_domain_config(int hvm_domid)
 
     /* get the pci pass-through parameter */
     if (pasprintf(&buf, "/local/domain/0/backend/pci/%u/%u/msitranslate",
-                  hvm_domid, pci_devid) == -1)
-        goto out;
+                  hvm_domid, pci_devid) != -1)
+    {
+        free(params);
+        params = xs_read(xsh, XBT_NULL, buf, &len);
+        if (params)
+            direct_pci_msitranslate = atoi(params);
+        else
+            direct_pci_msitranslate = PT_PCI_MSITRANSLATE_DEFAULT;
+    }
 
-    free(params);
-    params = xs_read(xsh, XBT_NULL, buf, &len);
-    if (params)
-        direct_pci_msitranslate = atoi(params);
-    else
-        direct_pci_msitranslate = PT_PCI_MSITRANSLATE_DEFAULT;
+    if (pasprintf(&buf, "/local/domain/0/backend/pci/%u/%u/power_mgmt",
+                  hvm_domid, pci_devid) != -1)
+    {
+        free(params);
+        params = xs_read(xsh, XBT_NULL, buf, &len);
+        if (params)
+            direct_pci_power_mgmt = atoi(params);
+        else
+            direct_pci_power_mgmt = PT_PCI_POWER_MANAGEMENT_DEFAULT;
+    }
 
  out:
     free(danger_type);

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [PATCH 2/2] ioemu: make management of PCI D-states by guest optional

[Yuji Shimada]

On Fri, 6 Mar 2009 14:26:37 +0900
Kouya Shimura <kouya@jp.fujitsu.com> wrote:

> +/* read Power Management Control/Status register */
> +static int pt_pmcsr_reg_read(struct pt_dev *ptdev,
> +        struct pt_reg_tbl *cfg_entry,
> +        uint16_t *value, uint16_t valid_mask)
> +{
> +    struct pt_reg_info_tbl *reg = cfg_entry->reg;
> +    uint16_t valid_emu_mask = reg->emu_mask;
> +
> +    if (!ptdev->power_mgmt)
> +        valid_emu_mask |= PCI_PM_CTRL_STATE_MASK;

(snip)

> @@ -3082,6 +3113,24 @@ static int pt_pmcsr_reg_write(struct pt_dev *ptdev,
>      struct pt_pm_info *pm_state = ptdev->pm_state;
>      uint16_t read_val = 0;
>  
> +    if (!ptdev->power_mgmt) {
> +        uint16_t emu_mask = 
> +            PCI_PM_CTRL_PME_STATUS | PCI_PM_CTRL_DATA_SCALE_MASK |
> +            PCI_PM_CTRL_PME_ENABLE |
> +            PCI_PM_CTRL_DATA_SEL_MASK | PCI_PM_CTRL_STATE_MASK;
> +        uint16_t ro_mask = PCI_PM_CTRL_DATA_SCALE_MASK;

Hi,

Why are Data_Scale field and Data_Select field emulated?
I think we can pass-through them as follows.

On the other hand, No_Soft_Reset field needs to be emulated and fixed
to 1. The reason is PowerState field is emulated, that means guest
software can change power state, but actual power state of device is
not changed. So internal reset never occurs.

As a result, the code will be as the following.

read:
      valid_emu_mask |= PCI_PM_CTRL_STATE_MASK |
             PCI_PM_CTRL_NO_SOFT_RESET;

write:
      uint16_t emu_mask = reg->emu_mask | PCI_PM_CTRL_STATE_MASK |
             PCI_PM_CTRL_NO_SOFT_RESET;
      uint16_t ro_mask = reg->ro_mask;

Thanks,
--
Yuji Shimada

Re: Re: [PATCH 2/2] ioemu: make management of PCI D-states by guest optional

[Kouya Shimura]

Yuji Shimada writes:
> Why are Data_Scale field and Data_Select field emulated?

No reason, I intended that emu_mask and ro_mask become
the same as before your PCI D-state patch.

> I think we can pass-through them as follows.
> 
> On the other hand, No_Soft_Reset field needs to be emulated and fixed
> to 1. The reason is PowerState field is emulated, that means guest
> software can change power state, but actual power state of device is
> not changed. So internal reset never occurs.
> 
> As a result, the code will be as the following.
> 
> read:
>       valid_emu_mask |= PCI_PM_CTRL_STATE_MASK |
>              PCI_PM_CTRL_NO_SOFT_RESET;
> 
> write:
>       uint16_t emu_mask = reg->emu_mask | PCI_PM_CTRL_STATE_MASK |
>              PCI_PM_CTRL_NO_SOFT_RESET;
>       uint16_t ro_mask = reg->ro_mask;

Thank you for pointing out. I'll fix it.

Thanks,
Kouya

[PATCH v2 2/2] ioemu: make management of PCI D-states by guest optional

[Kouya Shimura]

[-- Attachment #1: message body text --]
[-- Type: text/plain, Size: 543 bytes --]

This is an updated patch per Shimada-san's advice.

Commit 8c771eb6294afc5b3754a9e3de51568d4e5986c2 enables the guest OS
to program D0-D3hot states of the assigned device, however,
D3hot state in some PCI devices causes the failure of domain
creation/destruction.

With this patch, we can configure a guest to manage the PCI D-states
or not for each PCI passthru device.

A corresponding change is committed to xen-unstable.hg.
    chageset: 19279:ec671455fb05ca6714deeaca78aacb1026ca4752

Signed-off-by: Kouya Shimura <kouya@jp.fujitsu.com>


[-- Attachment #2: pci_pm_option_ioemu_v2.patch --]
[-- Type: text/x-patch, Size: 7873 bytes --]

diff --git a/hw/pass-through.c b/hw/pass-through.c
index 4a86309..78a8e8f 100644
--- a/hw/pass-through.c
+++ b/hw/pass-through.c
@@ -105,6 +105,9 @@ static int pt_long_reg_read(struct pt_dev *ptdev,
 static int pt_bar_reg_read(struct pt_dev *ptdev,
     struct pt_reg_tbl *cfg_entry,
     uint32_t *value, uint32_t valid_mask);
+static int pt_pmcsr_reg_read(struct pt_dev *ptdev,
+    struct pt_reg_tbl *cfg_entry,
+    uint16_t *value, uint16_t valid_mask);
 static int pt_byte_reg_write(struct pt_dev *ptdev,
     struct pt_reg_tbl *cfg_entry,
     uint8_t *value, uint8_t dev_value, uint8_t valid_mask);
@@ -407,7 +410,7 @@ static struct pt_reg_info_tbl pt_emu_reg_pm_tbl[] = {
         .ro_mask    = 0xE1FC,
         .emu_mask   = 0x8100,
         .init       = pt_pmcsr_reg_init,
-        .u.w.read   = pt_word_reg_read,
+        .u.w.read   = pt_pmcsr_reg_read,
         .u.w.write  = pt_pmcsr_reg_write,
         .u.w.restore  = pt_pmcsr_reg_restore,
     },
@@ -2341,6 +2344,9 @@ static uint32_t pt_pmc_reg_init(struct pt_dev *ptdev,
 {
     PCIDevice *d = &ptdev->dev;
 
+    if (!ptdev->power_mgmt)
+        return reg->init_val;
+
     /* set Power Management Capabilities register */
     ptdev->pm_state->pmc_field = *(uint16_t *)(d->config + real_offset);
 
@@ -2354,6 +2360,9 @@ static uint32_t pt_pmcsr_reg_init(struct pt_dev *ptdev,
     PCIDevice *d = &ptdev->dev;
     uint16_t cap_ver  = 0;
 
+    if (!ptdev->power_mgmt)
+        return reg->init_val;
+
     /* check PCI Power Management support version */
     cap_ver = ptdev->pm_state->pmc_field & PCI_PM_CAP_VER_MASK;
 
@@ -2553,6 +2562,9 @@ static uint8_t pt_reg_grp_size_init(struct pt_dev *ptdev,
 static uint8_t pt_pm_size_init(struct pt_dev *ptdev,
         struct pt_reg_grp_info_tbl *grp_reg, uint32_t base_offset)
 {
+    if (!ptdev->power_mgmt)
+        return grp_reg->grp_size;
+
     ptdev->pm_state = qemu_mallocz(sizeof(struct pt_pm_info));
     if (!ptdev->pm_state)
     {
@@ -2806,6 +2818,25 @@ static int pt_bar_reg_read(struct pt_dev *ptdev,
    return 0;
 }
 
+
+/* read Power Management Control/Status register */
+static int pt_pmcsr_reg_read(struct pt_dev *ptdev,
+        struct pt_reg_tbl *cfg_entry,
+        uint16_t *value, uint16_t valid_mask)
+{
+    struct pt_reg_info_tbl *reg = cfg_entry->reg;
+    uint16_t valid_emu_mask = reg->emu_mask;
+
+    if (!ptdev->power_mgmt)
+        valid_emu_mask |= PCI_PM_CTRL_STATE_MASK | PCI_PM_CTRL_NO_SOFT_RESET;
+
+    valid_emu_mask = valid_emu_mask & valid_mask ;
+    *value = PT_MERGE_VALUE(*value, cfg_entry->data, ~valid_emu_mask);
+
+    return 0;
+}
+
+
 /* write byte size emulate register */
 static int pt_byte_reg_write(struct pt_dev *ptdev,
         struct pt_reg_tbl *cfg_entry,
@@ -3077,19 +3108,26 @@ static int pt_pmcsr_reg_write(struct pt_dev *ptdev,
 {
     struct pt_reg_info_tbl *reg = cfg_entry->reg;
     PCIDevice *d = &ptdev->dev;
+    uint16_t emu_mask = reg->emu_mask;
     uint16_t writable_mask = 0;
     uint16_t throughable_mask = 0;
     struct pt_pm_info *pm_state = ptdev->pm_state;
     uint16_t read_val = 0;
 
+    if (!ptdev->power_mgmt)
+        emu_mask |= PCI_PM_CTRL_STATE_MASK | PCI_PM_CTRL_NO_SOFT_RESET;
+
     /* modify emulate register */
-    writable_mask = reg->emu_mask & ~reg->ro_mask & valid_mask;
+    writable_mask = emu_mask & ~reg->ro_mask & valid_mask;
     cfg_entry->data = PT_MERGE_VALUE(*value, cfg_entry->data, writable_mask);
 
     /* create value for writing to I/O device register */
-    throughable_mask = ~reg->emu_mask & valid_mask;
+    throughable_mask = ~emu_mask & valid_mask;
     *value = PT_MERGE_VALUE(*value, dev_value, throughable_mask);
 
+    if (!ptdev->power_mgmt)
+        return 0;
+
     /* set I/O device power state */
     pm_state->cur_state = (dev_value & PCI_PM_CTRL_STATE_MASK);
 
@@ -3564,7 +3602,7 @@ struct pt_dev * register_real_device(PCIBus *e_bus,
     struct pci_config_cf8 machine_bdf;
     int free_pci_slot = -1;
     char *key, *val;
-    int msi_translate;
+    int msi_translate, power_mgmt;
 
     PT_LOG("Assigning real physical device %02x:%02x.%x ...\n",
         r_bus, r_dev, r_func);
@@ -3597,6 +3635,7 @@ struct pt_dev * register_real_device(PCIBus *e_bus,
     }
 
     msi_translate = direct_pci_msitranslate;
+    power_mgmt = direct_pci_power_mgmt;
     while (opt) {
         if (get_next_keyval(&opt, &key, &val)) {
             PT_LOG("Error: unrecognized PCI assignment option \"%s\"\n", opt);
@@ -3618,6 +3657,21 @@ struct pt_dev * register_real_device(PCIBus *e_bus,
             else
                 PT_LOG("Error: unrecognized value for msitranslate=\n");
         }
+        else if (strcmp(key, "power_mgmt") == 0)
+        {
+            if (strcmp(val, "0") == 0)
+            {
+                PT_LOG("Disable PCI Power Management\n");
+                power_mgmt = 0;
+            }
+            else if (strcmp(val, "1") == 0)
+            {
+                PT_LOG("Enable PCI Power Management\n");
+                power_mgmt = 1;
+            }
+            else
+                PT_LOG("Error: unrecognized value for power_mgmt=\n");
+        }
         else
             PT_LOG("Error: unrecognized PCI assignment option \"%s=%s\"\n", key, val);
 
@@ -3639,6 +3693,7 @@ struct pt_dev * register_real_device(PCIBus *e_bus,
 
     assigned_device->pci_dev = pci_dev;
     assigned_device->msi_trans_cap = msi_translate;
+    assigned_device->power_mgmt = power_mgmt;
 
     /* Assign device */
     machine_bdf.reg = 0;
diff --git a/hw/pass-through.h b/hw/pass-through.h
index e86d311..b7b5a79 100644
--- a/hw/pass-through.h
+++ b/hw/pass-through.h
@@ -217,6 +217,7 @@ struct pt_dev {
     /* Physical MSI to guest INTx translation when possible */
     int msi_trans_cap;
     int msi_trans_en;
+    int power_mgmt;
     struct pt_pm_info *pm_state;                /* PM virtualization */
 };
 
diff --git a/hw/pci.h b/hw/pci.h
index 2800499..10fa601 100644
--- a/hw/pci.h
+++ b/hw/pci.h
@@ -65,6 +65,7 @@ struct PCIDevice {
 
 extern char direct_pci_str[];
 extern int direct_pci_msitranslate;
+extern int direct_pci_power_mgmt;
 
 PCIDevice *pci_register_device(PCIBus *bus, const char *name,
                                int instance_size, int devfn,
diff --git a/xenstore.c b/xenstore.c
index 928e950..4ee6ceb 100644
--- a/xenstore.c
+++ b/xenstore.c
@@ -303,8 +303,10 @@ const char *xenstore_get_guest_uuid(void)
 
 #define DIRECT_PCI_STR_LEN 512
 #define PT_PCI_MSITRANSLATE_DEFAULT 1
+#define PT_PCI_POWER_MANAGEMENT_DEFAULT 0
 char direct_pci_str[DIRECT_PCI_STR_LEN];
 int direct_pci_msitranslate;
+int direct_pci_power_mgmt;
 void xenstore_parse_domain_config(int hvm_domid)
 {
     char **e_danger = NULL;
@@ -603,15 +605,26 @@ void xenstore_parse_domain_config(int hvm_domid)
 
     /* get the pci pass-through parameter */
     if (pasprintf(&buf, "/local/domain/0/backend/pci/%u/%u/msitranslate",
-                  hvm_domid, pci_devid) == -1)
-        goto out;
+                  hvm_domid, pci_devid) != -1)
+    {
+        free(params);
+        params = xs_read(xsh, XBT_NULL, buf, &len);
+        if (params)
+            direct_pci_msitranslate = atoi(params);
+        else
+            direct_pci_msitranslate = PT_PCI_MSITRANSLATE_DEFAULT;
+    }
 
-    free(params);
-    params = xs_read(xsh, XBT_NULL, buf, &len);
-    if (params)
-        direct_pci_msitranslate = atoi(params);
-    else
-        direct_pci_msitranslate = PT_PCI_MSITRANSLATE_DEFAULT;
+    if (pasprintf(&buf, "/local/domain/0/backend/pci/%u/%u/power_mgmt",
+                  hvm_domid, pci_devid) != -1)
+    {
+        free(params);
+        params = xs_read(xsh, XBT_NULL, buf, &len);
+        if (params)
+            direct_pci_power_mgmt = atoi(params);
+        else
+            direct_pci_power_mgmt = PT_PCI_POWER_MANAGEMENT_DEFAULT;
+    }
 
  out:
     free(danger_type);

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [PATCH 0/2] passthrough: make management of PCI D-states by guest optional

[Yuji Shimada]

On Fri, 6 Mar 2009 14:23:20 +0900
Kouya Shimura <kouya@jp.fujitsu.com> wrote:

> Using D3hot state of PCI devices in xen is not mature yet.
> I met domain destruction/creation troubles in some PCI cards since the
> PCI config registers can't be read with D3hot state. xend.log is attached.

xend.log shows error occurred on resetting the device.
I think programming the device to D0 state is needed before resetting
the device.

In addition to this, config register values should be saved
when domain 0 starts, because in some devices config registers
can't be read with D3 hot state. To achieve this, it is good that
pciback driver saves config register values when it is bound to the
device, resets the device, and restores config register after
resetting.

Actually resetting and saving/restoring in pciback were discussed on
xen-devel. But unfortunately they seem not to be under developing.

Another approach is that qemu programs the device to D0 state on
shutdowning of guest domain, instead of booting of guest domain.
Then, xend can reset device successfully.

Thanks,
--
Yuji Shimada

RE: Re: [PATCH 0/2] passthrough: make management of PCI D-states by guest optional

[Cui, Dexuan]

Yuji Shimada wrote:
> On Fri, 6 Mar 2009 14:23:20 +0900
> Kouya Shimura <kouya@jp.fujitsu.com> wrote:
> >
>> Using D3hot state of PCI devices in xen is not mature yet.
>> I met domain destruction/creation troubles in some PCI cards since
>> the PCI config registers can't be read with D3hot state. xend.log is
>> attached. 
>>
>> This patch set makes the management of PCI D-states by guest optional.
>> 
>> The default is "pci_power_mgmt=0" which disables the guest OS from
>> managing D-states because it would be better to avoid the trouble than
>> advantage of low power consumption.
Is this the only reason to add the guest config parameter?
If yes, I don't think it is worthwhile to have two over-200-lines patches.
IMO we should fix the code in xend.

> 
> xend.log shows error occurred on resetting the device.
> I think programming the device to D0 state is needed before resetting
> the device.
I think so, too. 

> Another approach is that qemu programs the device to D0 state on
> shutdowning of guest domain, instead of booting of guest domain.
> Then, xend can reset device successfully.
I think it's not good to do this in ioemu. One reason is: ioemu doesn't always know when a guest is being shut down.e.g., we may "xm destroy" a guest.
xend should be a better place.

Thanks,
-- Dexuan

RE: Re: [PATCH 0/2] passthrough: make management of PCI D-states by guest optional

[Kouya Shimura]

Cui, Dexuan writes:
> Yuji Shimada wrote:
> > On Fri, 6 Mar 2009 14:23:20 +0900
> > Kouya Shimura <kouya@jp.fujitsu.com> wrote:
> > >
> >> Using D3hot state of PCI devices in xen is not mature yet.
> >> I met domain destruction/creation troubles in some PCI cards since
> >> the PCI config registers can't be read with D3hot state. xend.log is
> >> attached. 
> >>
> >> This patch set makes the management of PCI D-states by guest optional.
> >> 
> >> The default is "pci_power_mgmt=0" which disables the guest OS from
> >> managing D-states because it would be better to avoid the trouble than
> >> advantage of low power consumption.
> Is this the only reason to add the guest config parameter?
> If yes, I don't think it is worthwhile to have two over-200-lines patches.

If the answer was yes, I would have proposed to revert the patches
about PCI D-states. This patch enables individual setting for each PCI
passthru device and that requires many lines.

Supposing a PCI device has a hardware bug, that is possible, don't you
wish to disable *only* it?  Actually, Q-logic FC card which has two
PCI functions looks ill.

Any way, tool part of this patch is committed.

> IMO we should fix the code in xend.

I hope so, too.  After that, turn on "pci_power_mgmt=1".

Thanks,
Kouya

Red Hat dropped XEN

[Venefax]

I was really dismayed today when I installed the latest Red Hat 5.3 and the
version of Xen included is 3.1. Apparently, the only distribution with Xen
3.3 is Suse. Am I reading here that Red Hat will never upgrade Xen to newer
versions? I installed the released version, and it still uses Xen 3.1.
Federico

RE: Red Hat dropped XEN

[Jayaraman, Bhaskar]

This means they don't spport VT-x and VT-d which are features that are available from later versions only?
Bhaskar.

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Venefax
Sent: Monday, March 09, 2009 6:16 AM
To: xen-devel@lists.xensource.com
Subject: [Xen-devel] Red Hat dropped XEN

I was really dismayed today when I installed the latest Red Hat 5.3 and the
version of Xen included is 3.1. Apparently, the only distribution with Xen
3.3 is Suse. Am I reading here that Red Hat will never upgrade Xen to newer
versions? I installed the released version, and it still uses Xen 3.1.
Federico


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Red Hat dropped XEN

[Keir Fraser]

Yes to VT-x, no to VT-d.

 -- Keir

On 09/03/2009 07:41, "Jayaraman, Bhaskar" <Bhaskar.Jayaraman@lsi.com> wrote:

> This means they don't spport VT-x and VT-d which are features that are
> available from later versions only?
> Bhaskar.
> 
> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com
> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Venefax
> Sent: Monday, March 09, 2009 6:16 AM
> To: xen-devel@lists.xensource.com
> Subject: [Xen-devel] Red Hat dropped XEN
> 
> I was really dismayed today when I installed the latest Red Hat 5.3 and the
> version of Xen included is 3.1. Apparently, the only distribution with Xen
> 3.3 is Suse. Am I reading here that Red Hat will never upgrade Xen to newer
> versions? I installed the released version, and it still uses Xen 3.1.
> Federico
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: Red Hat dropped XEN

[Keir Fraser]

I don't think they'd upgrade a package version in a RHEL stable release
cycle -- just backport lots of patches.

 K.

On 09/03/2009 00:46, "Venefax" <venefax@gmail.com> wrote:

> I was really dismayed today when I installed the latest Red Hat 5.3 and the
> version of Xen included is 3.1. Apparently, the only distribution with Xen
> 3.3 is Suse. Am I reading here that Red Hat will never upgrade Xen to newer
> versions? I installed the released version, and it still uses Xen 3.1.
> Federico
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: Red Hat dropped XEN

[Daniel P. Berrange]

On Sun, Mar 08, 2009 at 08:46:22PM -0400, Venefax wrote:
> I was really dismayed today when I installed the latest Red Hat 5.3 and the
> version of Xen included is 3.1. Apparently, the only distribution with Xen
> 3.3 is Suse. Am I reading here that Red Hat will never upgrade Xen to newer
> versions? I installed the released version, and it still uses Xen 3.1.

Things are not quite that simple. The RHEL-5 userspace tools are mostly on 
a 3.0.3 base, while the hypervisor and kernel are on a 3.1.0 base. The base
versions are typically not upgraded during the lifetime of a major RHEL
release series. That said, we do have a large number of carefully backported 
features & patches from newer versions, eg to add support for NPT/EPT, 
improved HVM support, hugepage support, and much more besides.  So just 
comparing version numbers won't give you a true picture of Xen features 
available in RHEL-5.

Regards,
Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

RE: Red Hat dropped XEN

[Venefax]

I was trying to use some of the command set, like "xm new -F" etc., and it
did not work, so I did checked and the xm options are limited, compared to
SLES 10 SP2 or the Beta SLES 11. It is an inferior product. For example,
what I typically do is this:
Create a paravirtualized domU
Xm list --long domname > domname.sxp
Edit domname.sxp and add (cpus 2-15)
Xm new -F domname.sxp

The question is: how do you achieve that with RHEL 5.3? I don't want my
Dumu's to use all the CPU's, since I restrict Dom0 to CPU's 0-1. Is there a
way, at all?
Federico


-----Original Message-----
From: Daniel P. Berrange [mailto:berrange@redhat.com] 
Sent: Monday, March 09, 2009 6:13 AM
To: Venefax
Cc: xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] Red Hat dropped XEN

On Sun, Mar 08, 2009 at 08:46:22PM -0400, Venefax wrote:
> I was really dismayed today when I installed the latest Red Hat 5.3 and
the
> version of Xen included is 3.1. Apparently, the only distribution with Xen
> 3.3 is Suse. Am I reading here that Red Hat will never upgrade Xen to
newer
> versions? I installed the released version, and it still uses Xen 3.1.

Things are not quite that simple. The RHEL-5 userspace tools are mostly on 
a 3.0.3 base, while the hypervisor and kernel are on a 3.1.0 base. The base
versions are typically not upgraded during the lifetime of a major RHEL
release series. That said, we do have a large number of carefully backported

features & patches from newer versions, eg to add support for NPT/EPT, 
improved HVM support, hugepage support, and much more besides.  So just 
comparing version numbers won't give you a true picture of Xen features 
available in RHEL-5.

Regards,
Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/
:|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org
:|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/
:|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505
:|

RE: Red Hat dropped XEN

[Mike Brady]

You could try upgrading to 3.3.1 from http://www.gitco.de/repo/

On Mon, 2009-03-09 at 06:22 -0400, Venefax wrote:
> I was trying to use some of the command set, like "xm new -F" etc., and it
> did not work, so I did checked and the xm options are limited, compared to
> SLES 10 SP2 or the Beta SLES 11. It is an inferior product. For example,
> what I typically do is this:
> Create a paravirtualized domU
> Xm list --long domname > domname.sxp
> Edit domname.sxp and add (cpus 2-15)
> Xm new -F domname.sxp
> 
> The question is: how do you achieve that with RHEL 5.3? I don't want my
> Dumu's to use all the CPU's, since I restrict Dom0 to CPU's 0-1. Is there a
> way, at all?
> Federico
> 
> 
> -----Original Message-----
> From: Daniel P. Berrange [mailto:berrange@redhat.com] 
> Sent: Monday, March 09, 2009 6:13 AM
> To: Venefax
> Cc: xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] Red Hat dropped XEN
> 
> On Sun, Mar 08, 2009 at 08:46:22PM -0400, Venefax wrote:
> > I was really dismayed today when I installed the latest Red Hat 5.3 and
> the
> > version of Xen included is 3.1. Apparently, the only distribution with Xen
> > 3.3 is Suse. Am I reading here that Red Hat will never upgrade Xen to
> newer
> > versions? I installed the released version, and it still uses Xen 3.1.
> 
> Things are not quite that simple. The RHEL-5 userspace tools are mostly on 
> a 3.0.3 base, while the hypervisor and kernel are on a 3.1.0 base. The base
> versions are typically not upgraded during the lifetime of a major RHEL
> release series. That said, we do have a large number of carefully backported
> 
> features & patches from newer versions, eg to add support for NPT/EPT, 
> improved HVM support, hugepage support, and much more besides.  So just 
> comparing version numbers won't give you a true picture of Xen features 
> available in RHEL-5.
> 
> Regards,
> Daniel

RE: Red Hat dropped XEN

[Venefax]

Are there any instructions as how to do that? Or it is just a case of
downloading the RPM's and installing them?
Federico

-----Original Message-----
From: Mike Brady [mailto:mike.brady@devnull.net.nz] 
Sent: Monday, March 09, 2009 12:41 PM
To: Venefax
Cc: 'Daniel P. Berrange'; xen-devel@lists.xensource.com
Subject: RE: [Xen-devel] Red Hat dropped XEN

You could try upgrading to 3.3.1 from http://www.gitco.de/repo/

On Mon, 2009-03-09 at 06:22 -0400, Venefax wrote:
> I was trying to use some of the command set, like "xm new -F" etc., and it
> did not work, so I did checked and the xm options are limited, compared to
> SLES 10 SP2 or the Beta SLES 11. It is an inferior product. For example,
> what I typically do is this:
> Create a paravirtualized domU
> Xm list --long domname > domname.sxp
> Edit domname.sxp and add (cpus 2-15)
> Xm new -F domname.sxp
> 
> The question is: how do you achieve that with RHEL 5.3? I don't want my
> Dumu's to use all the CPU's, since I restrict Dom0 to CPU's 0-1. Is there
a
> way, at all?
> Federico
> 
> 
> -----Original Message-----
> From: Daniel P. Berrange [mailto:berrange@redhat.com] 
> Sent: Monday, March 09, 2009 6:13 AM
> To: Venefax
> Cc: xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] Red Hat dropped XEN
> 
> On Sun, Mar 08, 2009 at 08:46:22PM -0400, Venefax wrote:
> > I was really dismayed today when I installed the latest Red Hat 5.3 and
> the
> > version of Xen included is 3.1. Apparently, the only distribution with
Xen
> > 3.3 is Suse. Am I reading here that Red Hat will never upgrade Xen to
> newer
> > versions? I installed the released version, and it still uses Xen 3.1.
> 
> Things are not quite that simple. The RHEL-5 userspace tools are mostly on

> a 3.0.3 base, while the hypervisor and kernel are on a 3.1.0 base. The
base
> versions are typically not upgraded during the lifetime of a major RHEL
> release series. That said, we do have a large number of carefully
backported
> 
> features & patches from newer versions, eg to add support for NPT/EPT, 
> improved HVM support, hugepage support, and much more besides.  So just 
> comparing version numbers won't give you a true picture of Xen features 
> available in RHEL-5.
> 
> Regards,
> Daniel

RE: Re: [PATCH 0/2] passthrough: make management of PCI D-states by guest optional

[Cui, Dexuan]

Kouya Shimura wrote:
> Cui, Dexuan writes:
>> Yuji Shimada wrote:
>>> On Fri, 6 Mar 2009 14:23:20 +0900
>>> Kouya Shimura <kouya@jp.fujitsu.com> wrote:
>>>> 
>>>> Using D3hot state of PCI devices in xen is not mature yet.
>>>> I met domain destruction/creation troubles in some PCI cards since
>>>> the PCI config registers can't be read with D3hot state. xend.log
>>>> is attached. 
>>>> 
>>>> This patch set makes the management of PCI D-states by guest
>>>> optional. 
>>>> 
>>>> The default is "pci_power_mgmt=0" which disables the guest OS from
>>>> managing D-states because it would be better to avoid the trouble
>>>> than advantage of low power consumption.
>> Is this the only reason to add the guest config parameter?
>> If yes, I don't think it is worthwhile to have two over-200-lines
>> patches. 
> 
> If the answer was yes, I would have proposed to revert the patches
> about PCI D-states. This patch enables individual setting for each PCI
> passthru device and that requires many lines.
> 
> Supposing a PCI device has a hardware bug, that is possible, don't you
> wish to disable *only* it?  Actually, Q-logic FC card which has two
> PCI functions looks ill.
I agree.

> 
> Any way, tool part of this patch is committed.
Now I think it's good. :-)

> 
>> IMO we should fix the code in xend.
> 
> I hope so, too.  After that, turn on "pci_power_mgmt=1".
Looks it's not very easy to fix xend here. :-(
e.g.,  I think the python error you see when guest is destroyed is caused by: xend tries to save the 256-byte pci config space of the assigned device before doing FLR, but at D3hot state, the config space is inaccessible.
So, if we allow guest to program D3/D0, the current "save just before do_FLR, do_FLR, restore" in xend doesn't work at all. We should save config space (into xenstore??) before assigning a deivce to guest, or we can save the proper values of config space into pciback.

Thanks,
 -- Dexuan

Re: [PATCH 0/2] passthrough: make management of PCI D-states by guest optional

[Yuji Shimada]

On Fri, 6 Mar 2009 14:23:20 +0900
Kouya Shimura <kouya@jp.fujitsu.com> wrote:

> Hi,
> 
> Using D3hot state of PCI devices in xen is not mature yet.
> I met domain destruction/creation troubles in some PCI cards since the
> PCI config registers can't be read with D3hot state. xend.log is attached.
> 
> This patch set makes the management of PCI D-states by guest optional.
> 
> The default is "pci_power_mgmt=0" which disables the guest OS from
> managing D-states because it would be better to avoid the trouble than
> advantage of low power consumption.
> 
> Thanks,
> Kouya
> 
> From xend.log:
> 
> [2009-03-04 09:49:24 4356] ERROR (XendDomainInfo:2584) XendDomainInfo.destroy: domain destruction failed.
> Traceback (most recent call last):
>   File "/usr/lib64/python/xen/xend/XendDomainInfo.py", line 2578, in destroy
>     do_FLR(self.domid)
>   File "/usr/lib64/python/xen/xend/XendDomainInfo.py", line 314, in do_FLR
>     raise VmError("pci: failed to locate device and "+
> VmError: pci: failed to locate device and parse it's resources - ord() expected a character, but string of length 0 found
> ...[snip]...
> [2009-03-04 10:31:24 4343] ERROR (XendDomainInfo:2445) XendDomainInfo.initDomain: exception occurred
> Traceback (most recent call last):
>   File "/usr/lib64/python/xen/xend/XendDomainInfo.py", line 2437, in _initDomain
>     self._createDevices()
>   File "/usr/lib64/python/xen/xend/XendDomainInfo.py", line 2049, in _createDevices
>     devid = self._createDevice(devclass, config)
>   File "/usr/lib64/python/xen/xend/XendDomainInfo.py", line 2016, in _createDevice
>     return self.getDeviceController(deviceClass).createDevice(devConfig)
>   File "/usr/lib64/python/xen/xend/server/DevController.py", line 67, in createDevice
>     self.setupDevice(config)
>   File "/usr/lib64/python/xen/xend/server/pciif.py", line 399, in setupDevice
>     raise VmError("pci: failed to locate device and "+
> VmError: pci: failed to locate device and parse it's resources - ord() expected a character, but string of length 0 found

Hi, Shimura-san

In my environment, I can't draw above error.

So could you test the following patches?

http://lists.xensource.com/archives/html/xen-devel/2009-04/msg00829.html

I think your D-state problem is resolved with these patches.

Thanks,
--
Yuji Shimada

Re: [PATCH 0/2] passthrough: make management of PCI D-states by guest optional

[Kouya Shimura]

Hi,

I tried your patches. Acutually, error messages in xend.log have gone.
So, the problem that PCI config can't be read correctly, is resolved.

However, my D-state problem is not resolved yet.
In a PCI HBA card, qemu-dm.log shows:

check_power_state: Error: Failed to change power state. [00:03.0][requested sta
te:3][current state:0]

Probably this is a hardware problem. After that, an HVM guest can't
boot up from its disk again. 

Thus, for the present, I'll still set 'pci_power_mgmt=0'.

Thanks,
Kouya

Yuji Shimada writes:
> On Fri, 6 Mar 2009 14:23:20 +0900
> Kouya Shimura <kouya@jp.fujitsu.com> wrote:
> 
> > Hi,
> > 
> > Using D3hot state of PCI devices in xen is not mature yet.
> > I met domain destruction/creation troubles in some PCI cards since the
> > PCI config registers can't be read with D3hot state. xend.log is attached.
> > 
> > This patch set makes the management of PCI D-states by guest optional.
> > 
> > The default is "pci_power_mgmt=0" which disables the guest OS from
> > managing D-states because it would be better to avoid the trouble than
> > advantage of low power consumption.
> > 
> > Thanks,
> > Kouya
> > 
> > From xend.log:
> > 
> > [2009-03-04 09:49:24 4356] ERROR (XendDomainInfo:2584) XendDomainInfo.destroy: domain destruction failed.
> > Traceback (most recent call last):
> >   File "/usr/lib64/python/xen/xend/XendDomainInfo.py", line 2578, in destroy
> >     do_FLR(self.domid)
> >   File "/usr/lib64/python/xen/xend/XendDomainInfo.py", line 314, in do_FLR
> >     raise VmError("pci: failed to locate device and "+
> > VmError: pci: failed to locate device and parse it's resources - ord() expected a character, but string of length 0 found
> > ...[snip]...
> > [2009-03-04 10:31:24 4343] ERROR (XendDomainInfo:2445) XendDomainInfo.initDomain: exception occurred
> > Traceback (most recent call last):
> >   File "/usr/lib64/python/xen/xend/XendDomainInfo.py", line 2437, in _initDomain
> >     self._createDevices()
> >   File "/usr/lib64/python/xen/xend/XendDomainInfo.py", line 2049, in _createDevices
> >     devid = self._createDevice(devclass, config)
> >   File "/usr/lib64/python/xen/xend/XendDomainInfo.py", line 2016, in _createDevice
> >     return self.getDeviceController(deviceClass).createDevice(devConfig)
> >   File "/usr/lib64/python/xen/xend/server/DevController.py", line 67, in createDevice
> >     self.setupDevice(config)
> >   File "/usr/lib64/python/xen/xend/server/pciif.py", line 399, in setupDevice
> >     raise VmError("pci: failed to locate device and "+
> > VmError: pci: failed to locate device and parse it's resources - ord() expected a character, but string of length 0 found
> 
> Hi, Shimura-san
> 
> In my environment, I can't draw above error.
> 
> So could you test the following patches?
> 
> http://lists.xensource.com/archives/html/xen-devel/2009-04/msg00829.html
> 
> I think your D-state problem is resolved with these patches.
> 
> Thanks,
> --
> Yuji Shimada