All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH]  Simulates the MSIx table read operation
@ 2010-08-04  2:35 Liu, Yuan B
  2010-08-06 15:01 ` Konrad Rzeszutek Wilk
  0 siblings, 1 reply; 6+ messages in thread
From: Liu, Yuan B @ 2010-08-04  2:35 UTC (permalink / raw)
  To: xen-devel; +Cc: Dong, Eddie


[-- Attachment #1.1: Type: text/plain, Size: 1973 bytes --]

Hi,
         This patch simulates the MSIx table read operation to avoid read traffic caused by guest Linux kernel in a multiple guests environments running with high interrupt rate workload.(We tested 24 guests with iperf by 10Gb workload)

[Background]
                   The assumptions about underlying hardware of OS running in the virtual machine environment would not hold for some cases. This is particularly perceived when considering the CPU virtualization that, the VCPU of the OS would be scheduled out while physical CPU of OS would never be. This cause the corner case trouble of OS designed inherently by the assumption targeting the physical CPU. We have seen the _lock-holder preemption_ case. Now SR-IOV issue is yet another one.
         [Issue]
                   Linux generic IRQ logic for edge interrupt, during the 'Writing EOI' period, has been written the way that in a high rate interrupt environment, the subsequent interrupt would cause the guest busy masking/unmasking interrupt if the previous one isn't handled immediately(For e.g. the guest is scheduled out).
The mask/unmask operation would cause a read operation to flush the previous PCI transactions to ensure the write is successful. This corner case isn't handled by the Xen which only intercept the Guests' mask/unmask operation and forward other requests(read/write table) to qemu.
                 This special case doesn't appear in the light workload but in the case of many (for e.g. 24) guests, it would cause the CPU utilization of Dom0 up to 140%(This is proportional to the number of the guests), which definitely limit the scalability and performance of virtualization technology.
       [Effect]
                 This patch emulates the read operation in the Xen and test showed that all the abnormal MMIO read operation is eliminated completely during iperf running in a heavy workload. The CPU utilization has been dropped to 60% in my test.

Thanks,
Yuan

[-- Attachment #1.2: Type: text/html, Size: 9278 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH]  Simulates the MSIx table read operation
  2010-08-04  2:35 [PATCH] Simulates the MSIx table read operation Liu, Yuan B
@ 2010-08-06 15:01 ` Konrad Rzeszutek Wilk
  0 siblings, 0 replies; 6+ messages in thread
From: Konrad Rzeszutek Wilk @ 2010-08-06 15:01 UTC (permalink / raw)
  To: Liu, Yuan B; +Cc: xen-devel, Dong, Eddie

On Wed, Aug 04, 2010 at 10:35:26AM +0800, Liu, Yuan B wrote:
> Hi,
>          This patch simulates the MSIx table read operation to avoid read traffic caused by guest Linux kernel in a multiple guests environments running with high interrupt rate workload.(We tested 24 guests with iperf by 10Gb workload)
> 
> [Background]
>                    The assumptions about underlying hardware of OS running in the virtual machine environment would not hold for some cases. This is particularly perceived when considering the CPU virtualization that, the VCPU of the OS would be scheduled out while physical CPU of OS would never be. This cause the corner case trouble of OS designed inherently by the assumption targeting the physical CPU. We have seen the _lock-holder preemption_ case. Now SR-IOV issue is yet another one.
>          [Issue]
>                    Linux generic IRQ logic for edge interrupt, during the 'Writing EOI' period, has been written the way that in a high rate interrupt environment, the subsequent interrupt would cause the guest busy masking/unmasking interrupt if the previous one isn't handled immediately(For e.g. the guest is scheduled out).
> The mask/unmask operation would cause a read operation to flush the previous PCI transactions to ensure the write is successful. This corner case isn't handled by the Xen which only intercept the Guests' mask/unmask operation and forward other requests(read/write table) to qemu.
>                  This special case doesn't appear in the light workload but in the case of many (for e.g. 24) guests, it would cause the CPU utilization of Dom0 up to 140%(This is proportional to the number of the guests), which definitely limit the scalability and performance of virtualization technology.
>        [Effect]
>                  This patch emulates the read operation in the Xen and test showed that all the abnormal MMIO read operation is eliminated completely during iperf running in a heavy workload. The CPU utilization has been dropped to 60% in my test.

I am having a hard time understanding this.  Is the issue here that
read/write of the MSI-X table is being done in QEMU, and it is much
better to do so in the hypervisor which traps already the mask/unmaks
operation so that QEMU is not overwhelmed by having to do this?

With this in patch in place, wouldn't QEMU still do the read operation?

> 
> Thanks,
> Yuan

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] Simulates the MSIx table read operation
@ 2010-08-10 16:04 Liu Yuan
  0 siblings, 0 replies; 6+ messages in thread
From: Liu Yuan @ 2010-08-10 16:04 UTC (permalink / raw)
  To: konrad.wilk; +Cc: xen-devel


[-- Attachment #1.1: Type: text/plain, Size: 793 bytes --]

>I am having a hard time understanding this.  Is the issue here that
>read/write of the MSI-X table is being done in QEMU, and it is much
>better to do so in the hypervisor which traps already the mask/unmaks
>operation so that QEMU is not overwhelmed by having to do this?

Yes, having QEMU do this read operation,it needs much more CPU cycles
and longer latency.

[before patch]
read path: Guest -> Xen -> QEMU -> XEN -> Guest
[after patch]
read path: Guest -> Xen -> Guest

Xen traps all the MSIx table operation. And before this patch, only
mask/unmask is simulated in
Xen. With this patch applied, READ will be simulated and WRITE will
*still* be forwarded to QEMU.

>
>With this in patch in place, wouldn't QEMU still do the read operation?

No

Thanks,
Yuan

-- 
I am Multician, really

[-- Attachment #1.2: Type: text/html, Size: 935 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

^ permalink raw reply	[flat|nested] 6+ messages in thread

* RE: [PATCH]  Simulates the MSIx table read operation
@ 2010-08-04  9:08 Liu, Yuan B
  0 siblings, 0 replies; 6+ messages in thread
From: Liu, Yuan B @ 2010-08-04  9:08 UTC (permalink / raw)
  To: xen-devel


[-- Attachment #1.1: Type: text/plain, Size: 4871 bytes --]

Sorry, patch code style cleaned up and rebased to the latest tip

---

VMSI: This patch simulate the MSIx table read operation

Signed-off-by: Liu Yuan <yuan.b.liu@intel.com>
Signed-off-by: Eddie Dong <eddie.dong@intel.com>

diff -r 38aee6139719 xen/arch/x86/hvm/vmsi.c
--- a/xen/arch/x86/hvm/vmsi.c   Tue Aug 03 21:03:09 2010 +0100
+++ b/xen/arch/x86/hvm/vmsi.c   Wed Aug 04 17:01:23 2010 +0800
@@ -159,7 +159,10 @@
     unsigned long gtable;       /* gpa of msix table */
     unsigned long table_len;
     unsigned long table_flags[MAX_MSIX_TABLE_ENTRIES / BITS_PER_LONG + 1];
-
+#define MAX_MSIX_ACC_ENTRIES 3
+    struct {
+        uint32_t msi_ad[3];    /* Shadow of address low, high and data */
+    } gentries[MAX_MSIX_ACC_ENTRIES];
     struct rcu_head rcu;
 };

@@ -205,9 +208,10 @@
     struct vcpu *v, unsigned long address,
     unsigned long len, unsigned long *pval)
 {
-    unsigned long offset;
+    unsigned long offset, val;
     struct msixtbl_entry *entry;
     void *virt;
+    int nr_entry, index;
     int r = X86EMUL_UNHANDLEABLE;

     rcu_read_lock(&msixtbl_rcu_lock);
@@ -215,18 +219,29 @@
     if ( len != 4 )
         goto out;

-    offset = address & (PCI_MSIX_ENTRY_SIZE - 1);
-    if ( offset != PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET)
-        goto out;
-
     entry = msixtbl_find_entry(v, address);
     virt = msixtbl_addr_to_virt(entry, address);
     if ( !virt )
         goto out;

-    *pval = readl(virt);
+    nr_entry = (address - entry->gtable) / PCI_MSIX_ENTRY_SIZE;
+    offset = address & (PCI_MSIX_ENTRY_SIZE - 1);
+    if ( nr_entry >= MAX_MSIX_ACC_ENTRIES &&
+         offset != PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET )
+        goto out;
+
+    val = readl(virt);
+    if ( offset != PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET )
+    {
+        index = offset / sizeof(uint32_t);
+        *pval = entry->gentries[nr_entry].msi_ad[index];
+    }
+    else
+    {
+        *pval = val;
+    }
+
     r = X86EMUL_OKAY;
-
 out:
     rcu_read_unlock(&msixtbl_rcu_lock);
     return r;
@@ -238,7 +253,7 @@
     unsigned long offset;
     struct msixtbl_entry *entry;
     void *virt;
-    int nr_entry;
+    int nr_entry, index;
     int r = X86EMUL_UNHANDLEABLE;

     rcu_read_lock(&msixtbl_rcu_lock);
@@ -252,6 +267,11 @@
     offset = address & (PCI_MSIX_ENTRY_SIZE - 1);
     if ( offset != PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET)
     {
+        if ( nr_entry < MAX_MSIX_ACC_ENTRIES )
+        {
+            index = offset / sizeof(uint32_t);
+            entry->gentries[nr_entry].msi_ad[index] = val;
+        }
         set_bit(nr_entry, &entry->table_flags);
         goto out;
     }




From: Liu, Yuan B
Sent: Wednesday, August 04, 2010 10:35 AM
To: 'xen-devel@lists.xensource.com'
Cc: Dong, Eddie
Subject: [PATCH] Simulates the MSIx table read operation

Hi,
         This patch simulates the MSIx table read operation to avoid read traffic caused by guest Linux kernel in a multiple guests environments running with high interrupt rate workload.(We tested 24 guests with iperf by 10Gb workload)

[Background]
                   The assumptions about underlying hardware of OS running in the virtual machine environment would not hold for some cases. This is particularly perceived when considering the CPU virtualization that, the VCPU of the OS would be scheduled out while physical CPU of OS would never be. This cause the corner case trouble of OS designed inherently by the assumption targeting the physical CPU. We have seen the _lock-holder preemption_ case. Now SR-IOV issue is yet another one.
         [Issue]
                   Linux generic IRQ logic for edge interrupt, during the 'Writing EOI' period, has been written the way that in a high rate interrupt environment, the subsequent interrupt would cause the guest busy masking/unmasking interrupt if the previous one isn't handled immediately(For e.g. the guest is scheduled out).
The mask/unmask operation would cause a read operation to flush the previous PCI transactions to ensure the write is successful. This corner case isn't handled by the Xen which only intercept the Guests' mask/unmask operation and forward other requests(read/write table) to qemu.
                 This special case doesn't appear in the light workload but in the case of many (for e.g. 24) guests, it would cause the CPU utilization of Dom0 up to 140%(This is proportional to the number of the guests), which definitely limit the scalability and performance of virtualization technology.
       [Effect]
                 This patch emulates the read operation in the Xen and test showed that all the abnormal MMIO read operation is eliminated completely during iperf running in a heavy workload. The CPU utilization has been dropped to 60% in my test.

Thanks,
Yuan

[-- Attachment #1.2: Type: text/html, Size: 30190 bytes --]

[-- Attachment #2: msi-mask-table.patch --]
[-- Type: application/octet-stream, Size: 2556 bytes --]

VMSI: This patch simulate the MSIx table read operation

Signed-off-by: Liu Yuan <yuan.b.liu@intel.com>
Signed-off-by: Eddie Dong <eddie.dong@intel.com>

diff -r 38aee6139719 xen/arch/x86/hvm/vmsi.c
--- a/xen/arch/x86/hvm/vmsi.c	Tue Aug 03 21:03:09 2010 +0100
+++ b/xen/arch/x86/hvm/vmsi.c	Wed Aug 04 17:01:23 2010 +0800
@@ -159,7 +159,10 @@
     unsigned long gtable;       /* gpa of msix table */
     unsigned long table_len;
     unsigned long table_flags[MAX_MSIX_TABLE_ENTRIES / BITS_PER_LONG + 1];
-
+#define MAX_MSIX_ACC_ENTRIES 3
+    struct { 
+        uint32_t msi_ad[3];	/* Shadow of address low, high and data */
+    } gentries[MAX_MSIX_ACC_ENTRIES];
     struct rcu_head rcu;
 };
 
@@ -205,9 +208,10 @@
     struct vcpu *v, unsigned long address,
     unsigned long len, unsigned long *pval)
 {
-    unsigned long offset;
+    unsigned long offset, val;
     struct msixtbl_entry *entry;
     void *virt;
+    int nr_entry, index;
     int r = X86EMUL_UNHANDLEABLE;
 
     rcu_read_lock(&msixtbl_rcu_lock);
@@ -215,18 +219,29 @@
     if ( len != 4 )
         goto out;
 
-    offset = address & (PCI_MSIX_ENTRY_SIZE - 1);
-    if ( offset != PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET)
-        goto out;
-
     entry = msixtbl_find_entry(v, address);
     virt = msixtbl_addr_to_virt(entry, address);
     if ( !virt )
         goto out;
 
-    *pval = readl(virt);
+    nr_entry = (address - entry->gtable) / PCI_MSIX_ENTRY_SIZE;
+    offset = address & (PCI_MSIX_ENTRY_SIZE - 1);
+    if ( nr_entry >= MAX_MSIX_ACC_ENTRIES && 
+         offset != PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET )
+        goto out;
+
+    val = readl(virt);
+    if ( offset != PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET )
+    {
+        index = offset / sizeof(uint32_t);
+        *pval = entry->gentries[nr_entry].msi_ad[index];
+    }
+    else 
+    {
+        *pval = val;
+    }
+    
     r = X86EMUL_OKAY;
-
 out:
     rcu_read_unlock(&msixtbl_rcu_lock);
     return r;
@@ -238,7 +253,7 @@
     unsigned long offset;
     struct msixtbl_entry *entry;
     void *virt;
-    int nr_entry;
+    int nr_entry, index;
     int r = X86EMUL_UNHANDLEABLE;
 
     rcu_read_lock(&msixtbl_rcu_lock);
@@ -252,6 +267,11 @@
     offset = address & (PCI_MSIX_ENTRY_SIZE - 1);
     if ( offset != PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET)
     {
+        if ( nr_entry < MAX_MSIX_ACC_ENTRIES ) 
+        {
+            index = offset / sizeof(uint32_t);
+            entry->gentries[nr_entry].msi_ad[index] = val;
+        }
         set_bit(nr_entry, &entry->table_flags);
         goto out;
     }

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

^ permalink raw reply	[flat|nested] 6+ messages in thread

* RE: [PATCH]  Simulates the MSIx table read operation
@ 2010-08-04  8:12 Liu, Yuan B
  0 siblings, 0 replies; 6+ messages in thread
From: Liu, Yuan B @ 2010-08-04  8:12 UTC (permalink / raw)
  To: xen-devel; +Cc: Dong, Eddie


[-- Attachment #1.1: Type: text/plain, Size: 2601 bytes --]

Rebased the patch and simplified some logic.

Regards,
Yuan

----

VMSI: This patch simulate the MSIx table read operation

Signed-off-by: Liu Yuan <yuan.b.liu@intel.com>
Signed-off-by: Eddie Dong <eddie.dong@intel.com>

diff -r ef5f25de00da xen/arch/x86/hvm/vmsi.c
--- a/xen/arch/x86/hvm/vmsi.c   Mon Aug 02 17:19:06 2010 +0100
+++ b/xen/arch/x86/hvm/vmsi.c   Wed Aug 04 16:04:49 2010 +0800
@@ -158,7 +158,10 @@
     unsigned long gtable;       /* gpa of msix table */
     unsigned long table_len;
     unsigned long table_flags[MAX_MSIX_TABLE_ENTRIES / BITS_PER_LONG + 1];
-
+#define MAX_MSIX_ACC_ENTRIES 3
+    struct {
+        uint32_t msi_ad[3];    /* Shadow of address low, high and data */
+    } gentries[MAX_MSIX_ACC_ENTRIES];
     struct rcu_head rcu;
 };

@@ -202,9 +205,10 @@
     struct vcpu *v, unsigned long address,
     unsigned long len, unsigned long *pval)
 {
-    unsigned long offset;
+    unsigned long offset, val;
     struct msixtbl_entry *entry;
     void *virt;
+    int nr_entry, index;
     int r = X86EMUL_UNHANDLEABLE;

     rcu_read_lock();
@@ -212,16 +216,27 @@
     if ( len != 4 )
         goto out;

-    offset = address & (PCI_MSIX_ENTRY_SIZE - 1);
-    if ( offset != PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET)
-        goto out;
-
     entry = msixtbl_find_entry(v, address);
     virt = msixtbl_addr_to_virt(entry, address);
     if ( !virt )
         goto out;

-    *pval = readl(virt);
+    nr_entry = (address - entry->gtable) / PCI_MSIX_ENTRY_SIZE;
+    offset = address & (PCI_MSIX_ENTRY_SIZE - 1);
+    if (nr_entry >= MAX_MSIX_ACC_ENTRIES &&
+        offset != PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET)
+        goto out;
+
+    val = readl(virt);
+    if ( offset != PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET){
+        index = offset / sizeof(uint32_t);
+        *pval = entry->gentries[nr_entry].msi_ad[index];
+    }
+    else
+    {
+        *pval = val;
+    }
+
     r = X86EMUL_OKAY;

 out:
@@ -235,7 +250,7 @@
     unsigned long offset;
     struct msixtbl_entry *entry;
     void *virt;
-    int nr_entry;
+    int nr_entry, index;
     int r = X86EMUL_UNHANDLEABLE;

     rcu_read_lock();
@@ -249,6 +264,10 @@
     offset = address & (PCI_MSIX_ENTRY_SIZE - 1);
     if ( offset != PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET)
     {
+        if (nr_entry < MAX_MSIX_ACC_ENTRIES) {
+            index = offset / sizeof(uint32_t);
+            entry->gentries[nr_entry].msi_ad[index] = val;
+        }
         set_bit(nr_entry, &entry->table_flags);
         goto out;
     }



[-- Attachment #1.2: Type: text/html, Size: 24507 bytes --]

[-- Attachment #2: msi-mask-table.patch --]
[-- Type: application/octet-stream, Size: 2444 bytes --]

VMSI: This patch simulate the MSIx table read operation

Signed-off-by: Liu Yuan <yuan.b.liu@intel.com>
Signed-off-by: Eddie Dong <eddie.dong@intel.com>

diff -r ef5f25de00da xen/arch/x86/hvm/vmsi.c
--- a/xen/arch/x86/hvm/vmsi.c	Mon Aug 02 17:19:06 2010 +0100
+++ b/xen/arch/x86/hvm/vmsi.c	Wed Aug 04 16:04:49 2010 +0800
@@ -158,7 +158,10 @@
     unsigned long gtable;       /* gpa of msix table */
     unsigned long table_len;
     unsigned long table_flags[MAX_MSIX_TABLE_ENTRIES / BITS_PER_LONG + 1];
-
+#define MAX_MSIX_ACC_ENTRIES 3
+    struct { 
+        uint32_t msi_ad[3];	/* Shadow of address low, high and data */
+    } gentries[MAX_MSIX_ACC_ENTRIES];
     struct rcu_head rcu;
 };
 
@@ -202,9 +205,10 @@
     struct vcpu *v, unsigned long address,
     unsigned long len, unsigned long *pval)
 {
-    unsigned long offset;
+    unsigned long offset, val;
     struct msixtbl_entry *entry;
     void *virt;
+    int nr_entry, index;
     int r = X86EMUL_UNHANDLEABLE;
 
     rcu_read_lock();
@@ -212,16 +216,27 @@
     if ( len != 4 )
         goto out;
 
-    offset = address & (PCI_MSIX_ENTRY_SIZE - 1);
-    if ( offset != PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET)
-        goto out;
-
     entry = msixtbl_find_entry(v, address);
     virt = msixtbl_addr_to_virt(entry, address);
     if ( !virt )
         goto out;
 
-    *pval = readl(virt);
+    nr_entry = (address - entry->gtable) / PCI_MSIX_ENTRY_SIZE;
+    offset = address & (PCI_MSIX_ENTRY_SIZE - 1);
+    if (nr_entry >= MAX_MSIX_ACC_ENTRIES && 
+        offset != PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET)
+        goto out;
+
+    val = readl(virt);
+    if ( offset != PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET){
+        index = offset / sizeof(uint32_t);
+        *pval = entry->gentries[nr_entry].msi_ad[index];
+    }
+    else 
+    {
+        *pval = val;
+    }
+    
     r = X86EMUL_OKAY;
 
 out:
@@ -235,7 +250,7 @@
     unsigned long offset;
     struct msixtbl_entry *entry;
     void *virt;
-    int nr_entry;
+    int nr_entry, index;
     int r = X86EMUL_UNHANDLEABLE;
 
     rcu_read_lock();
@@ -249,6 +264,10 @@
     offset = address & (PCI_MSIX_ENTRY_SIZE - 1);
     if ( offset != PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET)
     {
+        if (nr_entry < MAX_MSIX_ACC_ENTRIES) {
+            index = offset / sizeof(uint32_t);
+            entry->gentries[nr_entry].msi_ad[index] = val;
+        }
         set_bit(nr_entry, &entry->table_flags);
         goto out;
     }

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

^ permalink raw reply	[flat|nested] 6+ messages in thread

* RE: [PATCH]  Simulates the MSIx table read operation
@ 2010-08-04  2:36 Liu, Yuan B
  0 siblings, 0 replies; 6+ messages in thread
From: Liu, Yuan B @ 2010-08-04  2:36 UTC (permalink / raw)
  To: xen-devel; +Cc: Dong, Eddie


[-- Attachment #1.1: Type: text/plain, Size: 2529 bytes --]

VMSI: This patch simulates the MSIx table read operation

Signed-off-by: Liu Yuan <yuan.b.liu@intel.com>
Signed-off-by: Eddie Dong <eddie.dong@intel.com>

diff -r 114f860872fb xen/arch/x86/hvm/vmsi.c
--- a/xen/arch/x86/hvm/vmsi.c
+++ b/xen/arch/x86/hvm/vmsi.c
@@ -158,7 +158,10 @@ struct msixtbl_entry
     unsigned long gtable;       /* gpa of msix table */
     unsigned long table_len;
     unsigned long table_flags[MAX_MSIX_TABLE_ENTRIES / BITS_PER_LONG + 1];
-
+#define MAX_MSIX_ACC_ENTRIES 3
+    struct {
+        uint32_t msi_ad[3];    /* Shadow of address low, high and data */
+    } gentries[MAX_MSIX_ACC_ENTRIES];
     struct rcu_head rcu;
 };

@@ -205,6 +208,7 @@ static int msixtbl_read(
     unsigned long offset;
     struct msixtbl_entry *entry;
     void *virt;
+    int nr_entry, index;
     int r = X86EMUL_UNHANDLEABLE;

     rcu_read_lock();
@@ -212,14 +216,22 @@ static int msixtbl_read(
     if ( len != 4 )
         goto out;

-    offset = address & (PCI_MSIX_ENTRY_SIZE - 1);
-    if ( offset != PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET)
-        goto out;
-
-    entry = msixtbl_find_entry(v, address);
     virt = msixtbl_addr_to_virt(entry, address);
     if ( !virt )
         goto out;
+
+    offset = address & (PCI_MSIX_ENTRY_SIZE - 1);
+    entry = msixtbl_find_entry(v, address);
+    if ( offset != PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET){
+        nr_entry = (address - entry->gtable) / PCI_MSIX_ENTRY_SIZE;
+        index = offset / sizeof(uint32_t);
+        if (nr_entry < MAX_MSIX_ACC_ENTRIES) {
+            *pval = entry->gentries[nr_entry].msi_ad[index];
+            readl(virt); /* Linux kernel needs this to flush the write */
+            r = X86EMUL_OKAY;
+        }
+        goto out;
+    }

     *pval = readl(virt);
     r = X86EMUL_OKAY;
@@ -235,7 +247,7 @@ static int msixtbl_write(struct vcpu *v,
     unsigned long offset;
     struct msixtbl_entry *entry;
     void *virt;
-    int nr_entry;
+    int nr_entry, index;
     int r = X86EMUL_UNHANDLEABLE;

     rcu_read_lock();
@@ -249,6 +261,10 @@ static int msixtbl_write(struct vcpu *v,
     offset = address & (PCI_MSIX_ENTRY_SIZE - 1);
     if ( offset != PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET)
     {
+        if (nr_entry < MAX_MSIX_ACC_ENTRIES) {
+                index = offset / sizeof(uint32_t);
+                entry->gentries[nr_entry].msi_ad[index] = val;
+        }
         set_bit(nr_entry, &entry->table_flags);
         goto out;
     }



[-- Attachment #1.2: Type: text/html, Size: 21693 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2010-08-10 16:04 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2010-08-04  2:35 [PATCH] Simulates the MSIx table read operation Liu, Yuan B
2010-08-06 15:01 ` Konrad Rzeszutek Wilk
2010-08-04  2:36 Liu, Yuan B
2010-08-04  8:12 Liu, Yuan B
2010-08-04  9:08 Liu, Yuan B
2010-08-10 16:04 Liu Yuan

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.