From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58002) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZwtV9-0007ab-4U for qemu-devel@nongnu.org; Thu, 12 Nov 2015 10:09:28 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZwtV4-00080w-Ok for qemu-devel@nongnu.org; Thu, 12 Nov 2015 10:09:27 -0500 Received: from mx1.redhat.com ([209.132.183.28]:43871) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZwtV4-00080l-Hc for qemu-devel@nongnu.org; Thu, 12 Nov 2015 10:09:22 -0500 Date: Thu, 12 Nov 2015 17:09:19 +0200 From: "Michael S. Tsirkin" Message-ID: <1447340097-18216-5-git-send-email-mst@redhat.com> References: <1447340097-18216-1-git-send-email-mst@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1447340097-18216-1-git-send-email-mst@redhat.com> Subject: [Qemu-devel] [PULL 04/11] virtio-pci: use zero length mmio eventfd for 1.0 notification cap when possible List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: Peter Maydell , Jason Wang From: Jason Wang We use data match eventfd for 1.0 notification currently. This could be slow since software decoding is needed for mmio exit. To speed this up, we can switch to use zero length mmio eventfd for 1.0 notification since we can examine the queue index directly from the writing address. KVM kernel module can utilize this by registering it to fast mmio bus which could be as fast as pio on ept capable machine when fast mmio is supported by host kernel. Lots of improvements were seen on a ept capable machine: Guest RX:(TCP) size/session/+throughput%/+cpu%/-+per cpu%/ 64/1/+1.6807%/[-16.2421%]/[+21.3984%]/ 64/2/+0.6091%/[-11.0187%]/[+13.0678%]/ 64/4/+0.0553%/[-5.9768%]/[+6.4155%]/ 64/8/+0.1206%/[-4.0057%]/[+4.2984%]/ 256/1/-0.0031%/[-10.1166%]/[+11.2517%]/ 256/2/-0.5058%/[-6.1656%]/+6.0317%]/ ... Guest TX:(TCP) size/session/+throughput%/+cpu%/-+per cpu%/ 64/1/[+18.9183%]/-0.2823%/[+19.2550%]/ 64/2/[+13.5714%]/[+2.2675%]/[+11.0533%]/ 64/4/[+13.1070%]/[+2.1817%]/[+10.6920%]/ 64/8/[+13.0426%]/[+2.0887%]/[+10.7299%]/ 256/1/[+36.2761%]/+6.3434%/[+28.1471%]/ ... 1024/1/[+44.8873%]/+2.0811%/[+41.9335%]/ ... 1024/4/+0.0228%/[-2.2044%]/[+2.2774%]/ ... 16384/2/+0.0127%/[-5.0346%]/[+5.3148%]/ ... 65535/1/[+0.0062%]/[-4.1183%]/[+4.3017%]/ 65535/2/+0.0004%/[-4.2311%]/[+4.4185%]/ 65535/4/+0.0107%/[-4.6106%]/[+4.8446%]/ 65535/8/-0.0090%/[-5.5178%]/[+5.8306%]/ Latency:(TCP_RR) size/session/+transaction rate%/+cpu%/-+per cpu%/ 64/1/[+6.5248%]/[-9.2882%]/[+17.4322%]/ 64/25/[+11.0854%]/[+0.8000%]/[+10.2038%]/ 64/50/[+12.1076%]/[+2.4627%]/[+9.4131%]/ 256/1/[+5.3677%]/[+10.5669%]/-4.7024%/ 256/25/[+5.6402%]/-0.8962%/[+6.5955%]/ 256/50/[+5.9685%]/[+1.7766%]/[+4.1188%]/ 4096/1/+0.2508%/[-10.4941%]/[+12.0047%]/ 4096/25/[+1.8533%]/-0.0273%/+1.8812%/ 4096/50/[+1.2156%]/-1.4134%/+2.6667%/ Notes: data with '[]' is the one whose significance is greater than 95%. Thanks Wenli Quan for the benchmarking. Cc: Michael S. Tsirkin Signed-off-by: Jason Wang Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- hw/virtio/virtio-pci.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c index ab2a372..9c3f6e3 100644 --- a/hw/virtio/virtio-pci.c +++ b/hw/virtio/virtio-pci.c @@ -270,6 +270,7 @@ static int virtio_pci_set_host_notifier_internal(VirtIOPCIProxy *proxy, EventNotifier *notifier = virtio_queue_get_host_notifier(vq); bool legacy = !(proxy->flags & VIRTIO_PCI_FLAG_DISABLE_LEGACY); bool modern = !(proxy->flags & VIRTIO_PCI_FLAG_DISABLE_MODERN); + bool fast_mmio = kvm_ioeventfd_any_length_enabled(); MemoryRegion *modern_mr = &proxy->notify.mr; MemoryRegion *legacy_mr = &proxy->bar; hwaddr modern_addr = QEMU_VIRTIO_PCI_QUEUE_MEM_MULT * @@ -286,8 +287,13 @@ static int virtio_pci_set_host_notifier_internal(VirtIOPCIProxy *proxy, } virtio_queue_set_host_notifier_fd_handler(vq, true, set_handler); if (modern) { - memory_region_add_eventfd(modern_mr, modern_addr, 2, - true, n, notifier); + if (fast_mmio) { + memory_region_add_eventfd(modern_mr, modern_addr, 0, + false, n, notifier); + } else { + memory_region_add_eventfd(modern_mr, modern_addr, 2, + false, n, notifier); + } } if (legacy) { memory_region_add_eventfd(legacy_mr, legacy_addr, 2, @@ -295,8 +301,13 @@ static int virtio_pci_set_host_notifier_internal(VirtIOPCIProxy *proxy, } } else { if (modern) { - memory_region_del_eventfd(modern_mr, modern_addr, 2, - true, n, notifier); + if (fast_mmio) { + memory_region_del_eventfd(modern_mr, modern_addr, 0, + false, n, notifier); + } else { + memory_region_del_eventfd(modern_mr, modern_addr, 2, + false, n, notifier); + } } if (legacy) { memory_region_del_eventfd(legacy_mr, legacy_addr, 2, -- MST