From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752739AbdKFJOx (ORCPT ); Mon, 6 Nov 2017 04:14:53 -0500 Received: from mx1.redhat.com ([209.132.183.28]:58590 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752651AbdKFJOs (ORCPT ); Mon, 6 Nov 2017 04:14:48 -0500 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com BFCE97EA90 Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=vkuznets@redhat.com From: Vitaly Kuznetsov To: Wanpeng Li Cc: "the arch\/x86 maintainers" , devel@linuxdriverproject.org, "linux-kernel\@vger.kernel.org" , "K. Y. Srinivasan" , Haiyang Zhang , Stephen Hemminger , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , Steven Rostedt , Jork Loeser , Simon Xiao , Andy Lutomirski , Andy Shevchenko , Paolo Bonzini , kvm Subject: Re: [PATCH v10 0/9] Hyper-V: paravirtualized remote TLB flushing and hypercall improvements References: <20170802160921.21791-1-vkuznets@redhat.com> Date: Mon, 06 Nov 2017 10:14:41 +0100 In-Reply-To: (Wanpeng Li's message of "Mon, 6 Nov 2017 16:43:51 +0800") Message-ID: <87d14vrcou.fsf@vitty.brq.redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Mon, 06 Nov 2017 09:14:48 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org --=-=-= Content-Type: text/plain Wanpeng Li writes: > 2017-08-03 0:09 GMT+08:00 Vitaly Kuznetsov : >> Changes since v9: >> - Rebase to 4.13-rc3. >> - Drop PATCH1 as it was already taken by Greg to char-misc tree. There're no >> functional dependencies on this patch so the series can go through a different tree >> (and it actually belongs to x86 if I got Ingo's comment right). >> - Add in missing void return type in PATCH1 [Colin King, Ingo Molnar, Greg KH] >> - A few minor fixes in what is now PATCH7: add pr_fmt, tiny style fix in >> hyperv_flush_tlb_others() [Andy Shevchenko] >> - Fix "error: implicit declaration of function 'virt_to_phys'" in PATCH2 >> reported by kbuild test robot (#include ) >> - Add Steven's 'Reviewed-by:' to PATCH9. >> >> Original description: >> >> Hyper-V supports hypercalls for doing local and remote TLB flushing and >> gives its guests hints when using hypercall is preferred. While doing >> hypercalls for local TLB flushes is probably not practical (and is not >> being suggested by modern Hyper-V versions) remote TLB flush with a >> hypercall brings significant improvement. >> >> To test the series I wrote a special 'TLB trasher': on a 16 vCPU guest I >> was creating 32 threads which were doing 100000 mmap/munmaps each on some >> big file. Here are the results: >> >> Before: >> # time ./pthread_mmap ./randfile >> real 3m33.118s >> user 0m3.698s >> sys 3m16.624s >> >> After: >> # time ./pthread_mmap ./randfile >> real 2m19.920s >> user 0m2.662s >> sys 2m9.948s >> >> This series brings a number of small improvements along the way: fast >> hypercall implementation and using it for event signaling, rep hypercalls >> implementation, hyperv tracing subsystem (which only traces the newly added >> remote TLB flush for now). >> > > Hi Vitaly, > > Could you attach your benchmark? I'm interested in to try the > implementation in paravirt kvm. > Oh, this would be cool) I briefly discussed the idea with Radim (one of KVM maintainers) during the last KVM Forum and he wasn't opposed to the idea. Need to talk to Paolo too. Good thing is that we have everything in place for guests now (HAVE_RCU_TABLE_FREE is enabled globaly on x86). Please see the microbenchmark attached. Adjust defines in the beginning to match your needs. It is not anything smart, basically just a TLB trasher. In theory, the best result is achived when we're overcommiting the host by running multiple vCPUs on each pCPU. In this case PV tlb flush avoids touching vCPUs which are not scheduled and avoid the wait on the main CPU. -- Vitaly --=-=-= Content-Type: text/plain Content-Disposition: inline; filename=pthread_mmap.c #include #include #include #include #include #include #include #include #include #define nthreads 48 #define pagecount 16384 #define nrounds 1000 #define nchunks 20 #define PAGE_SIZE 4096 int fd; unsigned long v; void *threadf(void *ptr) { unsigned long *addr[nchunks]; int i, j, k; struct timespec ts = {0}; int ret; ts.tv_nsec = random() % 1024; for (j = 0; j < nrounds; j++) { for (i = 0; i < nchunks; i++) { addr[i] = mmap(NULL, PAGE_SIZE * pagecount, PROT_READ, MAP_SHARED, fd, i * PAGE_SIZE); if (addr[i] == MAP_FAILED) { fprintf(stderr, "mmap\n"); exit(1); } } nanosleep(&ts, NULL); for (i = 0; i < nchunks; i++) { v += *addr[i]; } nanosleep(&ts, NULL); for (i = 0; i < nchunks; i++) { munmap(addr[i], PAGE_SIZE * pagecount); } } } int main(int argc, char *argv[]) { pthread_t thr[nthreads]; int i; srandom(time(NULL)); if (argc < 2) { fprintf(stderr, "usage: %s \n", argv[0]); exit(1); } fd = open(argv[1], O_RDONLY); if (fd < 0) { fprintf(stderr, "open\n"); exit(1); } for (i = 0; i < nthreads; i++) { if(pthread_create(&thr[i], NULL, threadf, NULL)) { fprintf(stderr, "pthread_create\n"); exit(1); } } for (i = 0; i < nthreads; i++) { if(pthread_join(thr[i], NULL)) { fprintf(stderr, "pthread_join\n"); exit(1); } } return 0; } --=-=-=--