From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751381AbdFAGxO (ORCPT ); Thu, 1 Jun 2017 02:53:14 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:47002 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751053AbdFAGxM (ORCPT ); Thu, 1 Jun 2017 02:53:12 -0400 Date: Thu, 1 Jun 2017 09:53:02 +0300 From: Mike Rapoport To: Michal Hocko Cc: Andrea Arcangeli , Vlastimil Babka , "Kirill A. Shutemov" , Andrew Morton , Arnd Bergmann , "Kirill A. Shutemov" , Pavel Emelyanov , linux-mm , lkml , Linux API Subject: Re: [PATCH] mm: introduce MADV_CLR_HUGEPAGE References: <20170524075043.GB3063@rapoport-lnx> <20170524103947.GC3063@rapoport-lnx> <20170524111800.GD14733@dhcp22.suse.cz> <20170524142735.GF3063@rapoport-lnx> <20170530074408.GA7969@dhcp22.suse.cz> <20170530101921.GA25738@rapoport-lnx> <20170530103930.GB7969@dhcp22.suse.cz> <20170530140456.GA8412@redhat.com> <20170530143941.GK7969@dhcp22.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170530143941.GK7969@dhcp22.suse.cz> User-Agent: Mutt/1.5.24 (2015-08-30) X-TM-AS-GCONF: 00 x-cbid: 17060106-0040-0000-0000-0000039E96E4 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17060106-0041-0000-0000-00002592DD95 Message-Id: <20170601065302.GA30495@rapoport-lnx> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-06-01_01:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1706010126 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, May 30, 2017 at 04:39:41PM +0200, Michal Hocko wrote: > On Tue 30-05-17 16:04:56, Andrea Arcangeli wrote: > > > > UFFDIO_COPY while not being a major slowdown for sure, it's likely > > measurable at the microbenchmark level because it would add a > > enter/exit kernel to every 4k memcpy. It's not hard to imagine that as > > measurable. How that impacts the total precopy time I don't know, it > > would need to be benchmarked to be sure. > > Yes, please! I've run a simple test (below) that fills 1G of memory either with memcpy of ioctl(UFFDIO_COPY) in 4K chunks. The machine I used has two "Intel(R) Xeon(R) CPU E5-2680 0 @ 2.70GHz" and 128G of RAM. I've averaged elapsed time reported by /usr/bin/time over 100 runs and here what I've got: memcpy with THP on: 0.3278 sec memcpy with THP off: 0.5295 sec UFFDIO_COPY: 0.44 sec That said, for the CRIU usecase UFFDIO_COPY seems faster that disabling THP and then doing memcpy. -- Sincerely yours, Mike. ---------------------------------------------------------- { ... src = mmap(NULL, page_size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); if (src == MAP_FAILED) fprintf(stderr, "map src failed\n"), exit(1); *((unsigned long *)src) = 1; if (disable_huge && prctl(PR_SET_THP_DISABLE, 1, 0, 0, 0)) fprintf(stderr, "ptctl failed\n"), exit(1); dst = mmap(NULL, page_size * nr_pages, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); if (dst == MAP_FAILED) fprintf(stderr, "map dst failed\n"), exit(1); if (use_uffd && userfaultfd_register(dst)) fprintf(stderr, "userfault_register failed\n"), exit(1); for (i = 0; i < nr_pages; i++) { char *address = dst + i * page_size; if (use_uffd) { struct uffdio_copy uffdio_copy; uffdio_copy.dst = (unsigned long)address; uffdio_copy.src = (unsigned long)src; uffdio_copy.len = page_size; uffdio_copy.mode = 0; uffdio_copy.copy = 0; ret = ioctl(uffd, UFFDIO_COPY, &uffdio_copy); if (ret) fprintf(stderr, "copy: %d, %d\n", ret, errno), exit(1); } else { memcpy(address, src, page_size); } } return 0; } From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mike Rapoport Subject: Re: [PATCH] mm: introduce MADV_CLR_HUGEPAGE Date: Thu, 1 Jun 2017 09:53:02 +0300 Message-ID: <20170601065302.GA30495@rapoport-lnx> References: <20170524075043.GB3063@rapoport-lnx> <20170524103947.GC3063@rapoport-lnx> <20170524111800.GD14733@dhcp22.suse.cz> <20170524142735.GF3063@rapoport-lnx> <20170530074408.GA7969@dhcp22.suse.cz> <20170530101921.GA25738@rapoport-lnx> <20170530103930.GB7969@dhcp22.suse.cz> <20170530140456.GA8412@redhat.com> <20170530143941.GK7969@dhcp22.suse.cz> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Content-Disposition: inline In-Reply-To: <20170530143941.GK7969@dhcp22.suse.cz> Sender: owner-linux-mm@kvack.org To: Michal Hocko Cc: Andrea Arcangeli , Vlastimil Babka , "Kirill A. Shutemov" , Andrew Morton , Arnd Bergmann , "Kirill A. Shutemov" , Pavel Emelyanov , linux-mm , lkml , Linux API List-Id: linux-api@vger.kernel.org On Tue, May 30, 2017 at 04:39:41PM +0200, Michal Hocko wrote: > On Tue 30-05-17 16:04:56, Andrea Arcangeli wrote: > > > > UFFDIO_COPY while not being a major slowdown for sure, it's likely > > measurable at the microbenchmark level because it would add a > > enter/exit kernel to every 4k memcpy. It's not hard to imagine that as > > measurable. How that impacts the total precopy time I don't know, it > > would need to be benchmarked to be sure. > > Yes, please! I've run a simple test (below) that fills 1G of memory either with memcpy of ioctl(UFFDIO_COPY) in 4K chunks. The machine I used has two "Intel(R) Xeon(R) CPU E5-2680 0 @ 2.70GHz" and 128G of RAM. I've averaged elapsed time reported by /usr/bin/time over 100 runs and here what I've got: memcpy with THP on: 0.3278 sec memcpy with THP off: 0.5295 sec UFFDIO_COPY: 0.44 sec That said, for the CRIU usecase UFFDIO_COPY seems faster that disabling THP and then doing memcpy. -- Sincerely yours, Mike. ---------------------------------------------------------- { ... src = mmap(NULL, page_size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); if (src == MAP_FAILED) fprintf(stderr, "map src failed\n"), exit(1); *((unsigned long *)src) = 1; if (disable_huge && prctl(PR_SET_THP_DISABLE, 1, 0, 0, 0)) fprintf(stderr, "ptctl failed\n"), exit(1); dst = mmap(NULL, page_size * nr_pages, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); if (dst == MAP_FAILED) fprintf(stderr, "map dst failed\n"), exit(1); if (use_uffd && userfaultfd_register(dst)) fprintf(stderr, "userfault_register failed\n"), exit(1); for (i = 0; i < nr_pages; i++) { char *address = dst + i * page_size; if (use_uffd) { struct uffdio_copy uffdio_copy; uffdio_copy.dst = (unsigned long)address; uffdio_copy.src = (unsigned long)src; uffdio_copy.len = page_size; uffdio_copy.mode = 0; uffdio_copy.copy = 0; ret = ioctl(uffd, UFFDIO_COPY, &uffdio_copy); if (ret) fprintf(stderr, "copy: %d, %d\n", ret, errno), exit(1); } else { memcpy(address, src, page_size); } } return 0; } -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org