* [PATCH RFC v2 0/2] Reduce DPDK initialization time @ 2015-11-20 1:53 Zhihong Wang 2015-11-20 1:53 ` [PATCH RFC v2 1/2] lib/librte_eal: Reduce timer " Zhihong Wang 2015-11-20 1:53 ` [PATCH RFC v2 2/2] lib/librte_eal: Remove unnecessary hugepage zero-filling Zhihong Wang 0 siblings, 2 replies; 7+ messages in thread From: Zhihong Wang @ 2015-11-20 1:53 UTC (permalink / raw) To: dev This RFC patch aims to reduce DPDK initialization time, which is important in cases such as micro service. Changes are: 1. Reduce timer initialization time 2. Remove unnecessary hugepage zero-filling operations With this patch: 1. Timer initialization time can be reduced by 4/10 second 2. Memory initialization time can be reduced nearly by half The 2nd topic has been brought up before in this thread: http://dpdk.org/dev/patchwork/patch/4219/ -------------- Changes in v2: 1. Use MAP_POPULATE flag to populate page tables 2. Add comments to avoid future misunderstanding Zhihong Wang (2): lib/librte_eal: Reduce timer initialization time lib/librte_eal: Remove unnecessary hugepage zero-filling lib/librte_eal/linuxapp/eal/eal_memory.c | 20 ++++++-------------- lib/librte_eal/linuxapp/eal/eal_timer.c | 2 +- 2 files changed, 7 insertions(+), 15 deletions(-) -- 2.5.0 ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH RFC v2 1/2] lib/librte_eal: Reduce timer initialization time 2015-11-20 1:53 [PATCH RFC v2 0/2] Reduce DPDK initialization time Zhihong Wang @ 2015-11-20 1:53 ` Zhihong Wang 2015-11-20 16:31 ` Stephen Hemminger 2015-11-20 1:53 ` [PATCH RFC v2 2/2] lib/librte_eal: Remove unnecessary hugepage zero-filling Zhihong Wang 1 sibling, 1 reply; 7+ messages in thread From: Zhihong Wang @ 2015-11-20 1:53 UTC (permalink / raw) To: dev Changing from 1/2 second to 1/10 doesn't compromise the precision, and a 4/10 second is worth saving. Signed-off-by: Zhihong Wang <zhihong.wang@intel.com> --- lib/librte_eal/linuxapp/eal/eal_timer.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/librte_eal/linuxapp/eal/eal_timer.c b/lib/librte_eal/linuxapp/eal/eal_timer.c index e0642de..4de0353 100644 --- a/lib/librte_eal/linuxapp/eal/eal_timer.c +++ b/lib/librte_eal/linuxapp/eal/eal_timer.c @@ -271,7 +271,7 @@ get_tsc_freq(void) #ifdef CLOCK_MONOTONIC_RAW #define NS_PER_SEC 1E9 - struct timespec sleeptime = {.tv_nsec = 5E8 }; /* 1/2 second */ + struct timespec sleeptime = {.tv_nsec = 1E8 }; /* 1/10 second */ struct timespec t_start, t_end; uint64_t tsc_hz; -- 2.5.0 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH RFC v2 1/2] lib/librte_eal: Reduce timer initialization time 2015-11-20 1:53 ` [PATCH RFC v2 1/2] lib/librte_eal: Reduce timer " Zhihong Wang @ 2015-11-20 16:31 ` Stephen Hemminger 2015-11-23 1:29 ` Wang, Zhihong 0 siblings, 1 reply; 7+ messages in thread From: Stephen Hemminger @ 2015-11-20 16:31 UTC (permalink / raw) To: Zhihong Wang; +Cc: dev On Thu, 19 Nov 2015 20:53:47 -0500 Zhihong Wang <zhihong.wang@intel.com> wrote: > + struct timespec sleeptime = {.tv_nsec = 1E8 }; /* 1/10 second */ Why not NSEC_PER_SEC / 10 ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH RFC v2 1/2] lib/librte_eal: Reduce timer initialization time 2015-11-20 16:31 ` Stephen Hemminger @ 2015-11-23 1:29 ` Wang, Zhihong 0 siblings, 0 replies; 7+ messages in thread From: Wang, Zhihong @ 2015-11-23 1:29 UTC (permalink / raw) To: Stephen Hemminger; +Cc: dev > -----Original Message----- > From: Stephen Hemminger [mailto:stephen@networkplumber.org] > Sent: Saturday, November 21, 2015 12:32 AM > To: Wang, Zhihong <zhihong.wang@intel.com> > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH RFC v2 1/2] lib/librte_eal: Reduce timer > initialization time > > On Thu, 19 Nov 2015 20:53:47 -0500 > Zhihong Wang <zhihong.wang@intel.com> wrote: > > > + struct timespec sleeptime = {.tv_nsec = 1E8 }; /* 1/10 second */ > > Why not NSEC_PER_SEC / 10 Yes, it looks better using macro :) ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH RFC v2 2/2] lib/librte_eal: Remove unnecessary hugepage zero-filling 2015-11-20 1:53 [PATCH RFC v2 0/2] Reduce DPDK initialization time Zhihong Wang 2015-11-20 1:53 ` [PATCH RFC v2 1/2] lib/librte_eal: Reduce timer " Zhihong Wang @ 2015-11-20 1:53 ` Zhihong Wang 2015-11-20 16:32 ` Stephen Hemminger 1 sibling, 1 reply; 7+ messages in thread From: Zhihong Wang @ 2015-11-20 1:53 UTC (permalink / raw) To: dev The kernel fills new allocated (huge) pages with zeros. DPDK just has to touch the pages to trigger the allocation. Signed-off-by: Zhihong Wang <zhihong.wang@intel.com> --- lib/librte_eal/linuxapp/eal/eal_memory.c | 20 ++++++-------------- 1 file changed, 6 insertions(+), 14 deletions(-) diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c b/lib/librte_eal/linuxapp/eal/eal_memory.c index 0de75cd..21a5146 100644 --- a/lib/librte_eal/linuxapp/eal/eal_memory.c +++ b/lib/librte_eal/linuxapp/eal/eal_memory.c @@ -399,8 +399,10 @@ map_all_hugepages(struct hugepage_file *hugepg_tbl, return -1; } + /* map the segment, and populate page tables, + * the kernel fills this segment with zeros */ virtaddr = mmap(vma_addr, hugepage_sz, PROT_READ | PROT_WRITE, - MAP_SHARED, fd, 0); + MAP_SHARED | MAP_POPULATE, fd, 0); if (virtaddr == MAP_FAILED) { RTE_LOG(ERR, EAL, "%s(): mmap failed: %s\n", __func__, strerror(errno)); @@ -410,7 +412,6 @@ map_all_hugepages(struct hugepage_file *hugepg_tbl, if (orig) { hugepg_tbl[i].orig_va = virtaddr; - memset(virtaddr, 0, hugepage_sz); } else { hugepg_tbl[i].final_va = virtaddr; @@ -529,22 +530,16 @@ remap_all_hugepages(struct hugepage_file *hugepg_tbl, struct hugepage_info *hpi) old_addr = vma_addr; - /* map new, bigger segment */ + /* map new, bigger segment, and populate page tables, + * the kernel fills this segment with zeros */ vma_addr = mmap(vma_addr, total_size, - PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); + PROT_READ | PROT_WRITE, MAP_SHARED | MAP_POPULATE, fd, 0); if (vma_addr == MAP_FAILED || vma_addr != old_addr) { RTE_LOG(ERR, EAL, "%s(): mmap failed: %s\n", __func__, strerror(errno)); close(fd); return -1; } - - /* touch the page. this is needed because kernel postpones mapping - * creation until the first page fault. with this, we pin down - * the page and it is marked as used and gets into process' pagemap. - */ - for (offset = 0; offset < total_size; offset += hugepage_sz) - *((volatile uint8_t*) RTE_PTR_ADD(vma_addr, offset)); } /* set shared flock on the file. */ @@ -592,9 +587,6 @@ remap_all_hugepages(struct hugepage_file *hugepg_tbl, struct hugepage_info *hpi) } } - /* zero out the whole segment */ - memset(hugepg_tbl[page_idx].final_va, 0, total_size); - page_idx++; } -- 2.5.0 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH RFC v2 2/2] lib/librte_eal: Remove unnecessary hugepage zero-filling 2015-11-20 1:53 ` [PATCH RFC v2 2/2] lib/librte_eal: Remove unnecessary hugepage zero-filling Zhihong Wang @ 2015-11-20 16:32 ` Stephen Hemminger 2015-11-23 1:31 ` Wang, Zhihong 0 siblings, 1 reply; 7+ messages in thread From: Stephen Hemminger @ 2015-11-20 16:32 UTC (permalink / raw) To: Zhihong Wang; +Cc: dev On Thu, 19 Nov 2015 20:53:48 -0500 Zhihong Wang <zhihong.wang@intel.com> wrote: > The kernel fills new allocated (huge) pages with zeros. > DPDK just has to touch the pages to trigger the allocation. > > Signed-off-by: Zhihong Wang <zhihong.wang@intel.com> The code is correct, and optimized. Don't you need to update the commit message since you no longer have to touch the pages. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH RFC v2 2/2] lib/librte_eal: Remove unnecessary hugepage zero-filling 2015-11-20 16:32 ` Stephen Hemminger @ 2015-11-23 1:31 ` Wang, Zhihong 0 siblings, 0 replies; 7+ messages in thread From: Wang, Zhihong @ 2015-11-23 1:31 UTC (permalink / raw) To: Stephen Hemminger; +Cc: dev > -----Original Message----- > From: Stephen Hemminger [mailto:stephen@networkplumber.org] > Sent: Saturday, November 21, 2015 12:33 AM > To: Wang, Zhihong <zhihong.wang@intel.com> > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH RFC v2 2/2] lib/librte_eal: Remove unnecessary > hugepage zero-filling > > On Thu, 19 Nov 2015 20:53:48 -0500 > Zhihong Wang <zhihong.wang@intel.com> wrote: > > > The kernel fills new allocated (huge) pages with zeros. > > DPDK just has to touch the pages to trigger the allocation. > > > > Signed-off-by: Zhihong Wang <zhihong.wang@intel.com> > > The code is correct, and optimized. > Don't you need to update the commit message since you no longer have to touch > the pages. Thanks for pointing this out! ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2015-11-23 1:32 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-11-20 1:53 [PATCH RFC v2 0/2] Reduce DPDK initialization time Zhihong Wang 2015-11-20 1:53 ` [PATCH RFC v2 1/2] lib/librte_eal: Reduce timer " Zhihong Wang 2015-11-20 16:31 ` Stephen Hemminger 2015-11-23 1:29 ` Wang, Zhihong 2015-11-20 1:53 ` [PATCH RFC v2 2/2] lib/librte_eal: Remove unnecessary hugepage zero-filling Zhihong Wang 2015-11-20 16:32 ` Stephen Hemminger 2015-11-23 1:31 ` Wang, Zhihong
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.