From: Jia He <hejianet@gmail.com>
To: netdev@vger.kernel.org
Cc: linux-sctp@vger.kernel.org, linux-kernel@vger.kernel.org,
davem@davemloft.net, Alexey Kuznetsov <kuznet@ms2.inr.ac.ru>,
James Morris <jmorris@namei.org>,
Hideaki YOSHIFUJI <yoshfuji@linux-ipv6.org>,
Patrick McHardy <kaber@trash.net>,
Vlad Yasevich <vyasevich@gmail.com>,
Neil Horman <nhorman@tuxdriver.com>,
Steffen Klassert <steffen.klassert@secunet.com>,
Herbert Xu <herbert@gondor.apana.org.au>,
Jia He <hejianet@gmail.com>
Subject: [RFC PATCH 0/6] Reduce cache miss for snmp_fold_field
Date: Tue, 30 Aug 2016 00:03:54 +0800 [thread overview]
Message-ID: <1472486640-1111-1-git-send-email-hejianet@gmail.com> (raw)
In a PowerPc server with large cpu number(160), besides commit
a3a773726c9f ("net: Optimize snmp stat aggregation by walking all
the percpu data at once"), I watched several other snmp_fold_field
callsites which will cause high cache miss rate.
#My simple test case, which read from the procfs items endlessly:
/***********************************************************/
#include <stdlib.h>
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#define LINELEN 2560
int main(int argc, char **argv)
{
int i;
int fd = -1 ;
int rdsize = 0;
char buf[LINELEN+1];
buf[LINELEN] = 0;
memset(buf,0,LINELEN);
if(1 >= argc) {
printf("file name empty\n");
return -1;
}
fd = open(argv[1], O_RDWR, 0644);
if(0 > fd){
printf("open error\n");
return -2;
}
for(i=0;i<0xffffffff;i++) {
while(0 < (rdsize = read(fd,buf,LINELEN))){
//nothing here
}
lseek(fd, 0, SEEK_SET);
}
close(fd);
return 0;
}
/**********************************************************/
#compile and run:
gcc test.c -o test
perf stat -d -e cache-misses ./test /proc/net/snmp
perf stat -d -e cache-misses ./test /proc/net/snmp6
perf stat -d -e cache-misses ./test /proc/net/netstat
perf stat -d -e cache-misses ./test /proc/net/sctp/snmp
perf stat -d -e cache-misses ./test /proc/net/xfrm_stat
#test results
I firstly test the correctness of data statistics.
As for performance, before the patch set:
====================
Performance counter stats for 'system wide':
355911097 cache-misses [40.08%]
2356829300 L1-dcache-loads [60.04%]
355642645 L1-dcache-load-misses # 15.09% of all L1-dcache hits [60.02%]
346544541 LLC-loads [59.97%]
389763 LLC-load-misses # 0.11% of all LL-cache hits [40.02%]
6.245162638 seconds time elapsed
After the patch set:
===================
Performance counter stats for 'system wide':
194992476 cache-misses [40.03%]
6718051877 L1-dcache-loads [60.07%]
194871921 L1-dcache-load-misses # 2.90% of all L1-dcache hits [60.11%]
187632232 LLC-loads [60.04%]
464466 LLC-load-misses # 0.25% of all LL-cache hits [39.89%]
6.868422769 seconds time elapsed
The cache-miss rate can be reduced from 15% to 2.9%
Jia He (6):
proc: Reduce cache miss in {snmp,netstat}_seq_show
proc: Reduce cache miss in snmp6_seq_show
proc: Reduce cache miss in sctp_snmp_seq_show
proc: Reduce cache miss in xfrm_statistics_seq_show
ipv6: Remove useless parameter in __snmp6_fill_statsdev
net: Suppress the "Comparison to NULL could be written" warning
net/ipv4/proc.c | 110 ++++++++++++++++++++++++++++++---------------------
net/ipv6/addrconf.c | 12 +++---
net/ipv6/proc.c | 47 ++++++++++++++++------
net/sctp/proc.c | 15 +++++--
net/xfrm/xfrm_proc.c | 15 +++++--
5 files changed, 131 insertions(+), 68 deletions(-)
--
1.8.3.1
next reply other threads:[~2016-08-29 16:04 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-08-29 16:03 Jia He [this message]
2016-08-29 16:03 ` [RFC PATCH 1/6] proc: Reduce cache miss in {snmp,netstat}_seq_show Jia He
2016-08-29 16:41 ` Eric Dumazet
2016-08-30 8:15 ` hejianet
2016-08-29 16:03 ` [RFC PATCH 2/6] proc: Reduce cache miss in snmp6_seq_show Jia He
2016-08-29 16:03 ` [RFC PATCH 3/6] proc: Reduce cache miss in sctp_snmp_seq_show Jia He
2016-08-29 16:03 ` [RFC PATCH 4/6] proc: Reduce cache miss in xfrm_statistics_seq_show Jia He
2016-08-29 16:03 ` [RFC PATCH 5/6] ipv6: Remove useless parameter in __snmp6_fill_statsdev Jia He
2016-08-29 16:04 ` [RFC PATCH 6/6] net: Suppress the "Comparison to NULL Jia He
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1472486640-1111-1-git-send-email-hejianet@gmail.com \
--to=hejianet@gmail.com \
--cc=davem@davemloft.net \
--cc=herbert@gondor.apana.org.au \
--cc=jmorris@namei.org \
--cc=kaber@trash.net \
--cc=kuznet@ms2.inr.ac.ru \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-sctp@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=nhorman@tuxdriver.com \
--cc=steffen.klassert@secunet.com \
--cc=vyasevich@gmail.com \
--cc=yoshfuji@linux-ipv6.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).