From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756806AbcH2WjE (ORCPT ); Mon, 29 Aug 2016 18:39:04 -0400 Received: from mail-eopbgr50124.outbound.protection.outlook.com ([40.107.5.124]:53708 "EHLO EUR03-VE1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1754688AbcH2WjC (ORCPT ); Mon, 29 Aug 2016 18:39:02 -0400 X-Greylist: delayed 3789 seconds by postgrey-1.27 at vger.kernel.org; Mon, 29 Aug 2016 18:39:01 EDT Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=ktkhai@virtuozzo.com; Subject: [PATCH RFC 0/2] net: Iterate over cpu_present_mask during calculation of percpu statistics From: Kirill Tkhai To: , CC: , , , , , , , Date: Mon, 29 Aug 2016 22:03:48 +0300 Message-ID: <147249730528.18175.4772805024092024580.stgit@pro> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-Originating-IP: [195.214.232.10] X-ClientProxiedBy: AM4PR01CA0021.eurprd01.prod.exchangelabs.com (10.164.74.159) To HE1PR0802MB2139.eurprd08.prod.outlook.com (10.172.126.11) X-MS-Office365-Filtering-Correlation-Id: 0f144d44-79c1-4113-0777-08d3d03f37ce X-Microsoft-Exchange-Diagnostics: 1;HE1PR0802MB2139;2:VlqTSChu/1o+4Bk9etBEDCs3zhuJI3dGf7RZtEJDqAwjRorTToqzRX1/4/L3ATWbCnpHcmr8FJV4gKa4KSssk5vAfByoXs9ytHpYaPc9D5Ti/xWG2s9SNnRoFydaK9GOZ4MowtRWddp8mzlfsUxYhjdMUleAGQCQ7KIel9S+RfhL9kOiZokL9llmD64DSVQM;3:O6QWdryHivulnfVkYzFwvVm0GScKOo/oRISwWW620pUyGFSlwU1gtnNT/kZrgULUA587igxj4QikzfHdKO/Ldm2d1NhUzPnZS0Ygdo1IwQYlNa3hvxdQ3WsNCQuWzSGH X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:HE1PR0802MB2139; X-Microsoft-Exchange-Diagnostics: 1;HE1PR0802MB2139;25:x0EDq+iIFEXTT/Kyh++i+XrxCwuHkfCPAstd73ipMsaTwm9WVShduK+h53m7B5fUDgqqAUhtRRr2vq9bRmnI3Uw/9VXUflYDc2YmgM8xAiJ/vZfNKercUkpL5r/G1JJdUqtyPJndz4uPba9aayqLvBxw2w/TNwCrbf0EwE91UW7eo3lEgFk1C6utiWyoI3gzZoGTXU+UqwUWriSH5+dLisFnUQQ/7Ze6tN3o8RuJbJGXRG+v9aCHqVUMePRqPQ8/p2IOuXgLKDzrEB6pI4/biEjWhBJCbbp7qhHyMp4Cux13YdJG5GYXESKPEdGDrtWcH2xREDW4AynF1EezaiqrMHJay6l2Ho/jp8ho+kE3D7rvxdtw17Z3oxEati5jDGw+AlUkPMsmEx2J0pYHZU6NyguiHF48c+E2DrPOFSbqVIYpooqrF7aYCsXIuoLrBn8YzEAHmGUjveteh1vZdUwOzzEq1t+1pQaotxuqNG8v5YhNP3phqaL5W29RUU35d9KJ1RMn6eqPNS3iAeEfSz528DWbJinK6GJBJHonkq45hJxu7N7s2sAYmLPiPHxnaFh9BzpdKMPe6JIoqoikfmJdP3YBXPknGh/YEKIIrmmWBCmlhxr5b8T5xmMhuFKO2Blwu6cdR4a2EzQDhVD1SY67Ropq2QpJrKWZuLQ7HwVtMf+Dp7k/ykQa8Bn0QAb1zwW122nyI8+lGCTbzZNnXdm0mQ==;31:J+0j5JtSXxRr4RUBVURppSTalxT1JImDtWJNbl9NVMzAVXSpfo1TnnebXmbdYeVkLTsTE0JMwSVut+5xsx3lwJkCM5NwpWRmu8PZkJ8/CnqiPyVpgmCxlrDdiDoOHA6ru3pR3stYDaDvUQOn/nI3nUuLZBDtz9FzFOCawLbU3qJiDTmjNIGdtTyc6KjI5xVmyakkKU+EAX4kXJojEidZrKpx5rah9oH6cGtJx494H8Y= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040176)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6043046)(6042046);SRVR:HE1PR0802MB2139;BCL:0;PCL:0;RULEID:;SRVR:HE1PR0802MB2139; X-Microsoft-Exchange-Diagnostics: 1;HE1PR0802MB2139;4:qZznu0f5ZNVDtWBgQx3sNYQPaUWQcd5dW1DDbQez+1ElbRHCWg0Jp1s6DMyEIy0F3IfAhShIBuo8eQ0GCTWmJJItybXN1iEhW3DbUfdZOlwnlD1i3y/eYlw24CZPf9Msjy56i0DIaHKREH2c7FYF/We5afqFtxISnj/dJVOOVzZTJlcPj+0K+EPHcAUyY3FbfEEP6G2LnApEVgjzbUTZRryBl1UbjGzbVWSVb4cQDZl9YOx8h95/A69rUI1EuV5OgwQ5TPtxh/6eCNbxKMVaFtK6sWAYl6NehfDLSLw+rYymQhyQbu8FdFvVCaRX0keaayWJZyCcVhD0aMoZiDlU7YYs+kdFP5KuVDTmidAvoMWYHyOZUla7BZZnkoOulSJuvkv0ePO4lKBYrF0BR/2ILswDFuwlbAvzOwzxG7K9NDg= X-Forefront-PRVS: 0049B3F387 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6009001)(6049001)(7916002)(189002)(199003)(8676002)(101416001)(7846002)(7736002)(97736004)(305945005)(107886002)(92566002)(4001350100001)(189998001)(5001770100001)(33646002)(105586002)(106356001)(42186005)(230700001)(586003)(23676002)(4001430100002)(50986999)(19580405001)(19580395003)(6116002)(3846002)(50466002)(103116003)(33716001)(68736007)(229853001)(86362001)(77096005)(54356999)(4326007)(5660300001)(2906002)(66066001)(83506001)(9686002)(81166006)(81156014)(47776003)(217873001);DIR:OUT;SFP:1102;SCL:1;SRVR:HE1PR0802MB2139;H:[127.0.1.1];FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtIRTFQUjA4MDJNQjIxMzk7MjM6ZlFScjRldnJWaXZuYW9yS2lEUmVOclVO?= =?utf-8?B?enpVV1hqdWtXSXdWeno4UStLbG9ab2d5cW9OYlRtTFl1SVpERVFQY3dGaEtQ?= =?utf-8?B?bDRZNEZMZlpKVGtObjlRdzNaZW5NY3NMZTdEYm91eFlIM2dnNytEWWJYWXZu?= =?utf-8?B?eG1WTUw3TzRBN3lWZ05VOERhQzVueUEybGRPOGpyR0J3c0JBTmtyQ01vM1JX?= =?utf-8?B?YlRDMFNSTXR3djVqWG15VDBEMVhEam9mc3ExK3NTUUFWQWNxcXVMMTdXUFc4?= =?utf-8?B?QXV2cUVIeHNpSWc2bzdVSHFWTUswbmkzV0J3T09uVGJFakFjMkhMMGFqT210?= =?utf-8?B?TW1OTXlnYnFkSFZoeGtuM2ZnTEN3VEpEcitXTmFmcnpLZDh5c0NiUll0SXpa?= =?utf-8?B?TTBVSU5QbWkyZ3lhMVdJWkpSN0N2VExRK3E5d0czcVhmY1NuMHYyMDJQUXNh?= =?utf-8?B?c3pjcDRweUNCQzQrU0VTRzlBTFpLZGM3NTYwVXlxYTNIYjFScm93SjZmQkNJ?= =?utf-8?B?NXBxUXNUMVd0WndDd2R3OXpLR0Y1bzJNZ2NmSHFmV2VURzJUSUE2UkNYbzc1?= =?utf-8?B?Z0VESXdwSE55YVRpaE9aMXBuRTcrZkpTVmZsMS9ua25pZ2RYNEFUTE41UFMv?= =?utf-8?B?SmhoWm02NXNGODVkbXlsQmR2aGNRV1dGS3BNZzhpU2RzaE9WK0lreFhoUGt3?= =?utf-8?B?VmhBRjE1NVk4SmZTc1pSaW1EOGczN0tjWWY4dTl0RUl4SnVqSDZGaUxKeGhH?= =?utf-8?B?dGkyOXFoQXhneHVOdlBYekxPay9CQmM5YmpEbHo2UlFJaWVWZHZwZ0Z6SFpQ?= =?utf-8?B?dlplYldCSVhzL3c2NGNNb0MvanpPY2ptYmtyQkNTSDcwQTZZcHd3L1g0RkFG?= =?utf-8?B?YzZKa1BFbUJISTQ5dWswZFVBQ1l1WVVwTzcyNjRGdTQyNlZSTjA5RUdxdEtw?= =?utf-8?B?QjNBM2llL0VVUWNqczE3Zyt1L0FSbkFBa0E5YlNtTlgzMktIc3hXR3l1Rjha?= =?utf-8?B?V3dock9KbjMyNU1TMC9uOVcvQmVaQUhtY2U0NHYwRVkwZ3I3RXEvRC9TOVp4?= =?utf-8?B?bjQxT1Z3TWFJZ1B2RlYzOUZDbXp2NFFLaGdqMkZTQzJmaWUxa0R0cGVmU0w5?= =?utf-8?B?WGZoZzMzQVNxNU1ja2hJOVNuTkFUeFdiVkRlZVlKTHVYQ3JtQXNRSXVEZnpu?= =?utf-8?B?NGdWbTEybDVKY2ROQlZIdlpHS0pOZzFMazE2bUdJTlVRcXJ6RFN6ZlhlUUJE?= =?utf-8?B?SHJTdmhDeTZKUVM0ODNvbUhLM2RSMStuWGdzNHhMbVhYcXRNcmh4ZE1oUDcx?= =?utf-8?B?T3BTdm9sQlRKbHdKUzEwUXlNa0xTZWJ1MHo3RExDSTdrOTZkYW1zRllJeWRQ?= =?utf-8?B?cFkra21NaUF6Mzl6VnowTWVwNnQ3blVQV2laSmJ3eUhSc1ZBcFd3dE5DcVBj?= =?utf-8?B?V1FBL0ViS2VOL3BZbGkxQm1Pa0pUVnFhck5GUVVDcHduNTVqV1JXTU1KeXNQ?= =?utf-8?B?bmMvZms3dHVySG03cmN4NEoycGV6V001Y3FKbWNXOGNxYTRvT29KeTA2RHVs?= =?utf-8?B?MkVKWjBaS2FxQXNuTkE5YkZyQmlTbmprVSt1eUo5YzUxQ0UxeXRVWS8yTkpP?= =?utf-8?B?QUYzaldCdERRdlYydFJQaitLZHRNRUlKTmwwUmhVdXE1SC9id3laYlo5QWc9?= =?utf-8?Q?=3D?= X-Microsoft-Exchange-Diagnostics: 1;HE1PR0802MB2139;6:2uUgB7NXRuV+w3LOEUg4TxgfjII12wRffP0vYhExRxdAZ6APPnrUuJw4nOrmS91nCA3/lyGOnieP+m4h0qreayWADLzqGPA62yneSaKTy1Z1qBpXCmKygE98ASpo48K0PmuSPuVyJ4spig3Y76BD8aFscUS7FwwePOcVOy4Bzdv4qRmrygP4f9xRcgaIj+gvfD0dhwMjy5bq6HfFYUNqCHGWRP9H3T9ECrKwe8wwiglMUXg2hBx4UWWc+brRO/QCIWinkt6dQqg/+8sjvvdRTwgQx0RFf/0DAk1/WjORUMa/SQnOCxLizopb35fmXujs;5:Sb5WWxl3XjS49AKw+7IOYs/iaF7SMcl8VkaiCWFzzHSsi5cAYL3J1f0lceKaKFeDDDfDeiE9lCfNuN6/PENMwwKdQ24hKFkmNqB4yGnRLadb17V5un4mr5+jm34C3/Zdo3PPy8jTqJd9rNFdn0aHvQ==;24:+uJvFEU+8Xq0hp60jX/ms3eRgSaVBv4oSSSdX0OYGYhJjwxJ3QGHhSf0sY9S6JAbaayjkTD+vtBM4g66t7JMdxxfvDXbDkOnNvNfsPVc6BA=;7:R8Ub7+YqKsEGJGuoVlDs7gv4nz2FZnGtjQXJZSaVxUeDp72Q876T0aOTdUVn8S8Gz4sCbmLGzOxzZr7iB4zpuImykzky/1RojdMK4hh2kwsI2/loypB5W/DyrXdTYW1OHRbGH1BhjHy1Q8OgxeC7XRWfQeku1Z/gM5Mg4lRWogqgt2RAv1BPK1r9ZETN3CzS2UuCMNblk+qNWILbCTozpR0ZaPHp8MRt+6VYmsMhYqzqYxmMowLAeRPEX4+CooeP SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;HE1PR0802MB2139;20:7qVBVP4Afd+H6aMytzb+F5iqKtNYGnIVkwKJStz1k4KAKotbfBQ/5qrI+CREd/2Dun9UhUEq3/Bao13E6KTqcgclGNkM203/gdmbDqu+yuJxdW7G5zSaLDmhtEG/UITlgDsfpK4pPYVk+Y4l0HGF0OUtut12gykyKAi7KSPmVvo= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Aug 2016 19:03:51.7830 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0802MB2139 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Many variables of statistics type are made percpu in kernel. This allows to do not make them atomic or to do not use synchronization. The result value is calculated as sum of values on every possible cpu. The problem is this scales bad. The calculations may took a lot of time. For example, some machine configurations have many possible cpus like below: "smpboot: Allowing 192 CPUs, 160 hotplug CPUs" There are only 32 real cpus, but 192 possible cpus. I had a report about very slow getifaddrs() on older kernel, when there are possible only 590 getifaddrs calls/second on Xeon(R) CPU E5-2667 v3 @ 3.20GHz. The patchset aims to begin solving of this problem. It makes possible to iterate over present cpus mask instead of possible. When cpu is going down, a statistics is being moved to an alive cpu. It's made in CPU_DYING callback, which happens when machine is stopped. So, iteration over present cpus mask is safe under preemption disabled. Patchset could exclude even offline cpus, but I didn't do that, because the main problem seems to be possible cpus. Also, this would require to do some changes in kernel/cpu.c, so I'd like to hear people opinion about expediency of this before. One more question is whether the whole kernel needs the same possibility and the patchset should be more generic. Please, comment! For the above configuration the patchset improves the below test in 2.9 times: #define _GNU_SOURCE /* To get defns of NI_MAXSERV and NI_MAXHOST */ #include #include #include #include #include #include #include #include int do_gia() { struct ifaddrs *ifaddr, *ifa; int family, s, n; char host[NI_MAXHOST]; if (getifaddrs(&ifaddr) == -1) { perror("getifaddrs"); exit(EXIT_FAILURE); } /* touch the data */ for (ifa = ifaddr, n = 0; ifa != NULL; ifa = ifa->ifa_next, n++) { if (ifa->ifa_addr == NULL) continue; family = ifa->ifa_addr->sa_family; } freeifaddrs(ifaddr); } int main(int argc, char *argv[]) { int i; for(i=0; i<10000; i++) do_gia(); } --- Kirill Tkhai (2): net: Implement net_stats callbacks net: Iterate over present cpus only during ipstats calculation include/net/stats.h | 9 ++++++ net/core/Makefile | 1 + net/core/stats.c | 83 +++++++++++++++++++++++++++++++++++++++++++++++++++ net/ipv6/addrconf.c | 4 ++ net/ipv6/af_inet6.c | 56 ++++++++++++++++++++++++++++++++++ 5 files changed, 152 insertions(+), 1 deletion(-) create mode 100644 include/net/stats.h create mode 100644 net/core/stats.c -- Signed-off-by: Kirill Tkhai