From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Google-Smtp-Source: ACJfBouXxbcN2PnDxQJ7ssd5PPsNuXpk55WjvmzQUc4r3RNI/42m8ZwnaphyUH61fI0fDfZ82EQx ARC-Seal: i=1; a=rsa-sha256; t=1516270617; cv=none; d=google.com; s=arc-20160816; b=oKd4gXmJcyPRl8CmaCev/DzrgPJmrkX4x0YB6tffaEJtmcMTS0dDzfzCxiemCqTpbq RI84SwjhSitIIkiUEIs4q6224eS6KNqYUGIJ1eOGZt8j+yajmhSi3MkQ8F/c/JywRQYO w5nIgfoA3FnyL/w8dOaiXLMTtAhVN2yXuEumMZzezpm/Dl6LjG0NnU/aWH2weFpz5Ozj nfkBvAMxGuRErm0++AyOhXhV7ONM2yslRbu/nnp0bpVGFge24vAIuyYTfMlpn5xHSHNV 9y1/iJe8L2pLxYlK2oDdNn59DyUTWX+3hbszdWnvQb6g6o7TBz8UBBU9VdabvWeiesGd Eabg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=spamdiagnosticmetadata:spamdiagnosticoutput :content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject :dkim-signature:arc-authentication-results; bh=u43Hvvuv7Ya3966NJzdNu/uFk7JemCbb2U/1vpqtpUI=; b=JsyFlvA9M7VVPtq/UsbAK5u/4JSTjU7jrgJsHyNM1Ry8xxxMC/d/LDgrPwPtbXpOUB Kp4/th/xfANv2lz3Kg6dXzNrUKcyEwY/M/aHEbZnTR2+OLzRpvyDeWsVLXLNSXZSfoqe Hq7jOoKzs5kCHHMF9yl3cKnTbu5BiGCqyCVi42VFMl22+SVGtvOgrxZcwMCZF4kD6sqC CWvCrBqVGhkxze+jaW0awtDz4JfToVR7EafBTdz4R9UTtU5lbMD3fCQ3rD0/W/cZ1kER FiYJw2vabTCMtFD/DV7AbS96Wg3tDZsEObttA9Z87U7YaLbU4+XKpWryhoOokmX3XWc9 bdEg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@virtuozzo.com header.s=selector1 header.b=ASrDncER; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 40.107.1.120 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com Authentication-Results: mx.google.com; dkim=pass header.i=@virtuozzo.com header.s=selector1 header.b=ASrDncER; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 40.107.1.120 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=ktkhai@virtuozzo.com; Subject: Re: [PATCH v2 05/31] net: Allow pernet_operations to be executed in parallel To: Andrei Vagin Cc: davem@davemloft.net, vyasevic@redhat.com, kstewart@linuxfoundation.org, pombredanne@nexb.com, vyasevich@gmail.com, mark.rutland@arm.com, gregkh@linuxfoundation.org, adobriyan@gmail.com, fw@strlen.de, nicolas.dichtel@6wind.com, xiyou.wangcong@gmail.com, roman.kapl@sysgo.com, paul@paul-moore.com, dsahern@gmail.com, daniel@iogearbox.net, lucien.xin@gmail.com, mschiffer@universe-factory.net, rshearma@brocade.com, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, ebiederm@xmission.com, gorcunov@virtuozzo.com, eric.dumazet@gmail.com, stephen@networkplumber.org References: <151120175301.3159.9577108443167812854.stgit@localhost.localdomain> <151120277590.3159.12461615068657469111.stgit@localhost.localdomain> <20180117183455.GA3189@outlook.office365.com> From: Kirill Tkhai Message-ID: <667f701d-4a53-650c-1289-edc8573e4751@virtuozzo.com> Date: Thu, 18 Jan 2018 13:16:49 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2 MIME-Version: 1.0 In-Reply-To: <20180117183455.GA3189@outlook.office365.com> Content-Type: text/plain; charset=koi8-r Content-Language: en-US Content-Transfer-Encoding: 7bit X-Originating-IP: [195.214.232.6] X-ClientProxiedBy: HE1PR05CA0228.eurprd05.prod.outlook.com (2603:10a6:3:fa::28) To HE1PR0801MB1338.eurprd08.prod.outlook.com (2603:10a6:3:39::28) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: df3148f1-12b7-43d5-52d8-08d55e5c9817 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(7020095)(4652020)(5600026)(4604075)(2017052603307)(7153060)(7193020);SRVR:HE1PR0801MB1338; X-Microsoft-Exchange-Diagnostics: 1;HE1PR0801MB1338;3:w/h7D3FzOVIA9JRC6nXzJceNkO2MGpsia5xADP76wVsfj/IbipmGyoOqkirs/YPeK2heXDHcyxD0H5XiGemYRwx0A6jk6LpgjXNLTsVy9LAPVbRvwizIeJkT3yPI03nAZZnAah0Y4Hs2lNDgoBZdt0T2/9jSnzXeCkFOGGrIfzTGkx5lfQ5LGVQew0tZQTbHGcBYwNef8PWLC42WAHxQa/xq9aZxH0ksL9jIrNTM/ONDsbGSmoEY62ULTpJTWiD7;25:r0cB2IMBe6ijzfm4AS3PzO6VA5TMtEEAEqkS4hlP6YhB03yVnFsdn11/elsslZ+Kt2obbCp76VjPzBRlc/wmU1SfF5ijlFF00vfDvSQ6YwdtzJuWemS2Y20+6HtqbaQ5a9WnDHo8tukk0kErJcgIHWGUIjioM41U0kb8BX2sZqFQJn7lOCNwjagvjxNs/+z72JKMIhBtFvg5doHrjc/NGMDMzQs8sRxv/ZvTb4P6aZfvcg5QKDZQgCYb9ew70Sat6hN7t5FSbcGRy9wqjW33tMlQgsJ21rhqaN6cWfyn6XL3TW7pnxCxmUsVNCrPK/KfCVs1YsvZ53aIZ68ERjUgjw==;31:IzZTo5vNVxzRoXYfW4fBu4892hLAQBrmif7hgKAX+nAEqULwp6JcnfyZU92bvmynJ+jhHYp5i9+4ca3RTU6jcHX8mUrCOs9V82c1HaMF3YpRNYX600zSnQl3VMi9h3GVHoVzAf8UEG/rhY5me1jpIbS4cAX338TMcbhci7lNRBkqj8kCIEFUfG1LXnkB7TzXCb4kGU4jncchrMtoZs74lqW/ZpHqxlbofUOkZWTN468= X-MS-TrafficTypeDiagnostic: HE1PR0801MB1338: X-Microsoft-Exchange-Diagnostics: 1;HE1PR0801MB1338;20:Qvy7/2A/BNMbNpktSE88WsPxi6iNkzbZ3jitYT+FLDz4k8bDY9tNuWUJV56ZKdDFv0ba868k4dFyQpLbiijVa9oTz73aR5oq4zgwoJYaFvRV1eR1vwCu8nzFYujvHfnzQJOI6XfxUmkOs5XTz7u+rrN7lgKw1hOGVS5kx31RsEtIeZa6Cx6oqQtotLi7lobXVb/Y+DhyHz2hkXUCd40ABkmQzkaZlgJcUUpTP45KfLCP6sR7l4pjilMAho0i/4KIW739EWV9kumJyVIfGq/fNJ0d4KFSgdhIswkW6LdmU+XmhoTXMsDtBwE7wkdheFuxWSVOJ4T+sNqwb1PujZP9YAxsWqGr8gefIdSrN6+bTV8XIC2lXunBjQXw/4Hc7wxewBrBeWEzDboaEPMa/T/cQgF/SaFQgb7JjIZXkNMQH/M=;4:Ik/CmFvJffSDd9h0TtLX9yH9WhayK7EGD1wNGQotmUBaiRSC4lRzl8vqXg4bzZFrL/U7K2jZR0oDLrD+ELOaE63S7LJQwlVgoUP9MWr3o4+/FMTZIiXECqL8efcDynx4+Ffswcl5U7mynH3vvNhgFlXOhTAn4RyiQaX8hVpR4PTUF9Rv67qrmhYSAQoK2i0ZwsUfH6sXqgz+arkvnYlPv5VluYMr1NxiEqauVHXEo93FZTECuQc+zgGO3puOh9NbZMzvkAtuP4pT/FE957Bbag== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040470)(2401047)(8121501046)(5005006)(3002001)(10201501046)(93006095)(93001095)(3231023)(2400060)(944501161)(6041268)(20161123562045)(20161123558120)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(6072148)(201708071742011);SRVR:HE1PR0801MB1338;BCL:0;PCL:0;RULEID:(100000803101)(100110400095);SRVR:HE1PR0801MB1338; X-Forefront-PRVS: 05568D1FF7 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(6049001)(39850400004)(346002)(396003)(39380400002)(376002)(366004)(189003)(199004)(83506002)(229853002)(8676002)(31686004)(86362001)(81156014)(53936002)(65806001)(65956001)(478600001)(3846002)(50466002)(316002)(26005)(16576012)(8936002)(81166006)(16526018)(31696002)(58126008)(23686003)(37006003)(66066001)(97736004)(64126003)(6246003)(36756003)(230700001)(25786009)(6862004)(39060400002)(4326008)(7416002)(2906002)(68736007)(7736002)(305945005)(47776003)(53546011)(386003)(65826007)(105586002)(106356001)(2950100002)(76176011)(59450400001)(6486002)(5660300001)(6636002)(77096007)(6116002)(52116002);DIR:OUT;SFP:1102;SCL:1;SRVR:HE1PR0801MB1338;H:[172.16.25.196];FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?koi8-r?Q?1;HE1PR0801MB1338;23:cp0frFOzO/vkqDjEUJOo74G4hSoE1/4/b+sMfWvJ4?= =?koi8-r?Q?29lA16SzV7ZKwCN6wtw7rnUv3R2nIk2T/7IOEfyqBG/X9qzV6OxyCxKC3nZHs0?= =?koi8-r?Q?kIv+lErpp7JkP8BS1OhM276CNbHb7HInZio8AMnX8CgBTkksjFhVXecNa2U2vz?= =?koi8-r?Q?7qENrZGH7UF06f8i2jcadzsBQxNynQ+oISW5MmRTndLDfEZcmHFqtI8Mk9A5L8?= =?koi8-r?Q?u+BPAwncmHt7Z6GVtoB1tO0jyqkqLmtJyNEZx/6xvSm96uwemThauXk/fxFWFL?= =?koi8-r?Q?5BzBuZjgQQxgz1NR8uKYDqza0t5b3PRbFZWIAEY7iVkwODQQCelkjwZQdMnXd9?= =?koi8-r?Q?NcRhPYSrd8PxemN7LgzRwmv5AQebJbD3kKNE7Fspqq2Ki6VIgpAe1s/PEIg/Wm?= =?koi8-r?Q?sra1Izib0eA/4BB9pLOfzg2TrEKZHmFB293HXh7gONUvEdiEHypukS0AKuHzTK?= =?koi8-r?Q?Y6IKDXlUpn5kud/Hf34jhv/WiE3gtPuvyIOtcW4obpEujiMtSYumT3z11xuOEl?= =?koi8-r?Q?1ANzxDd0AEDFdYDc1lADmUTPwdRIuZYM9a4/4JB7QvMPNQfOhD7A7yNw476Rz4?= =?koi8-r?Q?/B33menP1QPjorA5T16oj3kd9E4AaxcaSy/W03fpek42xWZ2bYHNkQCYcwzNtY?= =?koi8-r?Q?ye5gXfB43SVAio3vyKbMQcgc4xTPQYVKKtd5z/wAIjAR3xvV9rbOQ6iq14/9Cz?= =?koi8-r?Q?o6un89/QC4W4q/kt/zJjMvos7nF33mrcPEvJtp2xI0pIGRFzR4E+calWiy9aMt?= =?koi8-r?Q?FYXUmkku7ojGFM08LsiMGK3sbUnFfJlJ1CWfGmvAKA59fXLb22VNCUDc7TiblH?= =?koi8-r?Q?FDTDzkrTJhQbNaIJwEEogMgOec6538YlB17MZZllYtlqBEgcVKVydLnA0YHOr3?= =?koi8-r?Q?aa7n/M8KvxxzYAn+eVePyFfRofCbNuhMgMNCJKPMFu9CueLT7vbkn4QZYCPGA5?= =?koi8-r?Q?cLOIkitZvrdYl841D31QESPDsqeI6wv3TVMpYmdZFjgTUG/g/XuZ3c+VrdzFNR?= =?koi8-r?Q?o1ZbSjscCpQb6DWvt3nom351bmUUm0tm3yeS6QQVZuR8aGU67TS5wdOYljtcuB?= =?koi8-r?Q?w3Sl6wvDgULRAqEQqbT61H8a7F6kriu15q923Ln9R7+Dd9EY6R7gKiKfr7h9/U?= =?koi8-r?Q?cVeyggtxsFtJE3+jPwOK83VskfZGPikNLpI9LuQZ91BagKJzbm12kiiMl0N2XD?= =?koi8-r?Q?7TT6PMI4EBLUfFkh9ItOQSWapMFy7SlW5cBVW+lg/GENHaqyVFKfnLVat4O6L9?= =?koi8-r?Q?JL0454sZKdCbzVES6vb5oxgQ7Fa4rLF+8tFKKih2f3sm7xQcLPLG8M5ArHpfgC?= =?koi8-r?Q?BLbN8t4vZGgCOXgom7cltnJ3hPTwwjKo6wD2ZQtPjuLqG/y2F1j3RJDv8lnCt0?= =?koi8-r?Q?98u?= X-Microsoft-Exchange-Diagnostics: 1;HE1PR0801MB1338;6:yK+8/6hmZCjPmHycOTmLs637vdXf7M1cii8+2ER1sCkGeyzXmxGo7ae2MxoyhF13XLUrMRfajNCF/id7pE9+mv9WJrgLsYV0ceLpAytDcV6adAnqAUS3/pubpf8NA+xzLe+fXB7HgeW/4YLJ80UZvnvWla9iNtsf7/+EhIop+D1T6FtFL5fR1Rlv+MTaNKFp9icIG7/xxV6iPL/xVzGyiUrBPP4GkqkRG4MAgV2cwsaYxS+W/6qoMeBgZhcELOrHp5h7RYy9VFphOwxL+f3Ut0ojSSuAyS8vfud7cEru3NnbegBaezttlbFMjXl9d0pm7KfUnjs1/WzlMP6CgRSvh9V4cBQ5RB8mBndrQtv5Ykw=;5:xvGwlM1VEkKUsEVlhax3SDjska7pG19IyFRVNYC/iEdXIfa6HNtVQoPHR1ln5lDCo6Vm0W/clpKSAyHWlAxhpnHGnI+fUTJzsDdrXaYYxHgcnhY3cqKLoLBZaNSu+HNFnplp9qX1RpBeznJzqkbtHH7E+2/i+HwvDeI5OKIqBF4=;24:uS1GRpjks+h0Obw8Q8cyFLZ33NU5HRay/nhJZCCkQ62vncYHgPPSYAWUJTSUFmbho4p9vpN+MkvYDPN4h3ror8ImEpEMao7Go5YcjDovah8=;7:vUEbOhTdVW/EyCMwVJVzPgYj1r5RPD3566P5yA2vobUuQH2dUHj1cYaXpf0DZHJOY94d0SW5tQLomrthXy2RbV2/EQOn2ME1yuEcfnlvqJjuutu4A2ncbgT1gKVvgtowyuEiD2EbNaDUEiH9+1feVzbNb5iffgITa5XUPrVZyCmhDUfxavohR+zuSA4ZJUwKl3r8pnh4YJNjfBXg7lgWvYJufJNsYnKUi6podJlynO/r/IXTUxBU/xcsnz2vcRUm SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;HE1PR0801MB1338;20:xoyZQvvLdHPiKSUqCsZvJr23jsCVjXhOKo/hKoVLSeRXHf8KeiVhDZUz0LXAWdwjUDET5lSBARcVGLZkdJCSzwEBEOmEEP6YA7oEPCAkYiLV1+jOiS3qOKQC4XsfVV9Qk2194yfgoDTxZcsJAWPAZsinMJVIO85AQRIRJZP34AA= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jan 2018 10:16:51.4147 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: df3148f1-12b7-43d5-52d8-08d55e5c9817 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0801MB1338 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: =?utf-8?q?1584610972271332451?= X-GMAIL-MSGID: =?utf-8?q?1589924979398733582?= X-Mailing-List: linux-kernel@vger.kernel.org List-ID: On 17.01.2018 21:34, Andrei Vagin wrote: > On Mon, Nov 20, 2017 at 09:32:55PM +0300, Kirill Tkhai wrote: >> This adds new pernet_operations::async flag to indicate operations, >> which ->init(), ->exit() and ->exit_batch() methods are allowed >> to be executed in parallel with the methods of any other pernet_operations. >> >> When there are only asynchronous pernet_operations in the system, >> net_mutex won't be taken for a net construction and destruction. >> >> Also, remove BUG_ON(mutex_is_locked()) from net_assign_generic() >> without replacing with the equivalent net_sem check, as there is >> one more lockdep assert below. >> >> Suggested-by: Eric W. Biederman >> Signed-off-by: Kirill Tkhai >> --- >> include/net/net_namespace.h | 6 ++++++ >> net/core/net_namespace.c | 29 +++++++++++++++++++---------- >> 2 files changed, 25 insertions(+), 10 deletions(-) >> >> diff --git a/include/net/net_namespace.h b/include/net/net_namespace.h >> index 10f99dafd5ac..db978c4755f7 100644 >> --- a/include/net/net_namespace.h >> +++ b/include/net/net_namespace.h >> @@ -303,6 +303,12 @@ struct pernet_operations { >> void (*exit_batch)(struct list_head *net_exit_list); >> unsigned int *id; >> size_t size; >> + /* >> + * Indicates above methods are allowe to be executed in parallel >> + * with methods of any other pernet_operations, i.e. they are not >> + * need synchronization via net_mutex. >> + */ >> + bool async; >> }; >> >> /* >> diff --git a/net/core/net_namespace.c b/net/core/net_namespace.c >> index c4f7452906bb..550c766f73aa 100644 >> --- a/net/core/net_namespace.c >> +++ b/net/core/net_namespace.c >> @@ -41,8 +41,9 @@ struct net init_net = { >> EXPORT_SYMBOL(init_net); >> >> static bool init_net_initialized; >> +static unsigned nr_sync_pernet_ops; >> /* >> - * net_sem: protects: pernet_list, net_generic_ids, >> + * net_sem: protects: pernet_list, net_generic_ids, nr_sync_pernet_ops, >> * init_net_initialized and first_device pointer. >> */ >> DECLARE_RWSEM(net_sem); >> @@ -70,11 +71,10 @@ static int net_assign_generic(struct net *net, unsigned int id, void *data) >> { >> struct net_generic *ng, *old_ng; >> >> - BUG_ON(!mutex_is_locked(&net_mutex)); >> BUG_ON(id < MIN_PERNET_OPS_ID); >> >> old_ng = rcu_dereference_protected(net->gen, >> - lockdep_is_held(&net_mutex)); >> + lockdep_is_held(&net_sem)); >> if (old_ng->s.len > id) { >> old_ng->ptr[id] = data; >> return 0; >> @@ -419,11 +419,14 @@ struct net *copy_net_ns(unsigned long flags, >> rv = down_read_killable(&net_sem); >> if (rv < 0) >> goto put_userns; >> - rv = mutex_lock_killable(&net_mutex); >> - if (rv < 0) >> - goto up_read; >> + if (nr_sync_pernet_ops) { >> + rv = mutex_lock_killable(&net_mutex); >> + if (rv < 0) >> + goto up_read; >> + } >> rv = setup_net(net, user_ns); >> - mutex_unlock(&net_mutex); >> + if (nr_sync_pernet_ops) >> + mutex_unlock(&net_mutex); >> up_read: >> up_read(&net_sem); >> if (rv < 0) { >> @@ -453,7 +456,8 @@ static void cleanup_net(struct work_struct *work) >> spin_unlock_irq(&cleanup_list_lock); >> >> down_read(&net_sem); >> - mutex_lock(&net_mutex); >> + if (nr_sync_pernet_ops) >> + mutex_lock(&net_mutex); >> >> /* Don't let anyone else find us. */ >> rtnl_lock(); >> @@ -489,7 +493,8 @@ static void cleanup_net(struct work_struct *work) >> list_for_each_entry_reverse(ops, &pernet_list, list) >> ops_exit_list(ops, &net_exit_list); >> >> - mutex_unlock(&net_mutex); >> + if (nr_sync_pernet_ops) >> + mutex_unlock(&net_mutex); >> >> /* Free the net generic variables */ >> list_for_each_entry_reverse(ops, &pernet_list, list) >> @@ -961,6 +966,9 @@ static int register_pernet_operations(struct list_head *list, >> rcu_barrier(); >> if (ops->id) >> ida_remove(&net_generic_ids, *ops->id); >> + } else if (!ops->async) { >> + pr_info_once("Pernet operations %ps are sync.\n", ops); > > As far as I understand, we have this sync mode for backward > compatibility with non-upstream modules, don't we? If the answer is yes, > it may be better to add WARN_ONCE here? There are 200+ more pernet operations requiring the review and making them async. This pr_info_once() is to help people find unconverted pernet_operations they use and start the work on converting them. Thanks, Kirill >> + nr_sync_pernet_ops++; >> } >> >> return error; >> @@ -968,7 +976,8 @@ static int register_pernet_operations(struct list_head *list, >> >> static void unregister_pernet_operations(struct pernet_operations *ops) >> { >> - >> + if (!ops->async) >> + BUG_ON(nr_sync_pernet_ops-- == 0); >> __unregister_pernet_operations(ops); >> rcu_barrier(); >> if (ops->id) >>