From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F114DC4332F for ; Thu, 17 Nov 2022 17:40:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239831AbiKQRkP (ORCPT ); Thu, 17 Nov 2022 12:40:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58908 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233439AbiKQRkM (ORCPT ); Thu, 17 Nov 2022 12:40:12 -0500 Received: from mail-yb1-xb35.google.com (mail-yb1-xb35.google.com [IPv6:2607:f8b0:4864:20::b35]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D5C9C1118 for ; Thu, 17 Nov 2022 09:40:10 -0800 (PST) Received: by mail-yb1-xb35.google.com with SMTP id 7so2696798ybp.13 for ; Thu, 17 Nov 2022 09:40:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=AKl1hHI5se5hw6UJ9agcMcck6mPAULEqYlgEMH+yNFQ=; b=Nbf/olEfLZZXz7uiFTazHgrt6PaxZPZNdS17D5K2J6t9KQIhcV+YJjWktIXwgjlTY0 GpuiEnVuIIP16rXL/3UbqClkbwxNCZrqzxScSlrZvLP42fuOr8KHRjAguMSSGODsgO3E /fsoB0TbOFNEtFPXrEqvkEgo7nSdpnV22auDYH5pLwzZ87DEOy5Cf2ivZtCDwIhXk0Eb x6+nYZAbCiRu1pxIB1t1ONlwggxNrmOTnUSmjPLsEdS+e9vJkZR8dWcOrYrcHKKgw5E7 HahCGTyComxOFcxrBBsX9VJWEwWA4Q2sWGTv1JkwK9NcWx8i+fxMsrczZWZaoL+5zpc4 hyxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=AKl1hHI5se5hw6UJ9agcMcck6mPAULEqYlgEMH+yNFQ=; b=ILcOrq7Hr1E/CbaedCOja48tbWMLOmDTsSfUQLx7GFY4PpDCifxow1h4rXIaEMAWoQ ytTMgZSibF815JwAopfBwPXZbPm5GK7Vw9fmPtjv9oaDfsDb1a54NL6KNGqmrlC2WV1P ULy8qHN4gaW+z4fxLDPNr5e2GeTq6kqtANmQCEuItOgxpclM8wfsHFXs7a/f/+y7vZfJ QpaEXK2Jw1UHNXBz2/qruPTkMr8jg1qHIzjlpJg+zFBkXybBpuf0yZq7JksB7FA3yfj6 qvtEO1a/RxGWFGnmpHI/DHWs3NJyyVQe9gBeqAvohSG4C6iWz74fXzCGPgbyNaVkeClg giyg== X-Gm-Message-State: ANoB5pnC2lETrHwePqk9rsWsTQJJfGMNp5RaxZePEnujkHqLdLy6VwmD mkJYj4QP65wwl1RIM9MROoGqt/2g/CqWLmJLv8fS6qEQ0iE= X-Google-Smtp-Source: AA0mqf6uttxSWotz+Oayev08TSggx/5dMxILtzURgKDg1t+nWq4ImxvQ68Ne+mSpn5oyL8Ufjg4XsybtkACdF+cJR5c= X-Received: by 2002:a05:6902:11cd:b0:6e7:f2ba:7c0f with SMTP id n13-20020a05690211cd00b006e7f2ba7c0fmr1182001ybu.55.1668706809796; Thu, 17 Nov 2022 09:40:09 -0800 (PST) MIME-Version: 1.0 References: <20221117031551.1142289-1-joel@joelfernandes.org> <20221117031551.1142289-3-joel@joelfernandes.org> In-Reply-To: From: Eric Dumazet Date: Thu, 17 Nov 2022 09:39:57 -0800 Message-ID: Subject: Re: [PATCH rcu/dev 3/3] net: Use call_rcu_flush() for dst_destroy_rcu To: Joel Fernandes Cc: linux-kernel@vger.kernel.org, Cong Wang , David Ahern , "David S. Miller" , Hideaki YOSHIFUJI , Jakub Kicinski , Jamal Hadi Salim , Jiri Pirko , netdev@vger.kernel.org, Paolo Abeni , rcu@vger.kernel.org, rostedt@goodmis.org, paulmck@kernel.org, fweisbec@gmail.com, jiejiang@google.com, Thomas Glexiner Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Nov 17, 2022 at 9:38 AM Joel Fernandes wrote: > > On Thu, Nov 17, 2022 at 5:17 PM Eric Dumazet wrote: > > > > On Thu, Nov 17, 2022 at 7:58 AM Joel Fernandes wrote: > > > > > > Hello Eric, > > > > > > On Wed, Nov 16, 2022 at 07:44:41PM -0800, Eric Dumazet wrote: > > > > On Wed, Nov 16, 2022 at 7:16 PM Joel Fernandes (Google) > > > > wrote: > > > > > > > > > > In a networking test on ChromeOS, we find that using the new CONFIG_RCU_LAZY > > > > > causes a networking test to fail in the teardown phase. > > > > > > > > > > The failure happens during: ip netns del > > > > > > > > And ? What happens then next ? > > > > > > The test is doing the 'ip netns del ' and then polling for the > > > disappearance of a network interface name for upto 5 seconds. I believe it is > > > using netlink to get a table of interfaces. That polling is timing out. > > > > > > Here is some more details from the test's owner (copy pasting from another > > > bug report): > > > In the cleanup, we remove the netns, and thus will cause the veth pair being > > > removed automatically, so we use a poll to check that if the veth in the root > > > netns still exists to know whether the cleanup is done. > > > > > > Here is a public link to the code that is failing (its in golang): > > > https://source.chromium.org/chromiumos/chromiumos/codesearch/+/main:src/platform/tast-tests/src/chromiumos/tast/local/network/virtualnet/env/env.go;drc=6c2841d6cc3eadd23e07912ec331943ee33d7de8;l=161 > > > > > > Here is a public link to the line of code in the actual test leading up to the above > > > path (this is the test that is run: > > > network.RoutingFallthrough.ipv4_only_primary) : > > > https://source.chromium.org/chromiumos/chromiumos/codesearch/+/main:src/platform/tast-tests/src/chromiumos/tast/local/bundles/cros/network/routing_fallthrough.go;drc=8fbf2c53960bc8917a6a01fda5405cad7c17201e;l=52 > > > > > > > > Using ftrace, I found the callbacks it was queuing which this series fixes. Use > > > > > call_rcu_flush() to revert to the old behavior. With that, the test passes. > > > > > > > > What is this test about ? What barrier was used to make it not flaky ? > > > > > > I provided the links above, let me know if you have any questions. > > > > > > > Was it depending on some undocumented RCU behavior ? > > > > > > This is a new RCU feature posted here for significant power-savings on > > > battery-powered devices: > > > https://lore.kernel.org/rcu/20221017140726.GG5600@paulmck-ThinkPad-P17-Gen-1/T/#m7a54809b8903b41538850194d67eb34f203c752a > > > > > > There is also an LPC presentation about the same, I can dig the link if you > > > are interested. > > > > > > > Maybe adding a sysctl to force the flush would be better for functional tests ? > > > > > > > > I would rather change the test(s), than adding call_rcu_flush(), > > > > adding merge conflicts to future backports. > > > > > > I am not too sure about that, I think a user might expect the network > > > interface to disappear from the networking tables quickly enough without > > > dealing with barriers or kernel iternals. However, I added the authors of the > > > test to this email in the hopes he can provide is point of views as well. > > > > > > The general approach we are taking with this sort of thing is to use > > > call_rcu_flush() which is basically the same as call_rcu() for systems with > > > CALL_RCU_LAZY=n. You can see some examples of that in the patch series link > > > above. Just to note, CALL_RCU_LAZY depends on CONFIG_RCU_NOCB_CPU so its only > > > Android and ChromeOS that are using it. I am adding Jie to share any input, > > > he is from the networking team and knows this test well. > > > > > > > > > > I do not know what is this RCU_LAZY thing, but IMO this should be opt-in > > You should read the links I sent you. We did already try opt-in, > Thomas Gleixner made a point at LPC that we should not add new APIs > for this purpose and confuse kernel developers. > > > For instance, only kfree_rcu() should use it. > > No. Most of the call_rcu() usages are for freeing memory, so the > consensus is we should apply this as opt out and fix issues along the > way. We already did a lot of research/diligence on seeing which users > need conversion. > > > We can not review hundreds of call_rcu() call sites and decide if > > adding arbitrary delays cou hurt . > > That work has already been done as much as possible, please read the > links I sent. Oh well. No. I will leave it to other folks dealing with this crazy thing.