From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55AF9C4320A for ; Fri, 13 Aug 2021 10:16:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 39EA5610CC for ; Fri, 13 Aug 2021 10:16:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240011AbhHMKRN (ORCPT ); Fri, 13 Aug 2021 06:17:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35636 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239613AbhHMKRM (ORCPT ); Fri, 13 Aug 2021 06:17:12 -0400 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1EAB2C061756; Fri, 13 Aug 2021 03:16:46 -0700 (PDT) Received: by mail-wm1-x331.google.com with SMTP id k5-20020a05600c1c85b02902e699a4d20cso6565022wms.2; Fri, 13 Aug 2021 03:16:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=qqw1S6wFqpcslkxegqQiXTxlXz74o59gYYNcnKgmY4g=; b=Iac72rAAdZ6AojYmmBS1b45TFPgBlke8aXSGKhPB+5ChYdcMcO7rrOnxRDymsPQTes Sfi0jSDobljIJhCJbdsbs6KE8Wo+0oGJyE0jPPl7KVeVVHxVjzpLS8r78IaunH+YGcvA Cyskeksk0kb8Lc4py22Ak+sLqRYBquFwbX9xBkFi6YUCplmh7QPeDhT+NupyF+dGJJQg a3VCGO977oB0ja8EHzCBvHK5VfyAw8jJpH4DIFE7/HQDTP3Cys8n/WyOt0OkfOvqgIzS UUrd0Pq8lKhUlL44AEvsXjjX2WJ9YT9JYnRG+6DZMzK9iahdYLOv/kww2MyxFbsTnyPK ZLmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=qqw1S6wFqpcslkxegqQiXTxlXz74o59gYYNcnKgmY4g=; b=MzuZ0S0kpzxK4mrJ+ec9TBeSu+GABSnOsVJhpCJtTmpagya/KA7tHNNr3ZNlptK5N1 75r3SWuSjueMbjObUWcFtVKE1IjyBgGe2GlaH6M+CREBV/pG1FpXiUezwlhn1iF1Tn6e wqlEUoKWkp80JQ053gC97N7QGUlj4W6FKiRynQC81RpkxH7SsRvse05E3GIzArGlbQ11 EQfpwpFiQnWrVXkbvTVXZN9KNoB4p2P892KP7QuCdCjBRaTf9keLuLBRE8XLi3ekm729 pqMxno/OjPQS9hZTRWFUge8guiS0MRG0vs8IbQ7vxtNTxdITKNa/PV6kbkbXHkhrG3Eo p6jA== X-Gm-Message-State: AOAM533PbFDJsYnxIkfpenWLOgS2vDCnLWgB058oI7763qMrCUJT/J1Z a6n1g6bm8TiKb8AiyMjCZQk= X-Google-Smtp-Source: ABdhPJxusspL3XxpzAX1smg8pNVYGaREmsqR1h9AnCCTJ/0mjdcoR9VTniawwni1Ad19eDHMKiwbBA== X-Received: by 2002:a05:600c:4fd6:: with SMTP id o22mr1880796wmq.45.1628849804786; Fri, 13 Aug 2021 03:16:44 -0700 (PDT) Received: from [10.0.0.3] ([37.165.153.200]) by smtp.gmail.com with ESMTPSA id p5sm1253942wrd.25.2021.08.13.03.16.35 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 13 Aug 2021 03:16:44 -0700 (PDT) Subject: Re: [PATCH] net: drop skbs in napi->rx_list when removing the napi context. To: Phi Nguyen , Eric Dumazet Cc: David Miller , Jakub Kicinski , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , kpsingh@kernel.org, Antoine Tenart , Alexander Lobakin , Wei Wang , Taehee Yoo , =?UTF-8?B?QmrDtnJuIFTDtnBlbA==?= , memxor@gmail.com, netdev , LKML , bpf , linux-kernel-mentees@lists.linuxfoundation.org, syzbot+989efe781c74de1ddb54@syzkaller.appspotmail.com, Mahesh Bandewar References: <20210811235959.1099333-1-phind.uet@gmail.com> <663ac8c4-b0c3-5af6-c3c3-f371e0410a43@gmail.com> From: Eric Dumazet Message-ID: <8f324e46-f05c-42d7-9599-a43de7be17dc@gmail.com> Date: Fri, 13 Aug 2021 12:16:34 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.12.0 MIME-Version: 1.0 In-Reply-To: <663ac8c4-b0c3-5af6-c3c3-f371e0410a43@gmail.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 8/12/21 9:17 PM, Phi Nguyen wrote: > On 8/12/2021 3:07 PM, Eric Dumazet wrote: >> Also I object to this fix. >> >> If packets have been stored temporarily in GRO, they should be >> released at some point, >> normally at the end of a napi poll. >> >> By released, I mean that these packets should reach the upper stack, >> instead of being dropped without >> any notification. >> >> It seems a call to gro_normal_list() is missing somewhere. >> >> Can you find where ? >> >> Thanks ! >> H Eric, > > I think the location that should have a call to gro_normal_list() is __netif_napi_del(). Let say, if the driver call a function that lead to gro_normal_one(), and add a skb to the rx_list while the napi poll is not scheduled, and the driver remove the napi context before a napi poll could be triggered, then the added skb will be lost. > > Actually, this was the first solution that I tried with syzbot (It passed the test too). > Best regards, > Phi I think the real bug is in drivers/net/tun.c It can call napi_gro_frags() and I do not see corresponding napi_complete() This seems completely bogus. Your patch only works around one the many bugs caused by commit 90e33d45940793def6f773b2d528e9f3c84ffdc7 tun: enable napi_gro_frags() for TUN/TAP driver I suggest not adding your patch, because we should fix the root cause. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.0 required=3.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32AF7C4338F for ; Fri, 13 Aug 2021 10:16:54 +0000 (UTC) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D65FA60F57 for ; Fri, 13 Aug 2021 10:16:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D65FA60F57 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.linuxfoundation.org Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 9DE5C40249; Fri, 13 Aug 2021 10:16:53 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 32fekb7dsH4w; Fri, 13 Aug 2021 10:16:49 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 979DE401AF; Fri, 13 Aug 2021 10:16:49 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 6FB08C001A; Fri, 13 Aug 2021 10:16:49 +0000 (UTC) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1104DC000E for ; Fri, 13 Aug 2021 10:16:48 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id EC7F4401C7 for ; Fri, 13 Aug 2021 10:16:47 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1npfVWQ8PhUb for ; Fri, 13 Aug 2021 10:16:46 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by smtp2.osuosl.org (Postfix) with ESMTPS id 75E4F401AF for ; Fri, 13 Aug 2021 10:16:46 +0000 (UTC) Received: by mail-wm1-x330.google.com with SMTP id l10-20020a05600c4f0a00b002e6be174c29so2538898wmq.0 for ; Fri, 13 Aug 2021 03:16:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=qqw1S6wFqpcslkxegqQiXTxlXz74o59gYYNcnKgmY4g=; b=Iac72rAAdZ6AojYmmBS1b45TFPgBlke8aXSGKhPB+5ChYdcMcO7rrOnxRDymsPQTes Sfi0jSDobljIJhCJbdsbs6KE8Wo+0oGJyE0jPPl7KVeVVHxVjzpLS8r78IaunH+YGcvA Cyskeksk0kb8Lc4py22Ak+sLqRYBquFwbX9xBkFi6YUCplmh7QPeDhT+NupyF+dGJJQg a3VCGO977oB0ja8EHzCBvHK5VfyAw8jJpH4DIFE7/HQDTP3Cys8n/WyOt0OkfOvqgIzS UUrd0Pq8lKhUlL44AEvsXjjX2WJ9YT9JYnRG+6DZMzK9iahdYLOv/kww2MyxFbsTnyPK ZLmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=qqw1S6wFqpcslkxegqQiXTxlXz74o59gYYNcnKgmY4g=; b=YI9/pSIyqkhdw77pr38XoWPYe3YMZX7CiKEIluSCj7uaXzuAVfliU19wpHvjw0W+t6 G0isN6jA89W8dTiMSevCWwsaMk782gcDCQ1bcbxS/iRfV55sI3+HFKLsIGGg5E1buFEg Aw/YkP3F/m3HfDbbj4ZaiyMGzpWFfjq4IaGRmpDK7CjnUMMT9soGsZ0su9yYgNgeN7LC H/vMyBgkAtifj36W3kTNnygeaNxUXD36TB/TwDFOM59Fgf0nTDGKF52L0+CWLij2K70Y 4fDDamSObBxjsOG42yvzLZKJOFA+iOjNLzIPi8xbEK655/DK5/LUYcwmbAUuxOcWtFYC fRWw== X-Gm-Message-State: AOAM532jfKI4YmbdTdnQ68cv5WZFLymeZ2Iojor008aiKndKSHWsXXxz RYML7JNgD0TW3KRCQfFs0CQ= X-Google-Smtp-Source: ABdhPJxusspL3XxpzAX1smg8pNVYGaREmsqR1h9AnCCTJ/0mjdcoR9VTniawwni1Ad19eDHMKiwbBA== X-Received: by 2002:a05:600c:4fd6:: with SMTP id o22mr1880796wmq.45.1628849804786; Fri, 13 Aug 2021 03:16:44 -0700 (PDT) Received: from [10.0.0.3] ([37.165.153.200]) by smtp.gmail.com with ESMTPSA id p5sm1253942wrd.25.2021.08.13.03.16.35 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 13 Aug 2021 03:16:44 -0700 (PDT) Subject: Re: [PATCH] net: drop skbs in napi->rx_list when removing the napi context. To: Phi Nguyen , Eric Dumazet References: <20210811235959.1099333-1-phind.uet@gmail.com> <663ac8c4-b0c3-5af6-c3c3-f371e0410a43@gmail.com> From: Eric Dumazet Message-ID: <8f324e46-f05c-42d7-9599-a43de7be17dc@gmail.com> Date: Fri, 13 Aug 2021 12:16:34 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.12.0 MIME-Version: 1.0 In-Reply-To: <663ac8c4-b0c3-5af6-c3c3-f371e0410a43@gmail.com> Content-Language: en-US Cc: Song Liu , Alexander Lobakin , syzbot+989efe781c74de1ddb54@syzkaller.appspotmail.com, Alexei Starovoitov , Wei Wang , Mahesh Bandewar , Daniel Borkmann , John Fastabend , Andrii Nakryiko , Yonghong Song , linux-kernel-mentees@lists.linuxfoundation.org, Jesper Dangaard Brouer , kpsingh@kernel.org, memxor@gmail.com, Jakub Kicinski , netdev , Antoine Tenart , LKML , David Miller , =?UTF-8?B?QmrDtnJuIFTDtnBlbA==?= , bpf , Martin KaFai Lau , Taehee Yoo X-BeenThere: linux-kernel-mentees@lists.linuxfoundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: linux-kernel-mentees-bounces@lists.linuxfoundation.org Sender: "Linux-kernel-mentees" On 8/12/21 9:17 PM, Phi Nguyen wrote: > On 8/12/2021 3:07 PM, Eric Dumazet wrote: >> Also I object to this fix. >> >> If packets have been stored temporarily in GRO, they should be >> released at some point, >> normally at the end of a napi poll. >> >> By released, I mean that these packets should reach the upper stack, >> instead of being dropped without >> any notification. >> >> It seems a call to gro_normal_list() is missing somewhere. >> >> Can you find where ? >> >> Thanks ! >> H Eric, > > I think the location that should have a call to gro_normal_list() is __netif_napi_del(). Let say, if the driver call a function that lead to gro_normal_one(), and add a skb to the rx_list while the napi poll is not scheduled, and the driver remove the napi context before a napi poll could be triggered, then the added skb will be lost. > > Actually, this was the first solution that I tried with syzbot (It passed the test too). > Best regards, > Phi I think the real bug is in drivers/net/tun.c It can call napi_gro_frags() and I do not see corresponding napi_complete() This seems completely bogus. Your patch only works around one the many bugs caused by commit 90e33d45940793def6f773b2d528e9f3c84ffdc7 tun: enable napi_gro_frags() for TUN/TAP driver I suggest not adding your patch, because we should fix the root cause. _______________________________________________ Linux-kernel-mentees mailing list Linux-kernel-mentees@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees