From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1DFC0C433F5 for ; Wed, 27 Apr 2022 16:54:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243643AbiD0Q5U (ORCPT ); Wed, 27 Apr 2022 12:57:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57082 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243537AbiD0Q5T (ORCPT ); Wed, 27 Apr 2022 12:57:19 -0400 Received: from mail-yw1-x1130.google.com (mail-yw1-x1130.google.com [IPv6:2607:f8b0:4864:20::1130]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2A945583B1 for ; Wed, 27 Apr 2022 09:54:08 -0700 (PDT) Received: by mail-yw1-x1130.google.com with SMTP id 00721157ae682-2ec42eae76bso25615847b3.10 for ; Wed, 27 Apr 2022 09:54:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=wfiHkBYhkuuwagiza//CJWlmobKsPGjvIgfj2Zl77i0=; b=AX0PVJkenkMDyDZfWx53hn0/wD0EZ3SA5eNMPnuVyG0rTAZ59N8AHf66J+mEF3o1d6 ymwrEPrOPEqkHgBwo2lSXDkpikJu6HDEzA6PRnkIhuzpEo+j+hw3tDekOejN+DUEIAwz 6NUCyHm4LHKpm+dIGtLjnZVPp2EP913smV3pmeOETnB09AojJC+WsIQa/HLnXQRsW8ba W0NiGvT+tY8xhsiJmMjlO9qxObMLNm0D8NCPMfZ69xghN0vCeWtJHUnNk88pe7uQ/elm RrVU23hW0WGzJt22dv6/yzcnggu7h8TQ15uyBd0LeLYks1DRVXCkphy7ZMsgbZ6GW/L/ PaeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=wfiHkBYhkuuwagiza//CJWlmobKsPGjvIgfj2Zl77i0=; b=SepskqW4a2EHe5XyHAiz/ccv2D2VYuMCwCKENgmDM5QUNXN4PH/Nio1ddZmsjE+g6R ymhqWEwnpGJia7Kqk9juJoM4L+J8MaZMhs9KYRbtwtWf9kluuQF0c5RU7+x4lpRFnXvJ ytS3fzXE3fmF5zpbLWL15VCLmQftUhFABa6AtO1CE9ciY0XR97Zd0de900u3fEXjciuo fr77x3Yq1XNdzN0vS1VFYSz29IW4eSuu6j6MWv6rWOwVk2BZ9O/pIbZWOz98+KItHQjZ 1Sx9vm7Cj4GdeX3ElSEEoGZj+ZTQhuFYQcxZT7l55Fef427u8JuI0R+NsCEzdEA4RiNl tgTQ== X-Gm-Message-State: AOAM531Q4J+Y9nW9s0MiKgeiRF85mkpEDUbfCWTKfd3KBnpgWSAzHtAD QX87nglffGCA6nSo8SDOlpIjpl1jniI6ZBJ8HsWLEA== X-Google-Smtp-Source: ABdhPJwv7IMex+4x/KuBwlwWVvzt7ZGLpS6u+P36j3lENQFeUqIcaCH3/qQxHaizbB4uHEz+yPW4f3CnmZfirjdKj9c= X-Received: by 2002:a0d:e8c7:0:b0:2f4:cd95:76d8 with SMTP id r190-20020a0de8c7000000b002f4cd9576d8mr27520111ywe.55.1651078447080; Wed, 27 Apr 2022 09:54:07 -0700 (PDT) MIME-Version: 1.0 References: <20220422201237.416238-1-eric.dumazet@gmail.com> In-Reply-To: From: Eric Dumazet Date: Wed, 27 Apr 2022 09:53:55 -0700 Message-ID: Subject: Re: [PATCH v2 net-next] net: generalize skb freeing deferral to per-cpu lists To: Ido Schimmel Cc: Eric Dumazet , "David S . Miller" , Jakub Kicinski , Paolo Abeni , netdev Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On Wed, Apr 27, 2022 at 8:34 AM Ido Schimmel wrote: > > > Eric, with this patch I'm seeing memory leaks such as these [1][2] after > boot. The system is using the igb driver for its management interface > [3]. The leaks disappear after reverting the patch. > > Any ideas? > No idea, skbs allocated to send an ACK can not be stored in receive queue, I guess this is a kmemleak false positive. Stress your host for hours, and check if there are real kmemleaks, as in memory being depleted. > Let me know if you need more info. I can easily test a patch. > > Thanks > > [1] > # cat /sys/kernel/debug/kmemleak > unreferenced object 0xffff888170143740 (size 216): > comm "softirq", pid 0, jiffies 4294825261 (age 95.244s) > hex dump (first 32 bytes): > 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00 00 17 0f 81 88 ff ff 00 00 00 00 00 00 00 00 ................ > backtrace: > [] napi_skb_cache_get+0xf0/0x180 > [] __napi_build_skb+0x1a/0x60 > [] napi_build_skb+0x23/0x350 > [] igb_poll+0x2b72/0x5880 [igb] > [] __napi_poll.constprop.0+0xb4/0x480 > [] net_rx_action+0x40a/0xc60 > [] __do_softirq+0x295/0x9fe > [] __irq_exit_rcu+0x11c/0x180 > [] irq_exit_rcu+0xa/0x20 > [] common_interrupt+0xa9/0xc0 > [] asm_common_interrupt+0x1e/0x40 > [] cpuidle_enter_state+0x27e/0xcb0 > [] cpuidle_enter+0x4f/0xa0 > [] do_idle+0x3b0/0x4b0 > [] cpu_startup_entry+0x19/0x20 > [] start_secondary+0x265/0x340 > > [2] > # cat /sys/kernel/debug/kmemleak > unreferenced object 0xffff88810ce3aac0 (size 216): > comm "softirq", pid 0, jiffies 4294861408 (age 64.607s) > hex dump (first 32 bytes): > 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00 c0 7b 07 81 88 ff ff 00 00 00 00 00 00 00 00 ..{............. > backtrace: > [] __alloc_skb+0x229/0x360 > [] __tcp_send_ack.part.0+0x6c/0x760 > [] tcp_send_ack+0x82/0xa0 > [] __tcp_ack_snd_check+0x15b/0xa00 > [] tcp_rcv_established+0x198e/0x2120 > [] tcp_v4_do_rcv+0x665/0x9a0 > [] tcp_v4_rcv+0x2c1e/0x32f0 > [] ip_protocol_deliver_rcu+0x53/0x2c0 > [] ip_local_deliver+0x3cb/0x620 > [] ip_sublist_rcv_finish+0x9f/0x2c0 > [] ip_list_rcv_finish.constprop.0+0x525/0x6f0 > [] ip_list_rcv+0x318/0x460 > [] __netif_receive_skb_list_core+0x541/0x8f0 > [] netif_receive_skb_list_internal+0x763/0xdc0 > [] napi_gro_complete.constprop.0+0x5a5/0x700 > [] dev_gro_receive+0xf2d/0x23f0 > unreferenced object 0xffff888175e1afc0 (size 216): > comm "sshd", pid 1024, jiffies 4294861424 (age 64.591s) > hex dump (first 32 bytes): > 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00 c0 7b 07 81 88 ff ff 00 00 00 00 00 00 00 00 ..{............. > backtrace: > [] __alloc_skb+0x229/0x360 > [] alloc_skb_with_frags+0x9c/0x720 > [] sock_alloc_send_pskb+0x7b3/0x940 > [] __ip_append_data+0x1874/0x36d0 > [] ip_make_skb+0x263/0x2e0 > [] udp_sendmsg+0x1c8a/0x29d0 > [] inet_sendmsg+0x9e/0xe0 > [] __sys_sendto+0x23d/0x360 > [] __x64_sys_sendto+0xe1/0x1b0 > [] do_syscall_64+0x35/0x80 > [] entry_SYSCALL_64_after_hwframe+0x44/0xae > > [3] > # ethtool -i enp8s0 > driver: igb > version: 5.18.0-rc3-custom-91743-g481c1b > firmware-version: 3.25, 0x80000708, 1.1824.0 > expansion-rom-version: > bus-info: 0000:08:00.0 > supports-statistics: yes > supports-test: yes > supports-eeprom-access: yes > supports-register-dump: yes > supports-priv-flags: yes