From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5BC87C432C0 for ; Thu, 21 Nov 2019 00:15:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3A6C020730 for ; Thu, 21 Nov 2019 00:15:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726910AbfKUAPJ (ORCPT ); Wed, 20 Nov 2019 19:15:09 -0500 Received: from winds.org ([68.75.195.9]:37910 "EHLO winds.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726343AbfKUAPJ (ORCPT ); Wed, 20 Nov 2019 19:15:09 -0500 Received: by winds.org (Postfix, from userid 100) id 58DC210D2363; Wed, 20 Nov 2019 19:15:07 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by winds.org (Postfix) with ESMTP id 53BBF10636C3; Wed, 20 Nov 2019 19:15:07 -0500 (EST) Date: Wed, 20 Nov 2019 19:15:07 -0500 (EST) From: Byron Stanoszek To: Florian Westphal cc: "David S. Miller" , linux-kernel@vger.kernel.org, netdev@vger.kernel.org Subject: Re: Kernel 5.4 regression - memory leak in network layer In-Reply-To: <20191120202822.GF20235@breakpoint.cc> Message-ID: References: <20191119162222.GA20235@breakpoint.cc> <20191120202822.GF20235@breakpoint.cc> User-Agent: Alpine 2.21.1 (LNX 202 2017-01-01) MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 20 Nov 2019, Florian Westphal wrote: > Florian Westphal wrote: >> Byron Stanoszek wrote: >>> unreferenced object 0xffff88821a48a180 (size 64): >>> comm "softirq", pid 0, jiffies 4294709480 (age 192.558s) >>> hex dump (first 32 bytes): >>> 01 00 00 00 01 06 ff ff 00 00 00 00 00 00 00 00 ................ >>> 00 20 72 3d 82 88 ff ff 00 00 00 00 00 00 00 00 . r=............ >>> backtrace: >>> [<00000000edf73c5e>] skb_ext_add+0xc0/0xf0 >>> [<00000000ca960770>] br_nf_pre_routing+0x171/0x489 >>> [<0000000063a55d83>] br_handle_frame+0x171/0x300 >> >> Brnf related, I will have a look. > > Not reproducible. > > I'm on > > c74386d50fbaf4a54fd3fe560f1abc709c0cff4b ("afs: Fix missing timeout reset"). I confirm I still see the issue on that commit. > Does your setup use any other settings (ethtool, sysctl, qdiscs, tunnels > and the like)? Yeah, I'm using macvlan. Here are my settings: $ ethtool -i eth0 driver: e1000e version: 3.2.6-k firmware-version: 0.13-4 expansion-rom-version: bus-info: 0000:00:1f.6 supports-statistics: yes supports-test: yes supports-eeprom-access: yes supports-register-dump: yes supports-priv-flags: no $ ethtool -i eth1 driver: igb version: 5.6.0-k firmware-version: 3.25, 0x800005cf expansion-rom-version: bus-info: 0000:01:00.0 supports-statistics: yes supports-test: yes supports-eeprom-access: yes supports-register-dump: yes supports-priv-flags: yes Commands to set up network: ethtool -K eth0 tx off rx off ethtool -K eth1 tx off rx off ifconfig lo 127.0.0.1 ifconfig eth0 up brctl addbr br0 brctl addif br0 eth0 brctl setfd br0 0 ifconfig eth1 up brctl addbr br1 brctl addif br1 eth1 brctl setfd br1 0 ifconfig br0 172.17.2.10 netmask 255.255.0.0 ifconfig br1 192.168.0.1 netmask 255.255.255.0 ip l add link br1 mac1 address BE:77:00:00:00:70 type macvlan mode bridge ip l set mac1 up ip a add 192.168.0.70/24 broadcast + dev mac1 $ iptables-save -c # Generated by iptables-save v1.8.3 on Wed Nov 20 17:26:29 2019 *raw :PREROUTING ACCEPT [3701999:2657924997] :OUTPUT ACCEPT [1122825:291796686] COMMIT # Completed on Wed Nov 20 17:26:29 2019 # Generated by iptables-save v1.8.3 on Wed Nov 20 17:26:29 2019 *nat :PREROUTING ACCEPT [612068:41087443] :INPUT ACCEPT [17:2254] :OUTPUT ACCEPT [55:3780] :POSTROUTING ACCEPT [36:2340] [0:0] -A PREROUTING -d 172.17.2.10/32 -i br0 -p tcp -m tcp --dport 102 -j DNAT --to-destination 192.168.0.2 [0:0] -A PREROUTING -d 172.17.2.10/32 -i br0 -p tcp -m tcp --dport 2222 -j DNAT --to-destination 192.168.0.2 [0:0] -A PREROUTING -d 172.17.2.10/32 -i br0 -p tcp -m tcp --dport 5900 -j DNAT --to-destination 192.168.0.4 [0:0] -A PREROUTING -d 172.17.2.10/32 -i br0 -p tcp -m tcp --dport 44818 -j DNAT --to-destination 192.168.0.2 [0:0] -A PREROUTING -d 172.17.2.10/32 -i br0 -p tcp -m tcp --dport 51234 -j DNAT --to-destination 192.168.0.9 [0:0] -A PREROUTING -d 172.17.2.10/32 -i br0 -p tcp -m tcp --dport 51235 -j DNAT --to-destination 192.168.0.9 [0:0] -A PREROUTING -d 172.17.2.10/32 -i br0 -p tcp -m tcp --dport 51236 -j DNAT --to-destination 192.168.0.9 [0:0] -A PREROUTING -d 172.17.2.10/32 -i br0 -p tcp -m tcp --dport 44444 -j DNAT --to-destination 192.168.0.9 [2:120] -A POSTROUTING -o br0 -j MASQUERADE [17:1320] -A POSTROUTING -o br1 -j MASQUERADE [0:0] -A POSTROUTING -o eth2 -j MASQUERADE COMMIT # Completed on Wed Nov 20 17:26:29 2019 # Generated by iptables-save v1.8.3 on Wed Nov 20 17:26:29 2019 *filter :INPUT ACCEPT [3093143:2617432037] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [1122803:291795238] COMMIT # Completed on Wed Nov 20 17:26:29 2019 Setting up another box as IP 172.17.2.11 and 192.168.0.99 and running this command from the original box reliably adds about 2MB of memory marked used according to "free": netperf -H 172.17.2.11 -t UDP_RR or netperf -H 192.168.0.99 -t UDP_RR Local /Remote Socket Size Request Resp. Elapsed Trans. Send Recv Size Size Time Rate bytes Bytes bytes bytes secs. per sec 212992 212992 1 1 10.00 4000.98 212992 212992 Nothing else at the moment is attached to the bridges: $ brctl show bridge name bridge id STP enabled interfaces br0 8000.2046a101b1fb no eth0 br1 8000.2046a101b1fc no eth1 As for network-related sysctls, I've got: # Enable IP Forwarding net.ipv4.ip_forward = 1 # Increase the number of in-flight AF_UNIX datagrams per socket net.unix.max_dgram_qlen = 1000 Regards, -Byron