From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95162C43331 for ; Wed, 25 Mar 2020 13:33:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 634872078A for ; Wed, 25 Mar 2020 13:33:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="unknown key version" (0-bit key) header.d=tlapnet.cz header.i=@tlapnet.cz header.b="AuD8Fe5X" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727316AbgCYNd1 (ORCPT ); Wed, 25 Mar 2020 09:33:27 -0400 Received: from mail-qk1-f173.google.com ([209.85.222.173]:34205 "EHLO mail-qk1-f173.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727021AbgCYNd1 (ORCPT ); Wed, 25 Mar 2020 09:33:27 -0400 Received: by mail-qk1-f173.google.com with SMTP id i6so2506132qke.1 for ; Wed, 25 Mar 2020 06:33:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tlapnet.cz; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=kprv2m0+xH6rw6KMPHdQkJlrvY3jozE/Ui8LMp4/cxU=; b=AuD8Fe5X60A4FBfq0yINhuH4fVh8O9zOuAVn4ixMyBOKfVhSpyh8BSa05qbIzGsBEs jbWWQ41wozVfcdT5yh4S4BHKXlz9z7t9UOAuaEvvR51wKcRFoTo1wSfkl+AJqkMT3lVW BJa81gR0B3pMqaLV7WN4DaAunEyhEwAuQTG08HAKU40vFMJtbbV9e0nwXlsFFYH0sNDA 1JVaiNCH0M45ggqjG8jBTIJYqD4vDWC+smw4e0PtuP9SotT32IqS4tmh/FicqQ791GME 4znc1/rtKO/vcjV97kwlad431nZHnSBWK4+kYHjr80/mTFFxzMALkPjTjuxQ/gj+ndOi migg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=kprv2m0+xH6rw6KMPHdQkJlrvY3jozE/Ui8LMp4/cxU=; b=ZD3O6K5dlVhNb8yXkRZRDPO0pL8Rg8QJm03fswzNv04JBsY3HlY2e6/qw8gholUqiC 2rMjQxsidzmtFmpGJWhFg9UGibWzP6JoX2fNHNBaS9QzrgJoMptG7S1IKjqyegj+cNNR uy+xhqeHbP7S5FUksy4pu8WxrYTbal3ERTOMSYrKLczf9LaNIWSZ1qTRenqehlTwHO6r eCGxtqnb4xNg2wxtXkFQZME89fnhNdoPB53mYMMnpZ5RRiPqarbVg8n8RNkiwbbbe6sL ElBYe64lTiLbyEzhjfyLFOdCz+JvZJqMe5Kkvpi5uBQL7D+EWelUjn/EZdV2XgOdQ1Wr KWfg== X-Gm-Message-State: ANhLgQ18p/VrWyJUEJDnPA7EOmb1O0VSfFfz6e62VR0SCu1YI6MS7mDl q/iElyU83YbWQfF2yvzz5537aeiSpXf3k1XvP7rAPA== X-Google-Smtp-Source: ADFU+vtUmRcpCaZhN/lUsx95//G2YmdM1GOs6nu9QG/5L1ZMzvYQ0R/30Kt3PSkycA7cM9YjMd3q8WqxVI29uhQfUlA= X-Received: by 2002:a05:620a:109c:: with SMTP id g28mr2849398qkk.409.1585143205314; Wed, 25 Mar 2020 06:33:25 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: =?UTF-8?Q?V=C3=A1clav_Zindulka?= Date: Wed, 25 Mar 2020 14:33:13 +0100 Message-ID: Subject: Re: iproute2: tc deletion freezes whole server To: Cong Wang Cc: Linux Kernel Network Developers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On Wed, Mar 25, 2020 at 12:27 PM V=C3=A1clav Zindulka wrote: > > > > > My testing setup consists of approx. 18k tc class rules and appro= x. > > > > > 13k tc qdisc rules and was altered only with different interface = name.... > > > > > > > > Please share you tc configurations (tc -s -d qd show dev ..., tc > > > > -s -d filter show dev...). > > > > > > I've placed whole reproducers into repository. Do you need exports of= rules too? > > > > > > > Also, it would be great if you can provide a minimal reproducer. > > > > > > I'm afraid that minor reproducer won't cause the problem. This was > > > happening mostly on servers with large tc rule setups. I was trying t= o > > > create small reproducer for nftables developer many times without > > > success. I can try to create reproducer as small as possible, but it > > > may still consist of thousands of rules. > > > > Yeah, this problem is probably TC specific, as we clean up from > > the top qdisc down to each class and each filter. > > As I mentioned earlier, this happens even with specific deletion of > smaller number of rules. Yet I don't oppose it may be caused by tc. > Just inability to process any packets is strange and I'm not sure it > is pure tc problem. > > > Can you try to reproduce the number of TC classes, for example, > > down to half, to see if the problem is gone? This could confirm > > whether it is related to the number of TC classes/filters. > > Sure. I'll try to reduce the size of tc rule set and test the problem fur= ther. I've tested it. Delay shortens and extends according to number of rules to delete. with approx. 3500 rules it took 2364ms with approx. 7000 rules it took 5526ms with approx. 14000 rules it took 11641ms with approx. 18000 rules it took 13302ms with approx. 31000 rules it took 22880ms Do you want me to test it with ifb interface too?