From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8997C43381 for ; Wed, 13 Feb 2019 21:55:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A81FF21872 for ; Wed, 13 Feb 2019 21:55:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=networkplumber-org.20150623.gappssmtp.com header.i=@networkplumber-org.20150623.gappssmtp.com header.b="wAltaOHC" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390000AbfBMVzj (ORCPT ); Wed, 13 Feb 2019 16:55:39 -0500 Received: from mail-pf1-f195.google.com ([209.85.210.195]:33892 "EHLO mail-pf1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388453AbfBMVzj (ORCPT ); Wed, 13 Feb 2019 16:55:39 -0500 Received: by mail-pf1-f195.google.com with SMTP id j18so1841049pfe.1 for ; Wed, 13 Feb 2019 13:55:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HbD1FI/pYz0PC93wjKI1+cpisdHVHluq3+EC2LN4hTY=; b=wAltaOHC0I58m78S66FtFEpt9conzaC+ZSpFmdkg+FxnKzbIVcDiAmyjE8RmG7Chy2 8Cf4MqEUbYk/s3ZPYys/YtPuOqrncKK6fmccCd+tVf+8nrn6z+3z/phxxhZhmChRA+C9 Z9mcruVhB5nFWKdJZ3VuwsImzYrNH1Si+YPuT/Azacn1bpuK36PBj4YjXwTnDm4ugjG3 JNiGq6GEu3VBzDg+XdRVFYg+O03hcFnIwMxrBQEZcrH7v51kVBqecL5tpxVV/tPKJSBM sfKgHhJZxDRB/2fsTZBV53xjxlQsWoUEyLruwap0pxwX5LqVoYv3LyGtgtTRF9JemWWy isCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HbD1FI/pYz0PC93wjKI1+cpisdHVHluq3+EC2LN4hTY=; b=Ci7uQOAjfcC3CbGWpKkm6U5QA3MDBolwxFdh+MxMZxdtSE5nAbMkFUM5+YA7YQxgFP PUYbrRpDs1RTN2mBkeJs3ecHFl3B3yp2Y8b/n9Jg35Oa8OSEdfalOmMnKjKngtsL6rar 9tbVJpKoibluETouqN/6l2A+OCb1ssUc5RU8pSxeD4W+yHfu5DI3o3glhC8ZQEV0HB3i 59zsO84HpEFiCZawqzOxKbnnufDZ9LNnO4/Q+YYqo/hggQxZ38m/cDxZVAI5noIkQize q3skvbzreollwnk3nokZl/oo75aI9XsFCYoJ+zxZq+sZvP/5zpygJbx/F56yGrh1ugpR UC0w== X-Gm-Message-State: AHQUAuZveVsy80y386WNl48MTbq5+cVl1PzC2PltChCWtFIf0n9A9vH7 Sae5QkRNVdHk9od1gQ56vCyImQ== X-Google-Smtp-Source: AHgI3IYIw0wxyMZU6+jPsHmTKkeJg1zf7MXg9IfptNzyRe3W0QWSL704ecBP1IZDJOKV0g8fHhXRLQ== X-Received: by 2002:a62:168e:: with SMTP id 136mr380548pfw.116.1550094938195; Wed, 13 Feb 2019 13:55:38 -0800 (PST) Received: from shemminger-XPS-13-9360 (204-195-22-127.wavecable.com. [204.195.22.127]) by smtp.gmail.com with ESMTPSA id k71sm331632pga.44.2019.02.13.13.55.37 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 13 Feb 2019 13:55:38 -0800 (PST) Date: Wed, 13 Feb 2019 13:55:34 -0800 From: Stephen Hemminger To: Stefano Brivio Cc: Eric Dumazet , netdev@vger.kernel.org, Sabrina Dubroca , David Ahern Subject: Re: [PATCH iproute2 net-next v2 3/4] ss: Buffer raw fields first, then render them as a table Message-ID: <20190213135534.01dacee5@shemminger-XPS-13-9360> In-Reply-To: <20190213221716.5f958c2a@redhat.com> References: <82f1bc98-df6d-2b0a-17e5-fa057563284e@gmail.com> <20190213093711.13ab560e@redhat.com> <20190213221716.5f958c2a@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On Wed, 13 Feb 2019 22:17:16 +0100 Stefano Brivio wrote: > On Wed, 13 Feb 2019 09:31:03 -0800 > Eric Dumazet wrote: > > > On 02/13/2019 12:37 AM, Stefano Brivio wrote: > > > On Tue, 12 Feb 2019 16:42:04 -0800 > > > Eric Dumazet wrote: > > > > > >> I do not get it. > > >> > > >> "ss -emoi " uses almost 1KB per socket. > > >> > > >> 10,000,000 sockets -> we need about 10GB of memory ??? > > >> > > >> This is a serious regression. > > > > > > I guess this is rather subjective: the worst case I considered back then > > > was the output of 'ss -tei0' (less than 500 bytes) for one million > > > sockets, which gives 500M of memory, which should in turn be fine on a > > > machine handling one million sockets. > > > > > > Now, if 'ss -emoi' on 10 million sockets is an actual use case (out of > > > curiosity: how are you going to process that output? Would JSON help?), > > > I see two easy options to solve this: > > > > > > ss -temoi | parser (written in shell or awk or whatever...) > > > > This is a use case, I just got bitten because using ss command > > actually OOM my container, while trying to debug a busy GFE. > > > > The host itself can have 10,000,000 TCP sockets, but usually sysadmin shells > > run in a container with no more than 500 MB available. > > > > Otherwise, it would be too easy for a buggy program to OOM the whole machine > > and have angry customers. > > > > > > > > 1. flush the output every time we reach a given buffer size (1M > > > perhaps). This might make the resulting blocks slightly unaligned, > > > with occasional loss of readability on lines occurring every 1k to > > > 10k sockets approximately, even though after 1k sockets column sizes > > > won't change much (it looks anyway better than the original), and I > > > don't expect anybody to actually scroll that output > > > > > > 2. add a switch for unbuffered output, but then you need to remember to > > > pass it manually, and the whole output would be as bad as the > > > original in case you need the switch. > > > > > > I'd rather go with 1., it's easy to implement (we already have partial > > > flushing with '--events') and it looks like a good compromise on > > > usability. Thoughts? > > > > > > > 1 seems fine, but a switch for 'please do not try to format' would be fine. > > > > I wonder why we try to 'format' when stdout is a pipe or a regular file . > > On a second thought: what about | less, or | grep [ports], > or > readable.log? I guess those might also be rather common use cases, > what do you think? > > I'm tempted to skip this for the moment and just go with option 1. > What I would favor: * use big enough columns that for the common case everything lines up fine * if column is to wide just print that element wider (which is what print %Ns does) and * add json output for programs that want to parse * use print_uint etc for that The buffering patch (in iproute2-next) can/will be reverted.