From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D937CC282C2 for ; Wed, 13 Feb 2019 08:37:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B091E222BE for ; Wed, 13 Feb 2019 08:37:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732742AbfBMIhS (ORCPT ); Wed, 13 Feb 2019 03:37:18 -0500 Received: from mx1.redhat.com ([209.132.183.28]:59702 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727345AbfBMIhS (ORCPT ); Wed, 13 Feb 2019 03:37:18 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8D75189ADD; Wed, 13 Feb 2019 08:37:17 +0000 (UTC) Received: from localhost (ovpn-200-19.brq.redhat.com [10.40.200.19]) by smtp.corp.redhat.com (Postfix) with ESMTPS id BA7B660FDF; Wed, 13 Feb 2019 08:37:15 +0000 (UTC) Date: Wed, 13 Feb 2019 09:37:11 +0100 From: Stefano Brivio To: Eric Dumazet Cc: Stephen Hemminger , netdev@vger.kernel.org, Sabrina Dubroca Subject: Re: [PATCH iproute2 net-next v2 3/4] ss: Buffer raw fields first, then render them as a table Message-ID: <20190213093711.13ab560e@redhat.com> In-Reply-To: <82f1bc98-df6d-2b0a-17e5-fa057563284e@gmail.com> References: <82f1bc98-df6d-2b0a-17e5-fa057563284e@gmail.com> Organization: Red Hat MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Wed, 13 Feb 2019 08:37:17 +0000 (UTC) Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On Tue, 12 Feb 2019 16:42:04 -0800 Eric Dumazet wrote: > I do not get it. > > "ss -emoi " uses almost 1KB per socket. > > 10,000,000 sockets -> we need about 10GB of memory ??? > > This is a serious regression. I guess this is rather subjective: the worst case I considered back then was the output of 'ss -tei0' (less than 500 bytes) for one million sockets, which gives 500M of memory, which should in turn be fine on a machine handling one million sockets. Now, if 'ss -emoi' on 10 million sockets is an actual use case (out of curiosity: how are you going to process that output? Would JSON help?), I see two easy options to solve this: 1. flush the output every time we reach a given buffer size (1M perhaps). This might make the resulting blocks slightly unaligned, with occasional loss of readability on lines occurring every 1k to 10k sockets approximately, even though after 1k sockets column sizes won't change much (it looks anyway better than the original), and I don't expect anybody to actually scroll that output 2. add a switch for unbuffered output, but then you need to remember to pass it manually, and the whole output would be as bad as the original in case you need the switch. I'd rather go with 1., it's easy to implement (we already have partial flushing with '--events') and it looks like a good compromise on usability. Thoughts? -- Stefano