From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ECC1AC282D9 for ; Thu, 31 Jan 2019 19:31:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B60C120B1F for ; Thu, 31 Jan 2019 19:31:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="UIe0DB2I" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729636AbfAaTbC (ORCPT ); Thu, 31 Jan 2019 14:31:02 -0500 Received: from mail-qk1-f176.google.com ([209.85.222.176]:32895 "EHLO mail-qk1-f176.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727949AbfAaTbB (ORCPT ); Thu, 31 Jan 2019 14:31:01 -0500 Received: by mail-qk1-f176.google.com with SMTP id d15so2622701qkj.0 for ; Thu, 31 Jan 2019 11:31:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:in-reply-to:references :organization:mime-version:content-transfer-encoding; bh=XaowyX4/dGVWMyVqibyZljBA9R8Kx5vJkd8c80XJdCs=; b=UIe0DB2IJYoRqCXDmY9GRhZLwlawOX/nvC646XbvZrxbxsRsyOsik4REQ9oXBmtNTN +iGD8H3WotPalFpnmNMtvbivQZzIPkl/LclQqw0A02/dkRGWt8FSBxWEV9KxIywi/UKJ JTHusri1G0CURXTisHPEmPevqOqfHIM1lx5KjEV9HjGlh04CcbJiaXoAAsEp9hvvOVYR 1Bnkl79fMT8PR5W6kVNMFR8zt+cK+VlQb9hUrTAjyQi5zBzw4dKspA52vfqUxEDJOiX9 Fx4SZTyslHsWZWaZB5Q8g9BMsQCKBvYl7jBcNJS8Si9n6eNcW+ftVVPw/HMOemaubbB+ ApIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:organization:mime-version:content-transfer-encoding; bh=XaowyX4/dGVWMyVqibyZljBA9R8Kx5vJkd8c80XJdCs=; b=IWwP9YsPbYH/GdOJu51EUndNox2fRVb3bIXoQq1h8XJw9IXqntVNcg4kcJVNbwCqrv WpqBw2d0pttT5lEXf5GgAh+3i4nBGgonUPp3wkkTOobfMEPd1R/rgkxcqrItKWqD9VVa S+HBEjKJILjx1RgfmYNcGMAP5G+wVi80d6IJPPtS4N1t2VQWL5sopx6XrTb+oK4FA61k nGAN6VkLTUSoWBTgaxAprjnYxgazQA+WHDXYFSDfI37PutS9i/X5JS8DlQKTfgtSNPwD LuIMnA6FbPFeBbyr79qf2BK6XS0B5sujZdvCjortkwBbQT5mkJ4hdd8D2KM/3zbU9EF/ UBYw== X-Gm-Message-State: AJcUukdXYRK7lZi4F46uXwKrhVDojmbUx1pJy1jtveHsJ3h8RCYSvnqe o6F3sos1xe73H0DsIMXooKLgAA== X-Google-Smtp-Source: ALg8bN74YMt37bWo6splmOEsf24MzkT+V7D7+Zus+mAnWPfQ54ozhuokzLZv1cDJsNnMXcYYQS2nWA== X-Received: by 2002:a37:7885:: with SMTP id t127mr33421185qkc.323.1548963060002; Thu, 31 Jan 2019 11:31:00 -0800 (PST) Received: from cakuba.hsd1.ca.comcast.net ([66.60.152.14]) by smtp.gmail.com with ESMTPSA id k22sm3738620qtm.73.2019.01.31.11.30.58 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 31 Jan 2019 11:30:59 -0800 (PST) Date: Thu, 31 Jan 2019 11:30:48 -0800 From: Jakub Kicinski To: Roopa Prabhu Cc: David Miller , oss-drivers@netronome.com, netdev , =?UTF-8?B?SmnFmcOtIFDDrXJrbw==?= , Florian Fainelli , Andrew Lunn , Michal Kubecek , David Ahern , Simon Horman , "Brandeburg, Jesse" , maciejromanfijalkowski@gmail.com, vasundhara-v.volam@broadcom.com, Michael Chan , shalomt@mellanox.com, Ido Schimmel Subject: Re: [RFC 00/14] netlink/hierarchical stats Message-ID: <20190131113048.45bd149a@cakuba.hsd1.ca.comcast.net> In-Reply-To: References: <20190128234507.32028-1-jakub.kicinski@netronome.com> <20190130162408.60f1f5dc@cakuba.hsd1.ca.comcast.net> Organization: Netronome Systems, Ltd. MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On Thu, 31 Jan 2019 08:31:51 -0800, Roopa Prabhu wrote: > On Thu, Jan 31, 2019 at 8:16 AM Roopa Prabhu wrote: > > On Wed, Jan 30, 2019 at 4:24 PM Jakub Kicinski wrote: > > > On Wed, 30 Jan 2019 14:14:34 -0800, Roopa Prabhu wrote: > > > > > > My thinking was that we should leave truly custom/strange stats to > > > ethtool API which works quite well for that and at the same time be > > > very accepting of people adding new IDs to HSTAT (only requirement is > > > basically defining the meaning very clearly). > > > > that sounds reasonable. But the 'defining meaning clearly' gets tricky > > sometimes. > > The vendor who gets their ID or meaning first wins :) and the rest > > will have to live with > > ethtool and explain to rest of the world that ethtool is more reliable > > for their hardware :) Right, that's the trade off inherent to standardization. I don't see any way to work around the fact that the definition may not fit all. What I want as a end user and what I want for my customers is the ability to switch the NIC on their system and not spend two months "integrating" into their automation :( If the definition of statistics is not solid we're back to square one. > > I am also concerned that this getting the ID into common HSTAT ID > > space will slow down the process of adding new counters > > for vendors. Which will lead to vendors sticking with ethtool API. I feel like whatever we did here will end up looking much like the ethtool interface, which is why I decided to leave that part out. Ethtool -S works pretty well for custom stats. Standard and structured stats don't fit with it in any way, the two seem best left separate. > > It would be great if people can get all stats in one place and not > > rely on another API for 'more'. One place in the driver or for the user? I'm happy to add the code to ethtool to also dump hstats and render them in a standard way. In fact the tool I have for testing has a "simplified" output format which looks exactly like ethtool -S. One place for the driver to report is hard, as I said I think the custom stats are best left with ethtool. Adding an extra incentive to standardize. > > > For the first stab I looked at two drivers and added all the stats that > > > were common. > > > > > > Given this set is identifying statistics by ID - how would we make that > > > extensible to drivers? Would we go back to strings or have some > > > "driver specific" ID space? > > > > I was looking for ideas from you really, to see if you had considered > > this. agree per driver ID space seems ugly. > > ethtool strings are great today...if we can control the duplication. > > But thinking some more..., i did see some > > patches recently for vendor specific parameter (with ID) space in > > devlink. maybe something like that will be > > reasonable ? I thought about this for a year and I basically came to the conclusion I can't find any perfect solution, if there is one. The devlink parameters are useful, but as anticipated they became the laziest excuse of an ABI... Don't get me started ;) > > > Is there any particular type of statistic you'd expect drivers to want > > > to add? For NICs I think IEEE/RMON should pretty much cover the > > > silicon ones, but I don't know much about switches :) > > > > I will have to go through the list. But switch asics do support > > flexible stats/counters that can be attached at various points. > > And new chip versions come with more support. Having that flexibility > > to expose/extend such stats incrementally is very valuable on a per > > hardware/vendor basis. Yes, I'm not too familiar with those counters. Do they need to be enabled to start counting? Do they have performance impact? Can the "sample" events perf-style? How is the condition on which they trigger defined? Is it maybe just "match a packet and increment a counter"? Would such counters benefit from hierarchical structure? I was trying to cover the long standing use cases - namely the IEEE/RMON stats which all MAC have had for years and per queue stats which all drivers have had for years. But if we can cater to more cases I'm open. > Just want to clarify that I am suggesting a nested HSTATS extension > infra for drivers (just like ethtool). > 'Common stats' stays at the top-level. I got a concept of groups here. The dump generally looks like this: [root group A (say MAC stats)] [sub group RX] [sub group TX] [root group B (say PCIe stats)] [sub group RX] [sub group TX] [root group C (say per-q driver stats] [sub group RX] [q1 group] [q2 group] [q3 group] [sub group TX] [q1 group] [q2 group] [q3 group] Each root group representing a "point in the pipeline". So it's not too hard to add a root group with whatever, the questions are move how would it benefit over existing ethtool if the stats are custom anyway? Hm..