From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B008FC43381 for ; Mon, 18 Feb 2019 18:56:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 882472177E for ; Mon, 18 Feb 2019 18:56:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727977AbfBRS4R (ORCPT ); Mon, 18 Feb 2019 13:56:17 -0500 Received: from dispatch1-us1.ppe-hosted.com ([67.231.154.164]:43550 "EHLO dispatch1-us1.ppe-hosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727414AbfBRS4R (ORCPT ); Mon, 18 Feb 2019 13:56:17 -0500 X-Virus-Scanned: Proofpoint Essentials engine Received: from webmail.solarflare.com (webmail.solarflare.com [12.187.104.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mx1-us3.ppe-hosted.com (Proofpoint Essentials ESMTP Server) with ESMTPS id 09AD46C0086; Mon, 18 Feb 2019 18:56:14 +0000 (UTC) Received: from ec-desktop.uk.solarflarecom.com (10.17.20.45) by ocex03.SolarFlarecom.com (10.20.40.36) with Microsoft SMTP Server (TLS) id 15.0.1395.4; Mon, 18 Feb 2019 10:56:08 -0800 Subject: Re: TC stats / hw offload question To: Jamal Hadi Salim , netdev CC: Jiri Pirko , Cong Wang , "Or Gerlitz" , Andy Gospodarek , "PJ Waskiewicz" , Anjali Singhai Jain , Jakub Kicinski References: <26f0cfc9-3bef-8579-72cc-aa6c5ccecd43@solarflare.com> <4cb765dd-453f-3139-bce6-6e0b31167aec@mojatatu.com> <70d77198-42fd-838a-d352-2647e7cad4d6@solarflare.com> <561205a6-101b-c86b-e77d-6ebdcf31a56d@mojatatu.com> <11ab2dc0-ec39-18cd-d170-0d5f954198b9@solarflare.com> <702dd5b7-c6ed-b669-8270-d44f5ff4fb30@mojatatu.com> From: Edward Cree Message-ID: <5930f8e7-3615-5470-ea09-f0e6f2a3a3d7@solarflare.com> Date: Mon, 18 Feb 2019 18:56:06 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2 MIME-Version: 1.0 In-Reply-To: <702dd5b7-c6ed-b669-8270-d44f5ff4fb30@mojatatu.com> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit Content-Language: en-GB X-Originating-IP: [10.17.20.45] X-TM-AS-Product-Ver: SMEX-12.5.0.1300-8.5.1010-24436.000 X-TM-AS-Result: No-13.887200-4.000000-10 X-TMASE-MatchedRID: 6lay9u8oTUMOwH4pD14DsPHkpkyUphL9XsJIQWO/qJXb6Y+fnTZULzuy SPjrCizKGlIk2Jn88imkwHiBmX7jpn1GcR5AeEs704Rmz/agfdzDHSNFHFxB80fLNMv2kATUe56 s8xvuUhDC96w3MrbH0Z7sSga5LeNvIwogOIk60zD0AHJaqpkQfKFI/hOhKe2iMYkmNIO7+hYgT2 /t3DW1rnd35mLO2cbtOdownEibelCf1aLEK0xXOx3EEAbn+GRb6J4k0JZAHPLMInipW2V983cht JI6MELj61nSqo5xsDPi6ToWRF9DCzsbw8P6b6JXFcvGRVnLvYUQtuqs6BbPJ+OxOq7LQlGLJDkG kQuDnwOeMCZsM21JcVPaP0oNjiJF0OxuGGtVaG/AzkaoCiomKkH7wsB5444wCcQK5sMS0d+jxYy RBa/qJUhfECd1rXxDtVC7UY1n6yLKayT/BQTiGgwWxr7XDKH8s5HVQbd+HZ7xmn/+xheKY4NboF gge70QpTWcgjzQSIm9nzfwkSfnjw== X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No X-TMASE-Result: 10--13.887200-4.000000 X-TMASE-Version: SMEX-12.5.0.1300-8.5.1010-24436.000 X-MDID: 1550516175-Bbc-jaKdmTQ4 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On 14/02/19 12:39, Jamal Hadi Salim wrote: > On 2019-02-11 6:44 a.m., Edward Cree wrote: >> My end goal is to implement TC offload in some hw we're designing >>   here at Solarflare.  So I'm trying to determine what hardware is >>   expected/required to do. >> It might be possible to design our new hw so that we can attach a >>   counter to every action, if that's what TC wants. > > It makes sense to have a counter on every action - even if it is > for debugging purposes. The two most basic actions are "drop" or > "accept". In TC speak the default action is "classid x:y" which > typically is to select a queue or give the flow some identity Perhaps I was insufficiently clear; we're only looking at the switching  side of things (e.g. OVS offload) right now; we don't yet have a plan  for 'delivery' filters (I imagine we'll probably initially port over  our ethtool ntuple filter handling from ef10, though we may end up  going down the TC route there). So I think at the moment 'classid' isn't relevant (?) > Note, your counters should also be shareable; example, count all > the drops in one counter across multiple flows as in the following > case where counter index 1 is used. > > tc flower match foo action drop index 1 > tc flower match bar action drop index 1 [...] > allow them to specify > the counter index (assuming you architecture has an indexed table > of counters). Our architecture allocates objects (including counters) and returns  opaque handles to them, so we'd need a software table to connect  counter index to FW counter ID. Also, sharing counters in hw causes extra work for the driver code  that keeps track of which encap actions are getting hit (so it can  keep the neighbour entries alive).  Maybe summing the shared  counters in sw is easier than that, I'm not sure (or maybe encap  action counters should just be kept separate from the counters we  report to TC). TBH I'm coming to the conclusion that what we should do for the first  version of our driver is just to create a counter per rule and report  it against the first action (only), and for now ignore the index (or  maybe require it to be set to some distinguished value, like 0, to  mean 'allocate', so that as a future extension we can support  shareable counters). -Ed