From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA653C43381 for ; Tue, 12 Mar 2019 02:11:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 559E1214AE for ; Tue, 12 Mar 2019 02:11:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="0ck0GQ7j" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726756AbfCLCLE (ORCPT ); Mon, 11 Mar 2019 22:11:04 -0400 Received: from mail-qk1-f194.google.com ([209.85.222.194]:39013 "EHLO mail-qk1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726727AbfCLCLD (ORCPT ); Mon, 11 Mar 2019 22:11:03 -0400 Received: by mail-qk1-f194.google.com with SMTP id c189so553037qke.6 for ; Mon, 11 Mar 2019 19:11:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:in-reply-to:references :organization:mime-version:content-transfer-encoding; bh=ofeK7EgGKgCetxYf5LzBMM8R4rg45KS6OX1UYxkRFCA=; b=0ck0GQ7j0vy+NRYkd6X7PAWaM4ilkjgaOJi36787bDVi+6TygI4T8jfoL6PIXSTrQL QKVGJYj7DIrTjDQLqXVf7Gj4flNkOzVWYJLMen0+C8c7n52IiRKOkA1RqEJnAqriH5HR +ytzI9PA/whv/6ZSgmFGjedkXGzd0Vr7Y5b78erF6DuDsAVtBzkOxBALbRN+8IE4F108 oF88ARWh56lFaU5/ObZS9u/yvEwcTeb3omIIjm5T0DdOCv85XiBo+km4KZ59y+XPE5fa jRbbMXxtKvRtI+L5Zx50wFo9HH9uPuGR2x3rjUS7GAZLqA2UZm4YB85+wqrmbECxy9it uIIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:organization:mime-version:content-transfer-encoding; bh=ofeK7EgGKgCetxYf5LzBMM8R4rg45KS6OX1UYxkRFCA=; b=XibG1I6f9g5SmbL9h10WsdRjqdG5ht55K253mWFpffUD9cAkhUrQMoqF9SFFB5rMBw +MjFbfl27vV5zrFqXaNxPb+GRT6g6xZTAL8oFhpwMqdLej2Ja36Yx/eUfsRXqME8aNc/ +n9fgjqDgcWe7+Bz063933AD/pgrpDoRVSlf5hHYSlHsUtmQaJT7KAQfI4GMVOwsPySt VAgXwASCahjYlbIdmKrJatyeNwXsVs8qOvL23XqHgAw6E44No4OenxPovbZ7l5zDhcRx ayWis9KCCwQZcd+68bw63IOxNKONX7u43xMjuMWWvYYQEDQilSz/Ez9U4ZOU4IL8b4B2 QPLg== X-Gm-Message-State: APjAAAVlwSoi4U5edvqrSiYVDDHokoKgTiJGXS2dmnBGDnh+oiGpT0xF PRKqCE5JhVg/BdkBVfzxWd3HJg== X-Google-Smtp-Source: APXvYqy7WRXmmRb5gDJNTsqBTNoP6CzmgYtB6Nw6G5s3XvgN0q8ZQBhCzIweWGjV+za+2kd6FMSSsw== X-Received: by 2002:a05:620a:1214:: with SMTP id u20mr11307298qkj.76.1552356662929; Mon, 11 Mar 2019 19:11:02 -0700 (PDT) Received: from cakuba.hsd1.ca.comcast.net ([66.60.152.14]) by smtp.gmail.com with ESMTPSA id i65sm8051956qki.32.2019.03.11.19.11.01 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 11 Mar 2019 19:11:02 -0700 (PDT) Date: Mon, 11 Mar 2019 19:10:54 -0700 From: Jakub Kicinski To: Jiri Pirko Cc: davem@davemloft.net, netdev@vger.kernel.org, oss-drivers@netronome.com Subject: Re: [PATCH net-next v2 4/7] devlink: allow subports on devlink PCI ports Message-ID: <20190311191054.36b801d6@cakuba.hsd1.ca.comcast.net> In-Reply-To: <20190311085204.GA2194@nanopsycho> References: <20190304075609.GV2314@nanopsycho> <20190304163302.7e40219e@cakuba.netronome.com> <20190305110601.GC2314@nanopsycho> <20190305091534.36200de6@cakuba.hsd1.ca.comcast.net> <20190306122037.GB2819@nanopsycho> <20190306095638.7c028bdd@cakuba.hsd1.ca.comcast.net> <20190307094816.GA2190@nanopsycho> <20190307185202.2db37490@cakuba.hsd1.ca.comcast.net> <20190308145421.GA2888@nanopsycho.orion> <20190308110943.2ee42bc0@cakuba.hsd1.ca.comcast.net> <20190311085204.GA2194@nanopsycho> Organization: Netronome Systems, Ltd. MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On Mon, 11 Mar 2019 09:52:04 +0100, Jiri Pirko wrote: > Fri, Mar 08, 2019 at 08:09:43PM CET, jakub.kicinski@netronome.com wrote: > >If the switchport is in the hypervisor then only the hypervisor can > >control switching/forwarding, correct? > > Correct. > > >The primary use case for partitioning within a VM (of a VF) would be > >containers (and DPDK)? > > Makes sense. > > >SR-IOV makes things harder. Splitting a PF is reasonably easy to grasp. > >I'm trying to get a sense of is how would we control an SR-IOV > >environment as a whole. > > You mean orchestration? Right, orchestration. To be clear on where I'm going with this - if we want to allow VFs to partition themselves then they have to control what is effectively a "nested" switch. A per-VF set of rules which would the get "flattened" into the main eswitch rule set. If I was to choose I'd really rather have this "flattening" be done on the (Linux) hypervisor and not in the vendor driver and firmware. I'd much rather have the VM make a "give me another NIC" orchestration call via some high level REST API than devlink. This makes the configuration strictly high level to low level: VM -> cloud net REST API -> cloud agent -> devlink/Linux -> FW -> HW Without round trips via firmware. This allows for easy policy enforcement, common code to be maintained in user space, in high level languages (no 0.5M LoC drivers and 10M LoC firmware for every driver). It can also be used with software paths like VirtIO.. Modelling and debugging a nested switch would be a nightmare. What follows is that we probably shouldn't deal with partitioning of VFs, but rather only partition via the PF devlink instance, and reassign the partitions to VMs. > I originally planned to implement sriov orchestration api in devlink too. Interesting, would you mind elaborating?