From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=3.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS, URIBL_BLOCKED,USER_AGENT_MUTT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9D37C43381 for ; Wed, 13 Mar 2019 16:33:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 753B22146E for ; Wed, 13 Mar 2019 16:33:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=resnulli-us.20150623.gappssmtp.com header.i=@resnulli-us.20150623.gappssmtp.com header.b="xjAz6C6x" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726730AbfCMQdK (ORCPT ); Wed, 13 Mar 2019 12:33:10 -0400 Received: from mail-wm1-f68.google.com ([209.85.128.68]:53044 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726078AbfCMQdJ (ORCPT ); Wed, 13 Mar 2019 12:33:09 -0400 Received: by mail-wm1-f68.google.com with SMTP id f65so2755707wma.2 for ; Wed, 13 Mar 2019 09:33:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=resnulli-us.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=MBYbQK4Zf6QizZTDOBmBfIcfXiVo/OeD8jpiPNhJ6aI=; b=xjAz6C6x3hbadamnd+kmdzOe1vikO4MEoQR4lrYoeNeBb9i9yjd/VRJlL+7PtyIZ9L 89mjYhiHumcySeOQZPwJ9FZ5qzeU70szCZsYS68RBLEzFflrosbT9Cr/ib7nZnHHGurz uj4ojxCqj8WJVHYmdUjZ/mOf5fGs4yE9HKeRLq7eqg/FZU17dCuWMUa5EpxA7umQdyzO Z6Uk6jPIRKiB7dIInu3jXjyVwBmZQzTooWsay2Lik2B631LnxjARGoph9yEzKq7pQlMM 3B5fAFabNumXTqwm0tZLXcRGQniU9p4mYPOoYLBedFSZVzeK/caD+hTQ+PY4JvdbYpE/ jZTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=MBYbQK4Zf6QizZTDOBmBfIcfXiVo/OeD8jpiPNhJ6aI=; b=JKb+/X6E7PcXpJ0CELQ6z1DG1YlX9VKBPoV8Cl4+nuIdF0DHPnyZ0WusC571FScIDn nOIlK1WTVVBABnpHAyJXqr+DZ0ctg9lSBBfITmpQ6gSe/PkfzT5qC75TM+t+0X53B8Fs Wh377JE2uyXuwmp5BfqoyvmnM5i4S+u72Z4GvZcMEnpzvBa27oWIBxkfEvBI9FpDX8NO psQi+S2K5HSuBKLZWKkQAe9wP/WOVaOxUZA4nQ1MxCg65p2ry5KVYRaD4wqn6GeB1IG5 GRJDrNI4KGkbf/m5sLmAc6HKCymqxJ4rL5JLisME7UVDXqqUpb+wXRbdWjbpMiuZUpzR KM3g== X-Gm-Message-State: APjAAAXdydlXmnAG14sLdTMkq5uAI0SQIQxpdt1xgUgPRY9zHQdkKeSG ofbxJNiQyYoHfpdn190A/eA5Sg== X-Google-Smtp-Source: APXvYqy3MFht9lIM47Ja0IGwrFw97UgJT0ItewiQIw8O83LXprPd6iJpybAoEdkMTxTkp5aKXATLew== X-Received: by 2002:a1c:721a:: with SMTP id n26mr1586388wmc.7.1552494787136; Wed, 13 Mar 2019 09:33:07 -0700 (PDT) Received: from localhost (mail.chocen-mesto.cz. [85.163.43.2]) by smtp.gmail.com with ESMTPSA id a22sm1255270wmj.35.2019.03.13.09.33.06 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 13 Mar 2019 09:33:06 -0700 (PDT) Date: Wed, 13 Mar 2019 17:22:43 +0100 From: Jiri Pirko To: Jakub Kicinski Cc: davem@davemloft.net, netdev@vger.kernel.org, oss-drivers@netronome.com Subject: Re: [PATCH net-next v2 4/7] devlink: allow subports on devlink PCI ports Message-ID: <20190313162243.GB2270@nanopsycho> References: <20190307094816.GA2190@nanopsycho> <20190307185202.2db37490@cakuba.hsd1.ca.comcast.net> <20190308145421.GA2888@nanopsycho.orion> <20190308110943.2ee42bc0@cakuba.hsd1.ca.comcast.net> <20190311085204.GA2194@nanopsycho> <20190311191054.36b801d6@cakuba.hsd1.ca.comcast.net> <20190312140239.GA2455@nanopsycho> <20190312135628.5250135b@cakuba.hsd1.ca.comcast.net> <20190313060701.GB2384@nanopsycho.orion> <20190313091731.76129ece@cakuba.attlocal.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190313091731.76129ece@cakuba.attlocal.net> User-Agent: Mutt/1.10.1 (2018-07-13) Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Wed, Mar 13, 2019 at 05:17:31PM CET, jakub.kicinski@netronome.com wrote: >On Wed, 13 Mar 2019 07:07:01 +0100, Jiri Pirko wrote: >> Tue, Mar 12, 2019 at 09:56:28PM CET, jakub.kicinski@netronome.com wrote: >> >On Tue, 12 Mar 2019 15:02:39 +0100, Jiri Pirko wrote: >> >> Tue, Mar 12, 2019 at 03:10:54AM CET, wrote: >> >> >On Mon, 11 Mar 2019 09:52:04 +0100, Jiri Pirko wrote: >> >> >> Fri, Mar 08, 2019 at 08:09:43PM CET, wrote: >> >> >> >If the switchport is in the hypervisor then only the hypervisor can >> >> >> >control switching/forwarding, correct? >> >> >> >> >> >> Correct. >> >> >> >> >> >> >The primary use case for partitioning within a VM (of a VF) would be >> >> >> >containers (and DPDK)? >> >> >> >> >> >> Makes sense. >> >> >> >> >> >> >SR-IOV makes things harder. Splitting a PF is reasonably easy to grasp. >> >> >> >I'm trying to get a sense of is how would we control an SR-IOV >> >> >> >environment as a whole. >> >> >> >> >> >> You mean orchestration? >> >> > >> >> >Right, orchestration. >> >> > >> >> >To be clear on where I'm going with this - if we want to allow VFs >> >> >to partition themselves then they have to control what is effectively >> >> >a "nested" switch. A per-VF set of rules which would the get >> >> >> >> Wait. If you allow to make VF subports (I believe that is what you ment >> >> by VFs partition themselves), that does not mean they will have a >> >> separate nested switch. They would still belong under the same one. >> > >> >But that existing switch is administered by the hypervisor, how would >> >the VF owners install forwarding rules in a switch they don't control? >> >> They won't. > >Argh. So how is forwarding configured if there are no rules? Are you >going to assume its switching on MACs? We're supposed to offload >software constructs. If its a software port it needs to be explicitly >switched. If it's not explicitly switched - we already have macvlan >offload. Wait a second. You configure the switch. And for that, you have the switchports (representors). What we are talking about are VF (VF subport) host legs. Am I missing something? > >> >> >"flattened" into the main eswitch rule set. If I was to choose I'd >> >> >really rather have this "flattening" be done on the (Linux) hypervisor >> >> >and not in the vendor driver and firmware. >> >> >> >> Agreed. Driver should provide one big switch. User should configure it. >> > >> >Cool, when you say user - is it the tenant or the provider? >> >> Whoever gets access to the instance. >> >> >> >I'd much rather have the VM make a "give me another NIC" orchestration >> >> >call via some high level REST API than devlink. This makes the >> >> >configuration strictly high level to low level: >> >> > >> >> > VM -> cloud net REST API -> cloud agent -> devlink/Linux -> FW -> HW >> >> > >> >> >Without round trips via firmware. >> >> >> >> Okay. So the "devlink/Linux -> FW" part is going to happen on baremetal. >> >> Makes sense. >> >> >> >> >This allows for easy policy enforcement, common code to be maintained >> >> >in user space, in high level languages (no 0.5M LoC drivers and 10M LoC >> >> >firmware for every driver). It can also be used with software paths >> >> >like VirtIO.. >> >> >> >> Agreed. >> >> >> >> >Modelling and debugging a nested switch would be a nightmare. What >> >> >follows is that we probably shouldn't deal with partitioning of VFs, >> >> >but rather only partition via the PF devlink instance, and reassign >> >> >the partitions to VMs. >> >> >> >> Agreed. That must be misunderstanding, I never suggested nested >> >> switches. >> > >> >Cool, yes, I was making sure we weren't going in that direction :) >> >> Okay. >> >> >> >> I originally planned to implement sriov orchestration api in devlink too. >> >> > >> >> >Interesting, would you mind elaborating? >> >> >> >> I have to think about it. But something like this: >> >> [...] >> > >> >I see thanks for the examples, they makes things clear! >> >> Okay. I will put together some documentation including this. I have some >> patches that implement some of the stuff. Your patchset also does some >> of that (considering you adjust a thing or two). Lets make this right. > >Yeah, I feel like I'm again getting further from clarity on what you're >trying to achieve. :)