From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2639DC43331 for ; Thu, 7 Nov 2019 20:32:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E795720869 for ; Thu, 7 Nov 2019 20:32:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="tfsumkoA" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725906AbfKGUco (ORCPT ); Thu, 7 Nov 2019 15:32:44 -0500 Received: from mail-pl1-f193.google.com ([209.85.214.193]:39930 "EHLO mail-pl1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725893AbfKGUco (ORCPT ); Thu, 7 Nov 2019 15:32:44 -0500 Received: by mail-pl1-f193.google.com with SMTP id o9so2372689plk.6 for ; Thu, 07 Nov 2019 12:32:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:in-reply-to:references :organization:mime-version:content-transfer-encoding; bh=H1xRS46fM5LD5K/ydfNvqg7rYgxWxCSqEht+PRGUa4U=; b=tfsumkoAdjTKHfYeisbsRDvaxQtbOoI/KGYjE81Yl0PG95lm/ujd3Y4G5wuC8c1nc1 HK0X8h6vOomissKrVlvXzUCVcVgVjU+Rc6vogymUT4FOKIACOC7Rhh/7fGMbcL/Dc8Q5 sBRRY3L8eX8HajS2sBlP0NEuDCge8l6wMteDEQsT5allc+AVpzFzPknkI7kMkp89BWPU Nexa7AwhxcBqhGulxhqBr2lGoNqjlalgk2QZMJHYS0kKDD4l1QS6643BNob8xEpA/C/W 3p/1adKgt8CBcD+Bt9sNv8+RP12YkmJwSE3XBVKqr0Z9ttUHYm4HnjcJebfdAPzxaiMt ulKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:organization:mime-version:content-transfer-encoding; bh=H1xRS46fM5LD5K/ydfNvqg7rYgxWxCSqEht+PRGUa4U=; b=Yx/1FBw7fATN5CdmvVCJT9KgRBjxoxOyOQw1biWN7+xrPiIXuIBwPINwRLJi24TlgH zZqh/7ets78mgKORlO+XyjYe4EGabTXNAt49f1IbaWtZf3XLRV32fWlEh1b1yhsHPnPk 8mBrE6RTodiQ82zNg/lkNCvIQRsd6y+jDdR/9yrwjm4lXrz89LVPg5VcV8doJg9Upnt6 CgAcQEeqqJPZjdzExaSCB5Bw7PeH25CDNcNT1Hhpdjjr9oauPdRAvgXP1+pnzgpbxd6D lYn0ODIKKmvS7K6Zgc0ZgGTBgMCn+3U7Xp5m5WL/PaSRUB3AKW1ZLHAseVstjlJ1pBRg glbA== X-Gm-Message-State: APjAAAXxCCFvO3u8vAKPgMkwLEC6PZFyBhzvHis9XYZholHtzFLiBBpY aQV6S4kLYhDz9yfglmg3N8QeHg== X-Google-Smtp-Source: APXvYqwhMCPR1WhyCfSCTY6X/ImHgiDS2TCZ50mSul13Rm3wAlTshDzZs6tMtEB+yT/FsJxEeI/Fyw== X-Received: by 2002:a17:902:a410:: with SMTP id p16mr5875809plq.184.1573158763275; Thu, 07 Nov 2019 12:32:43 -0800 (PST) Received: from cakuba.netronome.com ([65.196.126.174]) by smtp.gmail.com with ESMTPSA id a66sm3627107pfb.166.2019.11.07.12.32.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Nov 2019 12:32:42 -0800 (PST) Date: Thu, 7 Nov 2019 15:32:34 -0500 From: Jakub Kicinski To: Parav Pandit Cc: alex.williamson@redhat.com, davem@davemloft.net, kvm@vger.kernel.org, netdev@vger.kernel.org, saeedm@mellanox.com, kwankhede@nvidia.com, leon@kernel.org, cohuck@redhat.com, jiri@mellanox.com, linux-rdma@vger.kernel.org, Or Gerlitz Subject: Re: [PATCH net-next 00/19] Mellanox, mlx5 sub function support Message-ID: <20191107153234.0d735c1f@cakuba.netronome.com> In-Reply-To: <20191107160448.20962-1-parav@mellanox.com> References: <20191107160448.20962-1-parav@mellanox.com> Organization: Netronome Systems, Ltd. MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org On Thu, 7 Nov 2019 10:04:48 -0600, Parav Pandit wrote: > Mellanox sub function capability allows users to create several hundreds > of networking and/or rdma devices without depending on PCI SR-IOV support. You call the new port type "sub function" but the devlink port flavour is mdev. As I'm sure you remember you nacked my patches exposing NFP's PCI sub functions which are just regions of the BAR without any mdev capability. Am I in the clear to repost those now? Jiri? > Overview: > --------- > Mellanox ConnectX sub functions are exposed to user as a mediated > device (mdev) [2] as discussed in RFC [3] and further during > netdevconf0x13 at [4]. > > mlx5 mediated device (mdev) enables users to create multiple netdevices > and/or RDMA devices from single PCI function. > > Each mdev maps to a mlx5 sub function. > mlx5 sub function is similar to PCI VF. However it doesn't have its own > PCI function and MSI-X vectors. > > mlx5 mdevs share common PCI resources such as PCI BAR region, > MSI-X interrupts. > > Each mdev has its own window in the PCI BAR region, which is > accessible only to that mdev and applications using it. > > Each mlx5 sub function has its own resource namespace for RDMA resources. > > mdevs are supported when eswitch mode of the devlink instance > is in switchdev mode described in devlink documentation [5]. So presumably the mdevs don't spawn their own devlink instance today, but once mapped via VIRTIO to a VM they will create one? It could be useful to specify. > Network side: > - By default the netdevice and the rdma device of mlx5 mdev cannot send or > receive any packets over the network or to any other mlx5 mdev. Does this mean the frames don't fall back to the repr by default?