Re: VDPA Debug/Statistics

From: Roopa Prabhu <roopa@nvidia.com>
To: "Michael S. Tsirkin" <mst@redhat.com>, Eli Cohen <elic@nvidia.com>
Cc: Jason Wang <jasowang@redhat.com>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	"virtualization@lists.linux-foundation.org" 
	<virtualization@lists.linux-foundation.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"eli@mellanox.com" <eli@mellanox.com>,
	"lulu@redhat.com" <lulu@redhat.com>,
	Majd Dibbiny <majd@nvidia.com>, "Maor Dickman" <maord@nvidia.com>,
	Shahaf Shuler <shahafs@mellanox.com>,
	"Parav Pandit" <parav@mellanox.com>
Subject: Re: VDPA Debug/Statistics
Date: Tue, 11 Aug 2020 09:10:52 -0700	[thread overview]
Message-ID: <16cef93e-7421-a151-65ab-ba21e44cd00f@nvidia.com> (raw)
In-Reply-To: <20200811083803-mutt-send-email-mst@kernel.org>


On 8/11/20 5:44 AM, Michael S. Tsirkin wrote:
> External email: Use caution opening links or attachments
>
>
> On Tue, Aug 11, 2020 at 11:58:23AM +0000, Eli Cohen wrote:
>> On Tue, Aug 11, 2020 at 11:26:20AM +0000, Eli Cohen wrote:
>>> Hi All
>>>
>>> Currently, the only statistics we get for a VDPA instance comes from the virtio_net device instance. Since VDPA involves hardware acceleration, there can be quite a lot of information that can be fetched from the underlying device. Currently there is no generic method to fetch this information.
>>>
>>> One way of doing this can be to create a the host, a net device for
>>> each VDPA instance, and use it to get this information or do some
>>> configuration. Ethtool can be used in such a case
>>>
>>> I would like to hear what you think about this or maybe you have some other ideas to address this topic.
>>>
>>> Thanks,
>>> Eli
>> Something I'm not sure I understand is how are vdpa instances created on mellanox cards? There's a devlink command for that, is that right?
>> Can that be extended for stats?
>>
>> Currently any VF will be probed as VDPA device. We're adding devlink support but I am not sure if devlink is suitable for displaying statistics. We will discuss internally but I wanted to know why you guys think.
> OK still things like specifying the mac are managed through rtnetlink,
> right?
>
> Right now it does not look like you can mix stats and vf, they are
> handled separately:
>
>          if (rtnl_fill_stats(skb, dev))
>                  goto nla_put_failure;
>
>          if (rtnl_fill_vf(skb, dev, ext_filter_mask))
>                  goto nla_put_failure;
>
> but ability to query vf stats on the host sounds useful generally.
>
> As another option, we could use a vdpa specific way to retrieve stats,
> and teach qemu to report them.

If you are looking for a place to add additional stats, please, check 
the RTM_*STATS api

https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/tree/net/core/rtnetlink.c#n5351
(Its a place where new interface and protocol stats are being added)