VDPA Debug/Statistics

VDPA Debug/Statistics

[Eli Cohen]

Hi All

Currently, the only statistics we get for a VDPA instance comes from the virtio_net device instance. Since VDPA involves hardware acceleration, there can be quite a lot of information that can be fetched from the underlying device. Currently there is no generic method to fetch this information.

One way of doing this can be to create a the host, a net device for each VDPA instance, and use it to get this information or do some configuration. Ethtool can be used in such a case

I would like to hear what you think about this or maybe you have some other ideas to address this topic.

Thanks,
Eli

Re: VDPA Debug/Statistics

[Michael S. Tsirkin]

On Tue, Aug 11, 2020 at 11:26:20AM +0000, Eli Cohen wrote:
> Hi All
> 
> Currently, the only statistics we get for a VDPA instance comes from the virtio_net device instance. Since VDPA involves hardware acceleration, there can be quite a lot of information that can be fetched from the underlying device. Currently there is no generic method to fetch this information.
> 
> One way of doing this can be to create a the host, a net device for each VDPA instance, and use it to get this information or do some configuration. Ethtool can be used in such a case
> 
> I would like to hear what you think about this or maybe you have some other ideas to address this topic.
> 
> Thanks,
> Eli

Something I'm not sure I understand is how are vdpa instances created
on mellanox cards? There's a devlink command for that, is that right?
Can that be extended for stats?

-- 
MST

RE: VDPA Debug/Statistics

[Eli Cohen]

On Tue, Aug 11, 2020 at 11:26:20AM +0000, Eli Cohen wrote:
> Hi All
> 
> Currently, the only statistics we get for a VDPA instance comes from the virtio_net device instance. Since VDPA involves hardware acceleration, there can be quite a lot of information that can be fetched from the underlying device. Currently there is no generic method to fetch this information.
> 
> One way of doing this can be to create a the host, a net device for 
> each VDPA instance, and use it to get this information or do some 
> configuration. Ethtool can be used in such a case
> 
> I would like to hear what you think about this or maybe you have some other ideas to address this topic.
> 
> Thanks,
> Eli

Something I'm not sure I understand is how are vdpa instances created on mellanox cards? There's a devlink command for that, is that right?
Can that be extended for stats?

Currently any VF will be probed as VDPA device. We're adding devlink support but I am not sure if devlink is suitable for displaying statistics. We will discuss internally but I wanted to know why you guys think.

--
MST

Re: VDPA Debug/Statistics

[Michael S. Tsirkin]

On Tue, Aug 11, 2020 at 11:58:23AM +0000, Eli Cohen wrote:
> On Tue, Aug 11, 2020 at 11:26:20AM +0000, Eli Cohen wrote:
> > Hi All
> > 
> > Currently, the only statistics we get for a VDPA instance comes from the virtio_net device instance. Since VDPA involves hardware acceleration, there can be quite a lot of information that can be fetched from the underlying device. Currently there is no generic method to fetch this information.
> > 
> > One way of doing this can be to create a the host, a net device for 
> > each VDPA instance, and use it to get this information or do some 
> > configuration. Ethtool can be used in such a case
> > 
> > I would like to hear what you think about this or maybe you have some other ideas to address this topic.
> > 
> > Thanks,
> > Eli
> 
> Something I'm not sure I understand is how are vdpa instances created on mellanox cards? There's a devlink command for that, is that right?
> Can that be extended for stats?
> 
> Currently any VF will be probed as VDPA device. We're adding devlink support but I am not sure if devlink is suitable for displaying statistics. We will discuss internally but I wanted to know why you guys think.

OK still things like specifying the mac are managed through rtnetlink,
right?

Right now it does not look like you can mix stats and vf, they are
handled separately:

        if (rtnl_fill_stats(skb, dev))
                goto nla_put_failure;

        if (rtnl_fill_vf(skb, dev, ext_filter_mask))
                goto nla_put_failure;

but ability to query vf stats on the host sounds useful generally.

As another option, we could use a vdpa specific way to retrieve stats,
and teach qemu to report them.




> --
> MST

Re: VDPA Debug/Statistics

[Roopa Prabhu]

On 8/11/20 5:44 AM, Michael S. Tsirkin wrote:
> External email: Use caution opening links or attachments
>
>
> On Tue, Aug 11, 2020 at 11:58:23AM +0000, Eli Cohen wrote:
>> On Tue, Aug 11, 2020 at 11:26:20AM +0000, Eli Cohen wrote:
>>> Hi All
>>>
>>> Currently, the only statistics we get for a VDPA instance comes from the virtio_net device instance. Since VDPA involves hardware acceleration, there can be quite a lot of information that can be fetched from the underlying device. Currently there is no generic method to fetch this information.
>>>
>>> One way of doing this can be to create a the host, a net device for
>>> each VDPA instance, and use it to get this information or do some
>>> configuration. Ethtool can be used in such a case
>>>
>>> I would like to hear what you think about this or maybe you have some other ideas to address this topic.
>>>
>>> Thanks,
>>> Eli
>> Something I'm not sure I understand is how are vdpa instances created on mellanox cards? There's a devlink command for that, is that right?
>> Can that be extended for stats?
>>
>> Currently any VF will be probed as VDPA device. We're adding devlink support but I am not sure if devlink is suitable for displaying statistics. We will discuss internally but I wanted to know why you guys think.
> OK still things like specifying the mac are managed through rtnetlink,
> right?
>
> Right now it does not look like you can mix stats and vf, they are
> handled separately:
>
>          if (rtnl_fill_stats(skb, dev))
>                  goto nla_put_failure;
>
>          if (rtnl_fill_vf(skb, dev, ext_filter_mask))
>                  goto nla_put_failure;
>
> but ability to query vf stats on the host sounds useful generally.
>
> As another option, we could use a vdpa specific way to retrieve stats,
> and teach qemu to report them.

If you are looking for a place to add additional stats, please, check 
the RTM_*STATS api

https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/tree/net/core/rtnetlink.c#n5351
(Its a place where new interface and protocol stats are being added)

Re: VDPA Debug/Statistics

[Jason Wang]

On 2020/8/11 下午7:58, Eli Cohen wrote:
> On Tue, Aug 11, 2020 at 11:26:20AM +0000, Eli Cohen wrote:
>> Hi All
>>
>> Currently, the only statistics we get for a VDPA instance comes from the virtio_net device instance. Since VDPA involves hardware acceleration, there can be quite a lot of information that can be fetched from the underlying device. Currently there is no generic method to fetch this information.
>>
>> One way of doing this can be to create a the host, a net device for
>> each VDPA instance, and use it to get this information or do some
>> configuration. Ethtool can be used in such a case


The problems are:

- vDPA is not net specific
- vDPA should be transparent to host networking stack


>>
>> I would like to hear what you think about this or maybe you have some other ideas to address this topic.
>>
>> Thanks,
>> Eli
> Something I'm not sure I understand is how are vdpa instances created on mellanox cards? There's a devlink command for that, is that right?
> Can that be extended for stats?
>
> Currently any VF will be probed as VDPA device. We're adding devlink support but I am not sure if devlink is suitable for displaying statistics. We will discuss internally but I wanted to know why you guys think.


I agree with Michael, if it's possible, integrating stats with devlink 
should be the best. Having another interface with is just for stats 
looks not good.

Thanks


>
> --
> MST
>