[virtio-dev] About custom device counter

[virtio-dev] About custom device counter

[Xuan Zhuo]

We hope to support device custom counter. That is, virtio spec provides a
channel for driver and device, and both key and value are provided by device.

We discussed this issue earlier, and after some internal practice, I think it is
still necessary to discuss this again.

It is very important, each cloud vendor will always have some special counters,
these counters may not exist in another vendor. At the same time, if we have
to discuss it in the spec every time we add a counter, or add a feature, I
think it is very inconvenient. Manufacturers may add some new counters at any
time based on some new requirements. Some counters may also be removed at any
time.

Of course I know that doing this might hurt migration. But what I want to say is,
why does it affect live migration? These counters we plan to give to users
through ethtool -S, and some changes have taken place in the ethtool counters
output by users. Does this have any practical impact? Or do we directly use
some other output in a way, we can clearly tell the user that these counters
may change during the migration process. For example, the driver is migrated
to some new devices. These devices support some new counters. I think users
should be able to see these new counters. These new counters may be the
purpose of the migration.

We don't need to support live migration at this point.

Thanks.

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-comment] About custom device counter

[Xuan Zhuo]

We hope to support device custom counter. That is, virtio spec provides a
channel for driver and device, and both key and value are provided by device.

We discussed this issue earlier, and after some internal practice, I think it is
still necessary to discuss this again.

It is very important, each cloud vendor will always have some special counters,
these counters may not exist in another vendor. At the same time, if we have
to discuss it in the spec every time we add a counter, or add a feature, I
think it is very inconvenient. Manufacturers may add some new counters at any
time based on some new requirements. Some counters may also be removed at any
time.

Of course I know that doing this might hurt migration. But what I want to say is,
why does it affect live migration? These counters we plan to give to users
through ethtool -S, and some changes have taken place in the ethtool counters
output by users. Does this have any practical impact? Or do we directly use
some other output in a way, we can clearly tell the user that these counters
may change during the migration process. For example, the driver is migrated
to some new devices. These devices support some new counters. I think users
should be able to see these new counters. These new counters may be the
purpose of the migration.

We don't need to support live migration at this point.

Thanks.

This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-dev] Re: About custom device counter

[Michael S. Tsirkin]

On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> We hope to support device custom counter. That is, virtio spec provides a
> channel for driver and device, and both key and value are provided by device.
> 
> We discussed this issue earlier, and after some internal practice, I think it is
> still necessary to discuss this again.
> 
> It is very important, each cloud vendor will always have some special counters,
> these counters may not exist in another vendor. At the same time, if we have
> to discuss it in the spec every time we add a counter, or add a feature, I
> think it is very inconvenient. Manufacturers may add some new counters at any
> time based on some new requirements. Some counters may also be removed at any
> time.
> 
> Of course I know that doing this might hurt migration. But what I want to say is,
> why does it affect live migration? These counters we plan to give to users
> through ethtool -S, and some changes have taken place in the ethtool counters
> output by users. Does this have any practical impact? Or do we directly use
> some other output in a way, we can clearly tell the user that these counters
> may change during the migration process. For example, the driver is migrated
> to some new devices. These devices support some new counters. I think users
> should be able to see these new counters. These new counters may be the
> purpose of the migration.
> 
> We don't need to support live migration at this point.
> 
> Thanks.


Why not just document the thing?

-- 
MST


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-comment] Re: About custom device counter

[Michael S. Tsirkin]

On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> We hope to support device custom counter. That is, virtio spec provides a
> channel for driver and device, and both key and value are provided by device.
> 
> We discussed this issue earlier, and after some internal practice, I think it is
> still necessary to discuss this again.
> 
> It is very important, each cloud vendor will always have some special counters,
> these counters may not exist in another vendor. At the same time, if we have
> to discuss it in the spec every time we add a counter, or add a feature, I
> think it is very inconvenient. Manufacturers may add some new counters at any
> time based on some new requirements. Some counters may also be removed at any
> time.
> 
> Of course I know that doing this might hurt migration. But what I want to say is,
> why does it affect live migration? These counters we plan to give to users
> through ethtool -S, and some changes have taken place in the ethtool counters
> output by users. Does this have any practical impact? Or do we directly use
> some other output in a way, we can clearly tell the user that these counters
> may change during the migration process. For example, the driver is migrated
> to some new devices. These devices support some new counters. I think users
> should be able to see these new counters. These new counters may be the
> purpose of the migration.
> 
> We don't need to support live migration at this point.
> 
> Thanks.


Why not just document the thing?

-- 
MST


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-dev] Re: [virtio-comment] Re: About custom device counter

[Xuan Zhuo]

On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > We hope to support device custom counter. That is, virtio spec provides a
> > channel for driver and device, and both key and value are provided by device.
> >
> > We discussed this issue earlier, and after some internal practice, I think it is
> > still necessary to discuss this again.
> >
> > It is very important, each cloud vendor will always have some special counters,
> > these counters may not exist in another vendor. At the same time, if we have
> > to discuss it in the spec every time we add a counter, or add a feature, I
> > think it is very inconvenient. Manufacturers may add some new counters at any
> > time based on some new requirements. Some counters may also be removed at any
> > time.
> >
> > Of course I know that doing this might hurt migration. But what I want to say is,
> > why does it affect live migration? These counters we plan to give to users
> > through ethtool -S, and some changes have taken place in the ethtool counters
> > output by users. Does this have any practical impact? Or do we directly use
> > some other output in a way, we can clearly tell the user that these counters
> > may change during the migration process. For example, the driver is migrated
> > to some new devices. These devices support some new counters. I think users
> > should be able to see these new counters. These new counters may be the
> > purpose of the migration.
> >
> > We don't need to support live migration at this point.
> >
> > Thanks.
>
>
> Why not just document the thing?

Device Counter or Device Stat, we discussed this issue earlier,
So I want to slove the problem migration firstly.


Thanks.


>
> --
> MST
>
>
> This publicly archived list offers a means to provide input to the
> OASIS Virtual I/O Device (VIRTIO) TC.
>
> In order to verify user consent to the Feedback License terms and
> to minimize spam in the list archive, subscription is required
> before posting.
>
> Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> List help: virtio-comment-help@lists.oasis-open.org
> List archive: https://lists.oasis-open.org/archives/virtio-comment/
> Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> Committee: https://www.oasis-open.org/committees/virtio/
> Join OASIS: https://www.oasis-open.org/join/
>

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-comment] Re: About custom device counter

[Xuan Zhuo]

On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > We hope to support device custom counter. That is, virtio spec provides a
> > channel for driver and device, and both key and value are provided by device.
> >
> > We discussed this issue earlier, and after some internal practice, I think it is
> > still necessary to discuss this again.
> >
> > It is very important, each cloud vendor will always have some special counters,
> > these counters may not exist in another vendor. At the same time, if we have
> > to discuss it in the spec every time we add a counter, or add a feature, I
> > think it is very inconvenient. Manufacturers may add some new counters at any
> > time based on some new requirements. Some counters may also be removed at any
> > time.
> >
> > Of course I know that doing this might hurt migration. But what I want to say is,
> > why does it affect live migration? These counters we plan to give to users
> > through ethtool -S, and some changes have taken place in the ethtool counters
> > output by users. Does this have any practical impact? Or do we directly use
> > some other output in a way, we can clearly tell the user that these counters
> > may change during the migration process. For example, the driver is migrated
> > to some new devices. These devices support some new counters. I think users
> > should be able to see these new counters. These new counters may be the
> > purpose of the migration.
> >
> > We don't need to support live migration at this point.
> >
> > Thanks.
>
>
> Why not just document the thing?

Device Counter or Device Stat, we discussed this issue earlier,
So I want to slove the problem migration firstly.


Thanks.


>
> --
> MST
>
>
> This publicly archived list offers a means to provide input to the
> OASIS Virtual I/O Device (VIRTIO) TC.
>
> In order to verify user consent to the Feedback License terms and
> to minimize spam in the list archive, subscription is required
> before posting.
>
> Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> List help: virtio-comment-help@lists.oasis-open.org
> List archive: https://lists.oasis-open.org/archives/virtio-comment/
> Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> Committee: https://www.oasis-open.org/committees/virtio/
> Join OASIS: https://www.oasis-open.org/join/
>

This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-dev] RE: About custom device counter

[Parav Pandit]

> From: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> Sent: Monday, June 12, 2023 2:13 AM
> 
> We hope to support device custom counter. That is, virtio spec provides a
> channel for driver and device, and both key and value are provided by device.
> 
> We discussed this issue earlier, and after some internal practice, I think it is still
> necessary to discuss this again.
> 
> It is very important, each cloud vendor will always have some special counters,
> these counters may not exist in another vendor. At the same time, if we have to
> discuss it in the spec every time we add a counter, or add a feature, I think it is
> very inconvenient. Manufacturers may add some new counters at any time
> based on some new requirements. Some counters may also be removed at any
> time.
>
Debug counters are useful to have. 
I agree that they may be vendor specific.
We often try to abstract and define as much as vendor neutral as possible.

So we should probably first put our efforts to see if more than one vendor can use it.

Assuming there is some vendor specific counters, I imagine to come from a separate vendor specific query command.
This is to keep standard counters to avoid mixing with vend_specific.

> Of course I know that doing this might hurt migration. But what I want to say is,
> why does it affect live migration? These counters we plan to give to users
> through ethtool -S, and some changes have taken place in the ethtool counters
> output by users. Does this have any practical impact? Or do we directly use
> some other output in a way, we can clearly tell the user that these counters may
> change during the migration process. For example, the driver is migrated to
> some new devices. These devices support some new counters. I think users
> should be able to see these new counters. These new counters may be the
> purpose of the migration.

Counters which are well defined non-vendor specific should be able to migrate to destination on best effort basis.
For well defined counters if device A at src support N counters, on migrated destination, it should be N.
This should be able to achieve with feature provisioning infra via group owner command.

> 
> We don't need to support live migration at this point.
When done on best effort level, infra exists, vendor has ability to incrementally implement it.

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-comment] RE: About custom device counter

[Parav Pandit]

> From: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> Sent: Monday, June 12, 2023 2:13 AM
> 
> We hope to support device custom counter. That is, virtio spec provides a
> channel for driver and device, and both key and value are provided by device.
> 
> We discussed this issue earlier, and after some internal practice, I think it is still
> necessary to discuss this again.
> 
> It is very important, each cloud vendor will always have some special counters,
> these counters may not exist in another vendor. At the same time, if we have to
> discuss it in the spec every time we add a counter, or add a feature, I think it is
> very inconvenient. Manufacturers may add some new counters at any time
> based on some new requirements. Some counters may also be removed at any
> time.
>
Debug counters are useful to have. 
I agree that they may be vendor specific.
We often try to abstract and define as much as vendor neutral as possible.

So we should probably first put our efforts to see if more than one vendor can use it.

Assuming there is some vendor specific counters, I imagine to come from a separate vendor specific query command.
This is to keep standard counters to avoid mixing with vend_specific.

> Of course I know that doing this might hurt migration. But what I want to say is,
> why does it affect live migration? These counters we plan to give to users
> through ethtool -S, and some changes have taken place in the ethtool counters
> output by users. Does this have any practical impact? Or do we directly use
> some other output in a way, we can clearly tell the user that these counters may
> change during the migration process. For example, the driver is migrated to
> some new devices. These devices support some new counters. I think users
> should be able to see these new counters. These new counters may be the
> purpose of the migration.

Counters which are well defined non-vendor specific should be able to migrate to destination on best effort basis.
For well defined counters if device A at src support N counters, on migrated destination, it should be N.
This should be able to achieve with feature provisioning infra via group owner command.

> 
> We don't need to support live migration at this point.
When done on best effort level, infra exists, vendor has ability to incrementally implement it.

This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-dev] Re: [virtio-comment] Re: About custom device counter

[Michael S. Tsirkin]

On Mon, Jun 12, 2023 at 07:27:15PM +0800, Xuan Zhuo wrote:
> On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > > We hope to support device custom counter. That is, virtio spec provides a
> > > channel for driver and device, and both key and value are provided by device.
> > >
> > > We discussed this issue earlier, and after some internal practice, I think it is
> > > still necessary to discuss this again.
> > >
> > > It is very important, each cloud vendor will always have some special counters,
> > > these counters may not exist in another vendor. At the same time, if we have
> > > to discuss it in the spec every time we add a counter, or add a feature, I
> > > think it is very inconvenient. Manufacturers may add some new counters at any
> > > time based on some new requirements. Some counters may also be removed at any
> > > time.
> > >
> > > Of course I know that doing this might hurt migration. But what I want to say is,
> > > why does it affect live migration? These counters we plan to give to users
> > > through ethtool -S, and some changes have taken place in the ethtool counters
> > > output by users. Does this have any practical impact? Or do we directly use
> > > some other output in a way, we can clearly tell the user that these counters
> > > may change during the migration process. For example, the driver is migrated
> > > to some new devices. These devices support some new counters. I think users
> > > should be able to see these new counters. These new counters may be the
> > > purpose of the migration.
> > >
> > > We don't need to support live migration at this point.
> > >
> > > Thanks.
> >
> >
> > Why not just document the thing?
> 
> Device Counter or Device Stat, we discussed this issue earlier,
> So I want to slove the problem migration firstly.
> 
> 
> Thanks.

Sorry have trouble parsing this.

Can we get examples of counters from various vendors, so that
it's clear what kind of thing this, and it becomes convincing
that abstracting this in virtio spec is pointless?

Also, am I right assuming that in virtualized settings this is mostly
relevant for hosts? E.g. guests see a vendor neutral virtio but
host wants to figure out source of issues?




> 
> >
> > --
> > MST
> >
> >
> > This publicly archived list offers a means to provide input to the
> > OASIS Virtual I/O Device (VIRTIO) TC.
> >
> > In order to verify user consent to the Feedback License terms and
> > to minimize spam in the list archive, subscription is required
> > before posting.
> >
> > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > List help: virtio-comment-help@lists.oasis-open.org
> > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > Committee: https://www.oasis-open.org/committees/virtio/
> > Join OASIS: https://www.oasis-open.org/join/
> >


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-comment] Re: About custom device counter

[Michael S. Tsirkin]

On Mon, Jun 12, 2023 at 07:27:15PM +0800, Xuan Zhuo wrote:
> On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > > We hope to support device custom counter. That is, virtio spec provides a
> > > channel for driver and device, and both key and value are provided by device.
> > >
> > > We discussed this issue earlier, and after some internal practice, I think it is
> > > still necessary to discuss this again.
> > >
> > > It is very important, each cloud vendor will always have some special counters,
> > > these counters may not exist in another vendor. At the same time, if we have
> > > to discuss it in the spec every time we add a counter, or add a feature, I
> > > think it is very inconvenient. Manufacturers may add some new counters at any
> > > time based on some new requirements. Some counters may also be removed at any
> > > time.
> > >
> > > Of course I know that doing this might hurt migration. But what I want to say is,
> > > why does it affect live migration? These counters we plan to give to users
> > > through ethtool -S, and some changes have taken place in the ethtool counters
> > > output by users. Does this have any practical impact? Or do we directly use
> > > some other output in a way, we can clearly tell the user that these counters
> > > may change during the migration process. For example, the driver is migrated
> > > to some new devices. These devices support some new counters. I think users
> > > should be able to see these new counters. These new counters may be the
> > > purpose of the migration.
> > >
> > > We don't need to support live migration at this point.
> > >
> > > Thanks.
> >
> >
> > Why not just document the thing?
> 
> Device Counter or Device Stat, we discussed this issue earlier,
> So I want to slove the problem migration firstly.
> 
> 
> Thanks.

Sorry have trouble parsing this.

Can we get examples of counters from various vendors, so that
it's clear what kind of thing this, and it becomes convincing
that abstracting this in virtio spec is pointless?

Also, am I right assuming that in virtualized settings this is mostly
relevant for hosts? E.g. guests see a vendor neutral virtio but
host wants to figure out source of issues?




> 
> >
> > --
> > MST
> >
> >
> > This publicly archived list offers a means to provide input to the
> > OASIS Virtual I/O Device (VIRTIO) TC.
> >
> > In order to verify user consent to the Feedback License terms and
> > to minimize spam in the list archive, subscription is required
> > before posting.
> >
> > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > List help: virtio-comment-help@lists.oasis-open.org
> > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > Committee: https://www.oasis-open.org/committees/virtio/
> > Join OASIS: https://www.oasis-open.org/join/
> >


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-dev] Re: [virtio-comment] Re: About custom device counter

[Xuan Zhuo]

On Tue, 13 Jun 2023 05:02:01 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Mon, Jun 12, 2023 at 07:27:15PM +0800, Xuan Zhuo wrote:
> > On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > > > We hope to support device custom counter. That is, virtio spec provides a
> > > > channel for driver and device, and both key and value are provided by device.
> > > >
> > > > We discussed this issue earlier, and after some internal practice, I think it is
> > > > still necessary to discuss this again.
> > > >
> > > > It is very important, each cloud vendor will always have some special counters,
> > > > these counters may not exist in another vendor. At the same time, if we have
> > > > to discuss it in the spec every time we add a counter, or add a feature, I
> > > > think it is very inconvenient. Manufacturers may add some new counters at any
> > > > time based on some new requirements. Some counters may also be removed at any
> > > > time.
> > > >
> > > > Of course I know that doing this might hurt migration. But what I want to say is,
> > > > why does it affect live migration? These counters we plan to give to users
> > > > through ethtool -S, and some changes have taken place in the ethtool counters
> > > > output by users. Does this have any practical impact? Or do we directly use
> > > > some other output in a way, we can clearly tell the user that these counters
> > > > may change during the migration process. For example, the driver is migrated
> > > > to some new devices. These devices support some new counters. I think users
> > > > should be able to see these new counters. These new counters may be the
> > > > purpose of the migration.
> > > >
> > > > We don't need to support live migration at this point.
> > > >
> > > > Thanks.
> > >
> > >
> > > Why not just document the thing?
> >
> > Device Counter or Device Stat, we discussed this issue earlier,
> > So I want to slove the problem migration firstly.
> >
> >
> > Thanks.
>
> Sorry have trouble parsing this.
>
> Can we get examples of counters from various vendors, so that
> it's clear what kind of thing this, and it becomes convincing
> that abstracting this in virtio spec is pointless?

Such as "limit", in a cloud scenario, multiple users purchase different VMs, and
these VMs share the capabilities of the same host. In order to ensure that each
VM will not affect others, the network card(virtio-net) capability of each VM is
limited. When these users purchase VMs, this limit has already been determined.
So if the network card traffic of a vm exceeds the upper limit, packet loss will
occur. It is necessary for us to count these packet losses. And the device
should expose to the user.


Such as "session", our dpu supports tcp connection tracking, but there is an
upper limit to the number of connections, and if it exceeds, packet loss will
also occur.

These are two commonly used scenarios, I hope they can help you.

Thanks.


>
> Also, am I right assuming that in virtualized settings this is mostly
> relevant for hosts? E.g. guests see a vendor neutral virtio but
> host wants to figure out source of issues?
>
>
>
>
> >
> > >
> > > --
> > > MST
> > >
> > >
> > > This publicly archived list offers a means to provide input to the
> > > OASIS Virtual I/O Device (VIRTIO) TC.
> > >
> > > In order to verify user consent to the Feedback License terms and
> > > to minimize spam in the list archive, subscription is required
> > > before posting.
> > >
> > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > List help: virtio-comment-help@lists.oasis-open.org
> > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > Committee: https://www.oasis-open.org/committees/virtio/
> > > Join OASIS: https://www.oasis-open.org/join/
> > >
>

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-comment] Re: About custom device counter

[Xuan Zhuo]

On Tue, 13 Jun 2023 05:02:01 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Mon, Jun 12, 2023 at 07:27:15PM +0800, Xuan Zhuo wrote:
> > On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > > > We hope to support device custom counter. That is, virtio spec provides a
> > > > channel for driver and device, and both key and value are provided by device.
> > > >
> > > > We discussed this issue earlier, and after some internal practice, I think it is
> > > > still necessary to discuss this again.
> > > >
> > > > It is very important, each cloud vendor will always have some special counters,
> > > > these counters may not exist in another vendor. At the same time, if we have
> > > > to discuss it in the spec every time we add a counter, or add a feature, I
> > > > think it is very inconvenient. Manufacturers may add some new counters at any
> > > > time based on some new requirements. Some counters may also be removed at any
> > > > time.
> > > >
> > > > Of course I know that doing this might hurt migration. But what I want to say is,
> > > > why does it affect live migration? These counters we plan to give to users
> > > > through ethtool -S, and some changes have taken place in the ethtool counters
> > > > output by users. Does this have any practical impact? Or do we directly use
> > > > some other output in a way, we can clearly tell the user that these counters
> > > > may change during the migration process. For example, the driver is migrated
> > > > to some new devices. These devices support some new counters. I think users
> > > > should be able to see these new counters. These new counters may be the
> > > > purpose of the migration.
> > > >
> > > > We don't need to support live migration at this point.
> > > >
> > > > Thanks.
> > >
> > >
> > > Why not just document the thing?
> >
> > Device Counter or Device Stat, we discussed this issue earlier,
> > So I want to slove the problem migration firstly.
> >
> >
> > Thanks.
>
> Sorry have trouble parsing this.
>
> Can we get examples of counters from various vendors, so that
> it's clear what kind of thing this, and it becomes convincing
> that abstracting this in virtio spec is pointless?

Such as "limit", in a cloud scenario, multiple users purchase different VMs, and
these VMs share the capabilities of the same host. In order to ensure that each
VM will not affect others, the network card(virtio-net) capability of each VM is
limited. When these users purchase VMs, this limit has already been determined.
So if the network card traffic of a vm exceeds the upper limit, packet loss will
occur. It is necessary for us to count these packet losses. And the device
should expose to the user.


Such as "session", our dpu supports tcp connection tracking, but there is an
upper limit to the number of connections, and if it exceeds, packet loss will
also occur.

These are two commonly used scenarios, I hope they can help you.

Thanks.


>
> Also, am I right assuming that in virtualized settings this is mostly
> relevant for hosts? E.g. guests see a vendor neutral virtio but
> host wants to figure out source of issues?
>
>
>
>
> >
> > >
> > > --
> > > MST
> > >
> > >
> > > This publicly archived list offers a means to provide input to the
> > > OASIS Virtual I/O Device (VIRTIO) TC.
> > >
> > > In order to verify user consent to the Feedback License terms and
> > > to minimize spam in the list archive, subscription is required
> > > before posting.
> > >
> > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > List help: virtio-comment-help@lists.oasis-open.org
> > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > Committee: https://www.oasis-open.org/committees/virtio/
> > > Join OASIS: https://www.oasis-open.org/join/
> > >
>

This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-dev] Re: RE: About custom device counter

[Xuan Zhuo]

On Tue, 13 Jun 2023 01:57:39 +0000, Parav Pandit <parav@nvidia.com> wrote:
>
>
> > From: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > Sent: Monday, June 12, 2023 2:13 AM
> >
> > We hope to support device custom counter. That is, virtio spec provides a
> > channel for driver and device, and both key and value are provided by device.
> >
> > We discussed this issue earlier, and after some internal practice, I think it is still
> > necessary to discuss this again.
> >
> > It is very important, each cloud vendor will always have some special counters,
> > these counters may not exist in another vendor. At the same time, if we have to
> > discuss it in the spec every time we add a counter, or add a feature, I think it is
> > very inconvenient. Manufacturers may add some new counters at any time
> > based on some new requirements. Some counters may also be removed at any
> > time.
> >
> Debug counters are useful to have.
> I agree that they may be vendor specific.
> We often try to abstract and define as much as vendor neutral as possible.
>
> So we should probably first put our efforts to see if more than one vendor can use it.

I agree.

>
> Assuming there is some vendor specific counters, I imagine to come from a separate vendor specific query command.
> This is to keep standard counters to avoid mixing with vend_specific.

Yes, I agree that these are two separate channels.

But I don't think we need to wait for the first job to finish.

It is conceivable that standard counters will be displayed on ethtool in the
future. For counters from different manufacturers, we can display them in
/sys/class/net/eth0/vendor_couters.

Here we can tell the user that these are only for the current device, and may
change during the migration process.


>
> > Of course I know that doing this might hurt migration. But what I want to say is,
> > why does it affect live migration? These counters we plan to give to users
> > through ethtool -S, and some changes have taken place in the ethtool counters
> > output by users. Does this have any practical impact? Or do we directly use
> > some other output in a way, we can clearly tell the user that these counters may
> > change during the migration process. For example, the driver is migrated to
> > some new devices. These devices support some new counters. I think users
> > should be able to see these new counters. These new counters may be the
> > purpose of the migration.
>
> Counters which are well defined non-vendor specific should be able to migrate to destination on best effort basis.
> For well defined counters if device A at src support N counters, on migrated destination, it should be N.
> This should be able to achieve with feature provisioning infra via group owner command.

Yes, for standard counters.

>
> >
> > We don't need to support live migration at this point.
> When done on best effort level, infra exists, vendor has ability to incrementally implement it.


For the standard counters, the manufacturer will be very willing to use it. I
just want to introduce the capabilities of the manufacturer's customized.

Thanks.

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-comment] Re: RE: About custom device counter

[Xuan Zhuo]

On Tue, 13 Jun 2023 01:57:39 +0000, Parav Pandit <parav@nvidia.com> wrote:
>
>
> > From: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > Sent: Monday, June 12, 2023 2:13 AM
> >
> > We hope to support device custom counter. That is, virtio spec provides a
> > channel for driver and device, and both key and value are provided by device.
> >
> > We discussed this issue earlier, and after some internal practice, I think it is still
> > necessary to discuss this again.
> >
> > It is very important, each cloud vendor will always have some special counters,
> > these counters may not exist in another vendor. At the same time, if we have to
> > discuss it in the spec every time we add a counter, or add a feature, I think it is
> > very inconvenient. Manufacturers may add some new counters at any time
> > based on some new requirements. Some counters may also be removed at any
> > time.
> >
> Debug counters are useful to have.
> I agree that they may be vendor specific.
> We often try to abstract and define as much as vendor neutral as possible.
>
> So we should probably first put our efforts to see if more than one vendor can use it.

I agree.

>
> Assuming there is some vendor specific counters, I imagine to come from a separate vendor specific query command.
> This is to keep standard counters to avoid mixing with vend_specific.

Yes, I agree that these are two separate channels.

But I don't think we need to wait for the first job to finish.

It is conceivable that standard counters will be displayed on ethtool in the
future. For counters from different manufacturers, we can display them in
/sys/class/net/eth0/vendor_couters.

Here we can tell the user that these are only for the current device, and may
change during the migration process.


>
> > Of course I know that doing this might hurt migration. But what I want to say is,
> > why does it affect live migration? These counters we plan to give to users
> > through ethtool -S, and some changes have taken place in the ethtool counters
> > output by users. Does this have any practical impact? Or do we directly use
> > some other output in a way, we can clearly tell the user that these counters may
> > change during the migration process. For example, the driver is migrated to
> > some new devices. These devices support some new counters. I think users
> > should be able to see these new counters. These new counters may be the
> > purpose of the migration.
>
> Counters which are well defined non-vendor specific should be able to migrate to destination on best effort basis.
> For well defined counters if device A at src support N counters, on migrated destination, it should be N.
> This should be able to achieve with feature provisioning infra via group owner command.

Yes, for standard counters.

>
> >
> > We don't need to support live migration at this point.
> When done on best effort level, infra exists, vendor has ability to incrementally implement it.


For the standard counters, the manufacturer will be very willing to use it. I
just want to introduce the capabilities of the manufacturer's customized.

Thanks.

This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-dev] Re: [virtio-comment] Re: About custom device counter

[Jason Wang]

On Thu, Jun 15, 2023 at 2:34 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>
> On Tue, 13 Jun 2023 05:02:01 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > On Mon, Jun 12, 2023 at 07:27:15PM +0800, Xuan Zhuo wrote:
> > > On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > > > > We hope to support device custom counter. That is, virtio spec provides a
> > > > > channel for driver and device, and both key and value are provided by device.
> > > > >
> > > > > We discussed this issue earlier, and after some internal practice, I think it is
> > > > > still necessary to discuss this again.
> > > > >
> > > > > It is very important, each cloud vendor will always have some special counters,
> > > > > these counters may not exist in another vendor. At the same time, if we have
> > > > > to discuss it in the spec every time we add a counter, or add a feature, I
> > > > > think it is very inconvenient. Manufacturers may add some new counters at any
> > > > > time based on some new requirements. Some counters may also be removed at any
> > > > > time.
> > > > >
> > > > > Of course I know that doing this might hurt migration. But what I want to say is,
> > > > > why does it affect live migration? These counters we plan to give to users
> > > > > through ethtool -S, and some changes have taken place in the ethtool counters
> > > > > output by users. Does this have any practical impact? Or do we directly use
> > > > > some other output in a way, we can clearly tell the user that these counters
> > > > > may change during the migration process. For example, the driver is migrated
> > > > > to some new devices. These devices support some new counters. I think users
> > > > > should be able to see these new counters. These new counters may be the
> > > > > purpose of the migration.
> > > > >
> > > > > We don't need to support live migration at this point.
> > > > >
> > > > > Thanks.
> > > >
> > > >
> > > > Why not just document the thing?
> > >
> > > Device Counter or Device Stat, we discussed this issue earlier,
> > > So I want to slove the problem migration firstly.
> > >
> > >
> > > Thanks.
> >
> > Sorry have trouble parsing this.
> >
> > Can we get examples of counters from various vendors, so that
> > it's clear what kind of thing this, and it becomes convincing
> > that abstracting this in virtio spec is pointless?
>
> Such as "limit", in a cloud scenario, multiple users purchase different VMs, and
> these VMs share the capabilities of the same host. In order to ensure that each
> VM will not affect others, the network card(virtio-net) capability of each VM is
> limited. When these users purchase VMs, this limit has already been determined.
> So if the network card traffic of a vm exceeds the upper limit, packet loss will
> occur. It is necessary for us to count these packet losses. And the device
> should expose to the user.
>
>
> Such as "session", our dpu supports tcp connection tracking, but there is an
> upper limit to the number of connections, and if it exceeds, packet loss will
> also occur.
>
> These are two commonly used scenarios, I hope they can help you.

But you didn't explain why these can't be abstracted in the virtio spec?

Thanks

>
> Thanks.
>
>
> >
> > Also, am I right assuming that in virtualized settings this is mostly
> > relevant for hosts? E.g. guests see a vendor neutral virtio but
> > host wants to figure out source of issues?
> >
> >
> >
> >
> > >
> > > >
> > > > --
> > > > MST
> > > >
> > > >
> > > > This publicly archived list offers a means to provide input to the
> > > > OASIS Virtual I/O Device (VIRTIO) TC.
> > > >
> > > > In order to verify user consent to the Feedback License terms and
> > > > to minimize spam in the list archive, subscription is required
> > > > before posting.
> > > >
> > > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > > List help: virtio-comment-help@lists.oasis-open.org
> > > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > > Committee: https://www.oasis-open.org/committees/virtio/
> > > > Join OASIS: https://www.oasis-open.org/join/
> > > >
> >
>


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-comment] Re: About custom device counter

[Jason Wang]

On Thu, Jun 15, 2023 at 2:34 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>
> On Tue, 13 Jun 2023 05:02:01 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > On Mon, Jun 12, 2023 at 07:27:15PM +0800, Xuan Zhuo wrote:
> > > On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > > > > We hope to support device custom counter. That is, virtio spec provides a
> > > > > channel for driver and device, and both key and value are provided by device.
> > > > >
> > > > > We discussed this issue earlier, and after some internal practice, I think it is
> > > > > still necessary to discuss this again.
> > > > >
> > > > > It is very important, each cloud vendor will always have some special counters,
> > > > > these counters may not exist in another vendor. At the same time, if we have
> > > > > to discuss it in the spec every time we add a counter, or add a feature, I
> > > > > think it is very inconvenient. Manufacturers may add some new counters at any
> > > > > time based on some new requirements. Some counters may also be removed at any
> > > > > time.
> > > > >
> > > > > Of course I know that doing this might hurt migration. But what I want to say is,
> > > > > why does it affect live migration? These counters we plan to give to users
> > > > > through ethtool -S, and some changes have taken place in the ethtool counters
> > > > > output by users. Does this have any practical impact? Or do we directly use
> > > > > some other output in a way, we can clearly tell the user that these counters
> > > > > may change during the migration process. For example, the driver is migrated
> > > > > to some new devices. These devices support some new counters. I think users
> > > > > should be able to see these new counters. These new counters may be the
> > > > > purpose of the migration.
> > > > >
> > > > > We don't need to support live migration at this point.
> > > > >
> > > > > Thanks.
> > > >
> > > >
> > > > Why not just document the thing?
> > >
> > > Device Counter or Device Stat, we discussed this issue earlier,
> > > So I want to slove the problem migration firstly.
> > >
> > >
> > > Thanks.
> >
> > Sorry have trouble parsing this.
> >
> > Can we get examples of counters from various vendors, so that
> > it's clear what kind of thing this, and it becomes convincing
> > that abstracting this in virtio spec is pointless?
>
> Such as "limit", in a cloud scenario, multiple users purchase different VMs, and
> these VMs share the capabilities of the same host. In order to ensure that each
> VM will not affect others, the network card(virtio-net) capability of each VM is
> limited. When these users purchase VMs, this limit has already been determined.
> So if the network card traffic of a vm exceeds the upper limit, packet loss will
> occur. It is necessary for us to count these packet losses. And the device
> should expose to the user.
>
>
> Such as "session", our dpu supports tcp connection tracking, but there is an
> upper limit to the number of connections, and if it exceeds, packet loss will
> also occur.
>
> These are two commonly used scenarios, I hope they can help you.

But you didn't explain why these can't be abstracted in the virtio spec?

Thanks

>
> Thanks.
>
>
> >
> > Also, am I right assuming that in virtualized settings this is mostly
> > relevant for hosts? E.g. guests see a vendor neutral virtio but
> > host wants to figure out source of issues?
> >
> >
> >
> >
> > >
> > > >
> > > > --
> > > > MST
> > > >
> > > >
> > > > This publicly archived list offers a means to provide input to the
> > > > OASIS Virtual I/O Device (VIRTIO) TC.
> > > >
> > > > In order to verify user consent to the Feedback License terms and
> > > > to minimize spam in the list archive, subscription is required
> > > > before posting.
> > > >
> > > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > > List help: virtio-comment-help@lists.oasis-open.org
> > > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > > Committee: https://www.oasis-open.org/committees/virtio/
> > > > Join OASIS: https://www.oasis-open.org/join/
> > > >
> >
>


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-dev] Re: [virtio-comment] Re: About custom device counter

[Xuan Zhuo]

On Fri, 16 Jun 2023 08:57:01 +0800, Jason Wang <jasowang@redhat.com> wrote:
> On Thu, Jun 15, 2023 at 2:34 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
> >
> > On Tue, 13 Jun 2023 05:02:01 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > On Mon, Jun 12, 2023 at 07:27:15PM +0800, Xuan Zhuo wrote:
> > > > On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > > On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > > > > > We hope to support device custom counter. That is, virtio spec provides a
> > > > > > channel for driver and device, and both key and value are provided by device.
> > > > > >
> > > > > > We discussed this issue earlier, and after some internal practice, I think it is
> > > > > > still necessary to discuss this again.
> > > > > >
> > > > > > It is very important, each cloud vendor will always have some special counters,
> > > > > > these counters may not exist in another vendor. At the same time, if we have
> > > > > > to discuss it in the spec every time we add a counter, or add a feature, I
> > > > > > think it is very inconvenient. Manufacturers may add some new counters at any
> > > > > > time based on some new requirements. Some counters may also be removed at any
> > > > > > time.
> > > > > >
> > > > > > Of course I know that doing this might hurt migration. But what I want to say is,
> > > > > > why does it affect live migration? These counters we plan to give to users
> > > > > > through ethtool -S, and some changes have taken place in the ethtool counters
> > > > > > output by users. Does this have any practical impact? Or do we directly use
> > > > > > some other output in a way, we can clearly tell the user that these counters
> > > > > > may change during the migration process. For example, the driver is migrated
> > > > > > to some new devices. These devices support some new counters. I think users
> > > > > > should be able to see these new counters. These new counters may be the
> > > > > > purpose of the migration.
> > > > > >
> > > > > > We don't need to support live migration at this point.
> > > > > >
> > > > > > Thanks.
> > > > >
> > > > >
> > > > > Why not just document the thing?
> > > >
> > > > Device Counter or Device Stat, we discussed this issue earlier,
> > > > So I want to slove the problem migration firstly.
> > > >
> > > >
> > > > Thanks.
> > >
> > > Sorry have trouble parsing this.
> > >
> > > Can we get examples of counters from various vendors, so that
> > > it's clear what kind of thing this, and it becomes convincing
> > > that abstracting this in virtio spec is pointless?
> >
> > Such as "limit", in a cloud scenario, multiple users purchase different VMs, and
> > these VMs share the capabilities of the same host. In order to ensure that each
> > VM will not affect others, the network card(virtio-net) capability of each VM is
> > limited. When these users purchase VMs, this limit has already been determined.
> > So if the network card traffic of a vm exceeds the upper limit, packet loss will
> > occur. It is necessary for us to count these packet losses. And the device
> > should expose to the user.
> >
> >
> > Such as "session", our dpu supports tcp connection tracking, but there is an
> > upper limit to the number of connections, and if it exceeds, packet loss will
> > also occur.
> >
> > These are two commonly used scenarios, I hope they can help you.
>
> But you didn't explain why these can't be abstracted in the virtio spec?


As far as I know, there are no such concepts, and I don't like to force these
concepts to be abstracted.

Instead of forcing abstraction, I think it is more convenient to directly
provide a vendor channel.

Thanks.


>
> Thanks
>
> >
> > Thanks.
> >
> >
> > >
> > > Also, am I right assuming that in virtualized settings this is mostly
> > > relevant for hosts? E.g. guests see a vendor neutral virtio but
> > > host wants to figure out source of issues?
> > >
> > >
> > >
> > >
> > > >
> > > > >
> > > > > --
> > > > > MST
> > > > >
> > > > >
> > > > > This publicly archived list offers a means to provide input to the
> > > > > OASIS Virtual I/O Device (VIRTIO) TC.
> > > > >
> > > > > In order to verify user consent to the Feedback License terms and
> > > > > to minimize spam in the list archive, subscription is required
> > > > > before posting.
> > > > >
> > > > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > > > List help: virtio-comment-help@lists.oasis-open.org
> > > > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > > > Committee: https://www.oasis-open.org/committees/virtio/
> > > > > Join OASIS: https://www.oasis-open.org/join/
> > > > >
> > >
> >
>

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-comment] Re: About custom device counter

[Xuan Zhuo]

On Fri, 16 Jun 2023 08:57:01 +0800, Jason Wang <jasowang@redhat.com> wrote:
> On Thu, Jun 15, 2023 at 2:34 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
> >
> > On Tue, 13 Jun 2023 05:02:01 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > On Mon, Jun 12, 2023 at 07:27:15PM +0800, Xuan Zhuo wrote:
> > > > On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > > On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > > > > > We hope to support device custom counter. That is, virtio spec provides a
> > > > > > channel for driver and device, and both key and value are provided by device.
> > > > > >
> > > > > > We discussed this issue earlier, and after some internal practice, I think it is
> > > > > > still necessary to discuss this again.
> > > > > >
> > > > > > It is very important, each cloud vendor will always have some special counters,
> > > > > > these counters may not exist in another vendor. At the same time, if we have
> > > > > > to discuss it in the spec every time we add a counter, or add a feature, I
> > > > > > think it is very inconvenient. Manufacturers may add some new counters at any
> > > > > > time based on some new requirements. Some counters may also be removed at any
> > > > > > time.
> > > > > >
> > > > > > Of course I know that doing this might hurt migration. But what I want to say is,
> > > > > > why does it affect live migration? These counters we plan to give to users
> > > > > > through ethtool -S, and some changes have taken place in the ethtool counters
> > > > > > output by users. Does this have any practical impact? Or do we directly use
> > > > > > some other output in a way, we can clearly tell the user that these counters
> > > > > > may change during the migration process. For example, the driver is migrated
> > > > > > to some new devices. These devices support some new counters. I think users
> > > > > > should be able to see these new counters. These new counters may be the
> > > > > > purpose of the migration.
> > > > > >
> > > > > > We don't need to support live migration at this point.
> > > > > >
> > > > > > Thanks.
> > > > >
> > > > >
> > > > > Why not just document the thing?
> > > >
> > > > Device Counter or Device Stat, we discussed this issue earlier,
> > > > So I want to slove the problem migration firstly.
> > > >
> > > >
> > > > Thanks.
> > >
> > > Sorry have trouble parsing this.
> > >
> > > Can we get examples of counters from various vendors, so that
> > > it's clear what kind of thing this, and it becomes convincing
> > > that abstracting this in virtio spec is pointless?
> >
> > Such as "limit", in a cloud scenario, multiple users purchase different VMs, and
> > these VMs share the capabilities of the same host. In order to ensure that each
> > VM will not affect others, the network card(virtio-net) capability of each VM is
> > limited. When these users purchase VMs, this limit has already been determined.
> > So if the network card traffic of a vm exceeds the upper limit, packet loss will
> > occur. It is necessary for us to count these packet losses. And the device
> > should expose to the user.
> >
> >
> > Such as "session", our dpu supports tcp connection tracking, but there is an
> > upper limit to the number of connections, and if it exceeds, packet loss will
> > also occur.
> >
> > These are two commonly used scenarios, I hope they can help you.
>
> But you didn't explain why these can't be abstracted in the virtio spec?


As far as I know, there are no such concepts, and I don't like to force these
concepts to be abstracted.

Instead of forcing abstraction, I think it is more convenient to directly
provide a vendor channel.

Thanks.


>
> Thanks
>
> >
> > Thanks.
> >
> >
> > >
> > > Also, am I right assuming that in virtualized settings this is mostly
> > > relevant for hosts? E.g. guests see a vendor neutral virtio but
> > > host wants to figure out source of issues?
> > >
> > >
> > >
> > >
> > > >
> > > > >
> > > > > --
> > > > > MST
> > > > >
> > > > >
> > > > > This publicly archived list offers a means to provide input to the
> > > > > OASIS Virtual I/O Device (VIRTIO) TC.
> > > > >
> > > > > In order to verify user consent to the Feedback License terms and
> > > > > to minimize spam in the list archive, subscription is required
> > > > > before posting.
> > > > >
> > > > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > > > List help: virtio-comment-help@lists.oasis-open.org
> > > > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > > > Committee: https://www.oasis-open.org/committees/virtio/
> > > > > Join OASIS: https://www.oasis-open.org/join/
> > > > >
> > >
> >
>

This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-dev] Re: [virtio-comment] Re: About custom device counter

[Michael S. Tsirkin]

On Thu, Jun 15, 2023 at 02:23:31PM +0800, Xuan Zhuo wrote:
> On Tue, 13 Jun 2023 05:02:01 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > On Mon, Jun 12, 2023 at 07:27:15PM +0800, Xuan Zhuo wrote:
> > > On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > > > > We hope to support device custom counter. That is, virtio spec provides a
> > > > > channel for driver and device, and both key and value are provided by device.
> > > > >
> > > > > We discussed this issue earlier, and after some internal practice, I think it is
> > > > > still necessary to discuss this again.
> > > > >
> > > > > It is very important, each cloud vendor will always have some special counters,
> > > > > these counters may not exist in another vendor. At the same time, if we have
> > > > > to discuss it in the spec every time we add a counter, or add a feature, I
> > > > > think it is very inconvenient. Manufacturers may add some new counters at any
> > > > > time based on some new requirements. Some counters may also be removed at any
> > > > > time.
> > > > >
> > > > > Of course I know that doing this might hurt migration. But what I want to say is,
> > > > > why does it affect live migration? These counters we plan to give to users
> > > > > through ethtool -S, and some changes have taken place in the ethtool counters
> > > > > output by users. Does this have any practical impact? Or do we directly use
> > > > > some other output in a way, we can clearly tell the user that these counters
> > > > > may change during the migration process. For example, the driver is migrated
> > > > > to some new devices. These devices support some new counters. I think users
> > > > > should be able to see these new counters. These new counters may be the
> > > > > purpose of the migration.
> > > > >
> > > > > We don't need to support live migration at this point.
> > > > >
> > > > > Thanks.
> > > >
> > > >
> > > > Why not just document the thing?
> > >
> > > Device Counter or Device Stat, we discussed this issue earlier,
> > > So I want to slove the problem migration firstly.
> > >
> > >
> > > Thanks.
> >
> > Sorry have trouble parsing this.
> >
> > Can we get examples of counters from various vendors, so that
> > it's clear what kind of thing this, and it becomes convincing
> > that abstracting this in virtio spec is pointless?
> 
> Such as "limit", in a cloud scenario, multiple users purchase different VMs, and
> these VMs share the capabilities of the same host. In order to ensure that each
> VM will not affect others, the network card(virtio-net) capability of each VM is
> limited. When these users purchase VMs, this limit has already been determined.
> So if the network card traffic of a vm exceeds the upper limit, packet loss will
> occur. It is necessary for us to count these packet losses. And the device
> should expose to the user.

OK so here, driver is expected to rate limit yes?

> Such as "session", our dpu supports tcp connection tracking, but there is an
> upper limit to the number of connections, and if it exceeds, packet loss will
> also occur.

What can be done here? Just don't run so many apps?

> These are two commonly used scenarios, I hope they can help you.
> 
> Thanks.
> 
> 
> >
> > Also, am I right assuming that in virtualized settings this is mostly
> > relevant for hosts? E.g. guests see a vendor neutral virtio but
> > host wants to figure out source of issues?
> >
> >
> >
> >
> > >
> > > >
> > > > --
> > > > MST
> > > >
> > > >
> > > > This publicly archived list offers a means to provide input to the
> > > > OASIS Virtual I/O Device (VIRTIO) TC.
> > > >
> > > > In order to verify user consent to the Feedback License terms and
> > > > to minimize spam in the list archive, subscription is required
> > > > before posting.
> > > >
> > > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > > List help: virtio-comment-help@lists.oasis-open.org
> > > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > > Committee: https://www.oasis-open.org/committees/virtio/
> > > > Join OASIS: https://www.oasis-open.org/join/
> > > >
> >


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-comment] Re: About custom device counter

[Michael S. Tsirkin]

On Thu, Jun 15, 2023 at 02:23:31PM +0800, Xuan Zhuo wrote:
> On Tue, 13 Jun 2023 05:02:01 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > On Mon, Jun 12, 2023 at 07:27:15PM +0800, Xuan Zhuo wrote:
> > > On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > > > > We hope to support device custom counter. That is, virtio spec provides a
> > > > > channel for driver and device, and both key and value are provided by device.
> > > > >
> > > > > We discussed this issue earlier, and after some internal practice, I think it is
> > > > > still necessary to discuss this again.
> > > > >
> > > > > It is very important, each cloud vendor will always have some special counters,
> > > > > these counters may not exist in another vendor. At the same time, if we have
> > > > > to discuss it in the spec every time we add a counter, or add a feature, I
> > > > > think it is very inconvenient. Manufacturers may add some new counters at any
> > > > > time based on some new requirements. Some counters may also be removed at any
> > > > > time.
> > > > >
> > > > > Of course I know that doing this might hurt migration. But what I want to say is,
> > > > > why does it affect live migration? These counters we plan to give to users
> > > > > through ethtool -S, and some changes have taken place in the ethtool counters
> > > > > output by users. Does this have any practical impact? Or do we directly use
> > > > > some other output in a way, we can clearly tell the user that these counters
> > > > > may change during the migration process. For example, the driver is migrated
> > > > > to some new devices. These devices support some new counters. I think users
> > > > > should be able to see these new counters. These new counters may be the
> > > > > purpose of the migration.
> > > > >
> > > > > We don't need to support live migration at this point.
> > > > >
> > > > > Thanks.
> > > >
> > > >
> > > > Why not just document the thing?
> > >
> > > Device Counter or Device Stat, we discussed this issue earlier,
> > > So I want to slove the problem migration firstly.
> > >
> > >
> > > Thanks.
> >
> > Sorry have trouble parsing this.
> >
> > Can we get examples of counters from various vendors, so that
> > it's clear what kind of thing this, and it becomes convincing
> > that abstracting this in virtio spec is pointless?
> 
> Such as "limit", in a cloud scenario, multiple users purchase different VMs, and
> these VMs share the capabilities of the same host. In order to ensure that each
> VM will not affect others, the network card(virtio-net) capability of each VM is
> limited. When these users purchase VMs, this limit has already been determined.
> So if the network card traffic of a vm exceeds the upper limit, packet loss will
> occur. It is necessary for us to count these packet losses. And the device
> should expose to the user.

OK so here, driver is expected to rate limit yes?

> Such as "session", our dpu supports tcp connection tracking, but there is an
> upper limit to the number of connections, and if it exceeds, packet loss will
> also occur.

What can be done here? Just don't run so many apps?

> These are two commonly used scenarios, I hope they can help you.
> 
> Thanks.
> 
> 
> >
> > Also, am I right assuming that in virtualized settings this is mostly
> > relevant for hosts? E.g. guests see a vendor neutral virtio but
> > host wants to figure out source of issues?
> >
> >
> >
> >
> > >
> > > >
> > > > --
> > > > MST
> > > >
> > > >
> > > > This publicly archived list offers a means to provide input to the
> > > > OASIS Virtual I/O Device (VIRTIO) TC.
> > > >
> > > > In order to verify user consent to the Feedback License terms and
> > > > to minimize spam in the list archive, subscription is required
> > > > before posting.
> > > >
> > > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > > List help: virtio-comment-help@lists.oasis-open.org
> > > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > > Committee: https://www.oasis-open.org/committees/virtio/
> > > > Join OASIS: https://www.oasis-open.org/join/
> > > >
> >


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-dev] Re: [virtio-comment] Re: About custom device counter

[Michael S. Tsirkin]

On Fri, Jun 16, 2023 at 09:36:45AM +0800, Xuan Zhuo wrote:
> On Fri, 16 Jun 2023 08:57:01 +0800, Jason Wang <jasowang@redhat.com> wrote:
> > On Thu, Jun 15, 2023 at 2:34 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
> > >
> > > On Tue, 13 Jun 2023 05:02:01 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > On Mon, Jun 12, 2023 at 07:27:15PM +0800, Xuan Zhuo wrote:
> > > > > On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > > > On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > > > > > > We hope to support device custom counter. That is, virtio spec provides a
> > > > > > > channel for driver and device, and both key and value are provided by device.
> > > > > > >
> > > > > > > We discussed this issue earlier, and after some internal practice, I think it is
> > > > > > > still necessary to discuss this again.
> > > > > > >
> > > > > > > It is very important, each cloud vendor will always have some special counters,
> > > > > > > these counters may not exist in another vendor. At the same time, if we have
> > > > > > > to discuss it in the spec every time we add a counter, or add a feature, I
> > > > > > > think it is very inconvenient. Manufacturers may add some new counters at any
> > > > > > > time based on some new requirements. Some counters may also be removed at any
> > > > > > > time.
> > > > > > >
> > > > > > > Of course I know that doing this might hurt migration. But what I want to say is,
> > > > > > > why does it affect live migration? These counters we plan to give to users
> > > > > > > through ethtool -S, and some changes have taken place in the ethtool counters
> > > > > > > output by users. Does this have any practical impact? Or do we directly use
> > > > > > > some other output in a way, we can clearly tell the user that these counters
> > > > > > > may change during the migration process. For example, the driver is migrated
> > > > > > > to some new devices. These devices support some new counters. I think users
> > > > > > > should be able to see these new counters. These new counters may be the
> > > > > > > purpose of the migration.
> > > > > > >
> > > > > > > We don't need to support live migration at this point.
> > > > > > >
> > > > > > > Thanks.
> > > > > >
> > > > > >
> > > > > > Why not just document the thing?
> > > > >
> > > > > Device Counter or Device Stat, we discussed this issue earlier,
> > > > > So I want to slove the problem migration firstly.
> > > > >
> > > > >
> > > > > Thanks.
> > > >
> > > > Sorry have trouble parsing this.
> > > >
> > > > Can we get examples of counters from various vendors, so that
> > > > it's clear what kind of thing this, and it becomes convincing
> > > > that abstracting this in virtio spec is pointless?
> > >
> > > Such as "limit", in a cloud scenario, multiple users purchase different VMs, and
> > > these VMs share the capabilities of the same host. In order to ensure that each
> > > VM will not affect others, the network card(virtio-net) capability of each VM is
> > > limited. When these users purchase VMs, this limit has already been determined.
> > > So if the network card traffic of a vm exceeds the upper limit, packet loss will
> > > occur. It is necessary for us to count these packet losses. And the device
> > > should expose to the user.
> > >
> > >
> > > Such as "session", our dpu supports tcp connection tracking, but there is an
> > > upper limit to the number of connections, and if it exceeds, packet loss will
> > > also occur.
> > >
> > > These are two commonly used scenarios, I hope they can help you.
> >
> > But you didn't explain why these can't be abstracted in the virtio spec?
> 
> 
> As far as I know, there are no such concepts, and I don't like to force these
> concepts to be abstracted.
> 
> Instead of forcing abstraction, I think it is more convenient to directly
> provide a vendor channel.
> 
> Thanks.

Bypassing standartiziation always seems convenient short term.


> 
> >
> > Thanks
> >
> > >
> > > Thanks.
> > >
> > >
> > > >
> > > > Also, am I right assuming that in virtualized settings this is mostly
> > > > relevant for hosts? E.g. guests see a vendor neutral virtio but
> > > > host wants to figure out source of issues?
> > > >
> > > >
> > > >
> > > >
> > > > >
> > > > > >
> > > > > > --
> > > > > > MST
> > > > > >
> > > > > >
> > > > > > This publicly archived list offers a means to provide input to the
> > > > > > OASIS Virtual I/O Device (VIRTIO) TC.
> > > > > >
> > > > > > In order to verify user consent to the Feedback License terms and
> > > > > > to minimize spam in the list archive, subscription is required
> > > > > > before posting.
> > > > > >
> > > > > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > > > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > > > > List help: virtio-comment-help@lists.oasis-open.org
> > > > > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > > > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > > > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > > > > Committee: https://www.oasis-open.org/committees/virtio/
> > > > > > Join OASIS: https://www.oasis-open.org/join/
> > > > > >
> > > >
> > >
> >


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-comment] Re: About custom device counter

[Michael S. Tsirkin]

On Fri, Jun 16, 2023 at 09:36:45AM +0800, Xuan Zhuo wrote:
> On Fri, 16 Jun 2023 08:57:01 +0800, Jason Wang <jasowang@redhat.com> wrote:
> > On Thu, Jun 15, 2023 at 2:34 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
> > >
> > > On Tue, 13 Jun 2023 05:02:01 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > On Mon, Jun 12, 2023 at 07:27:15PM +0800, Xuan Zhuo wrote:
> > > > > On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > > > On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > > > > > > We hope to support device custom counter. That is, virtio spec provides a
> > > > > > > channel for driver and device, and both key and value are provided by device.
> > > > > > >
> > > > > > > We discussed this issue earlier, and after some internal practice, I think it is
> > > > > > > still necessary to discuss this again.
> > > > > > >
> > > > > > > It is very important, each cloud vendor will always have some special counters,
> > > > > > > these counters may not exist in another vendor. At the same time, if we have
> > > > > > > to discuss it in the spec every time we add a counter, or add a feature, I
> > > > > > > think it is very inconvenient. Manufacturers may add some new counters at any
> > > > > > > time based on some new requirements. Some counters may also be removed at any
> > > > > > > time.
> > > > > > >
> > > > > > > Of course I know that doing this might hurt migration. But what I want to say is,
> > > > > > > why does it affect live migration? These counters we plan to give to users
> > > > > > > through ethtool -S, and some changes have taken place in the ethtool counters
> > > > > > > output by users. Does this have any practical impact? Or do we directly use
> > > > > > > some other output in a way, we can clearly tell the user that these counters
> > > > > > > may change during the migration process. For example, the driver is migrated
> > > > > > > to some new devices. These devices support some new counters. I think users
> > > > > > > should be able to see these new counters. These new counters may be the
> > > > > > > purpose of the migration.
> > > > > > >
> > > > > > > We don't need to support live migration at this point.
> > > > > > >
> > > > > > > Thanks.
> > > > > >
> > > > > >
> > > > > > Why not just document the thing?
> > > > >
> > > > > Device Counter or Device Stat, we discussed this issue earlier,
> > > > > So I want to slove the problem migration firstly.
> > > > >
> > > > >
> > > > > Thanks.
> > > >
> > > > Sorry have trouble parsing this.
> > > >
> > > > Can we get examples of counters from various vendors, so that
> > > > it's clear what kind of thing this, and it becomes convincing
> > > > that abstracting this in virtio spec is pointless?
> > >
> > > Such as "limit", in a cloud scenario, multiple users purchase different VMs, and
> > > these VMs share the capabilities of the same host. In order to ensure that each
> > > VM will not affect others, the network card(virtio-net) capability of each VM is
> > > limited. When these users purchase VMs, this limit has already been determined.
> > > So if the network card traffic of a vm exceeds the upper limit, packet loss will
> > > occur. It is necessary for us to count these packet losses. And the device
> > > should expose to the user.
> > >
> > >
> > > Such as "session", our dpu supports tcp connection tracking, but there is an
> > > upper limit to the number of connections, and if it exceeds, packet loss will
> > > also occur.
> > >
> > > These are two commonly used scenarios, I hope they can help you.
> >
> > But you didn't explain why these can't be abstracted in the virtio spec?
> 
> 
> As far as I know, there are no such concepts, and I don't like to force these
> concepts to be abstracted.
> 
> Instead of forcing abstraction, I think it is more convenient to directly
> provide a vendor channel.
> 
> Thanks.

Bypassing standartiziation always seems convenient short term.


> 
> >
> > Thanks
> >
> > >
> > > Thanks.
> > >
> > >
> > > >
> > > > Also, am I right assuming that in virtualized settings this is mostly
> > > > relevant for hosts? E.g. guests see a vendor neutral virtio but
> > > > host wants to figure out source of issues?
> > > >
> > > >
> > > >
> > > >
> > > > >
> > > > > >
> > > > > > --
> > > > > > MST
> > > > > >
> > > > > >
> > > > > > This publicly archived list offers a means to provide input to the
> > > > > > OASIS Virtual I/O Device (VIRTIO) TC.
> > > > > >
> > > > > > In order to verify user consent to the Feedback License terms and
> > > > > > to minimize spam in the list archive, subscription is required
> > > > > > before posting.
> > > > > >
> > > > > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > > > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > > > > List help: virtio-comment-help@lists.oasis-open.org
> > > > > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > > > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > > > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > > > > Committee: https://www.oasis-open.org/committees/virtio/
> > > > > > Join OASIS: https://www.oasis-open.org/join/
> > > > > >
> > > >
> > >
> >


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-dev] Re: [virtio-comment] Re: About custom device counter

[Xuan Zhuo]

On Fri, 16 Jun 2023 10:13:23 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Thu, Jun 15, 2023 at 02:23:31PM +0800, Xuan Zhuo wrote:
> > On Tue, 13 Jun 2023 05:02:01 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > On Mon, Jun 12, 2023 at 07:27:15PM +0800, Xuan Zhuo wrote:
> > > > On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > > On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > > > > > We hope to support device custom counter. That is, virtio spec provides a
> > > > > > channel for driver and device, and both key and value are provided by device.
> > > > > >
> > > > > > We discussed this issue earlier, and after some internal practice, I think it is
> > > > > > still necessary to discuss this again.
> > > > > >
> > > > > > It is very important, each cloud vendor will always have some special counters,
> > > > > > these counters may not exist in another vendor. At the same time, if we have
> > > > > > to discuss it in the spec every time we add a counter, or add a feature, I
> > > > > > think it is very inconvenient. Manufacturers may add some new counters at any
> > > > > > time based on some new requirements. Some counters may also be removed at any
> > > > > > time.
> > > > > >
> > > > > > Of course I know that doing this might hurt migration. But what I want to say is,
> > > > > > why does it affect live migration? These counters we plan to give to users
> > > > > > through ethtool -S, and some changes have taken place in the ethtool counters
> > > > > > output by users. Does this have any practical impact? Or do we directly use
> > > > > > some other output in a way, we can clearly tell the user that these counters
> > > > > > may change during the migration process. For example, the driver is migrated
> > > > > > to some new devices. These devices support some new counters. I think users
> > > > > > should be able to see these new counters. These new counters may be the
> > > > > > purpose of the migration.
> > > > > >
> > > > > > We don't need to support live migration at this point.
> > > > > >
> > > > > > Thanks.
> > > > >
> > > > >
> > > > > Why not just document the thing?
> > > >
> > > > Device Counter or Device Stat, we discussed this issue earlier,
> > > > So I want to slove the problem migration firstly.
> > > >
> > > >
> > > > Thanks.
> > >
> > > Sorry have trouble parsing this.
> > >
> > > Can we get examples of counters from various vendors, so that
> > > it's clear what kind of thing this, and it becomes convincing
> > > that abstracting this in virtio spec is pointless?
> >
> > Such as "limit", in a cloud scenario, multiple users purchase different VMs, and
> > these VMs share the capabilities of the same host. In order to ensure that each
> > VM will not affect others, the network card(virtio-net) capability of each VM is
> > limited. When these users purchase VMs, this limit has already been determined.
> > So if the network card traffic of a vm exceeds the upper limit, packet loss will
> > occur. It is necessary for us to count these packet losses. And the device
> > should expose to the user.
>
> OK so here, driver is expected to rate limit yes?

The counter is for the user, not for the driver. Let the user know that some
packets are dropped by rate limit. If they need more traffic, then users may
need to buy a larger network card.

The driver does not know about this.


>
> > Such as "session", our dpu supports tcp connection tracking, but there is an
> > upper limit to the number of connections, and if it exceeds, packet loss will
> > also occur.
>
> What can be done here? Just don't run so many apps?

This function will be used in some special scenarios, I am not particularly
sure. But we do have such a need.

I just gave two examples. In fact, there are many other scenarios. In the cloud
scenario, we still have many strange features for customers. We want to embrace
the virtio, so we're not trying to do something that isn't compliant, but we do
currently have the problem that in our backend, we have a lot of functionality
that can't be exposed to the driver or the guest os by the standard virtio
channels. This actually affects the use of the virtio standard in cloud vendors.
For example, some cloud manufacturers are turning to use custom equipment.

We believe that virtio/virtio-net should also accelerate its evolution and
become more modern, so as to meet the needs of the market.

Thanks.


>
> > These are two commonly used scenarios, I hope they can help you.
> >
> > Thanks.
> >
> >
> > >
> > > Also, am I right assuming that in virtualized settings this is mostly
> > > relevant for hosts? E.g. guests see a vendor neutral virtio but
> > > host wants to figure out source of issues?
> > >
> > >
> > >
> > >
> > > >
> > > > >
> > > > > --
> > > > > MST
> > > > >
> > > > >
> > > > > This publicly archived list offers a means to provide input to the
> > > > > OASIS Virtual I/O Device (VIRTIO) TC.
> > > > >
> > > > > In order to verify user consent to the Feedback License terms and
> > > > > to minimize spam in the list archive, subscription is required
> > > > > before posting.
> > > > >
> > > > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > > > List help: virtio-comment-help@lists.oasis-open.org
> > > > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > > > Committee: https://www.oasis-open.org/committees/virtio/
> > > > > Join OASIS: https://www.oasis-open.org/join/
> > > > >
> > >
>

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-comment] Re: About custom device counter

[Xuan Zhuo]

On Fri, 16 Jun 2023 10:13:23 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Thu, Jun 15, 2023 at 02:23:31PM +0800, Xuan Zhuo wrote:
> > On Tue, 13 Jun 2023 05:02:01 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > On Mon, Jun 12, 2023 at 07:27:15PM +0800, Xuan Zhuo wrote:
> > > > On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > > On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > > > > > We hope to support device custom counter. That is, virtio spec provides a
> > > > > > channel for driver and device, and both key and value are provided by device.
> > > > > >
> > > > > > We discussed this issue earlier, and after some internal practice, I think it is
> > > > > > still necessary to discuss this again.
> > > > > >
> > > > > > It is very important, each cloud vendor will always have some special counters,
> > > > > > these counters may not exist in another vendor. At the same time, if we have
> > > > > > to discuss it in the spec every time we add a counter, or add a feature, I
> > > > > > think it is very inconvenient. Manufacturers may add some new counters at any
> > > > > > time based on some new requirements. Some counters may also be removed at any
> > > > > > time.
> > > > > >
> > > > > > Of course I know that doing this might hurt migration. But what I want to say is,
> > > > > > why does it affect live migration? These counters we plan to give to users
> > > > > > through ethtool -S, and some changes have taken place in the ethtool counters
> > > > > > output by users. Does this have any practical impact? Or do we directly use
> > > > > > some other output in a way, we can clearly tell the user that these counters
> > > > > > may change during the migration process. For example, the driver is migrated
> > > > > > to some new devices. These devices support some new counters. I think users
> > > > > > should be able to see these new counters. These new counters may be the
> > > > > > purpose of the migration.
> > > > > >
> > > > > > We don't need to support live migration at this point.
> > > > > >
> > > > > > Thanks.
> > > > >
> > > > >
> > > > > Why not just document the thing?
> > > >
> > > > Device Counter or Device Stat, we discussed this issue earlier,
> > > > So I want to slove the problem migration firstly.
> > > >
> > > >
> > > > Thanks.
> > >
> > > Sorry have trouble parsing this.
> > >
> > > Can we get examples of counters from various vendors, so that
> > > it's clear what kind of thing this, and it becomes convincing
> > > that abstracting this in virtio spec is pointless?
> >
> > Such as "limit", in a cloud scenario, multiple users purchase different VMs, and
> > these VMs share the capabilities of the same host. In order to ensure that each
> > VM will not affect others, the network card(virtio-net) capability of each VM is
> > limited. When these users purchase VMs, this limit has already been determined.
> > So if the network card traffic of a vm exceeds the upper limit, packet loss will
> > occur. It is necessary for us to count these packet losses. And the device
> > should expose to the user.
>
> OK so here, driver is expected to rate limit yes?

The counter is for the user, not for the driver. Let the user know that some
packets are dropped by rate limit. If they need more traffic, then users may
need to buy a larger network card.

The driver does not know about this.


>
> > Such as "session", our dpu supports tcp connection tracking, but there is an
> > upper limit to the number of connections, and if it exceeds, packet loss will
> > also occur.
>
> What can be done here? Just don't run so many apps?

This function will be used in some special scenarios, I am not particularly
sure. But we do have such a need.

I just gave two examples. In fact, there are many other scenarios. In the cloud
scenario, we still have many strange features for customers. We want to embrace
the virtio, so we're not trying to do something that isn't compliant, but we do
currently have the problem that in our backend, we have a lot of functionality
that can't be exposed to the driver or the guest os by the standard virtio
channels. This actually affects the use of the virtio standard in cloud vendors.
For example, some cloud manufacturers are turning to use custom equipment.

We believe that virtio/virtio-net should also accelerate its evolution and
become more modern, so as to meet the needs of the market.

Thanks.


>
> > These are two commonly used scenarios, I hope they can help you.
> >
> > Thanks.
> >
> >
> > >
> > > Also, am I right assuming that in virtualized settings this is mostly
> > > relevant for hosts? E.g. guests see a vendor neutral virtio but
> > > host wants to figure out source of issues?
> > >
> > >
> > >
> > >
> > > >
> > > > >
> > > > > --
> > > > > MST
> > > > >
> > > > >
> > > > > This publicly archived list offers a means to provide input to the
> > > > > OASIS Virtual I/O Device (VIRTIO) TC.
> > > > >
> > > > > In order to verify user consent to the Feedback License terms and
> > > > > to minimize spam in the list archive, subscription is required
> > > > > before posting.
> > > > >
> > > > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > > > List help: virtio-comment-help@lists.oasis-open.org
> > > > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > > > Committee: https://www.oasis-open.org/committees/virtio/
> > > > > Join OASIS: https://www.oasis-open.org/join/
> > > > >
> > >
>

This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-dev] RE: [virtio-comment] Re: About custom device counter

[Parav Pandit]

> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Friday, June 16, 2023 10:13 AM
> > Such as "limit", in a cloud scenario, multiple users purchase
> > different VMs, and these VMs share the capabilities of the same host.
> > In order to ensure that each VM will not affect others, the network
> > card(virtio-net) capability of each VM is limited. When these users purchase
> VMs, this limit has already been determined.
> > So if the network card traffic of a vm exceeds the upper limit, packet
> > loss will occur. It is necessary for us to count these packet losses.
> > And the device should expose to the user.
> 
> OK so here, driver is expected to rate limit yes?
> 
Some implementations of txq are lossy and some are not creating backpressure/flow control to driver so driver can rate limit it naturally.
So a tx packet drop counter is needed to cover the lossy implementations which is abstract enough regardless of reason on why device dropped it.
A more granular counter becomes vendor specific that we can possibly avoid or place under different command.

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

RE: [virtio-comment] Re: About custom device counter

[Parav Pandit]

> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Friday, June 16, 2023 10:13 AM
> > Such as "limit", in a cloud scenario, multiple users purchase
> > different VMs, and these VMs share the capabilities of the same host.
> > In order to ensure that each VM will not affect others, the network
> > card(virtio-net) capability of each VM is limited. When these users purchase
> VMs, this limit has already been determined.
> > So if the network card traffic of a vm exceeds the upper limit, packet
> > loss will occur. It is necessary for us to count these packet losses.
> > And the device should expose to the user.
> 
> OK so here, driver is expected to rate limit yes?
> 
Some implementations of txq are lossy and some are not creating backpressure/flow control to driver so driver can rate limit it naturally.
So a tx packet drop counter is needed to cover the lossy implementations which is abstract enough regardless of reason on why device dropped it.
A more granular counter becomes vendor specific that we can possibly avoid or place under different command.

This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-dev] Re: [virtio-comment] Re: About custom device counter

[Michael S. Tsirkin]

On Mon, Jun 19, 2023 at 12:33:33PM +0000, Parav Pandit wrote:
> 
> 
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Friday, June 16, 2023 10:13 AM
> > > Such as "limit", in a cloud scenario, multiple users purchase
> > > different VMs, and these VMs share the capabilities of the same host.
> > > In order to ensure that each VM will not affect others, the network
> > > card(virtio-net) capability of each VM is limited. When these users purchase
> > VMs, this limit has already been determined.
> > > So if the network card traffic of a vm exceeds the upper limit, packet
> > > loss will occur. It is necessary for us to count these packet losses.
> > > And the device should expose to the user.
> > 
> > OK so here, driver is expected to rate limit yes?
> > 
> Some implementations of txq are lossy and some are not creating backpressure/flow control to driver so driver can rate limit it naturally.
> So a tx packet drop counter is needed to cover the lossy implementations which is abstract enough regardless of reason on why device dropped it.


Yes but Xuan Zhuo here is saying that he wants a register that is
like link speed, not a counter.


> A more granular counter becomes vendor specific that we can possibly avoid or place under different command.


-- 
MST


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-comment] Re: About custom device counter

[Michael S. Tsirkin]

On Mon, Jun 19, 2023 at 12:33:33PM +0000, Parav Pandit wrote:
> 
> 
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Friday, June 16, 2023 10:13 AM
> > > Such as "limit", in a cloud scenario, multiple users purchase
> > > different VMs, and these VMs share the capabilities of the same host.
> > > In order to ensure that each VM will not affect others, the network
> > > card(virtio-net) capability of each VM is limited. When these users purchase
> > VMs, this limit has already been determined.
> > > So if the network card traffic of a vm exceeds the upper limit, packet
> > > loss will occur. It is necessary for us to count these packet losses.
> > > And the device should expose to the user.
> > 
> > OK so here, driver is expected to rate limit yes?
> > 
> Some implementations of txq are lossy and some are not creating backpressure/flow control to driver so driver can rate limit it naturally.
> So a tx packet drop counter is needed to cover the lossy implementations which is abstract enough regardless of reason on why device dropped it.


Yes but Xuan Zhuo here is saying that he wants a register that is
like link speed, not a counter.


> A more granular counter becomes vendor specific that we can possibly avoid or place under different command.


-- 
MST


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-dev] Re: [virtio-comment] Re: About custom device counter

[Michael S. Tsirkin]

On Thu, Jun 15, 2023 at 02:23:31PM +0800, Xuan Zhuo wrote:
> On Tue, 13 Jun 2023 05:02:01 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > On Mon, Jun 12, 2023 at 07:27:15PM +0800, Xuan Zhuo wrote:
> > > On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > > > > We hope to support device custom counter. That is, virtio spec provides a
> > > > > channel for driver and device, and both key and value are provided by device.
> > > > >
> > > > > We discussed this issue earlier, and after some internal practice, I think it is
> > > > > still necessary to discuss this again.
> > > > >
> > > > > It is very important, each cloud vendor will always have some special counters,
> > > > > these counters may not exist in another vendor. At the same time, if we have
> > > > > to discuss it in the spec every time we add a counter, or add a feature, I
> > > > > think it is very inconvenient. Manufacturers may add some new counters at any
> > > > > time based on some new requirements. Some counters may also be removed at any
> > > > > time.
> > > > >
> > > > > Of course I know that doing this might hurt migration. But what I want to say is,
> > > > > why does it affect live migration? These counters we plan to give to users
> > > > > through ethtool -S, and some changes have taken place in the ethtool counters
> > > > > output by users. Does this have any practical impact? Or do we directly use
> > > > > some other output in a way, we can clearly tell the user that these counters
> > > > > may change during the migration process. For example, the driver is migrated
> > > > > to some new devices. These devices support some new counters. I think users
> > > > > should be able to see these new counters. These new counters may be the
> > > > > purpose of the migration.
> > > > >
> > > > > We don't need to support live migration at this point.
> > > > >
> > > > > Thanks.
> > > >
> > > >
> > > > Why not just document the thing?
> > >
> > > Device Counter or Device Stat, we discussed this issue earlier,
> > > So I want to slove the problem migration firstly.
> > >
> > >
> > > Thanks.
> >
> > Sorry have trouble parsing this.
> >
> > Can we get examples of counters from various vendors, so that
> > it's clear what kind of thing this, and it becomes convincing
> > that abstracting this in virtio spec is pointless?
> 
> Such as "limit", in a cloud scenario, multiple users purchase different VMs, and
> these VMs share the capabilities of the same host. In order to ensure that each
> VM will not affect others, the network card(virtio-net) capability of each VM is
> limited. When these users purchase VMs, this limit has already been determined.
> So if the network card traffic of a vm exceeds the upper limit, packet loss will
> occur. It is necessary for us to count these packet losses. And the device
> should expose to the user.
> 
> 
> Such as "session", our dpu supports tcp connection tracking, but there is an
> upper limit to the number of connections, and if it exceeds, packet loss will
> also occur.
> 
> These are two commonly used scenarios, I hope they can help you.
> 
> Thanks.


So not a counter at all really. More a capability.

My rule of thumb is that if something can be relevant to multiple
vendors then it should not be a vendor specific capaiblity.
What you have described can easily apply to a software
virtio imlementation, so it's relevant for multiple
vendors.

So please try to add this to the spec as a generic capability.
Yes adding vendor specific extensions is of course less work
for vendors. But the ecosystem is only strong if everyone is
working on the same spec and codebase.

> 
> >
> > Also, am I right assuming that in virtualized settings this is mostly
> > relevant for hosts? E.g. guests see a vendor neutral virtio but
> > host wants to figure out source of issues?
> >
> >
> >
> >
> > >
> > > >
> > > > --
> > > > MST
> > > >
> > > >
> > > > This publicly archived list offers a means to provide input to the
> > > > OASIS Virtual I/O Device (VIRTIO) TC.
> > > >
> > > > In order to verify user consent to the Feedback License terms and
> > > > to minimize spam in the list archive, subscription is required
> > > > before posting.
> > > >
> > > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > > List help: virtio-comment-help@lists.oasis-open.org
> > > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > > Committee: https://www.oasis-open.org/committees/virtio/
> > > > Join OASIS: https://www.oasis-open.org/join/
> > > >
> >


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-comment] Re: About custom device counter

[Michael S. Tsirkin]

On Thu, Jun 15, 2023 at 02:23:31PM +0800, Xuan Zhuo wrote:
> On Tue, 13 Jun 2023 05:02:01 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > On Mon, Jun 12, 2023 at 07:27:15PM +0800, Xuan Zhuo wrote:
> > > On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > > > > We hope to support device custom counter. That is, virtio spec provides a
> > > > > channel for driver and device, and both key and value are provided by device.
> > > > >
> > > > > We discussed this issue earlier, and after some internal practice, I think it is
> > > > > still necessary to discuss this again.
> > > > >
> > > > > It is very important, each cloud vendor will always have some special counters,
> > > > > these counters may not exist in another vendor. At the same time, if we have
> > > > > to discuss it in the spec every time we add a counter, or add a feature, I
> > > > > think it is very inconvenient. Manufacturers may add some new counters at any
> > > > > time based on some new requirements. Some counters may also be removed at any
> > > > > time.
> > > > >
> > > > > Of course I know that doing this might hurt migration. But what I want to say is,
> > > > > why does it affect live migration? These counters we plan to give to users
> > > > > through ethtool -S, and some changes have taken place in the ethtool counters
> > > > > output by users. Does this have any practical impact? Or do we directly use
> > > > > some other output in a way, we can clearly tell the user that these counters
> > > > > may change during the migration process. For example, the driver is migrated
> > > > > to some new devices. These devices support some new counters. I think users
> > > > > should be able to see these new counters. These new counters may be the
> > > > > purpose of the migration.
> > > > >
> > > > > We don't need to support live migration at this point.
> > > > >
> > > > > Thanks.
> > > >
> > > >
> > > > Why not just document the thing?
> > >
> > > Device Counter or Device Stat, we discussed this issue earlier,
> > > So I want to slove the problem migration firstly.
> > >
> > >
> > > Thanks.
> >
> > Sorry have trouble parsing this.
> >
> > Can we get examples of counters from various vendors, so that
> > it's clear what kind of thing this, and it becomes convincing
> > that abstracting this in virtio spec is pointless?
> 
> Such as "limit", in a cloud scenario, multiple users purchase different VMs, and
> these VMs share the capabilities of the same host. In order to ensure that each
> VM will not affect others, the network card(virtio-net) capability of each VM is
> limited. When these users purchase VMs, this limit has already been determined.
> So if the network card traffic of a vm exceeds the upper limit, packet loss will
> occur. It is necessary for us to count these packet losses. And the device
> should expose to the user.
> 
> 
> Such as "session", our dpu supports tcp connection tracking, but there is an
> upper limit to the number of connections, and if it exceeds, packet loss will
> also occur.
> 
> These are two commonly used scenarios, I hope they can help you.
> 
> Thanks.


So not a counter at all really. More a capability.

My rule of thumb is that if something can be relevant to multiple
vendors then it should not be a vendor specific capaiblity.
What you have described can easily apply to a software
virtio imlementation, so it's relevant for multiple
vendors.

So please try to add this to the spec as a generic capability.
Yes adding vendor specific extensions is of course less work
for vendors. But the ecosystem is only strong if everyone is
working on the same spec and codebase.

> 
> >
> > Also, am I right assuming that in virtualized settings this is mostly
> > relevant for hosts? E.g. guests see a vendor neutral virtio but
> > host wants to figure out source of issues?
> >
> >
> >
> >
> > >
> > > >
> > > > --
> > > > MST
> > > >
> > > >
> > > > This publicly archived list offers a means to provide input to the
> > > > OASIS Virtual I/O Device (VIRTIO) TC.
> > > >
> > > > In order to verify user consent to the Feedback License terms and
> > > > to minimize spam in the list archive, subscription is required
> > > > before posting.
> > > >
> > > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > > List help: virtio-comment-help@lists.oasis-open.org
> > > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > > Committee: https://www.oasis-open.org/committees/virtio/
> > > > Join OASIS: https://www.oasis-open.org/join/
> > > >
> >


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-dev] RE: RE: About custom device counter

[Parav Pandit]

> From: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> Sent: Thursday, June 15, 2023 2:35 AM

> > Debug counters are useful to have.
> > I agree that they may be vendor specific.
> > We often try to abstract and define as much as vendor neutral as possible.
> >
> > So we should probably first put our efforts to see if more than one vendor can
> use it.
> 
> I agree.
> 
> >
> > Assuming there is some vendor specific counters, I imagine to come from a
> separate vendor specific query command.
> > This is to keep standard counters to avoid mixing with vend_specific.
> 
> Yes, I agree that these are two separate channels.
> 
> But I don't think we need to wait for the first job to finish.
>
I am saying if you have a counter in mind, lets discuss if we can generalize it, and if we can it falls in the first general bucket.
If we cannot it falls in second vendor specific bucket.
 
> It is conceivable that standard counters will be displayed on ethtool in the
> future. For counters from different manufacturers, we can display them in
> /sys/class/net/eth0/vendor_couters.
> 
I understood that you want to show device specific counters.
We can avoid bringing up the sw interface for a moment.
For above specific suggestion, vendor counters in sysfs for sure will not be acceptable as no vendor specific sysfs can exist anymore.

One can implement devlink health reporting counters as needed.

> Here we can tell the user that these are only for the current device, and may
> change during the migration process.
>
Sure.
 
> 
> >
> > > Of course I know that doing this might hurt migration. But what I
> > > want to say is, why does it affect live migration? These counters we
> > > plan to give to users through ethtool -S, and some changes have
> > > taken place in the ethtool counters output by users. Does this have
> > > any practical impact? Or do we directly use some other output in a
> > > way, we can clearly tell the user that these counters may change
> > > during the migration process. For example, the driver is migrated to
> > > some new devices. These devices support some new counters. I think
> > > users should be able to see these new counters. These new counters may be
> the purpose of the migration.
> >
> > Counters which are well defined non-vendor specific should be able to
> migrate to destination on best effort basis.
> > For well defined counters if device A at src support N counters, on migrated
> destination, it should be N.
> > This should be able to achieve with feature provisioning infra via group owner
> command.
> 
> Yes, for standard counters.
> 
> >
> > >
> > > We don't need to support live migration at this point.
> > When done on best effort level, infra exists, vendor has ability to
> incrementally implement it.
> 
> 
> For the standard counters, the manufacturer will be very willing to use it. I just
> want to introduce the capabilities of the manufacturer's customized.
> 
Lets try to generalize first and if we fail consider the vendor specific counters.

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-comment] RE: RE: About custom device counter

[Parav Pandit]

> From: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> Sent: Thursday, June 15, 2023 2:35 AM

> > Debug counters are useful to have.
> > I agree that they may be vendor specific.
> > We often try to abstract and define as much as vendor neutral as possible.
> >
> > So we should probably first put our efforts to see if more than one vendor can
> use it.
> 
> I agree.
> 
> >
> > Assuming there is some vendor specific counters, I imagine to come from a
> separate vendor specific query command.
> > This is to keep standard counters to avoid mixing with vend_specific.
> 
> Yes, I agree that these are two separate channels.
> 
> But I don't think we need to wait for the first job to finish.
>
I am saying if you have a counter in mind, lets discuss if we can generalize it, and if we can it falls in the first general bucket.
If we cannot it falls in second vendor specific bucket.
 
> It is conceivable that standard counters will be displayed on ethtool in the
> future. For counters from different manufacturers, we can display them in
> /sys/class/net/eth0/vendor_couters.
> 
I understood that you want to show device specific counters.
We can avoid bringing up the sw interface for a moment.
For above specific suggestion, vendor counters in sysfs for sure will not be acceptable as no vendor specific sysfs can exist anymore.

One can implement devlink health reporting counters as needed.

> Here we can tell the user that these are only for the current device, and may
> change during the migration process.
>
Sure.
 
> 
> >
> > > Of course I know that doing this might hurt migration. But what I
> > > want to say is, why does it affect live migration? These counters we
> > > plan to give to users through ethtool -S, and some changes have
> > > taken place in the ethtool counters output by users. Does this have
> > > any practical impact? Or do we directly use some other output in a
> > > way, we can clearly tell the user that these counters may change
> > > during the migration process. For example, the driver is migrated to
> > > some new devices. These devices support some new counters. I think
> > > users should be able to see these new counters. These new counters may be
> the purpose of the migration.
> >
> > Counters which are well defined non-vendor specific should be able to
> migrate to destination on best effort basis.
> > For well defined counters if device A at src support N counters, on migrated
> destination, it should be N.
> > This should be able to achieve with feature provisioning infra via group owner
> command.
> 
> Yes, for standard counters.
> 
> >
> > >
> > > We don't need to support live migration at this point.
> > When done on best effort level, infra exists, vendor has ability to
> incrementally implement it.
> 
> 
> For the standard counters, the manufacturer will be very willing to use it. I just
> want to introduce the capabilities of the manufacturer's customized.
> 
Lets try to generalize first and if we fail consider the vendor specific counters.

This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-dev] RE: [virtio-comment] Re: About custom device counter

[Parav Pandit]

> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, June 19, 2023 10:21 AM
 
> 
> Yes but Xuan Zhuo here is saying that he wants a register that is like link speed,
> not a counter.
>
I understand it as counter because Xuan is saying counter.

He wrote " The counter is for the user, not for the driver".

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

RE: [virtio-comment] Re: About custom device counter

[Parav Pandit]

> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, June 19, 2023 10:21 AM
 
> 
> Yes but Xuan Zhuo here is saying that he wants a register that is like link speed,
> not a counter.
>
I understand it as counter because Xuan is saying counter.

He wrote " The counter is for the user, not for the driver".

This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-dev] Re: RE: RE: About custom device counter

[Xuan Zhuo]

On Mon, 19 Jun 2023 20:40:54 +0000, Parav Pandit <parav@nvidia.com> wrote:
>
>
> > From: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > Sent: Thursday, June 15, 2023 2:35 AM
>
> > > Debug counters are useful to have.
> > > I agree that they may be vendor specific.
> > > We often try to abstract and define as much as vendor neutral as possible.
> > >
> > > So we should probably first put our efforts to see if more than one vendor can
> > use it.
> >
> > I agree.
> >
> > >
> > > Assuming there is some vendor specific counters, I imagine to come from a
> > separate vendor specific query command.
> > > This is to keep standard counters to avoid mixing with vend_specific.
> >
> > Yes, I agree that these are two separate channels.
> >
> > But I don't think we need to wait for the first job to finish.
> >
> I am saying if you have a counter in mind, lets discuss if we can generalize it, and if we can it falls in the first general bucket.
> If we cannot it falls in second vendor specific bucket.


That is ok.


>
> > It is conceivable that standard counters will be displayed on ethtool in the
> > future. For counters from different manufacturers, we can display them in
> > /sys/class/net/eth0/vendor_couters.
> >
> I understood that you want to show device specific counters.
> We can avoid bringing up the sw interface for a moment.
> For above specific suggestion, vendor counters in sysfs for sure will not be acceptable as no vendor specific sysfs can exist anymore.
>
> One can implement devlink health reporting counters as needed.


I am ok for any way to show this to the users.


>
> > Here we can tell the user that these are only for the current device, and may
> > change during the migration process.
> >
> Sure.
>
> >
> > >
> > > > Of course I know that doing this might hurt migration. But what I
> > > > want to say is, why does it affect live migration? These counters we
> > > > plan to give to users through ethtool -S, and some changes have
> > > > taken place in the ethtool counters output by users. Does this have
> > > > any practical impact? Or do we directly use some other output in a
> > > > way, we can clearly tell the user that these counters may change
> > > > during the migration process. For example, the driver is migrated to
> > > > some new devices. These devices support some new counters. I think
> > > > users should be able to see these new counters. These new counters may be
> > the purpose of the migration.
> > >
> > > Counters which are well defined non-vendor specific should be able to
> > migrate to destination on best effort basis.
> > > For well defined counters if device A at src support N counters, on migrated
> > destination, it should be N.
> > > This should be able to achieve with feature provisioning infra via group owner
> > command.
> >
> > Yes, for standard counters.
> >
> > >
> > > >
> > > > We don't need to support live migration at this point.
> > > When done on best effort level, infra exists, vendor has ability to
> > incrementally implement it.
> >
> >
> > For the standard counters, the manufacturer will be very willing to use it. I just
> > want to introduce the capabilities of the manufacturer's customized.
> >
> Lets try to generalize first and if we fail consider the vendor specific counters.


OK

Thanks.



---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-comment] Re: RE: RE: About custom device counter

[Xuan Zhuo]

On Mon, 19 Jun 2023 20:40:54 +0000, Parav Pandit <parav@nvidia.com> wrote:
>
>
> > From: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > Sent: Thursday, June 15, 2023 2:35 AM
>
> > > Debug counters are useful to have.
> > > I agree that they may be vendor specific.
> > > We often try to abstract and define as much as vendor neutral as possible.
> > >
> > > So we should probably first put our efforts to see if more than one vendor can
> > use it.
> >
> > I agree.
> >
> > >
> > > Assuming there is some vendor specific counters, I imagine to come from a
> > separate vendor specific query command.
> > > This is to keep standard counters to avoid mixing with vend_specific.
> >
> > Yes, I agree that these are two separate channels.
> >
> > But I don't think we need to wait for the first job to finish.
> >
> I am saying if you have a counter in mind, lets discuss if we can generalize it, and if we can it falls in the first general bucket.
> If we cannot it falls in second vendor specific bucket.


That is ok.


>
> > It is conceivable that standard counters will be displayed on ethtool in the
> > future. For counters from different manufacturers, we can display them in
> > /sys/class/net/eth0/vendor_couters.
> >
> I understood that you want to show device specific counters.
> We can avoid bringing up the sw interface for a moment.
> For above specific suggestion, vendor counters in sysfs for sure will not be acceptable as no vendor specific sysfs can exist anymore.
>
> One can implement devlink health reporting counters as needed.


I am ok for any way to show this to the users.


>
> > Here we can tell the user that these are only for the current device, and may
> > change during the migration process.
> >
> Sure.
>
> >
> > >
> > > > Of course I know that doing this might hurt migration. But what I
> > > > want to say is, why does it affect live migration? These counters we
> > > > plan to give to users through ethtool -S, and some changes have
> > > > taken place in the ethtool counters output by users. Does this have
> > > > any practical impact? Or do we directly use some other output in a
> > > > way, we can clearly tell the user that these counters may change
> > > > during the migration process. For example, the driver is migrated to
> > > > some new devices. These devices support some new counters. I think
> > > > users should be able to see these new counters. These new counters may be
> > the purpose of the migration.
> > >
> > > Counters which are well defined non-vendor specific should be able to
> > migrate to destination on best effort basis.
> > > For well defined counters if device A at src support N counters, on migrated
> > destination, it should be N.
> > > This should be able to achieve with feature provisioning infra via group owner
> > command.
> >
> > Yes, for standard counters.
> >
> > >
> > > >
> > > > We don't need to support live migration at this point.
> > > When done on best effort level, infra exists, vendor has ability to
> > incrementally implement it.
> >
> >
> > For the standard counters, the manufacturer will be very willing to use it. I just
> > want to introduce the capabilities of the manufacturer's customized.
> >
> Lets try to generalize first and if we fail consider the vendor specific counters.


OK

Thanks.



This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-dev] Re: [virtio-comment] Re: About custom device counter

[Xuan Zhuo]

On Mon, 19 Jun 2023 10:20:56 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Mon, Jun 19, 2023 at 12:33:33PM +0000, Parav Pandit wrote:
> >
> >
> > > From: Michael S. Tsirkin <mst@redhat.com>
> > > Sent: Friday, June 16, 2023 10:13 AM
> > > > Such as "limit", in a cloud scenario, multiple users purchase
> > > > different VMs, and these VMs share the capabilities of the same host.
> > > > In order to ensure that each VM will not affect others, the network
> > > > card(virtio-net) capability of each VM is limited. When these users purchase
> > > VMs, this limit has already been determined.
> > > > So if the network card traffic of a vm exceeds the upper limit, packet
> > > > loss will occur. It is necessary for us to count these packet losses.
> > > > And the device should expose to the user.
> > >
> > > OK so here, driver is expected to rate limit yes?
> > >
> > Some implementations of txq are lossy and some are not creating backpressure/flow control to driver so driver can rate limit it naturally.
> > So a tx packet drop counter is needed to cover the lossy implementations which is abstract enough regardless of reason on why device dropped it.
>
>
> Yes but Xuan Zhuo here is saying that he wants a register that is
> like link speed, not a counter.
>

A counter.

This is a capability, but it only exists between the device and the user. The
capability is set when the user buys a network card, so it has nothing to do
with the driver. As a result, some packets were lost, and we hope that these
counter can be passed to users. Because this capability has nothing to do
with the virtio spec, we always want it to be a vendor custom counter.

Of course, some parameters can be abstracted, and we are very happy to do so.
But I am worried that this will lead to inaccurate expressions of meaning.

Anyway we can try something out, I'll sort out some of the counters we'd like to
have. There are also some that I think we want to customize. We discuss further
based on these counters.


Thanks.


>
> > A more granular counter becomes vendor specific that we can possibly avoid or place under different command.
>
>
> --
> MST
>

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-comment] Re: About custom device counter

[Xuan Zhuo]

On Mon, 19 Jun 2023 10:20:56 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Mon, Jun 19, 2023 at 12:33:33PM +0000, Parav Pandit wrote:
> >
> >
> > > From: Michael S. Tsirkin <mst@redhat.com>
> > > Sent: Friday, June 16, 2023 10:13 AM
> > > > Such as "limit", in a cloud scenario, multiple users purchase
> > > > different VMs, and these VMs share the capabilities of the same host.
> > > > In order to ensure that each VM will not affect others, the network
> > > > card(virtio-net) capability of each VM is limited. When these users purchase
> > > VMs, this limit has already been determined.
> > > > So if the network card traffic of a vm exceeds the upper limit, packet
> > > > loss will occur. It is necessary for us to count these packet losses.
> > > > And the device should expose to the user.
> > >
> > > OK so here, driver is expected to rate limit yes?
> > >
> > Some implementations of txq are lossy and some are not creating backpressure/flow control to driver so driver can rate limit it naturally.
> > So a tx packet drop counter is needed to cover the lossy implementations which is abstract enough regardless of reason on why device dropped it.
>
>
> Yes but Xuan Zhuo here is saying that he wants a register that is
> like link speed, not a counter.
>

A counter.

This is a capability, but it only exists between the device and the user. The
capability is set when the user buys a network card, so it has nothing to do
with the driver. As a result, some packets were lost, and we hope that these
counter can be passed to users. Because this capability has nothing to do
with the virtio spec, we always want it to be a vendor custom counter.

Of course, some parameters can be abstracted, and we are very happy to do so.
But I am worried that this will lead to inaccurate expressions of meaning.

Anyway we can try something out, I'll sort out some of the counters we'd like to
have. There are also some that I think we want to customize. We discuss further
based on these counters.


Thanks.


>
> > A more granular counter becomes vendor specific that we can possibly avoid or place under different command.
>
>
> --
> MST
>

This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-dev] Re: [virtio-comment] Re: About custom device counter

[Xuan Zhuo]

On Mon, 19 Jun 2023 10:25:38 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Thu, Jun 15, 2023 at 02:23:31PM +0800, Xuan Zhuo wrote:
> > On Tue, 13 Jun 2023 05:02:01 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > On Mon, Jun 12, 2023 at 07:27:15PM +0800, Xuan Zhuo wrote:
> > > > On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > > On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > > > > > We hope to support device custom counter. That is, virtio spec provides a
> > > > > > channel for driver and device, and both key and value are provided by device.
> > > > > >
> > > > > > We discussed this issue earlier, and after some internal practice, I think it is
> > > > > > still necessary to discuss this again.
> > > > > >
> > > > > > It is very important, each cloud vendor will always have some special counters,
> > > > > > these counters may not exist in another vendor. At the same time, if we have
> > > > > > to discuss it in the spec every time we add a counter, or add a feature, I
> > > > > > think it is very inconvenient. Manufacturers may add some new counters at any
> > > > > > time based on some new requirements. Some counters may also be removed at any
> > > > > > time.
> > > > > >
> > > > > > Of course I know that doing this might hurt migration. But what I want to say is,
> > > > > > why does it affect live migration? These counters we plan to give to users
> > > > > > through ethtool -S, and some changes have taken place in the ethtool counters
> > > > > > output by users. Does this have any practical impact? Or do we directly use
> > > > > > some other output in a way, we can clearly tell the user that these counters
> > > > > > may change during the migration process. For example, the driver is migrated
> > > > > > to some new devices. These devices support some new counters. I think users
> > > > > > should be able to see these new counters. These new counters may be the
> > > > > > purpose of the migration.
> > > > > >
> > > > > > We don't need to support live migration at this point.
> > > > > >
> > > > > > Thanks.
> > > > >
> > > > >
> > > > > Why not just document the thing?
> > > >
> > > > Device Counter or Device Stat, we discussed this issue earlier,
> > > > So I want to slove the problem migration firstly.
> > > >
> > > >
> > > > Thanks.
> > >
> > > Sorry have trouble parsing this.
> > >
> > > Can we get examples of counters from various vendors, so that
> > > it's clear what kind of thing this, and it becomes convincing
> > > that abstracting this in virtio spec is pointless?
> >
> > Such as "limit", in a cloud scenario, multiple users purchase different VMs, and
> > these VMs share the capabilities of the same host. In order to ensure that each
> > VM will not affect others, the network card(virtio-net) capability of each VM is
> > limited. When these users purchase VMs, this limit has already been determined.
> > So if the network card traffic of a vm exceeds the upper limit, packet loss will
> > occur. It is necessary for us to count these packet losses. And the device
> > should expose to the user.
> >
> >
> > Such as "session", our dpu supports tcp connection tracking, but there is an
> > upper limit to the number of connections, and if it exceeds, packet loss will
> > also occur.
> >
> > These are two commonly used scenarios, I hope they can help you.
> >
> > Thanks.
>
>
> So not a counter at all really. More a capability.

Explain in other mail.

>
> My rule of thumb is that if something can be relevant to multiple
> vendors then it should not be a vendor specific capaiblity.
> What you have described can easily apply to a software
> virtio imlementation, so it's relevant for multiple
> vendors.

Great!!

>
> So please try to add this to the spec as a generic capability.
> Yes adding vendor specific extensions is of course less work
> for vendors. But the ecosystem is only strong if everyone is
> working on the same spec and codebase.

I agree.

Thank.


>
> >
> > >
> > > Also, am I right assuming that in virtualized settings this is mostly
> > > relevant for hosts? E.g. guests see a vendor neutral virtio but
> > > host wants to figure out source of issues?
> > >
> > >
> > >
> > >
> > > >
> > > > >
> > > > > --
> > > > > MST
> > > > >
> > > > >
> > > > > This publicly archived list offers a means to provide input to the
> > > > > OASIS Virtual I/O Device (VIRTIO) TC.
> > > > >
> > > > > In order to verify user consent to the Feedback License terms and
> > > > > to minimize spam in the list archive, subscription is required
> > > > > before posting.
> > > > >
> > > > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > > > List help: virtio-comment-help@lists.oasis-open.org
> > > > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > > > Committee: https://www.oasis-open.org/committees/virtio/
> > > > > Join OASIS: https://www.oasis-open.org/join/
> > > > >
> > >
>

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-comment] Re: About custom device counter

[Xuan Zhuo]

On Mon, 19 Jun 2023 10:25:38 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Thu, Jun 15, 2023 at 02:23:31PM +0800, Xuan Zhuo wrote:
> > On Tue, 13 Jun 2023 05:02:01 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > On Mon, Jun 12, 2023 at 07:27:15PM +0800, Xuan Zhuo wrote:
> > > > On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > > On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > > > > > We hope to support device custom counter. That is, virtio spec provides a
> > > > > > channel for driver and device, and both key and value are provided by device.
> > > > > >
> > > > > > We discussed this issue earlier, and after some internal practice, I think it is
> > > > > > still necessary to discuss this again.
> > > > > >
> > > > > > It is very important, each cloud vendor will always have some special counters,
> > > > > > these counters may not exist in another vendor. At the same time, if we have
> > > > > > to discuss it in the spec every time we add a counter, or add a feature, I
> > > > > > think it is very inconvenient. Manufacturers may add some new counters at any
> > > > > > time based on some new requirements. Some counters may also be removed at any
> > > > > > time.
> > > > > >
> > > > > > Of course I know that doing this might hurt migration. But what I want to say is,
> > > > > > why does it affect live migration? These counters we plan to give to users
> > > > > > through ethtool -S, and some changes have taken place in the ethtool counters
> > > > > > output by users. Does this have any practical impact? Or do we directly use
> > > > > > some other output in a way, we can clearly tell the user that these counters
> > > > > > may change during the migration process. For example, the driver is migrated
> > > > > > to some new devices. These devices support some new counters. I think users
> > > > > > should be able to see these new counters. These new counters may be the
> > > > > > purpose of the migration.
> > > > > >
> > > > > > We don't need to support live migration at this point.
> > > > > >
> > > > > > Thanks.
> > > > >
> > > > >
> > > > > Why not just document the thing?
> > > >
> > > > Device Counter or Device Stat, we discussed this issue earlier,
> > > > So I want to slove the problem migration firstly.
> > > >
> > > >
> > > > Thanks.
> > >
> > > Sorry have trouble parsing this.
> > >
> > > Can we get examples of counters from various vendors, so that
> > > it's clear what kind of thing this, and it becomes convincing
> > > that abstracting this in virtio spec is pointless?
> >
> > Such as "limit", in a cloud scenario, multiple users purchase different VMs, and
> > these VMs share the capabilities of the same host. In order to ensure that each
> > VM will not affect others, the network card(virtio-net) capability of each VM is
> > limited. When these users purchase VMs, this limit has already been determined.
> > So if the network card traffic of a vm exceeds the upper limit, packet loss will
> > occur. It is necessary for us to count these packet losses. And the device
> > should expose to the user.
> >
> >
> > Such as "session", our dpu supports tcp connection tracking, but there is an
> > upper limit to the number of connections, and if it exceeds, packet loss will
> > also occur.
> >
> > These are two commonly used scenarios, I hope they can help you.
> >
> > Thanks.
>
>
> So not a counter at all really. More a capability.

Explain in other mail.

>
> My rule of thumb is that if something can be relevant to multiple
> vendors then it should not be a vendor specific capaiblity.
> What you have described can easily apply to a software
> virtio imlementation, so it's relevant for multiple
> vendors.

Great!!

>
> So please try to add this to the spec as a generic capability.
> Yes adding vendor specific extensions is of course less work
> for vendors. But the ecosystem is only strong if everyone is
> working on the same spec and codebase.

I agree.

Thank.


>
> >
> > >
> > > Also, am I right assuming that in virtualized settings this is mostly
> > > relevant for hosts? E.g. guests see a vendor neutral virtio but
> > > host wants to figure out source of issues?
> > >
> > >
> > >
> > >
> > > >
> > > > >
> > > > > --
> > > > > MST
> > > > >
> > > > >
> > > > > This publicly archived list offers a means to provide input to the
> > > > > OASIS Virtual I/O Device (VIRTIO) TC.
> > > > >
> > > > > In order to verify user consent to the Feedback License terms and
> > > > > to minimize spam in the list archive, subscription is required
> > > > > before posting.
> > > > >
> > > > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > > > List help: virtio-comment-help@lists.oasis-open.org
> > > > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > > > Committee: https://www.oasis-open.org/committees/virtio/
> > > > > Join OASIS: https://www.oasis-open.org/join/
> > > > >
> > >
>

This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/