[virtio-dev] Re: [virtio-comment] Re: [PATCH 09/11] transport-pci: Describe PCI MMR dev config registers

[Jason Wang <jasowang@redhat.com>]

On Fri, Apr 14, 2023 at 3:39 AM Parav Pandit <parav@nvidia.com> wrote:
>
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Thursday, April 13, 2023 1:20 PM
> >
> > On Thu, Apr 13, 2023 at 01:14:15PM +0800, Jason Wang wrote:
> > > > >>>> The proposed solution in this series enables it and avoid per
> > > > >>>> field sw
> > > > >>> interpretation and mediation in parsing values etc.
> >
> > ... except for reset, notifications, and maybe more down the road.
> >
> Your AQ proposal addresses reset too.
> Nothing extra for the notifications, as comes for free from the device side.
> >
> > > > >>> I don't think it's possible. See the discussion about
> > > > >>> ORDER_PLATFORM and ACCESS_PLATFORM in previous threads.
> > > > >>>
> > > > >> I have read the previous thread.
> > > > >> Hypervisor will be limiting to those platforms where ORDER_PLATFORM
> > is not needed.
> > > > >
> > > > > So you introduce a bunch of new facilities that only work on some
> > > > > specific archs. This breaks the architecture independence of
> > > > > virtio since 1.0.
> > > > The defined spec for PCI device does not work today for transitional
> > > > device for virtualization. Only works in limited PF case.
> > > > Hence this update.
> > >
> > > I fully understand the motivation. I just want to say
> > >
> > > 1) compare to the MMIO ar BAR0, this proposal doesn't provide much
> > > advantages
> > > 2) mediate on top of modern devices allows us to not worry about the
> > > device design which is hard for legacy
> >
> > I begin to think so, too. When I proposed this it looked like just a single
> > capability will be enough, without a lot of mess.  But it seems that addressing
> > this fully is getting more and more complex.
> > The one thing we can't do in software is different header size for virtio net. For
> > starters, let's add a capability to address that?
>
> Hdr bit doesn't solve it because hypervisor is not involved in any trapping of feature bits, cvq or other vqs.
> It is unified code for 1.x and transitional in hypervisor.
>
> We have two options to satisfy the requirements.
> (partly taken/repeated from Jason's yday email).
>
> 1. AQ (solves reset) + notification for building non transitional device that support perform well and it is both backward and forward compat
> Pros:
> a. efficient device reset.
> b. efficient notifications from OS to device
> c. device vendor doesn't need to build transitional configuration space.
> d. works without any mediation in hv for 1.x and non 1.x for all non-legacy interfaces (vqs, config space, cvq, and future features).

Without mediation, how could you forward guest config access to admin
virtqueue? Or you mean:

1) hypervisor medate for legacy
2) otherwise the modern BARs are assigned to the guest

For 2) as we discussed, we can't have such an assumption as

1) spec doesn't enforce the size of a specific structure
2) bing vendor locked thus a blocker for live migration as it mandate
the layout for the guest, mediation layer is a must in this case to
maintain cross vendor compatibility

Hypervisor needs to start from a mediation method and do BAR
assignment only when possible.

> e. can work with non-Linux guest VMs too
>
> Cons:
> a. More AQ commands work in sw

Note that this needs to be done on top of the transport virtqueue. And
we need to carefully design the command sets since they could be
mutually exclusive.

Thanks


> b. Does not work for bare metal PFs
>
> 2. Allowing MMIO BAR0 on transitional device as SHOULD requirement with larger BAR size.
> Pros:
> a. Can work with Linux bare-metal and Linux guest VMs as one of the wider uses case
> b. in-efficient device handling for notifications
> c. Works without mediation like 1.d.
> d. Also works without HV mediation.
>
> Cons:
> a. device reset implementation is very for the hw.
> b. requires transitional device to be built.
> c. Notification performance may suffer.
>
> For Marvell and us #1 works well.
> I am evaluating #2 and get back.
>


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org