Re: [RFC PATCH 10/24] vdpa: introduce config operations for associating ASID to a virtqueue group

From: Jason Wang <jasowang@redhat.com>
To: Eli Cohen <elic@nvidia.com>
Cc: mst@redhat.com, lulu@redhat.com, kvm@vger.kernel.org,
	virtualization@lists.linux-foundation.org,
	netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
	rob.miller@broadcom.com, lingshan.zhu@intel.com,
	eperezma@redhat.com, hanand@xilinx.com, mhabets@solarflare.com,
	amorenoz@redhat.com, maxime.coquelin@redhat.com,
	stefanha@redhat.com, sgarzare@redhat.com
Subject: Re: [RFC PATCH 10/24] vdpa: introduce config operations for associating ASID to a virtqueue group
Date: Mon, 12 Oct 2020 15:45:10 +0800	[thread overview]
Message-ID: <b1ac150b-0845-874f-75d0-7440133a1d41@redhat.com> (raw)
In-Reply-To: <20201012065931.GA42327@mtl-vdi-166.wap.labs.mlnx>

On 2020/10/12 下午2:59, Eli Cohen wrote:
> On Fri, Oct 09, 2020 at 11:56:45AM +0800, Jason Wang wrote:
>> On 2020/10/1 下午9:29, Eli Cohen wrote:
>>> On Thu, Sep 24, 2020 at 11:21:11AM +0800, Jason Wang wrote:
>>>> This patch introduces a new bus operation to allow the vDPA bus driver
>>>> to associate an ASID to a virtqueue group.
>>>>
>>> So in case of virtio_net, I would expect that all the data virtqueues
>>> will be associated with the same address space identifier.
>>
>> Right.
>>
>> I will add the codes to do this in the next version. It should be more
>> explicit than have this assumption by default.
>>
>>
>>> Moreover,
>>> this assignment should be provided before the set_map call that provides
>>> the iotlb for the address space, correct?
>>
>> I think it's better not have this limitation, note that set_map() now takes
>> a asid argument.
>>
>> So for hardware if the associated as is changed, the driver needs to program
>> the hardware to switch to the new mapping.
>>
>> Does this work for mlx5?
>>
> So in theory we can have several asid's (for different virtqueues), each
> one should be followed by a specific set_map call. If this is so, how do
> I know if I met all the conditions run my driver? Maybe we need another
> callback to let the driver know it should not expect more set_maps().

This should work similarly as in the past. Two parts of the work is 
expected to be done by the driver:

1) store the mapping somewhere (e.g hardware) during set_map()
2) associating mapping with a specific virtqueue

The only difference is that more than one mapping is used now.

For the issue of more set_maps(), driver should be always ready for the 
new set_maps() call instead of not expecting new set_maps() since guest 
memory topology could be changed due to several reasons.

Qemu or vhost-vDPA will try their best to avoid the frequency of 
set_maps() for better performance (e.g through batched IOTLB updating). 
E.g there should be at most one set_map() during one time of guest booting.

Thanks

>