From: Auger Eric <eric.auger@redhat.com>
To: Bharat Bhushan <bharatb.linux@gmail.com>
Cc: "Zhong, Yang" <yang.zhong@intel.com>,
Jean-Philippe Brucker <jean-philippe@linaro.org>,
kevin.tian@intel.com, tnowicki@marvell.com, mst@redhat.com,
Peter Maydell <peter.maydell@linaro.org>,
drjones@redhat.com, peterx@redhat.com, qemu-devel@nongnu.org,
alex.williamson@redhat.com, qemu-arm@nongnu.org,
Bharat Bhushan <bbhushan2@marvell.com>,
linuc.decode@gmail.com, eric.auger.pro@gmail.com
Subject: Re: [PATCH v9 8/9] virtio-iommu: Implement probe request
Date: Thu, 7 May 2020 16:42:54 +0200 [thread overview]
Message-ID: <44bd0135-89ec-caa0-bb47-f3a8c69860e7@redhat.com> (raw)
In-Reply-To: <CAAeCc_nPrmiHDe61drkFmFVVWqOg6yvmC13TVfKmiZn1GF2eXA@mail.gmail.com>
Hi Bharat,
On 5/5/20 11:06 AM, Bharat Bhushan wrote:
> On Fri, Apr 24, 2020 at 7:22 PM Auger Eric <eric.auger@redhat.com> wrote:
>>
>> Hi Bharat,
>> On 4/23/20 6:09 PM, Jean-Philippe Brucker wrote:
>>> Hi Bharat,
>>>
>>> A few more things found while rebasing
>>>
>>> On Mon, Mar 23, 2020 at 02:16:16PM +0530, Bharat Bhushan wrote:
>>>> This patch implements the PROBE request. Currently supported
>>>> page size mask per endpoint is returned. Also append a NONE
>>>> property in the end.
>>>>
>>>> Signed-off-by: Bharat Bhushan <bbhushan2@marvell.com>
>>>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>>> ---
>>>> include/standard-headers/linux/virtio_iommu.h | 6 +
>>>> hw/virtio/virtio-iommu.c | 161 +++++++++++++++++-
>>>> hw/virtio/trace-events | 2 +
>>>> 3 files changed, 166 insertions(+), 3 deletions(-)
>>>>
>>>> diff --git a/include/standard-headers/linux/virtio_iommu.h b/include/standard-headers/linux/virtio_iommu.h
>>>> index b9443b83a1..8a0d47b907 100644
>>>> --- a/include/standard-headers/linux/virtio_iommu.h
>>>> +++ b/include/standard-headers/linux/virtio_iommu.h
>>>> @@ -111,6 +111,7 @@ struct virtio_iommu_req_unmap {
>>>>
>>>> #define VIRTIO_IOMMU_PROBE_T_NONE 0
>>>> #define VIRTIO_IOMMU_PROBE_T_RESV_MEM 1
>>>> +#define VIRTIO_IOMMU_PROBE_T_PAGE_SIZE_MASK 2
>>>>
>>>> #define VIRTIO_IOMMU_PROBE_T_MASK 0xfff
>>>>
>>>> @@ -130,6 +131,11 @@ struct virtio_iommu_probe_resv_mem {
>>>> uint64_t end;
>>>> };
>>>>
>>>> +struct virtio_iommu_probe_pgsize_mask {
>>>> + struct virtio_iommu_probe_property head;
>>>> + uint64_t pgsize_bitmap;
>>>> +};
>>>> +
>>>> struct virtio_iommu_req_probe {
>>>> struct virtio_iommu_req_head head;
>>>> uint32_t endpoint;
>>>> diff --git a/hw/virtio/virtio-iommu.c b/hw/virtio/virtio-iommu.c
>>>> index 747e3cf1da..63fbacdcdc 100644
>>>> --- a/hw/virtio/virtio-iommu.c
>>>> +++ b/hw/virtio/virtio-iommu.c
>>>> @@ -38,6 +38,10 @@
>>>>
>>>> /* Max size */
>>>> #define VIOMMU_DEFAULT_QUEUE_SIZE 256
>>>> +#define VIOMMU_PROBE_SIZE 512
>>>> +
>>>> +#define SUPPORTED_PROBE_PROPERTIES (\
>>>> + 1 << VIRTIO_IOMMU_PROBE_T_PAGE_SIZE_MASK)
>>>>
>>>> typedef struct VirtIOIOMMUDomain {
>>>> uint32_t id;
>>>> @@ -62,6 +66,13 @@ typedef struct VirtIOIOMMUMapping {
>>>> uint32_t flags;
>>>> } VirtIOIOMMUMapping;
>>>>
>>>> +typedef struct VirtIOIOMMUPropBuffer {
>>>> + VirtIOIOMMUEndpoint *endpoint;
>>>> + size_t filled;
>>>> + uint8_t *start;
>>>> + bool error;
>>>
>>> It doesn't seem like bufstate->error gets used anywhere
>> maybe rebase your work on
>> [PATCH for-4.2 v10 10/15] virtio-iommu: Implement probe request
>> which tests it.
>
> This was the staring point for me, As of now i moved away from "error"
> from above struct.
>
>>
>> Also in
>> [Qemu-devel] [PATCH for-4.2 v10 10/15] virtio-iommu: Implement probe request
>> I changed the implementation to keep it simpler.
In [PATCH 2/5] virtio-iommu: Implement RESV_MEM probe request
I reworked the original implementation. I think this version is simpler
to start from. I removed the VirtIOIOMMUPropBuffer struct which looks
more complex than requested.
Thanks
Eric
>>
>> Thanks
>>
>> Eric
>>>
>>>> +} VirtIOIOMMUPropBuffer;
>>>> +
>>>> static inline uint16_t virtio_iommu_get_bdf(IOMMUDevice *dev)
>>>> {
>>>> return PCI_BUILD_BDF(pci_bus_num(dev->bus), dev->devfn);
>>>> @@ -490,6 +501,114 @@ static int virtio_iommu_unmap(VirtIOIOMMU *s,
>>>> return ret;
>>>> }
>>>>
>>>> +static int virtio_iommu_fill_none_prop(VirtIOIOMMUPropBuffer *bufstate)
>>>> +{
>>>> + struct virtio_iommu_probe_property *prop;
>>>> +
>>>> + prop = (struct virtio_iommu_probe_property *)
>>>> + (bufstate->start + bufstate->filled);
>>>> + prop->type = 0;
>>>> + prop->length = 0;
>>>> + bufstate->filled += sizeof(*prop);
>>>> + trace_virtio_iommu_fill_none_property(bufstate->endpoint->id);
>>>> + return 0;
>>>> +}
>>>> +
>>>> +static int virtio_iommu_fill_page_size_mask(VirtIOIOMMUPropBuffer *bufstate)
>>>> +{
>>>> + struct virtio_iommu_probe_pgsize_mask *page_size_mask;
>>>> + size_t prop_size = sizeof(*page_size_mask);
>>>> + VirtIOIOMMUEndpoint *ep = bufstate->endpoint;
>>>> + VirtIOIOMMU *s = ep->viommu;
>>>> + IOMMUDevice *sdev;
>>>> +
>>>> + if (bufstate->filled + prop_size >= VIOMMU_PROBE_SIZE) {
>>>> + bufstate->error = true;
>>>> + /* get the traversal stopped by returning true */
>>>> + return true;
>>>> + }
>>>> +
>>>> + page_size_mask = (struct virtio_iommu_probe_pgsize_mask *)
>>>> + (bufstate->start + bufstate->filled);
>>>> +
>>>> + page_size_mask->head.type = VIRTIO_IOMMU_PROBE_T_PAGE_SIZE_MASK;
>>>> + page_size_mask->head.length = prop_size;
>>>> + QLIST_FOREACH(sdev, &s->notifiers_list, next) {
>>>> + if (ep->id == sdev->devfn) {
>>>> + page_size_mask->pgsize_bitmap = sdev->page_size_mask;
>>>
>>> Do we need a cpu_to_le64 here?
>
> Ack, yes, even head.type and head.length needed cpu_to_le16().
>
>>>
>>>> + }
>>>> + }
>>>> + bufstate->filled += sizeof(*page_size_mask);
>>>> + trace_virtio_iommu_fill_pgsize_mask_property(bufstate->endpoint->id,
>>>> + page_size_mask->pgsize_bitmap,
>>>> + bufstate->filled);
>>>> + return false;
>>>> +}
>>>> +
>>>> +/* Fill the properties[] buffer with properties of type @type */
>>>> +static int virtio_iommu_fill_property(int type,
>>>> + VirtIOIOMMUPropBuffer *bufstate)
>>>> +{
>>>> + int ret = -ENOSPC;
>>>> +
>>>> + if (bufstate->filled + sizeof(struct virtio_iommu_probe_property)
>>>> + >= VIOMMU_PROBE_SIZE) {
>>>> + /* no space left for the header */
>>>> + bufstate->error = true;
>>>> + goto out;
>>>> + }
>>>> +
>>>> + switch (type) {
>>>> + case VIRTIO_IOMMU_PROBE_T_NONE:
>>>> + ret = virtio_iommu_fill_none_prop(bufstate);
>>>> + break;
>>>> + case VIRTIO_IOMMU_PROBE_T_PAGE_SIZE_MASK:
>>>> + {
>>>> + ret = virtio_iommu_fill_page_size_mask(bufstate);
>>>> + break;
>>>> + }
>>>> + default:
>>>> + ret = -ENOENT;
>>>> + break;
>>>> + }
>>>> +out:
>>>> + if (ret) {
>>>> + error_report("%s property of type=%d could not be filled (%d),"
>>>> + " remaining size = 0x%lx",
>>>> + __func__, type, ret, bufstate->filled);
>>>> + }
>>>> + return ret;
>>>> +}
>>>> +
>>>> +/**
>>>> + * virtio_iommu_probe - Fill the probe request buffer with all
>>>> + * the properties the device is able to return and add a NONE
>>>> + * property at the end. @buf points to properties[].
>>>> + */
>>>> +static int virtio_iommu_probe(VirtIOIOMMU *s,
>>>> + struct virtio_iommu_req_probe *req,
>>>> + uint8_t *buf)
>>>> +{
>>>> + uint32_t ep_id = le32_to_cpu(req->endpoint);
>>>> + VirtIOIOMMUEndpoint *ep = virtio_iommu_get_endpoint(s, ep_id);
>>>> + int16_t prop_types = SUPPORTED_PROBE_PROPERTIES, type;
>>>> + VirtIOIOMMUPropBuffer bufstate = {.start = buf, .filled = 0,
>>>> + .error = false, .endpoint = ep};
>>>
>>> Probably need to check if ep is valid before going further
>
> yes, will take care
>
>>>
>>>> +
>>>> + while ((type = ctz32(prop_types)) != 32) {
>>>> + if (virtio_iommu_fill_property(type, &bufstate)) {
>>>> + goto failure;
>>>> + }
>>>> + prop_types &= ~(1 << type);
>>>> + }
>>>> + if (virtio_iommu_fill_property(VIRTIO_IOMMU_PROBE_T_NONE, &bufstate)) {
>>>> + goto failure;
>>>> + }
>>>
>>> We got rid of the NONE property in the spec, now there is:
>>>
>>> "If the device doesn’t fill all probe_size bytes with properties, it
>>> SHOULD fill the remaining bytes of properties with zeroes."
>>>
>>> So I think you can get rid of virtio_iommu_fill_none_prop() and fill the
>>> rest of the buffer with zeroes here instead.
>
> will do changes accordingly.
>
> Stay Safe !!
>
> Thanks
> -Bharat
>>>
>>> Thanks,
>>> Jean
>>>
>>>> + return VIRTIO_IOMMU_S_OK;
>>>> +failure:
>>>> + return VIRTIO_IOMMU_S_INVAL;
>>>> +}
>>>> +
>>>> static int virtio_iommu_iov_to_req(struct iovec *iov,
>>>> unsigned int iov_cnt,
>>>> void *req, size_t req_sz)
>>>> @@ -519,6 +638,17 @@ virtio_iommu_handle_req(detach)
>>>> virtio_iommu_handle_req(map)
>>>> virtio_iommu_handle_req(unmap)
>>>>
>>>> +static int virtio_iommu_handle_probe(VirtIOIOMMU *s,
>>>> + struct iovec *iov,
>>>> + unsigned int iov_cnt,
>>>> + uint8_t *buf)
>>>> +{
>>>> + struct virtio_iommu_req_probe req;
>>>> + int ret = virtio_iommu_iov_to_req(iov, iov_cnt, &req, sizeof(req));
>>>> +
>>>> + return ret ? ret : virtio_iommu_probe(s, &req, buf);
>>>> +}
>>>> +
>>>> static void virtio_iommu_handle_command(VirtIODevice *vdev, VirtQueue *vq)
>>>> {
>>>> VirtIOIOMMU *s = VIRTIO_IOMMU(vdev);
>>>> @@ -564,17 +694,33 @@ static void virtio_iommu_handle_command(VirtIODevice *vdev, VirtQueue *vq)
>>>> case VIRTIO_IOMMU_T_UNMAP:
>>>> tail.status = virtio_iommu_handle_unmap(s, iov, iov_cnt);
>>>> break;
>>>> + case VIRTIO_IOMMU_T_PROBE:
>>>> + {
>>>> + struct virtio_iommu_req_tail *ptail;
>>>> + uint8_t *buf = g_malloc0(s->config.probe_size + sizeof(tail));
>>>> +
>>>> + ptail = (struct virtio_iommu_req_tail *)
>>>> + (buf + s->config.probe_size);
>>>> + ptail->status = virtio_iommu_handle_probe(s, iov, iov_cnt, buf);
>>>> +
>>>> + sz = iov_from_buf(elem->in_sg, elem->in_num, 0,
>>>> + buf, s->config.probe_size + sizeof(tail));
>>>> + g_free(buf);
>>>> + assert(sz == s->config.probe_size + sizeof(tail));
>>>> + goto push;
>>>> + }
>>>> default:
>>>> tail.status = VIRTIO_IOMMU_S_UNSUPP;
>>>> }
>>>> - qemu_mutex_unlock(&s->mutex);
>>>>
>>>> out:
>>>> sz = iov_from_buf(elem->in_sg, elem->in_num, 0,
>>>> &tail, sizeof(tail));
>>>> assert(sz == sizeof(tail));
>>>>
>>>> - virtqueue_push(vq, elem, sizeof(tail));
>>>> +push:
>>>> + qemu_mutex_unlock(&s->mutex);
>>>> + virtqueue_push(vq, elem, sz);
>>>> virtio_notify(vdev, vq);
>>>> g_free(elem);
>>>> }
>>>> @@ -634,16 +780,23 @@ static IOMMUTLBEntry virtio_iommu_translate(IOMMUMemoryRegion *mr, hwaddr addr,
>>>> VirtIOIOMMUEndpoint *ep;
>>>> uint32_t sid, flags;
>>>> bool bypass_allowed;
>>>> + hwaddr addr_mask;
>>>> bool found;
>>>>
>>>> interval.low = addr;
>>>> interval.high = addr + 1;
>>>>
>>>> + if (sdev->page_size_mask) {
>>>> + addr_mask = (1 << ctz32(sdev->page_size_mask)) - 1;
>>>> + } else {
>>>> + addr_mask = (1 << ctz32(s->config.page_size_mask)) - 1;
>>>> + }
>>>> +
>>>> IOMMUTLBEntry entry = {
>>>> .target_as = &address_space_memory,
>>>> .iova = addr,
>>>> .translated_addr = addr,
>>>> - .addr_mask = (1 << ctz32(s->config.page_size_mask)) - 1,
>>>> + .addr_mask = addr_mask,
>>>> .perm = IOMMU_NONE,
>>>> };
>>>>
>>>> @@ -831,6 +984,7 @@ static void virtio_iommu_device_realize(DeviceState *dev, Error **errp)
>>>> s->config.page_size_mask = TARGET_PAGE_MASK;
>>>> s->config.input_range.end = -1UL;
>>>> s->config.domain_range.end = 32;
>>>> + s->config.probe_size = VIOMMU_PROBE_SIZE;
>>>>
>>>> virtio_add_feature(&s->features, VIRTIO_RING_F_EVENT_IDX);
>>>> virtio_add_feature(&s->features, VIRTIO_RING_F_INDIRECT_DESC);
>>>> @@ -840,6 +994,7 @@ static void virtio_iommu_device_realize(DeviceState *dev, Error **errp)
>>>> virtio_add_feature(&s->features, VIRTIO_IOMMU_F_MAP_UNMAP);
>>>> virtio_add_feature(&s->features, VIRTIO_IOMMU_F_BYPASS);
>>>> virtio_add_feature(&s->features, VIRTIO_IOMMU_F_MMIO);
>>>> + virtio_add_feature(&s->features, VIRTIO_IOMMU_F_PROBE);
>>>>
>>>> qemu_mutex_init(&s->mutex);
>>>>
>>>> diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events
>>>> index 8bae651191..b0a6e4bda3 100644
>>>> --- a/hw/virtio/trace-events
>>>> +++ b/hw/virtio/trace-events
>>>> @@ -76,3 +76,5 @@ virtio_iommu_report_fault(uint8_t reason, uint32_t flags, uint32_t endpoint, uin
>>>> virtio_iommu_notify_map(const char *name, uint64_t iova, uint64_t paddr, uint64_t map_size) "mr=%s iova=0x%"PRIx64" pa=0x%" PRIx64" size=0x%"PRIx64
>>>> virtio_iommu_notify_unmap(const char *name, uint64_t iova, uint64_t map_size) "mr=%s iova=0x%"PRIx64" size=0x%"PRIx64
>>>> virtio_iommu_remap(uint64_t iova, uint64_t pa, uint64_t size) "iova=0x%"PRIx64" pa=0x%" PRIx64" size=0x%"PRIx64""
>>>> +virtio_iommu_fill_none_property(uint32_t devid) "devid=%d"
>>>> +virtio_iommu_fill_pgsize_mask_property(uint32_t devid, uint64_t pgsize_mask, size_t filled) "dev= %d, pgsize_mask=0x%"PRIx64" filled=0x%lx"
>>>> --
>>>> 2.17.1
>>>>
>>>
>>
>
next prev parent reply other threads:[~2020-05-07 14:44 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-23 8:46 [PATCH v9 0/9] virtio-iommu: VFIO integration Bharat Bhushan
2020-03-23 8:46 ` [PATCH v9 1/9] hw/vfio/common: Remove error print on mmio region translation by viommu Bharat Bhushan
2020-03-23 23:08 ` Alex Williamson
2020-03-26 17:35 ` Auger Eric
2020-03-26 17:53 ` Alex Williamson
2020-03-27 5:50 ` [EXT] " Bharat Bhushan
2020-04-02 9:01 ` Bharat Bhushan
2020-04-24 14:17 ` Auger Eric
2020-05-05 9:25 ` Bharat Bhushan
2020-05-05 9:30 ` Auger Eric
2020-05-05 9:46 ` Bharat Bhushan
2020-05-05 10:18 ` Bharat Bhushan
2020-05-05 12:05 ` Auger Eric
2020-05-07 14:40 ` Auger Eric
2020-03-23 8:46 ` [PATCH v9 2/9] memory: Add interface to set iommu page size mask Bharat Bhushan
2020-03-26 16:06 ` Auger Eric
2020-03-27 5:33 ` [EXT] " Bharat Bhushan
2020-03-27 8:27 ` Auger Eric
2020-03-23 8:46 ` [PATCH v9 3/9] vfio: set iommu page size as per host supported page size Bharat Bhushan
2020-03-23 8:46 ` [PATCH v9 4/9] virtio-iommu: set supported page size mask Bharat Bhushan
2020-03-26 15:51 ` Auger Eric
2020-03-27 5:13 ` [EXT] " Bharat Bhushan
2020-03-27 8:28 ` Auger Eric
2020-03-23 8:46 ` [PATCH v9 5/9] virtio-iommu: Add iommu notifier for map/unmap Bharat Bhushan
2020-03-23 8:46 ` [PATCH v9 6/9] virtio-iommu: Call iommu notifier for attach/detach Bharat Bhushan
2020-03-23 8:46 ` [PATCH v9 7/9] virtio-iommu: add iommu replay Bharat Bhushan
2020-03-23 8:46 ` [PATCH v9 8/9] virtio-iommu: Implement probe request Bharat Bhushan
2020-03-26 15:48 ` Auger Eric
2020-03-27 5:40 ` [EXT] " Bharat Bhushan
2020-03-27 8:34 ` Auger Eric
2020-04-23 16:09 ` Jean-Philippe Brucker
2020-04-24 13:51 ` Auger Eric
2020-05-05 9:06 ` Bharat Bhushan
2020-05-07 14:42 ` Auger Eric [this message]
2020-03-23 8:46 ` [PATCH v9 9/9] virtio-iommu: add iommu notifier memory-region Bharat Bhushan
2020-03-23 9:52 ` [PATCH v9 0/9] virtio-iommu: VFIO integration no-reply
2020-03-23 9:59 ` no-reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=44bd0135-89ec-caa0-bb47-f3a8c69860e7@redhat.com \
--to=eric.auger@redhat.com \
--cc=alex.williamson@redhat.com \
--cc=bbhushan2@marvell.com \
--cc=bharatb.linux@gmail.com \
--cc=drjones@redhat.com \
--cc=eric.auger.pro@gmail.com \
--cc=jean-philippe@linaro.org \
--cc=kevin.tian@intel.com \
--cc=linuc.decode@gmail.com \
--cc=mst@redhat.com \
--cc=peter.maydell@linaro.org \
--cc=peterx@redhat.com \
--cc=qemu-arm@nongnu.org \
--cc=qemu-devel@nongnu.org \
--cc=tnowicki@marvell.com \
--cc=yang.zhong@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).