From: Halil Pasic <pasic@linux.ibm.com> To: "Jason J. Herne" <jjherne@linux.ibm.com> Cc: Cornelia Huck <cohuck@redhat.com>, kvm@vger.kernel.org, linux-s390@vger.kernel.org, Martin Schwidefsky <schwidefsky@de.ibm.com>, Sebastian Ott <sebott@linux.ibm.com>, virtualization@lists.linux-foundation.org, "Michael S. Tsirkin" <mst@redhat.com>, Christoph Hellwig <hch@infradead.org>, Thomas Huth <thuth@redhat.com>, Christian Borntraeger <borntraeger@de.ibm.com>, Viktor Mihajlovski <mihajlov@linux.ibm.com>, Vasily Gorbik <gor@linux.ibm.com>, Janosch Frank <frankja@linux.ibm.com>, Claudio Imbrenda <imbrenda@linux.ibm.com>, Farhan Ali <alifm@linux.ibm.com>, Eric Farman <farman@linux.ibm.com> Subject: Re: [PATCH 06/10] s390/cio: add basic protected virtualization support Date: Wed, 15 May 2019 23:08:17 +0200 [thread overview] Message-ID: <20190515230817.2f8a8a5d.pasic@linux.ibm.com> (raw) In-Reply-To: <d0ffefec-a14e-ee83-0aae-df288c3ffda4@linux.ibm.com> On Tue, 14 May 2019 10:47:34 -0400 "Jason J. Herne" <jjherne@linux.ibm.com> wrote: > On 5/13/19 5:41 AM, Cornelia Huck wrote: > > On Fri, 26 Apr 2019 20:32:41 +0200 > > Halil Pasic <pasic@linux.ibm.com> wrote: > > > >> As virtio-ccw devices are channel devices, we need to use the dma area > >> for any communication with the hypervisor. > >> > >> This patch addresses the most basic stuff (mostly what is required for > >> virtio-ccw), and does take care of QDIO or any devices. > > > > "does not take care of QDIO", surely? (Also, what does "any devices" > > mean? Do you mean "every arbitrary device", perhaps?) > > > >> > >> An interesting side effect is that virtio structures are now going to > >> get allocated in 31 bit addressable storage. > > > > Hm... > > > >> > >> Signed-off-by: Halil Pasic <pasic@linux.ibm.com> > >> --- > >> arch/s390/include/asm/ccwdev.h | 4 +++ > >> drivers/s390/cio/ccwreq.c | 8 ++--- > >> drivers/s390/cio/device.c | 65 +++++++++++++++++++++++++++++++++------- > >> drivers/s390/cio/device_fsm.c | 40 ++++++++++++------------- > >> drivers/s390/cio/device_id.c | 18 +++++------ > >> drivers/s390/cio/device_ops.c | 21 +++++++++++-- > >> drivers/s390/cio/device_pgid.c | 20 ++++++------- > >> drivers/s390/cio/device_status.c | 24 +++++++-------- > >> drivers/s390/cio/io_sch.h | 21 +++++++++---- > >> drivers/s390/virtio/virtio_ccw.c | 10 ------- > >> 10 files changed, 148 insertions(+), 83 deletions(-) > > > > (...) > > > >> diff --git a/drivers/s390/virtio/virtio_ccw.c b/drivers/s390/virtio/virtio_ccw.c > >> index 6d989c360f38..bb7a92316fc8 100644 > >> --- a/drivers/s390/virtio/virtio_ccw.c > >> +++ b/drivers/s390/virtio/virtio_ccw.c > >> @@ -66,7 +66,6 @@ struct virtio_ccw_device { > >> bool device_lost; > >> unsigned int config_ready; > >> void *airq_info; > >> - u64 dma_mask; > >> }; > >> > >> struct vq_info_block_legacy { > >> @@ -1255,16 +1254,7 @@ static int virtio_ccw_online(struct ccw_device *cdev) > >> ret = -ENOMEM; > >> goto out_free; > >> } > >> - > >> vcdev->vdev.dev.parent = &cdev->dev; > >> - cdev->dev.dma_mask = &vcdev->dma_mask; > >> - /* we are fine with common virtio infrastructure using 64 bit DMA */ > >> - ret = dma_set_mask_and_coherent(&cdev->dev, DMA_BIT_MASK(64)); > >> - if (ret) { > >> - dev_warn(&cdev->dev, "Failed to enable 64-bit DMA.\n"); > >> - goto out_free; > >> - } > > > > This means that vring structures now need to fit into 31 bits as well, > > I think? Is there any way to reserve the 31 bit restriction for channel > > subsystem structures and keep vring in the full 64 bit range? (Or am I > > fundamentally misunderstanding something?) > > > > I hope I've understood everything... I'm new to virtio. But from what I'm understanding, > the vring structure (a.k.a. the VirtQueue) needs to be accessed and modified by both host > and guest. Therefore the page(s) holding that data need to be marked shared if using > protected virtualization. This patch set makes use of DMA pages by way of swiotlb (always > below 32-bit line right?) for shared memory. The last sentence is wrong. You have to differentiate between stuff that is mapped as DMA and that is allocated as DMA. The mapped stuff is handled via swiotlb and bouncing. But that can not work for vring stuff which needs to be allocated as DMA. > Therefore, a side effect is that all shared > memory, including VirtQueue data will be in the DMA zone and in 32-bit memory. > Consequently wrong. The reason I explained in a reply to Connie (see there). > I don't see any restrictions on sharing pages above the 32-bit line. So it seems possible. > I'm not sure how much more work it would be. I wonder if Halil has considered this? I did consider this, the RFC was doing this (again see other mail). > Are we > worried that virtio data structures are going to be a burden on the 31-bit address space? > > That is a good question I can not answer. Since it is currently at least a page per queue (because we use dma direct, right Mimu?), I am concerned about this. Connie, what is your opinion? Regards, Halil
WARNING: multiple messages have this Message-ID (diff)
From: Halil Pasic <pasic@linux.ibm.com> To: "Jason J. Herne" <jjherne@linux.ibm.com> Cc: Vasily Gorbik <gor@linux.ibm.com>, linux-s390@vger.kernel.org, Thomas Huth <thuth@redhat.com>, Claudio Imbrenda <imbrenda@linux.ibm.com>, kvm@vger.kernel.org, Sebastian Ott <sebott@linux.ibm.com>, "Michael S. Tsirkin" <mst@redhat.com>, Cornelia Huck <cohuck@redhat.com>, Eric Farman <farman@linux.ibm.com>, virtualization@lists.linux-foundation.org, Christoph Hellwig <hch@infradead.org>, Martin Schwidefsky <schwidefsky@de.ibm.com>, Farhan Ali <alifm@linux.ibm.com>, Viktor Mihajlovski <mihajlov@linux.ibm.com>, Janosch Frank <frankja@linux.ibm.com> Subject: Re: [PATCH 06/10] s390/cio: add basic protected virtualization support Date: Wed, 15 May 2019 23:08:17 +0200 [thread overview] Message-ID: <20190515230817.2f8a8a5d.pasic@linux.ibm.com> (raw) In-Reply-To: <d0ffefec-a14e-ee83-0aae-df288c3ffda4@linux.ibm.com> On Tue, 14 May 2019 10:47:34 -0400 "Jason J. Herne" <jjherne@linux.ibm.com> wrote: > On 5/13/19 5:41 AM, Cornelia Huck wrote: > > On Fri, 26 Apr 2019 20:32:41 +0200 > > Halil Pasic <pasic@linux.ibm.com> wrote: > > > >> As virtio-ccw devices are channel devices, we need to use the dma area > >> for any communication with the hypervisor. > >> > >> This patch addresses the most basic stuff (mostly what is required for > >> virtio-ccw), and does take care of QDIO or any devices. > > > > "does not take care of QDIO", surely? (Also, what does "any devices" > > mean? Do you mean "every arbitrary device", perhaps?) > > > >> > >> An interesting side effect is that virtio structures are now going to > >> get allocated in 31 bit addressable storage. > > > > Hm... > > > >> > >> Signed-off-by: Halil Pasic <pasic@linux.ibm.com> > >> --- > >> arch/s390/include/asm/ccwdev.h | 4 +++ > >> drivers/s390/cio/ccwreq.c | 8 ++--- > >> drivers/s390/cio/device.c | 65 +++++++++++++++++++++++++++++++++------- > >> drivers/s390/cio/device_fsm.c | 40 ++++++++++++------------- > >> drivers/s390/cio/device_id.c | 18 +++++------ > >> drivers/s390/cio/device_ops.c | 21 +++++++++++-- > >> drivers/s390/cio/device_pgid.c | 20 ++++++------- > >> drivers/s390/cio/device_status.c | 24 +++++++-------- > >> drivers/s390/cio/io_sch.h | 21 +++++++++---- > >> drivers/s390/virtio/virtio_ccw.c | 10 ------- > >> 10 files changed, 148 insertions(+), 83 deletions(-) > > > > (...) > > > >> diff --git a/drivers/s390/virtio/virtio_ccw.c b/drivers/s390/virtio/virtio_ccw.c > >> index 6d989c360f38..bb7a92316fc8 100644 > >> --- a/drivers/s390/virtio/virtio_ccw.c > >> +++ b/drivers/s390/virtio/virtio_ccw.c > >> @@ -66,7 +66,6 @@ struct virtio_ccw_device { > >> bool device_lost; > >> unsigned int config_ready; > >> void *airq_info; > >> - u64 dma_mask; > >> }; > >> > >> struct vq_info_block_legacy { > >> @@ -1255,16 +1254,7 @@ static int virtio_ccw_online(struct ccw_device *cdev) > >> ret = -ENOMEM; > >> goto out_free; > >> } > >> - > >> vcdev->vdev.dev.parent = &cdev->dev; > >> - cdev->dev.dma_mask = &vcdev->dma_mask; > >> - /* we are fine with common virtio infrastructure using 64 bit DMA */ > >> - ret = dma_set_mask_and_coherent(&cdev->dev, DMA_BIT_MASK(64)); > >> - if (ret) { > >> - dev_warn(&cdev->dev, "Failed to enable 64-bit DMA.\n"); > >> - goto out_free; > >> - } > > > > This means that vring structures now need to fit into 31 bits as well, > > I think? Is there any way to reserve the 31 bit restriction for channel > > subsystem structures and keep vring in the full 64 bit range? (Or am I > > fundamentally misunderstanding something?) > > > > I hope I've understood everything... I'm new to virtio. But from what I'm understanding, > the vring structure (a.k.a. the VirtQueue) needs to be accessed and modified by both host > and guest. Therefore the page(s) holding that data need to be marked shared if using > protected virtualization. This patch set makes use of DMA pages by way of swiotlb (always > below 32-bit line right?) for shared memory. The last sentence is wrong. You have to differentiate between stuff that is mapped as DMA and that is allocated as DMA. The mapped stuff is handled via swiotlb and bouncing. But that can not work for vring stuff which needs to be allocated as DMA. > Therefore, a side effect is that all shared > memory, including VirtQueue data will be in the DMA zone and in 32-bit memory. > Consequently wrong. The reason I explained in a reply to Connie (see there). > I don't see any restrictions on sharing pages above the 32-bit line. So it seems possible. > I'm not sure how much more work it would be. I wonder if Halil has considered this? I did consider this, the RFC was doing this (again see other mail). > Are we > worried that virtio data structures are going to be a burden on the 31-bit address space? > > That is a good question I can not answer. Since it is currently at least a page per queue (because we use dma direct, right Mimu?), I am concerned about this. Connie, what is your opinion? Regards, Halil
next prev parent reply other threads:[~2019-05-15 21:08 UTC|newest] Thread overview: 182+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-04-26 18:32 [PATCH 00/10] s390: virtio: support protected virtualization Halil Pasic 2019-04-26 18:32 ` Halil Pasic 2019-04-26 18:32 ` [PATCH 01/10] virtio/s390: use vring_create_virtqueue Halil Pasic 2019-04-26 18:32 ` Halil Pasic 2019-05-03 9:17 ` Cornelia Huck 2019-05-03 20:04 ` Michael S. Tsirkin 2019-05-03 20:04 ` Michael S. Tsirkin 2019-05-04 14:03 ` Halil Pasic 2019-05-04 14:03 ` Halil Pasic 2019-05-05 11:15 ` Cornelia Huck 2019-05-05 11:15 ` Cornelia Huck 2019-05-07 13:58 ` Christian Borntraeger 2019-05-07 13:58 ` Christian Borntraeger 2019-05-08 20:12 ` Halil Pasic 2019-05-08 20:12 ` Halil Pasic 2019-05-10 14:07 ` Cornelia Huck 2019-05-10 14:07 ` Cornelia Huck 2019-05-12 16:47 ` Michael S. Tsirkin 2019-05-12 16:47 ` Michael S. Tsirkin 2019-05-13 9:52 ` Cornelia Huck 2019-05-13 9:52 ` Cornelia Huck 2019-05-13 12:27 ` Michael Mueller 2019-05-13 12:27 ` Michael Mueller 2019-05-13 12:29 ` Cornelia Huck 2019-05-13 12:29 ` Cornelia Huck 2019-04-26 18:32 ` [PATCH 02/10] virtio/s390: DMA support for virtio-ccw Halil Pasic 2019-04-26 18:32 ` Halil Pasic 2019-05-03 9:31 ` Cornelia Huck 2019-04-26 18:32 ` [PATCH 03/10] virtio/s390: enable packed ring Halil Pasic 2019-04-26 18:32 ` Halil Pasic 2019-05-03 9:44 ` Cornelia Huck 2019-05-05 15:13 ` Thomas Huth 2019-05-05 15:13 ` Thomas Huth 2019-04-26 18:32 ` [PATCH 04/10] s390/mm: force swiotlb for protected virtualization Halil Pasic 2019-04-26 18:32 ` Halil Pasic 2019-04-26 19:27 ` Christoph Hellwig 2019-04-26 19:27 ` Christoph Hellwig 2019-04-29 13:59 ` Halil Pasic 2019-04-29 13:59 ` Halil Pasic 2019-04-29 14:05 ` Christian Borntraeger 2019-04-29 14:05 ` Christian Borntraeger 2019-05-13 12:50 ` Michael Mueller 2019-05-13 12:50 ` Michael Mueller 2019-05-08 13:15 ` Claudio Imbrenda 2019-05-08 13:15 ` Claudio Imbrenda 2019-05-09 22:34 ` Halil Pasic 2019-05-09 22:34 ` Halil Pasic 2019-05-15 14:15 ` Michael Mueller 2019-05-15 14:15 ` Michael Mueller [not found] ` <ad23f5e7-dc78-04af-c892-47bbc65134c6@linux.ibm.com> 2019-05-09 18:05 ` Jason J. Herne 2019-05-09 18:05 ` Jason J. Herne 2019-05-09 18:05 ` Jason J. Herne 2019-05-10 7:49 ` Claudio Imbrenda 2019-05-10 7:49 ` Claudio Imbrenda 2019-04-26 18:32 ` [PATCH 05/10] s390/cio: introduce DMA pools to cio Halil Pasic 2019-04-26 18:32 ` Halil Pasic 2019-05-08 13:18 ` Sebastian Ott 2019-05-08 13:18 ` Sebastian Ott 2019-05-08 21:22 ` Halil Pasic 2019-05-08 21:22 ` Halil Pasic 2019-05-09 8:40 ` Sebastian Ott 2019-05-09 8:40 ` Sebastian Ott 2019-05-09 10:11 ` Cornelia Huck 2019-05-09 10:11 ` Cornelia Huck 2019-05-09 22:11 ` Halil Pasic 2019-05-09 22:11 ` Halil Pasic 2019-05-10 14:10 ` Cornelia Huck 2019-05-10 14:10 ` Cornelia Huck 2019-05-12 18:22 ` Halil Pasic 2019-05-12 18:22 ` Halil Pasic 2019-05-13 13:29 ` Cornelia Huck 2019-05-13 13:29 ` Cornelia Huck 2019-05-15 17:12 ` Halil Pasic 2019-05-15 17:12 ` Halil Pasic 2019-05-16 6:13 ` Cornelia Huck 2019-05-16 6:13 ` Cornelia Huck 2019-05-16 13:59 ` Sebastian Ott 2019-05-16 13:59 ` Sebastian Ott 2019-05-20 12:13 ` Halil Pasic 2019-05-20 12:13 ` Halil Pasic 2019-05-21 8:46 ` Michael Mueller 2019-05-21 8:46 ` Michael Mueller 2019-05-22 12:07 ` Sebastian Ott 2019-05-22 12:07 ` Sebastian Ott 2019-05-22 22:12 ` Halil Pasic 2019-05-22 22:12 ` Halil Pasic 2019-05-23 15:17 ` Halil Pasic 2019-05-23 15:17 ` Halil Pasic 2019-04-26 18:32 ` [PATCH 06/10] s390/cio: add basic protected virtualization support Halil Pasic 2019-04-26 18:32 ` Halil Pasic 2019-05-08 13:46 ` Sebastian Ott 2019-05-08 13:46 ` Sebastian Ott 2019-05-08 13:54 ` Christoph Hellwig 2019-05-08 13:54 ` Christoph Hellwig 2019-05-08 21:08 ` Halil Pasic 2019-05-08 21:08 ` Halil Pasic 2019-05-09 8:52 ` Sebastian Ott 2019-05-09 8:52 ` Sebastian Ott 2019-05-08 14:23 ` Pierre Morel 2019-05-08 14:23 ` Pierre Morel 2019-05-13 9:41 ` Cornelia Huck 2019-05-13 9:41 ` Cornelia Huck 2019-05-14 14:47 ` Jason J. Herne 2019-05-14 14:47 ` Jason J. Herne 2019-05-15 21:08 ` Halil Pasic [this message] 2019-05-15 21:08 ` Halil Pasic 2019-05-16 6:32 ` Cornelia Huck 2019-05-16 6:32 ` Cornelia Huck 2019-05-16 13:42 ` Halil Pasic 2019-05-16 13:42 ` Halil Pasic 2019-05-16 13:54 ` Cornelia Huck 2019-05-16 13:54 ` Cornelia Huck 2019-05-15 20:51 ` Halil Pasic 2019-05-15 20:51 ` Halil Pasic 2019-05-16 6:29 ` Cornelia Huck 2019-05-16 6:29 ` Cornelia Huck 2019-05-18 18:11 ` Halil Pasic 2019-05-18 18:11 ` Halil Pasic 2019-05-20 10:21 ` Cornelia Huck 2019-05-20 10:21 ` Cornelia Huck 2019-05-20 12:34 ` Halil Pasic 2019-05-20 12:34 ` Halil Pasic 2019-05-20 13:43 ` Cornelia Huck 2019-05-20 13:43 ` Cornelia Huck 2019-04-26 18:32 ` [PATCH 07/10] s390/airq: use DMA memory for adapter interrupts Halil Pasic 2019-04-26 18:32 ` Halil Pasic 2019-05-08 13:58 ` Sebastian Ott 2019-05-08 13:58 ` Sebastian Ott 2019-05-09 11:37 ` Cornelia Huck 2019-05-09 11:37 ` Cornelia Huck 2019-05-13 12:59 ` Cornelia Huck 2019-05-13 12:59 ` Cornelia Huck 2019-04-26 18:32 ` [PATCH 08/10] virtio/s390: add indirection to indicators access Halil Pasic 2019-04-26 18:32 ` Halil Pasic 2019-05-08 14:31 ` Pierre Morel 2019-05-08 14:31 ` Pierre Morel 2019-05-09 12:01 ` Pierre Morel 2019-05-09 12:01 ` Pierre Morel 2019-05-09 18:26 ` Halil Pasic 2019-05-09 18:26 ` Halil Pasic 2019-05-10 7:43 ` Pierre Morel 2019-05-10 7:43 ` Pierre Morel 2019-05-10 11:54 ` Halil Pasic 2019-05-10 11:54 ` Halil Pasic 2019-05-10 15:36 ` Pierre Morel 2019-05-10 15:36 ` Pierre Morel 2019-05-13 10:15 ` Cornelia Huck 2019-05-13 10:15 ` Cornelia Huck 2019-05-16 15:24 ` Pierre Morel 2019-05-16 15:24 ` Pierre Morel 2019-04-26 18:32 ` [PATCH 09/10] virtio/s390: use DMA memory for ccw I/O and classic notifiers Halil Pasic 2019-04-26 18:32 ` Halil Pasic 2019-05-08 14:46 ` Pierre Morel 2019-05-08 14:46 ` Pierre Morel 2019-05-09 13:30 ` Pierre Morel 2019-05-09 13:30 ` Pierre Morel 2019-05-09 18:30 ` Halil Pasic 2019-05-09 18:30 ` Halil Pasic 2019-05-13 13:54 ` Cornelia Huck 2019-05-13 13:54 ` Cornelia Huck 2019-04-26 18:32 ` [PATCH 10/10] virtio/s390: make airq summary indicators DMA Halil Pasic 2019-04-26 18:32 ` Halil Pasic 2019-05-08 15:11 ` Pierre Morel 2019-05-08 15:11 ` Pierre Morel 2019-05-15 13:33 ` Michael Mueller 2019-05-15 13:33 ` Michael Mueller 2019-05-15 17:23 ` Halil Pasic 2019-05-15 17:23 ` Halil Pasic 2019-05-13 12:20 ` Cornelia Huck 2019-05-13 12:20 ` Cornelia Huck 2019-05-15 13:43 ` Michael Mueller 2019-05-15 13:43 ` Michael Mueller 2019-05-15 13:50 ` Cornelia Huck 2019-05-15 13:50 ` Cornelia Huck 2019-05-15 17:18 ` Halil Pasic 2019-05-15 17:18 ` Halil Pasic 2019-05-03 9:55 ` [PATCH 00/10] s390: virtio: support protected virtualization Cornelia Huck 2019-05-03 10:03 ` Juergen Gross 2019-05-03 13:33 ` Cornelia Huck 2019-05-03 13:33 ` Cornelia Huck 2019-05-04 13:58 ` Halil Pasic 2019-05-04 13:58 ` Halil Pasic
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20190515230817.2f8a8a5d.pasic@linux.ibm.com \ --to=pasic@linux.ibm.com \ --cc=alifm@linux.ibm.com \ --cc=borntraeger@de.ibm.com \ --cc=cohuck@redhat.com \ --cc=farman@linux.ibm.com \ --cc=frankja@linux.ibm.com \ --cc=gor@linux.ibm.com \ --cc=hch@infradead.org \ --cc=imbrenda@linux.ibm.com \ --cc=jjherne@linux.ibm.com \ --cc=kvm@vger.kernel.org \ --cc=linux-s390@vger.kernel.org \ --cc=mihajlov@linux.ibm.com \ --cc=mst@redhat.com \ --cc=schwidefsky@de.ibm.com \ --cc=sebott@linux.ibm.com \ --cc=thuth@redhat.com \ --cc=virtualization@lists.linux-foundation.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.