virtio fixes pull for 4.0?

virtio fixes pull for 4.0?

[Michael S. Tsirkin]

Hi Rusty!
There are a bunch of (mostly virtio 1.0 related) fixes for virtio
that need to go into 4.0 I think.
	virtio_blk: typo fix
	virtio_blk: fix comment for virtio 1.0
	virtio_console: init work unconditionally
	virtio_console: avoid config access from irq
	virtio_balloon: set DRIVER_OK before using device

seem ready?

	virtio_mmio: generation support
	virtio_mmio: fix endian-ness for mmio these two are waiting for ack by Pawel

These two fix bugs in virtio 1.0 code for mmio.
Host code for that was AFAIK not posted, so I can't test properly.
Pawel?

	virtio-balloon: do not call blocking ops when !TASK_RUNNING

Rusty, it seems you prefer a different fix for this issue,
while Cornelia prefers mine. Maybe both me and Cornelia misunderstand the
issue? I know you dealt with a ton of similar issues recently
so you are more likely to be right, but I'd like to understand
what's going on better all the same. Could you help please?

	virtio_rpmsg: set DRIVER_OK before using device

Just posted this, but seems pretty obvious.

I think it's a good idea to merge these patches (maybe except the
!TASK_RUNNING thing) sooner rather than later, to make sure people have
the time to test the fixes properly.  Would you like me to pack up (some
of them) them up and do a pull request?

Thanks,
-- 
MST

[Qemu-devel] virtio fixes pull for 4.0?

[Michael S. Tsirkin]

Hi Rusty!
There are a bunch of (mostly virtio 1.0 related) fixes for virtio
that need to go into 4.0 I think.
	virtio_blk: typo fix
	virtio_blk: fix comment for virtio 1.0
	virtio_console: init work unconditionally
	virtio_console: avoid config access from irq
	virtio_balloon: set DRIVER_OK before using device

seem ready?

	virtio_mmio: generation support
	virtio_mmio: fix endian-ness for mmio these two are waiting for ack by Pawel

These two fix bugs in virtio 1.0 code for mmio.
Host code for that was AFAIK not posted, so I can't test properly.
Pawel?

	virtio-balloon: do not call blocking ops when !TASK_RUNNING

Rusty, it seems you prefer a different fix for this issue,
while Cornelia prefers mine. Maybe both me and Cornelia misunderstand the
issue? I know you dealt with a ton of similar issues recently
so you are more likely to be right, but I'd like to understand
what's going on better all the same. Could you help please?

	virtio_rpmsg: set DRIVER_OK before using device

Just posted this, but seems pretty obvious.

I think it's a good idea to merge these patches (maybe except the
!TASK_RUNNING thing) sooner rather than later, to make sure people have
the time to test the fixes properly.  Would you like me to pack up (some
of them) them up and do a pull request?

Thanks,
-- 
MST

Re: virtio fixes pull for 4.0?

[Rusty Russell]

"Michael S. Tsirkin" <mst@redhat.com> writes:
> Hi Rusty!
> There are a bunch of (mostly virtio 1.0 related) fixes for virtio
> that need to go into 4.0 I think.
> 	virtio_blk: typo fix
> 	virtio_blk: fix comment for virtio 1.0

OK, I've added these two.  I tend to be overcautious after the merge
window.

> 	virtio_console: init work unconditionally
> 	virtio_console: avoid config access from irq
> 	virtio_balloon: set DRIVER_OK before using device
>
> seem ready?

These are in my virtio-next tree already.  

> 	virtio_mmio: generation support
> 	virtio_mmio: fix endian-ness for mmio these two are waiting for ack by Pawel
>
> These two fix bugs in virtio 1.0 code for mmio.
> Host code for that was AFAIK not posted, so I can't test properly.
> Pawel?

I'm waiting on Acks for these two.

> 	virtio-balloon: do not call blocking ops when !TASK_RUNNING
>
> Rusty, it seems you prefer a different fix for this issue,
> while Cornelia prefers mine. Maybe both me and Cornelia misunderstand the
> issue? I know you dealt with a ton of similar issues recently
> so you are more likely to be right, but I'd like to understand
> what's going on better all the same. Could you help please?

In the longer run, we should handle failures from these callbacks.  But
we don't need to do that right now.  So we want the minimal fix.

And an annotation is the minimal fix.  The bug has been there for ages;
it's just the warning that is new (if we *always* slept, we would
spin, but in practice we'll rarely sleep).

> 	virtio_rpmsg: set DRIVER_OK before using device
>
> Just posted this, but seems pretty obvious.

Yep, I've applied this too.  Thanks!

> I think it's a good idea to merge these patches (maybe except the
> !TASK_RUNNING thing) sooner rather than later, to make sure people have
> the time to test the fixes properly.  Would you like me to pack up (some
> of them) them up and do a pull request?

I'm waiting a bit longer, since they seem to still be tricking in.

I'm still chasing a QEMU bug, where it seems to fill in a number too
large in the 'len' field for a block device.  It should be 1 byte for a
block device write, for example.  See patch which causes assert() in
qemu, but I had to stop at that point (should get back tomorrow I hope).

Thanks,
Rusty.

diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index 882a31b..98e99b8 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -243,16 +243,21 @@ int virtio_queue_empty(VirtQueue *vq)
 }
 
 void virtqueue_fill(VirtQueue *vq, const VirtQueueElement *elem,
-                    unsigned int len, unsigned int idx)
+                    unsigned int len_written, unsigned int idx)
 {
-    unsigned int offset;
+    unsigned int offset, tot_wlen;
     int i;
 
-    trace_virtqueue_fill(vq, elem, len, idx);
+    trace_virtqueue_fill(vq, elem, len_written, idx);
+
+    for (tot_wlen = i = 0; i < elem->out_num; i++) {
+        tot_wlen += elem->out_sg[i].iov_len;
+    }
+    assert(len_written <= tot_wlen);
 
     offset = 0;
     for (i = 0; i < elem->in_num; i++) {
-        size_t size = MIN(len - offset, elem->in_sg[i].iov_len);
+        size_t size = MIN(len_written - offset, elem->in_sg[i].iov_len);
 
         cpu_physical_memory_unmap(elem->in_sg[i].iov_base,
                                   elem->in_sg[i].iov_len,
@@ -270,7 +275,7 @@ void virtqueue_fill(VirtQueue *vq, const VirtQueueElement *elem,
 
     /* Get a pointer to the next entry in the used ring. */
     vring_used_ring_id(vq, idx, elem->index);
-    vring_used_ring_len(vq, idx, len);
+    vring_used_ring_len(vq, idx, len_written);
 }
 
 void virtqueue_flush(VirtQueue *vq, unsigned int count)
@@ -288,9 +293,9 @@ void virtqueue_flush(VirtQueue *vq, unsigned int count)
 }
 
 void virtqueue_push(VirtQueue *vq, const VirtQueueElement *elem,
-                    unsigned int len)
+                    unsigned int len_written)
 {
-    virtqueue_fill(vq, elem, len, 0);
+    virtqueue_fill(vq, elem, len_written, 0);
     virtqueue_flush(vq, 1);
 }
 
diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index df09993..153374f 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -191,10 +191,10 @@ VirtQueue *virtio_add_queue(VirtIODevice *vdev, int queue_size,
 void virtio_del_queue(VirtIODevice *vdev, int n);
 
 void virtqueue_push(VirtQueue *vq, const VirtQueueElement *elem,
-                    unsigned int len);
+                    unsigned int len_written);
 void virtqueue_flush(VirtQueue *vq, unsigned int count);
 void virtqueue_fill(VirtQueue *vq, const VirtQueueElement *elem,
-                    unsigned int len, unsigned int idx);
+                    unsigned int len_written, unsigned int idx);
 
 void virtqueue_map_sg(struct iovec *sg, hwaddr *addr,
     size_t num_sg, int is_write);

Re: [Qemu-devel] virtio fixes pull for 4.0?

[Rusty Russell]

"Michael S. Tsirkin" <mst@redhat.com> writes:
> Hi Rusty!
> There are a bunch of (mostly virtio 1.0 related) fixes for virtio
> that need to go into 4.0 I think.
> 	virtio_blk: typo fix
> 	virtio_blk: fix comment for virtio 1.0

OK, I've added these two.  I tend to be overcautious after the merge
window.

> 	virtio_console: init work unconditionally
> 	virtio_console: avoid config access from irq
> 	virtio_balloon: set DRIVER_OK before using device
>
> seem ready?

These are in my virtio-next tree already.  

> 	virtio_mmio: generation support
> 	virtio_mmio: fix endian-ness for mmio these two are waiting for ack by Pawel
>
> These two fix bugs in virtio 1.0 code for mmio.
> Host code for that was AFAIK not posted, so I can't test properly.
> Pawel?

I'm waiting on Acks for these two.

> 	virtio-balloon: do not call blocking ops when !TASK_RUNNING
>
> Rusty, it seems you prefer a different fix for this issue,
> while Cornelia prefers mine. Maybe both me and Cornelia misunderstand the
> issue? I know you dealt with a ton of similar issues recently
> so you are more likely to be right, but I'd like to understand
> what's going on better all the same. Could you help please?

In the longer run, we should handle failures from these callbacks.  But
we don't need to do that right now.  So we want the minimal fix.

And an annotation is the minimal fix.  The bug has been there for ages;
it's just the warning that is new (if we *always* slept, we would
spin, but in practice we'll rarely sleep).

> 	virtio_rpmsg: set DRIVER_OK before using device
>
> Just posted this, but seems pretty obvious.

Yep, I've applied this too.  Thanks!

> I think it's a good idea to merge these patches (maybe except the
> !TASK_RUNNING thing) sooner rather than later, to make sure people have
> the time to test the fixes properly.  Would you like me to pack up (some
> of them) them up and do a pull request?

I'm waiting a bit longer, since they seem to still be tricking in.

I'm still chasing a QEMU bug, where it seems to fill in a number too
large in the 'len' field for a block device.  It should be 1 byte for a
block device write, for example.  See patch which causes assert() in
qemu, but I had to stop at that point (should get back tomorrow I hope).

Thanks,
Rusty.

diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index 882a31b..98e99b8 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -243,16 +243,21 @@ int virtio_queue_empty(VirtQueue *vq)
 }
 
 void virtqueue_fill(VirtQueue *vq, const VirtQueueElement *elem,
-                    unsigned int len, unsigned int idx)
+                    unsigned int len_written, unsigned int idx)
 {
-    unsigned int offset;
+    unsigned int offset, tot_wlen;
     int i;
 
-    trace_virtqueue_fill(vq, elem, len, idx);
+    trace_virtqueue_fill(vq, elem, len_written, idx);
+
+    for (tot_wlen = i = 0; i < elem->out_num; i++) {
+        tot_wlen += elem->out_sg[i].iov_len;
+    }
+    assert(len_written <= tot_wlen);
 
     offset = 0;
     for (i = 0; i < elem->in_num; i++) {
-        size_t size = MIN(len - offset, elem->in_sg[i].iov_len);
+        size_t size = MIN(len_written - offset, elem->in_sg[i].iov_len);
 
         cpu_physical_memory_unmap(elem->in_sg[i].iov_base,
                                   elem->in_sg[i].iov_len,
@@ -270,7 +275,7 @@ void virtqueue_fill(VirtQueue *vq, const VirtQueueElement *elem,
 
     /* Get a pointer to the next entry in the used ring. */
     vring_used_ring_id(vq, idx, elem->index);
-    vring_used_ring_len(vq, idx, len);
+    vring_used_ring_len(vq, idx, len_written);
 }
 
 void virtqueue_flush(VirtQueue *vq, unsigned int count)
@@ -288,9 +293,9 @@ void virtqueue_flush(VirtQueue *vq, unsigned int count)
 }
 
 void virtqueue_push(VirtQueue *vq, const VirtQueueElement *elem,
-                    unsigned int len)
+                    unsigned int len_written)
 {
-    virtqueue_fill(vq, elem, len, 0);
+    virtqueue_fill(vq, elem, len_written, 0);
     virtqueue_flush(vq, 1);
 }
 
diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index df09993..153374f 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -191,10 +191,10 @@ VirtQueue *virtio_add_queue(VirtIODevice *vdev, int queue_size,
 void virtio_del_queue(VirtIODevice *vdev, int n);
 
 void virtqueue_push(VirtQueue *vq, const VirtQueueElement *elem,
-                    unsigned int len);
+                    unsigned int len_written);
 void virtqueue_flush(VirtQueue *vq, unsigned int count);
 void virtqueue_fill(VirtQueue *vq, const VirtQueueElement *elem,
-                    unsigned int len, unsigned int idx);
+                    unsigned int len_written, unsigned int idx);
 
 void virtqueue_map_sg(struct iovec *sg, hwaddr *addr,
     size_t num_sg, int is_write);

Re: virtio fixes pull for 4.0?

[Cornelia Huck]

On Mon, 09 Mar 2015 17:43:19 +1030
Rusty Russell <rusty@rustcorp.com.au> wrote:

> "Michael S. Tsirkin" <mst@redhat.com> writes:

> > 	virtio-balloon: do not call blocking ops when !TASK_RUNNING
> >
> > Rusty, it seems you prefer a different fix for this issue,
> > while Cornelia prefers mine. Maybe both me and Cornelia misunderstand the
> > issue? I know you dealt with a ton of similar issues recently
> > so you are more likely to be right, but I'd like to understand
> > what's going on better all the same. Could you help please?
> 
> In the longer run, we should handle failures from these callbacks.  But
> we don't need to do that right now.  So we want the minimal fix.
> 
> And an annotation is the minimal fix.  The bug has been there for ages;
> it's just the warning that is new (if we *always* slept, we would
> spin, but in practice we'll rarely sleep).

I don't think doing_io() in virtio-ccw will sleep often, although it
might happen under loads that delay posting the interrupt.

I understand why you'd like to do the minimal fix, and it's not likely
that new problems start popping up now.

So I'm OK with doing the annotation for 4.0 and more involved changes
for 4.1.

Re: [Qemu-devel] virtio fixes pull for 4.0?

[Cornelia Huck]

On Mon, 09 Mar 2015 17:43:19 +1030
Rusty Russell <rusty@rustcorp.com.au> wrote:

> "Michael S. Tsirkin" <mst@redhat.com> writes:

> > 	virtio-balloon: do not call blocking ops when !TASK_RUNNING
> >
> > Rusty, it seems you prefer a different fix for this issue,
> > while Cornelia prefers mine. Maybe both me and Cornelia misunderstand the
> > issue? I know you dealt with a ton of similar issues recently
> > so you are more likely to be right, but I'd like to understand
> > what's going on better all the same. Could you help please?
> 
> In the longer run, we should handle failures from these callbacks.  But
> we don't need to do that right now.  So we want the minimal fix.
> 
> And an annotation is the minimal fix.  The bug has been there for ages;
> it's just the warning that is new (if we *always* slept, we would
> spin, but in practice we'll rarely sleep).

I don't think doing_io() in virtio-ccw will sleep often, although it
might happen under loads that delay posting the interrupt.

I understand why you'd like to do the minimal fix, and it's not likely
that new problems start popping up now.

So I'm OK with doing the annotation for 4.0 and more involved changes
for 4.1.

Re: virtio fixes pull for 4.0?

[Michael S. Tsirkin]

On Mon, Mar 09, 2015 at 05:43:19PM +1030, Rusty Russell wrote:
> > I think it's a good idea to merge these patches (maybe except the
> > !TASK_RUNNING thing) sooner rather than later, to make sure people have
> > the time to test the fixes properly.  Would you like me to pack up (some
> > of them) them up and do a pull request?
> 
> I'm waiting a bit longer, since they seem to still be tricking in.

I don't think there was a pull request since rc1 which
makes me a bit anxious. Any outstanding issues?

-- 
MST

Re: [Qemu-devel] virtio fixes pull for 4.0?

[Michael S. Tsirkin]

On Mon, Mar 09, 2015 at 05:43:19PM +1030, Rusty Russell wrote:
> > I think it's a good idea to merge these patches (maybe except the
> > !TASK_RUNNING thing) sooner rather than later, to make sure people have
> > the time to test the fixes properly.  Would you like me to pack up (some
> > of them) them up and do a pull request?
> 
> I'm waiting a bit longer, since they seem to still be tricking in.

I don't think there was a pull request since rc1 which
makes me a bit anxious. Any outstanding issues?

-- 
MST

Re: virtio fixes pull for 4.0?

[Rusty Russell]

"Michael S. Tsirkin" <mst@redhat.com> writes:
> On Mon, Mar 09, 2015 at 05:43:19PM +1030, Rusty Russell wrote:
>> > I think it's a good idea to merge these patches (maybe except the
>> > !TASK_RUNNING thing) sooner rather than later, to make sure people have
>> > the time to test the fixes properly.  Would you like me to pack up (some
>> > of them) them up and do a pull request?
>> 
>> I'm waiting a bit longer, since they seem to still be tricking in.
>
> I don't think there was a pull request since rc1 which
> makes me a bit anxious. Any outstanding issues?

Well, I was hoping for an ack on your mmio patches.  But the weekend
is over, so I've pushed and sent Linus a pull request.

Cheers,
Rusty.

Re: [Qemu-devel] virtio fixes pull for 4.0?

[Rusty Russell]

"Michael S. Tsirkin" <mst@redhat.com> writes:
> On Mon, Mar 09, 2015 at 05:43:19PM +1030, Rusty Russell wrote:
>> > I think it's a good idea to merge these patches (maybe except the
>> > !TASK_RUNNING thing) sooner rather than later, to make sure people have
>> > the time to test the fixes properly.  Would you like me to pack up (some
>> > of them) them up and do a pull request?
>> 
>> I'm waiting a bit longer, since they seem to still be tricking in.
>
> I don't think there was a pull request since rc1 which
> makes me a bit anxious. Any outstanding issues?

Well, I was hoping for an ack on your mmio patches.  But the weekend
is over, so I've pushed and sent Linus a pull request.

Cheers,
Rusty.

Re: virtio fixes pull for 4.0?

[Rusty Russell]

"Michael S. Tsirkin" <mst@redhat.com> writes:
> On Mon, Mar 09, 2015 at 05:43:19PM +1030, Rusty Russell wrote:
>> > I think it's a good idea to merge these patches (maybe except the
>> > !TASK_RUNNING thing) sooner rather than later, to make sure people have
>> > the time to test the fixes properly.  Would you like me to pack up (some
>> > of them) them up and do a pull request?
>> 
>> I'm waiting a bit longer, since they seem to still be tricking in.
>
> I don't think there was a pull request since rc1 which
> makes me a bit anxious. Any outstanding issues?

Well, I was hoping for an ack on your mmio patches.  But the weekend
is over, so I've pushed and sent Linus a pull request.

Cheers,
Rusty.

Re: virtio fixes pull for 4.0?

[Pawel Moll]

On Mon, 2015-03-09 at 07:13 +0000, Rusty Russell wrote:
> > 	virtio_mmio: generation support
> > 	virtio_mmio: fix endian-ness for mmio these two are waiting for ack by Pawel
> >
> > These two fix bugs in virtio 1.0 code for mmio.
> > Host code for that was AFAIK not posted, so I can't test properly.
> > Pawel?
> 
> I'm waiting on Acks for these two.

Right, sorry about being silent for a while - I forked and was on
paternity leave...

Will go through the thread and respond today.

Pawel

Re: [Qemu-devel] virtio fixes pull for 4.0?

[Pawel Moll]

On Mon, 2015-03-09 at 07:13 +0000, Rusty Russell wrote:
> > 	virtio_mmio: generation support
> > 	virtio_mmio: fix endian-ness for mmio these two are waiting for ack by Pawel
> >
> > These two fix bugs in virtio 1.0 code for mmio.
> > Host code for that was AFAIK not posted, so I can't test properly.
> > Pawel?
> 
> I'm waiting on Acks for these two.

Right, sorry about being silent for a while - I forked and was on
paternity leave...

Will go through the thread and respond today.

Pawel

Re: virtio fixes pull for 4.0?

[Pawel Moll]

On Mon, 2015-03-09 at 07:13 +0000, Rusty Russell wrote:
> > 	virtio_mmio: generation support
> > 	virtio_mmio: fix endian-ness for mmio these two are waiting for ack by Pawel
> >
> > These two fix bugs in virtio 1.0 code for mmio.
> > Host code for that was AFAIK not posted, so I can't test properly.
> > Pawel?
> 
> I'm waiting on Acks for these two.

Right, sorry about being silent for a while - I forked and was on
paternity leave...

Will go through the thread and respond today.

Pawel

virtio fixes pull for 4.0?

[Michael S. Tsirkin]

Hi Rusty!
There are a bunch of (mostly virtio 1.0 related) fixes for virtio
that need to go into 4.0 I think.
	virtio_blk: typo fix
	virtio_blk: fix comment for virtio 1.0
	virtio_console: init work unconditionally
	virtio_console: avoid config access from irq
	virtio_balloon: set DRIVER_OK before using device

seem ready?

	virtio_mmio: generation support
	virtio_mmio: fix endian-ness for mmio these two are waiting for ack by Pawel

These two fix bugs in virtio 1.0 code for mmio.
Host code for that was AFAIK not posted, so I can't test properly.
Pawel?

	virtio-balloon: do not call blocking ops when !TASK_RUNNING

Rusty, it seems you prefer a different fix for this issue,
while Cornelia prefers mine. Maybe both me and Cornelia misunderstand the
issue? I know you dealt with a ton of similar issues recently
so you are more likely to be right, but I'd like to understand
what's going on better all the same. Could you help please?

	virtio_rpmsg: set DRIVER_OK before using device

Just posted this, but seems pretty obvious.

I think it's a good idea to merge these patches (maybe except the
!TASK_RUNNING thing) sooner rather than later, to make sure people have
the time to test the fixes properly.  Would you like me to pack up (some
of them) them up and do a pull request?

Thanks,
-- 
MST