flow control in vmbus ring buffers

flow control in vmbus ring buffers

[Roman Kagan]

I'm trying to get my head around how the flow control in vmbus ring
buffers works.

In particular, I'm failing to see how the following can be prevented:


     producer                     |       consumer
==================================================================
read read_index                   |
(enough room for packet)          |
write pending_send_sz = 0         |
write packet                      |
update write_index                |
                                  | read write_index
read read_index                   | read packet
(not enough room for packet)      | update read_index (= write_index)
                                  | read pending_send_sz = 0
write pending_send_sz = X         | skip notification
go to sleep                       | go to sleep
                                stall

Could anybody please shed some light on how it's supposed to work?

Thanks,
Roman.

Re: flow control in vmbus ring buffers

[Roman Kagan]

On Tue, Dec 24, 2019 at 10:56:08AM +0000, Roman Kagan wrote:
> I'm trying to get my head around how the flow control in vmbus ring
> buffers works.
> 
> In particular, I'm failing to see how the following can be prevented:
> 
> 
>      producer                     |       consumer
> ==================================================================
> read read_index                   |
> (enough room for packet)          |
> write pending_send_sz = 0         |
> write packet                      |
> update write_index                |
>                                   | read write_index
> read read_index                   | read packet
> (not enough room for packet)      | update read_index (= write_index)
>                                   | read pending_send_sz = 0
> write pending_send_sz = X         | skip notification
> go to sleep                       | go to sleep
>                                 stall
> 
> Could anybody please shed some light on how it's supposed to work?

Sorry to reply to myself, but looks like the answer is to re-read
read_index on the producer side after setting pending_send_sz.

Thanks and sorry for the noise,
Roman.

RE: flow control in vmbus ring buffers

[Michael Kelley]

From: Roman Kagan <rkagan@virtuozzo.com> Sent: Tuesday, December 24, 2019 8:29 AM
> 
> On Tue, Dec 24, 2019 at 10:56:08AM +0000, Roman Kagan wrote:
> > I'm trying to get my head around how the flow control in vmbus ring
> > buffers works.
> >
> > In particular, I'm failing to see how the following can be prevented:
> >
> >
> >      producer                     |       consumer
> > ==================================================================
> > read read_index                   |
> > (enough room for packet)          |
> > write pending_send_sz = 0         |
> > write packet                      |
> > update write_index                |
> >                                   | read write_index
> > read read_index                   | read packet
> > (not enough room for packet)      | update read_index (= write_index)
> >                                   | read pending_send_sz = 0
> > write pending_send_sz = X         | skip notification
> > go to sleep                       | go to sleep
> >                                 stall
> >
> > Could anybody please shed some light on how it's supposed to work?
> 
> Sorry to reply to myself, but looks like the answer is to re-read
> read_index on the producer side after setting pending_send_sz.
> 

Are you seeing a problem in the Linux guest code in drivers/hv/ring_buffer.c?
The only time the Linux guest as the producer sets pending_send_sz is in the
hv_socket code.  Or are you looking at the KVM code that does emulation of
Hyper-V, where KVM is acting as the producer?

Just curious as to what prompted your question.  I did the most recent fixes a
couple years ago to the Linux guest code for handling the ring buffer interaction
with Hyper-V, and if there is still a problem, I'm super interested. :-)

Michael

Re: flow control in vmbus ring buffers

[Roman Kagan]

On Tue, Dec 24, 2019 at 06:03:15PM +0000, Michael Kelley wrote:
> From: Roman Kagan <rkagan@virtuozzo.com> Sent: Tuesday, December 24, 2019 8:29 AM
> > 
> > On Tue, Dec 24, 2019 at 10:56:08AM +0000, Roman Kagan wrote:
> > > I'm trying to get my head around how the flow control in vmbus ring
> > > buffers works.
> > >
> > > In particular, I'm failing to see how the following can be prevented:
> > >
> > >
> > >      producer                     |       consumer
> > > ==================================================================
> > > read read_index                   |
> > > (enough room for packet)          |
> > > write pending_send_sz = 0         |
> > > write packet                      |
> > > update write_index                |
> > >                                   | read write_index
> > > read read_index                   | read packet
> > > (not enough room for packet)      | update read_index (= write_index)
> > >                                   | read pending_send_sz = 0
> > > write pending_send_sz = X         | skip notification
> > > go to sleep                       | go to sleep
> > >                                 stall
> > >
> > > Could anybody please shed some light on how it's supposed to work?
> > 
> > Sorry to reply to myself, but looks like the answer is to re-read
> > read_index on the producer side after setting pending_send_sz.
> > 
> 
> Are you seeing a problem in the Linux guest code in drivers/hv/ring_buffer.c?

No, nothing in particular.

> The only time the Linux guest as the producer sets pending_send_sz is in the
> hv_socket code.

Indeed.  This surprized me somewhat, but, well...

> Or are you looking at the KVM code that does emulation of Hyper-V,
> where KVM is acting as the producer?

Yes, at the QEMU code, more precisely.  I wrote it a while back and now
have some spare cycles to at last submit it to mainline QEMU; while
reviewing it I noticed this part that I did wrong.

Thanks,
Roman.