From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Michael S. Tsirkin" Subject: Re: [virtio-dev] packed ring layout proposal v3 Date: Sun, 1 Oct 2017 07:08:29 +0300 Message-ID: <20171001063817-mutt-send-email-mst@kernel.org> References: <20160915223915.qjlnlvf2w7u37bu3@redhat.com> <20170926011826-mutt-send-email-mst@kernel.org> <7A0DC0C9-F148-4161-B2D1-8D8D14D8B9A1@cisco.com> <20170928023112-mutt-send-email-mst@kernel.org> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Content-Disposition: inline In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: virtualization-bounces@lists.linux-foundation.org Errors-To: virtualization-bounces@lists.linux-foundation.org To: "Liang, Cunming" Cc: "virtio-dev@lists.oasis-open.org" , "Steven Luong (sluong)" , "virtualization@lists.linux-foundation.org" List-Id: virtualization@lists.linuxfoundation.org T24gVGh1LCBTZXAgMjgsIDIwMTcgYXQgMDk6NDQ6MzVBTSArMDAwMCwgTGlhbmcsIEN1bm1pbmcg d3JvdGU6Cj4gCj4gR2V0IGl0IG5vdy4gUGxlYXNlIGNvcnJlY3QgbWUgaWYgSSBtaXNzaW5nIHNv bWV0aGluZy4KPiAKPiAKPiBGbGFncyBzdGF0dXMgaGludHMsCj4gCj4gLSBERVNDX0RSSVZFUiBv bmx5OiBkcml2ZXIgb3ducyB0aGUgZGVzY3JpcHRvciB3L28gYXZhaWxhYmxlIGluZm8gcmVhZHkg Zm9yIGRldmljZSB0byB1c2UKPiAKPiAtIERFU0NfRFJJVkVSIHwgREVTQ19XUkFQOiBkcml2ZXIg aGFzIHByZXBhcmVkIGFuIGF2YWlsYWJsZSBkZXNjcmlwdG9yLCBkZXZpY2UgaGFzbid0IHVzZWQg aXQgeWV0Cj4gCj4gLSBOb25lOiBkZXZpY2UgaGFzIHVzZWQgdGhlIGRlc2NyaXB0b3IsIGFuZCB3 cml0ZSBkZXNjcmlwdG9yIG91dAo+IAo+IC0gREVTQ19XUkFQIG9ubHk6IHNoYWxsIG5vdCBoYXBw ZW4sIGRldmljZSBtYWtlIHN1cmUgdG8gY2xlYXIgaXQKPiAKPiAKPiBQb2xsaW5nIGJlaGF2aW9y IGlzLAo+IAo+IC0gRGV2aWNlIG1vbml0b3IgREVTQ19XUkFQIGJpdCBzZXQgb3Igbm90OyBJZiBz ZXQsIGdvIHRvIHVzZSBkZXNjcmlwdG9yIGFuZCBjbGVhciBERVNDX0RSSVZFUiBiaXQgaW4gdGhl IGVuZCAobm90ZTogYWx3YXlzIG5lZWQgdG8gY2xlYXIgREVTQ19XUkFQKQo+IAo+IC0gRHJpdmVy IG1vbml0b3IgREVTQ19EUklWRVIgYml0IGNsZWFyZWQgb3Igbm90OyBJZiBjbGVhcmVkLCByZWNs YWltIGRlc2NyaXB0b3Ioc2V0IERFU0NfRFJJVkVSKSBhbmQgc2V0IERFU0NfV1JBUCBvbmNlIG5l dyBhdmFpbGFibGUgZGVzY3JpcHRvciBnZXQgcmVhZHkgdG8gZ28KPiAKPiAKPiAtLQo+IFN0ZXZl CgoKSG1tIG5vLCBub3Qgd2hhdCBJIGhhZCBpbiBtaW5kLgoKREVTQ19EUklWRVI6IHVzZWQgYnkg ZHJpdmVyIHRvIHBvbGwuIERyaXZlciBzZXRzIGl0IHdoZW4gd3JpdGluZyBhCmRlc2NyaXB0b3Iu ICBEZXZpY2UgY2xlYXJzIGl0IHdoZW4gb3ZlcndyaXRpbmcgYSBkZXNjcmlwdG9yLgpUaHVzIGRy aXZlciB1c2VzIERFU0NfRFJJVkVSIHRvIGRldGVjdCB0aGF0IGRldmljZSBkYXRhIGluCmRlc2Ny aXB0b3IgaXMgdmFsaWQuCgoKCkRFU0NfV1JBUDogdXNlZCBieSBkZXZpY2UgdG8gcG9sbC4gRHJp dmVyIHNldHMgaXQgdG8gYSAqZGlmZmVyZW50Kgp2YWx1ZSBldmVyeSB0aW1lIGl0IG92ZXJ3cml0 ZXMgYSBkZXNjcmlwdG9yLiBIb3cgdG8gYWNoaWV2ZSBpdD8Kc2luY2UgZGVzY3JpcHRvcnMgYXJl IHdyaXR0ZW4gb3V0IGluIHJpbmcgb3JkZXIsCnNpbXBseSBtYWludGFpbiB0aGUgY3VycmVudCB2 YWx1ZSBpbnRlcm5hbGx5IChzdGFydCB2YWx1ZSAxKSBhbmQgZmxpcCBpdApldmVyeSB0aW1lIHlv dSBvdmVyd3JpdGUgdGhlIGZpcnN0IGRlc2NyaXB0b3IuCkRldmljZSBsZWF2ZXMgaXQgaW50YWN0 IHdoZW4gb3ZlcndyaXRpbmcgYSBkZXNjcmlwdG9yLgoKCkFmdGVyIHdyaXRpbmcgZG93biB0aGlz IGV4cGxhbmF0aW9uLCBJIHRoaW5rIHRoZSBuYW1lcyBhcmVuJ3QKZ3JlYXQuCgoKCkxldCBtZSB0 cnkgYW4gYWx0ZXJuYXRpdmUgZXhwbGFuYXRpb24uCgotLS0tLS0tLS0tLS0tLS0KQSB0d28tYml0 IGZpZWxkLCBEUklWRVJfT1dORVIsIHNpZ25hbHMgdGhlIGJ1ZmZlciBvd25lcnNoaXAuCkl0IGhh cyA0IHBvc3NpYmxlIHZhbHVlczoKdmFsdWVzIDB4MSwgMHgxMSBhcmUgd3JpdHRlbiBieSBkcml2 ZXIKdmFsdWVzIDB4MCwgMHgxMCBhcmUgd3JpdHRlbiBieSBkZXZpY2UKCmVhY2ggdGltZSBkcml2 ZXIgd3JpdGVzIG91dCBhIGRlc2NyaXB0b3IsIGl0IG11c3QgbWFrZSBzdXJlCnRoYXQgdGhlIGhp Z2ggYml0IGluIE9XTkVSIGNoYW5nZXMuCgplYWNoIHRpbWUgZGV2aWNlIHdyaXRlcyBvdXQgYSBk ZXNjcmlwdG9yLCBpdCBtdXN0IG1ha2Ugc3VyZQp0aGF0IHRoZSBoaWdoIGJpdCBpbiBPV05FUiBk b2VzIG5vdCBjaGFuZ2UuCgoKCnRoaXMgaXMgZXhhY3RseSB0aGUgc2FtZSBmdW5jdGlvbmFsbHks IERSSVZFUiBpcyBoaWdoIGJpdCBhbmQKV1JBUCBpcyB0aGUgbG93IGJpdC4gIERvZXMgdGhpcyBt YWtlIHRoaW5ncyBjbGVhcmVyPwotLS0tLS0tLS0tLS0tLS0KCgoKTWF5YmUgdGhlIGRpZmZlcmVu Y2UgYmV0d2VlbiBkZXZpY2UgYW5kIGRyaXZlcgppcyBjb25mdXNpbmcuIFdlIGNhbiBmaXggdGhh dCBieSBjaGFuZ2luZyB0aGUgdmFsdWVzLgpIZXJlIGlzIGFuIGFsdGVybmF0aXZlLiBMZXQgbWUg a25vdyBpZiB5b3UgbGlrZSBpdCBiZXR0ZXIgLQpJIG5lZWQgdG8gdGhpbmsgYSBiaXQgbW9yZSB0 byBtYWtlIHN1cmUgaXQgd29ya3MsCmJ1dCB0byBnaXZlIHlvdSBhbiBpZGVhOgoKCi0tLS0tLS0t LS0tLS0tLQpBIHR3by1iaXQgZmllbGQsIERSSVZFUl9PV05FUiwgc2lnbmFscyB0aGUgYnVmZmVy IG93bmVyc2hpcC4KSXQgaGFzIDQgcG9zc2libGUgdmFsdWVzOgp2YWx1ZXMgMHgxLCAweDEwIGFy ZSB3cml0dGVuIGJ5IGRyaXZlcgp2YWx1ZXMgMHgwLCAweDExIGFyZSB3cml0dGVuIGJ5IGRldmlj ZQoKZWFjaCB0aW1lIGRyaXZlciB3cml0ZXMgb3V0IGEgZGVzY3JpcHRvciwgaXQgbXVzdCBtYWtl IHN1cmUKdGhhdCB0aGUgaGlnaCBiaXQgaW4gT1dORVIgY2hhbmdlcy4gVGh1cyBmaXJzdCB0aW1l Cml0IHdyaXRlcyAweDEwLCBuZXh0IHRpbWUgMHgxLCB0aGVuIDB4MTAgYWdhaW4uCgplYWNoIHRp bWUgZGV2aWNlIHdyaXRlcyBvdXQgYSBkZXNjcmlwdG9yLCBpdCBtdXN0IG1ha2Ugc3VyZQp0aGF0 IHRoZSBsb3cgYml0IGluIE9XTkVSIGNoYW5nZXMuICBUaHVzIGZpcnN0IHRpbWUKaXQgd3JpdGVz IDB4MTEsIG5leHQgdGltZSAweDAsIHRoZW4gMHgxMSBhZ2Fpbi4KCi0tLS0tLS0tLS0tLS0tLQoK CgoKCgoKCgoKCgoKCgoKCgoKPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tCj4gPiBGcm9t OiBNaWNoYWVsIFMuIFRzaXJraW4gW21haWx0bzptc3RAcmVkaGF0LmNvbV0KPiA+IFNlbnQ6IFRo dXJzZGF5LCBTZXB0ZW1iZXIgMjgsIDIwMTcgNzo0OSBBTQo+ID4gVG86IFN0ZXZlbiBMdW9uZyAo c2x1b25nKQo+ID4gQ2M6IExpYW5nLCBDdW5taW5nOyB2aXJ0aW8tZGV2QGxpc3RzLm9hc2lzLW9w ZW4ub3JnOwo+ID4gdmlydHVhbGl6YXRpb25AbGlzdHMubGludXgtZm91bmRhdGlvbi5vcmcKPiA+ IFN1YmplY3Q6IFJlOiBbdmlydGlvLWRldl0gcGFja2VkIHJpbmcgbGF5b3V0IHByb3Bvc2FsIHYz Cj4gPiAKPiA+IE9uIFR1ZSwgU2VwIDI2LCAyMDE3IGF0IDExOjM4OjE4UE0gKzAwMDAsIFN0ZXZl biBMdW9uZyAoc2x1b25nKSB3cm90ZToKPiA+ID4gTWljaGFlbCwKPiA+ID4KPiA+ID4gV291bGQg eW91IHBsZWFzZSBnaXZlIGFuIGV4YW1wbGUgb3IgdHdvIGhvdyB0aGVzZSB0d28gZmxhZ3MKPiA+ IERFU0NfRFJJVkVSIGFuZCBERVNDX1dSQVAgYXJlIHVzZWQgdG9nZXRoZXI/IExpa2Ugb3RoZXJz LCBJIGFtCj4gPiBjb25mdXNlZCBieSB0aGUgZGVzY3JpcHRpb24gYW5kIHN0aWxsIGRvbuKAmXQg cXVpdGUgZ3JvayBpdC4KPiA+ID4KPiA+ID4gU3RldmVuCj4gPiAKPiA+IE15IGJhZCwgSSB3aWxs IG5lZWQgdG8gd29yayBvbiBpdC4gSGVyZSBpcyBhbiBleGFtcGxlOgo+ID4gCj4gPiBMZXQncyBh c3N1bWUgZGV2aWNlIHByb21pc2VkIHRvIGNvbnN1bWUgcGFja2V0cyBpbiBvcmRlcgo+ID4gCj4g PiByaW5nIHNpemUgPSAyCj4gPiAKPiA+IFJpbmcgaXMgMCBpbml0aWFsaXplZC4KPiA+IAo+ID4g RGV2aWNlIGluaXRpYWxseSBwb2xscyBERVNDWzBdLmZsYWdzIGZvciBXUkFQIGJpdCB0byBjaGFu Z2UuCj4gPiAKPiA+IGRyaXZlciBhZGRzOgo+ID4gCj4gPiBERVNDWzBdLmFkZHIgPSAxMjM0Cj4g PiBERVNDWzBdLmlkID0gMAo+ID4gREVTQ1swXS5mbGFncyA9IERFU0NfRFJJVkVSIHwgREVTQ19O RVhUIHwgREVTQ19XUkFQCj4gPiAKPiA+IGFuZAo+ID4gCj4gPiBERVNDWzBdLmFkZHIgPSA1Njc4 Cj4gPiBERVNDWzFdLmlkID0gMQo+ID4gREVTQ1sxXS5mbGFncyA9IERFU0NfRFJJVkVSIHwgREVT Q19XUkFQCj4gPiAKPiA+IAo+ID4gaXQgbm93IHN0YXJ0cyBwb2xsaW5nIERFU0NbMF0gZmxhZ3Mu Cj4gPiAKPiA+IAo+ID4gRGV2aWNlIHJlYWRzIDEyMzQsIGV4ZWN1dGVzIGl0LCBkb2VzIG5vdCB1 c2UgaXQuCj4gPiAKPiA+IERldmljZSByZWFkcyA1Njc4LCBleGVjdXRlcyBpdCwgYW5kIHVzZXMg aXQ6Cj4gPiAKPiA+IERFU0NbMF0uaWQgPSAxCj4gPiBERVNDWzBdLmZsYWdzID0gMAo+ID4gCj4g PiBEZXZpY2Ugbm93IHBvbGxzIERFU0NbMF0uZmxhZ3MgZm9yIFdSQVAgYml0IHRvIGNoYW5nZS4K PiA+IAo+ID4gTm93IGRyaXZlciBzZWVzIHRoYXQgRFJJVkVSIGJpdCBoYXMgYmVlbiBjbGVhcmVk LCBzbyBpdCBub3dzIHRoYXQgaWQgaXMgdmFsaWQuIEkKPiA+IHNlZXMgaWQgMSwgdGhlcmVmb3Jl IGlkIDAgYW5kIDEgaGFzIGJlZW4gcmVhZCBhbmQgYXJlIHNhZmUgdG8gb3ZlcndyaXRlLgo+ID4g Cj4gPiBTbyBpdCB3cml0ZXMgaXQgb3V0LiBJdCB3cmFwcGVkIGFyb3VuZCB0byBiZWdpbm5pbmcg b2YgcmluZywgc28gaXQgZmxpcHMgdGhlCj4gPiBXUkFQIGJpdCB0byAwIG9uIGFsbCBkZXNjcmlw dG9ycyBub3c6Cj4gPiAKPiA+IERFU0NbMF0uYWRkciA9IDlBQkMKPiA+IERFU0NbMF0uaWQgPSAw Cj4gPiBERVNDWzBdLmZsYWdzID0gREVTQ19EUklWRVIgfCBERVNDX05FWFQKPiA+IAo+ID4gCj4g PiBERVNDWzBdLmFkZHIgPSBERUYwCj4gPiBERVNDWzBdLmlkID0gMQo+ID4gREVTQ1swXS5mbGFn cyA9IERFU0NfRFJJVkVSCj4gPiAKPiA+IAo+ID4gTmV4dCByb3VuZCB3cmFwIHdpbGwgYmUgMSBh Z2Fpbi4KPiA+IAo+ID4gCj4gPiBUbyBzdW1tYXJpc2U6Cj4gPiAKPiA+IERSSVZFUiBiaXQgaXMg dXNlZCBieSBkcml2ZXIgdG8gZGV0ZWN0IGRldmljZSBoYXMgdXNlZCBvbmUgb3IgbW9yZQo+ID4g ZGVzY3JpcHRvcnMuICBXUkFQIGlzIGlzIHVzZWQgYnkgZGV2aWNlIHRvIGRldGVjdCBkcml2ZXIg aGFzIG1hZGUgYSBuZXcKPiA+IGRlc2NyaXB0b3IgYXZhaWxhYmxlLgo+ID4gCj4gPiAKPiA+IC0t Cj4gPiBNU1QKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18K VmlydHVhbGl6YXRpb24gbWFpbGluZyBsaXN0ClZpcnR1YWxpemF0aW9uQGxpc3RzLmxpbnV4LWZv dW5kYXRpb24ub3JnCmh0dHBzOi8vbGlzdHMubGludXhmb3VuZGF0aW9uLm9yZy9tYWlsbWFuL2xp c3RpbmZvL3ZpcnR1YWxpemF0aW9u From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: virtio-dev-return-2589-cohuck=redhat.com@lists.oasis-open.org Sender: List-Post: List-Help: List-Unsubscribe: List-Subscribe: Received: from lists.oasis-open.org (oasis-open.org [66.179.20.138]) by lists.oasis-open.org (Postfix) with ESMTP id AB1D55818C66 for ; Sat, 30 Sep 2017 21:08:32 -0700 (PDT) Date: Sun, 1 Oct 2017 07:08:29 +0300 From: "Michael S. Tsirkin" Message-ID: <20171001063817-mutt-send-email-mst@kernel.org> References: <20160915223915.qjlnlvf2w7u37bu3@redhat.com> <20170926011826-mutt-send-email-mst@kernel.org> <7A0DC0C9-F148-4161-B2D1-8D8D14D8B9A1@cisco.com> <20170928023112-mutt-send-email-mst@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Subject: Re: [virtio-dev] packed ring layout proposal v3 To: "Liang, Cunming" Cc: "Steven Luong (sluong)" , "virtio-dev@lists.oasis-open.org" , "virtualization@lists.linux-foundation.org" List-ID: On Thu, Sep 28, 2017 at 09:44:35AM +0000, Liang, Cunming wrote: > > Get it now. Please correct me if I missing something. > > > Flags status hints, > > - DESC_DRIVER only: driver owns the descriptor w/o available info ready for device to use > > - DESC_DRIVER | DESC_WRAP: driver has prepared an available descriptor, device hasn't used it yet > > - None: device has used the descriptor, and write descriptor out > > - DESC_WRAP only: shall not happen, device make sure to clear it > > > Polling behavior is, > > - Device monitor DESC_WRAP bit set or not; If set, go to use descriptor and clear DESC_DRIVER bit in the end (note: always need to clear DESC_WRAP) > > - Driver monitor DESC_DRIVER bit cleared or not; If cleared, reclaim descriptor(set DESC_DRIVER) and set DESC_WRAP once new available descriptor get ready to go > > > -- > Steve Hmm no, not what I had in mind. DESC_DRIVER: used by driver to poll. Driver sets it when writing a descriptor. Device clears it when overwriting a descriptor. Thus driver uses DESC_DRIVER to detect that device data in descriptor is valid. DESC_WRAP: used by device to poll. Driver sets it to a *different* value every time it overwrites a descriptor. How to achieve it? since descriptors are written out in ring order, simply maintain the current value internally (start value 1) and flip it every time you overwrite the first descriptor. Device leaves it intact when overwriting a descriptor. After writing down this explanation, I think the names aren't great. Let me try an alternative explanation. --------------- A two-bit field, DRIVER_OWNER, signals the buffer ownership. It has 4 possible values: values 0x1, 0x11 are written by driver values 0x0, 0x10 are written by device each time driver writes out a descriptor, it must make sure that the high bit in OWNER changes. each time device writes out a descriptor, it must make sure that the high bit in OWNER does not change. this is exactly the same functionally, DRIVER is high bit and WRAP is the low bit. Does this make things clearer? --------------- Maybe the difference between device and driver is confusing. We can fix that by changing the values. Here is an alternative. Let me know if you like it better - I need to think a bit more to make sure it works, but to give you an idea: --------------- A two-bit field, DRIVER_OWNER, signals the buffer ownership. It has 4 possible values: values 0x1, 0x10 are written by driver values 0x0, 0x11 are written by device each time driver writes out a descriptor, it must make sure that the high bit in OWNER changes. Thus first time it writes 0x10, next time 0x1, then 0x10 again. each time device writes out a descriptor, it must make sure that the low bit in OWNER changes. Thus first time it writes 0x11, next time 0x0, then 0x11 again. --------------- > > -----Original Message----- > > From: Michael S. Tsirkin [mailto:mst@redhat.com] > > Sent: Thursday, September 28, 2017 7:49 AM > > To: Steven Luong (sluong) > > Cc: Liang, Cunming; virtio-dev@lists.oasis-open.org; > > virtualization@lists.linux-foundation.org > > Subject: Re: [virtio-dev] packed ring layout proposal v3 > > > > On Tue, Sep 26, 2017 at 11:38:18PM +0000, Steven Luong (sluong) wrote: > > > Michael, > > > > > > Would you please give an example or two how these two flags > > DESC_DRIVER and DESC_WRAP are used together? Like others, I am > > confused by the description and still don’t quite grok it. > > > > > > Steven > > > > My bad, I will need to work on it. Here is an example: > > > > Let's assume device promised to consume packets in order > > > > ring size = 2 > > > > Ring is 0 initialized. > > > > Device initially polls DESC[0].flags for WRAP bit to change. > > > > driver adds: > > > > DESC[0].addr = 1234 > > DESC[0].id = 0 > > DESC[0].flags = DESC_DRIVER | DESC_NEXT | DESC_WRAP > > > > and > > > > DESC[0].addr = 5678 > > DESC[1].id = 1 > > DESC[1].flags = DESC_DRIVER | DESC_WRAP > > > > > > it now starts polling DESC[0] flags. > > > > > > Device reads 1234, executes it, does not use it. > > > > Device reads 5678, executes it, and uses it: > > > > DESC[0].id = 1 > > DESC[0].flags = 0 > > > > Device now polls DESC[0].flags for WRAP bit to change. > > > > Now driver sees that DRIVER bit has been cleared, so it nows that id is valid. I > > sees id 1, therefore id 0 and 1 has been read and are safe to overwrite. > > > > So it writes it out. It wrapped around to beginning of ring, so it flips the > > WRAP bit to 0 on all descriptors now: > > > > DESC[0].addr = 9ABC > > DESC[0].id = 0 > > DESC[0].flags = DESC_DRIVER | DESC_NEXT > > > > > > DESC[0].addr = DEF0 > > DESC[0].id = 1 > > DESC[0].flags = DESC_DRIVER > > > > > > Next round wrap will be 1 again. > > > > > > To summarise: > > > > DRIVER bit is used by driver to detect device has used one or more > > descriptors. WRAP is is used by device to detect driver has made a new > > descriptor available. > > > > > > -- > > MST --------------------------------------------------------------------- To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org