From mboxrd@z Thu Jan 1 00:00:00 1970 From: Siwei Liu Subject: Re: [virtio-dev] Re: [Qemu-devel] [PATCH] qemu: Introduce VIRTIO_NET_F_STANDBY feature bit to virtio_net Date: Thu, 14 Jun 2018 18:57:11 -0700 Message-ID: References: <1525734594-11134-1-git-send-email-sridhar.samudrala@intel.com> <20180611202207-mutt-send-email-mst@kernel.org> <20180612051432-mutt-send-email-mst@kernel.org> <23fc4aa4-ec41-d6e2-3354-10cbfc13b7ec@intel.com> <20180612142557-mutt-send-email-mst@kernel.org> <20180614120231.0a72bd5f.cohuck@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Cc: Alexander Duyck , virtio-dev@lists.oasis-open.org, Jiri Pirko , "Michael S. Tsirkin" , Jakub Kicinski , "Samudrala, Sridhar" , qemu-devel@nongnu.org, virtualization@lists.linux-foundation.org, Netdev , aaron.f.brown@intel.com To: Cornelia Huck Return-path: In-Reply-To: <20180614120231.0a72bd5f.cohuck@redhat.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: virtualization-bounces@lists.linux-foundation.org Errors-To: virtualization-bounces@lists.linux-foundation.org List-Id: netdev.vger.kernel.org VGhhbmsgeW91IGZvciBzaGFyaW5nIHlvdXIgdGhvdWdodHMsIENvcm5lbGlhLiBXaXRoIHF1ZXN0 aW9ucyBiZWxvdywgSQp0aGluayB5b3UgcmFpc2VkIHJlYWxseSBnb29kIHBvaW50cywgc29tZSBv ZiB3aGljaCBJIGRvbid0IGhhdmUgYW5zd2VyCnlldCBhbmQgd291bGQgYWxzbyBsaWtlIHRvIGV4 cGxvcmUgaGVyZS4KCkZpcnN0IG9mZiwgSSBkb24ndCB3YW50IHRvIHB1c2ggdGhlIGRpc2N1c3Np b24gdG8gdGhlIGV4dHJlbWUgYXQgdGhpcwpwb2ludCwgb3Igc2VsbCBhbnl0aGluZyBhYm91dCBo YXZpbmcgUUVNVSBtYW5hZ2UgZXZlcnl0aGluZwphdXRvbWF0aWNhbGx5LiBEb24ndCBnZXQgbWUg d3JvbmcsIGl0J3Mgbm90IHRoZXJlIHlldC4gTGV0J3MgZG9uJ3QKYXNzdW1lIHdlIGFyZSB0aWVk IHRvIGEgc3BlY2lmaWMgb3IgY29uY2VydGUgc29sdXRpb24uIEkgdGhpbmsgdGhlIGtleQpmb3Ig b3VyIGRpc2N1c3Npb24gbWlnaHQgYmUgdG8gZGVmaW5lIG9yIHJlZmluZSB0aGUgYm91bmRhcnkg YmV0d2VlbgpWTSBhbmQgZ3Vlc3QsICBlLmcuIHdoYXQgZWFjaCBsYXllciBpcyBleHBlY3RlZCB0 byBjb250cm9sIGFuZCBtYW5hZ2UKZXhhY3RseS4KCkluIG15IHZpZXcsIHRoZXJlIG1pZ2h0IGJl IHBvc3NpYmx5IDMgZGlmZmVyZW50IG9wdGlvbnMgdG8gcmVwcmVzZW50CnRoZSBmYWlsb3ZlciBk ZXZpY2UgY29uY2VpcHQgdG8gUUVNVSBhbmQgbGlidmlydCAob3IgYW55IHVwcGVyIGxheWVyCnNv ZnR3YXJlKToKCmEuIFNlcGVyYXRlIGRldmljZTogaW4gdGhpcyBtb2RlbCwgdmlydGlvIGFuZCBw YXNzdGhvdWdoIHJlbWFpbnMKc2VwYXJhdGUgZGV2aWNlcyBqdXN0IGFzIHRvZGF5LiBRRU1VIGV4 cG9zZXMgdGhlIHN0YW5kYnkgZmVhdHVyZSBiaXQKZm9yIHZpcnRpbywgYW5kIHB1Ymxpc2ggc3Rh dHVzL2V2ZW50IGFyb3VuZCB0aGUgbmVnb3RpYXRpb24gcHJvY2VzcyBvZgp0aGlzIGZlYXR1cmUg Yml0IGZvciBsaWJ2aXJ0IHRvIHJlYWN0IHVwb24uIFNpbmNlIExpYnZpcnQgaGFzIHRoZQpwYWly aW5nIHJlbGF0aW9uc2hpcCBpdHNlbGYsIG1heWJlIHRocm91Z2ggTUFDIGFkZHJlc3Mgb3Igc29t ZXRoaW5nCmVsc2UsIGl0IGNhbiBjb250cm9sIHRoZSBwcmVzZW5jZSBvZiBwcmltYXJ5IGJ5IGhv dCBwbHVnZ2luZyBvcgp1bnBsdWdnaW5nIHRoZSBwYXNzdGhyb3VnaCBkZXZpY2UsIGFsdGhvdWdo IGl0IGhhcyB0byB3b3JrIHRpZ2h0bHkKd2l0aCB2aXJ0aW8ncyBmZWF0dXJlIG5lZ290YXRpb24g cHJvY2Vzcy4gTm90IGp1c3QgZm9yIG1pZ3JhdGlvbiBidXQKYWxzbyB2YXJpb3VzIGNvcm5lciBz Y2VuYXJpb3MgKGRyaXZlci9mZWF0dXJlIG9rLCBkZXZpY2UgcmVzZXQsCnJlYm9vdCwgbGVnYWN5 IGd1ZXN0IGV0YykgYWxvbmcgdmlydGlvJ3MgZmVhdHVyZSBuZWdvdGlhdGlvbi4KCmIuIENvdXBs ZWQgZGV2aWNlOiBpbiB0aGlzIG1vZGVsLCB2aXJ0aW8gYW5kIHBhc3N0aG91Z2ggZGV2aWNlcyBh cmUKd2Vha2x5IGNvdXBsZWQgdXNpbmcgc29tZSBncm91cCBJRCwgaS5lLiBRRU1VIG1hdGNoIHRo ZSBwYXNzdGhvdWdoCmRldmljZSBmb3IgYSBzdGFuZGJ5IHZpcnRpbyBpbnN0YW5jZSBieSBjb21w YXJpbmcgdGhlIGdyb3VwIElEIHZhbHVlCnByZXNlbnQgYmVoaW5kIGVhY2ggZGV2aWNlJ3MgYnJp ZGdlLiBMaWJ2aXJ0IHByb3ZpZGVzIFFFTVUgdGhlIGdyb3VwCklEIGZvciBib3RoIHR5cGUgb2Yg ZGV2aWNlcywgYW5kIG9ubHkgZGVhbHMgd2l0aCBob3QgcGx1ZyBmb3IKbWlncmF0aW9uLCBieSBj aGVja2luZyBzb21lIG1pZ3JhdGlvbiBzdGF0dXMgZXhwb3NlZCAoZS5nLiB0aGUgZmVhdHVyZQpu ZWdvdGlhdGlvbiBzdGF0dXMgb24gdGhlIHZpcnRpbyBkZXZpY2UpIGJ5IFFFTVUuIFFFTVUgbWFu YWdlcyB0aGUKdmlzaWJpbGl0eSBvZiB0aGUgcHJpbWFyeSBpbiBndWVzdCBhbG9uZyB2aXJ0aW8n cyBmZWF0dXJlIG5lZ290aWF0aW9uCnByb2Nlc3MuCgpjLiBGdWxseSBjb21iaW5lZCBkZXZpY2U6 IGluIHRoaXMgbW9kZWwsIHZpcnRpbyBhbmQgcGFzc3Rob3VnaCBkZXZpY2VzCmFyZSB2aWV3ZWQg YXMgYSBzaW5nbGUgVk0gaW50ZXJmYWNlIGFsdG9ndGhlci4gUUVNVSBub3QganVzdCBjb250cm9s cwp0aGUgdmlzaWJpbGl0eSBvZiB0aGUgcHJpbWFyeSBpbiBndWVzdCwgYnV0IGNhbiBhbHNvIG1h bmFnZSB0aGUKZXhwb3N1cmUgb2YgdGhlIHBhc3N0aHJvdWdoIGZvciBtaWdyYXRhYmlsaXR5LiBJ dCBjYW4gYmUgbGlrZSB0aGF0CmxpYnZpcnQgc3VwcGxpZXMgdGhlIGdyb3VwIElEIHRvIFFFTVUu IE9yIGxpYnZpcnQgZG9lcyBub3QgZXZlbiBoYXZlCnRvIHByb3ZpZGUgZ3JvdXAgSUQgZm9yIGdy b3VwaW5nIHRoZSB0d28gZGV2aWNlcywgaWYganVzdCBvbmUgc2luZ2xlCmNvbWJpbmVkIGRldmlj ZSBpcyBleHBvc2VkIGJ5IFFFTVUuIEluIGVpdGhlciBjYXNlLCBRRU1VIG1hbmFnZXMgYWxsCmFz cGVjdCBvZiBzdWNoIGludGVybmFsIGNvbnN0cnVjdCwgaW5jbHVkaW5nIHZpcnRpbyBmZWF0dXJl Cm5lZ290aWF0aW9uLCBwcmVzZW5jZSBvZiB0aGUgcHJpbWFyeSwgYW5kIGxpdmUgbWlncmF0aW9u LgoKSXQgbG9va3MgbGlrZSB0byBtZSB0aGF0LCBpbiB5b3VyIG9waW5pb24sIHlvdSBzZWVtIHRv IHByZWZlciBnbyB3aXRoCihhKS4gV2hpbGUgSSdtIGFjdHVhbGx5IG9rYXkgd2l0aCBlaXRoZXIg KGIpIG9yIChjKS4gRG8gSSB1bmRlcnN0YW5kCnlvdXIgcG9pbnQgY29ycmVjdGx5PwoKVGhlIHJl YXNvbiB0aGF0IEkgZmVlbCB0aGF0IChhKSBtaWdodCBub3QgYmUgaWRlYWwsIGp1c3QgYXMgTWlj aGFlbAphbGx1ZGVkIHRvIChxdW90aW5nIGJlbG93KSwgaXMgdGhhdCBhcyBtYW5hZ2VtZW50IHN0 YWNrLCBpdCByZWFsbHkKZG9lc24ndCBuZWVkIHRvIGNhcmUgYWJvdXQgdGhlIGRldGFpbGVkIHBy b2Nlc3Mgb2YgZmVhdHVyZSBuZWdvdGlhdGlvbgooaWYgd2UgdmlldyB0aGUgZ3Vlc3QgcHJlc2Vu Y2Ugb2YgdGhlIHByaW1hcnkgYXMgcGFydCBvZiBmZWF0dXJlCm5lZ290aWF0aW9uIGF0IGFuIGV4 dGVuZGVkIGxldmVsIG5vdCBqdXN0IHZpcnRpbykuIEFsbCBpdCBuZWVkcyB0byBiZQpkb25lIGlz IHRvIGhhbmQgaW4gdGhlIHJlcXVpcmVkIGRldmljZXMgdG8gUUVNVSBhbmQgdGhhdCdzIGFsbC4g V2h5IGRvCndlIG5lZWQgdG8gYWRkZCB2YXJpb3VzIGhvb2tzLCBldmVudHMgZm9yIHdoaWNoZXZl ciBoYXBwZW5zIGludGVybmFsbHkKd2l0aGluIHRoZSBndWVzdD8KCicnClByaW1hcnkgZGV2aWNl IGlzIGFkZGVkIHdpdGggYSBzcGVjaWFsICJwcmltYXJ5LWZhaWxvdmVyIiBmbGFnLgpBIHZpcnR1 YWwgbWFjaGluZSBpcyB0aGVuIGluaXRpYWxpemVkIHdpdGgganVzdCBhIHN0YW5kYnkgdmlydGlv CmRldmljZS4gUHJpbWFyeSBpcyBub3QgeWV0IGFkZGVkLgoKTGF0ZXIgUUVNVSBkZXRlY3RzIHRo YXQgZ3Vlc3QgZHJpdmVyIGRldmljZSBzZXQgRFJJVkVSX09LLgpJdCB0aGVuIGV4cG9zZXMgdGhl IHByaW1hcnkgZGV2aWNlIHRvIHRoZSBndWVzdCwgYW5kIHRyaWdnZXJzCmEgZGV2aWNlIGFkZGl0 aW9uIGV2ZW50IChob3QtcGx1ZyBldmVudCkgZm9yIGl0LgoKSWYgUUVNVSBkZXRlY3RzIGd1ZXN0 IGRyaXZlciByZW1vdmFsLCBpdCBpbml0aWF0ZXMgYSBob3QtdW5wbHVnIHNlcXVlbmNlCnRvIHJl bW92ZSB0aGUgcHJpbWFyeSBkcml2ZXIuICBJbiBwYXJ0aWN1bGFyLCBpZiBRRU1VIGRldGVjdHMg Z3Vlc3QKcmUtaW5pdGlhbGl6YXRpb24gKGUuZy4gYnkgZGV0ZWN0aW5nIGd1ZXN0IHJlc2V0KSBp dCBpbW1lZGlhdGVseSByZW1vdmVzCnRoZSBwcmltYXJ5IGRldmljZS4KJycKCmFuZCwKCicnCm1h bmFnZW1lbnQganVzdCB3YW50cyB0byBnaXZlIHRoZSBwcmltYXJ5IHRvIGd1ZXN0IGFuZCBsYXRl ciB0YWtlIGl0IGJhY2ssCml0IHJlYWxseSBkb2VzIG5vdCBjYXJlIGFib3V0IHRoZSBkZXRhaWxz IG9mIHRoZSBwcm9jZXNzLApzbyBJIGRvbid0IHNlZSB3aGF0IGRvZXMgcHVzaGluZyBpdCB1cCB0 aGUgc3RhY2sgYnV5IHlvdS4KClNvIEkgZG9uJ3QgdGhpbmsgaXQgKm5lZWRzKiB0byBiZSBkb25l IGluIGxpYnZpcnQuIEl0IHByb2JhYmx5IGNhbiBpZiB5b3UKYWRkIGEgYnVuY2ggb2YgaG9va3Mg c28gaXQga25vd3Mgd2hlbmV2ZXIgdm0gcmVib290cywgZHJpdmVyIGJpbmRzIGFuZAp1bmJpbmRz IGZyb20gZGV2aWNlLCBhbmQgY2FuIGNoZWNrIHRoYXQgYmFja3VwIGZsYWcgd2FzIHNldC4KSWYg eW91IGFyZSBwdXNoaW5nIGZvciBhIHNldHVwIGxpa2UgdGhhdCBwbGVhc2UgZ2V0IGEgYnV5LWlu CmZyb20gbGlidmlydCBtYWludGFpbmVycyBvciBiZXR0ZXIgd3JpdGUgYSBwYXRjaC4KJycKCkxl dCBtZSBrbm93IGlmIG15IGNsYXJpZmljYXRpb25zIHNvdW5kIGNsZWFyIHRvIHlvdSBub3cuCgoK VGhhbmtzLAotU2l3ZWkKCgpPbiBUaHUsIEp1biAxNCwgMjAxOCBhdCAzOjAyIEFNLCBDb3JuZWxp YSBIdWNrIDxjb2h1Y2tAcmVkaGF0LmNvbT4gd3JvdGU6Cj4gSSd2ZSBiZWVuIHBvaW50ZWQgdG8g dGhpcyBkaXNjdXNzaW9uICh3aGljaCBJIGhhZCBtaXNzZWQgcHJldmlvdXNseSkKPiBhbmQgSSdt IGdldHRpbmcgYSBoZWFkYWNoZS4gTGV0IG1lIGZpcnN0IHN1bW1hcml6ZSBob3cgSSB1bmRlcnN0 YW5kIGhvdwo+IHRoaXMgZmVhdHVyZSBpcyBzdXBwb3NlZCB0byB3b3JrLCB0aGVuIEknbGwgcmVz cG9uZCB0byBzb21lIGluZGl2aWR1YWwKPiBwb2ludHMuCj4KPiBUaGUgYmFzaWMgaWRlYSBpcyB0 byBlbmFibGUgZ3Vlc3RzIHRvIG1pZ3JhdGUgc2VhbWxlc3NseSwgd2hpbGUgc3RpbGwKPiBtYWtp bmcgaXQgcG9zc2libGUgZm9yIHRoZW0gdG8gdXNlIGEgcGFzc2VkLXRocm91Z2ggZGV2aWNlIGZv ciBtb3JlCj4gcGVyZm9ybWFuY2UgZXRjLiBUaGUgbWVhbnMgdG8gZG8gc28gaXMgdG8gaG9vayBh IHZpcnRpby1uZXQgZGV2aWNlCj4gdG9nZXRoZXIgd2l0aCBhIG5ldHdvcmsgZGV2aWNlIHBhc3Nl ZCB0aHJvdWdoIHZpYSB2ZmlvLiBUaGUKPiB2ZmlvLWhhbmRsZWQgZGV2aWNlIGlzIHRoZXJlIGZv ciBwZXJmb3JtYW5jZSwgdGhlIHZpcnRpbyBkZXZpY2UgZm9yCj4gbWlncmF0YWJpbGl0eS4gV2Ug aGF2ZSBhIG5ldyB2aXJ0aW8gZmVhdHVyZSBiaXQgZm9yIHRoYXQgd2hpY2ggbmVlZHMgdG8KPiBi ZSBuZWdvdGlhdGVkIGZvciB0aGF0ICdjb21iaW5lZCcgZGV2aWNlIHRvIGJlIGF2YWlsYWJsZS4g V2UgaGF2ZSB0bwo+IGNvbnNpZGVyIHR3byBjYXNlczoKPgo+IC0gT2xkZXIgZ3Vlc3RzIHRoYXQg ZG8gbm90IHN1cHBvcnQgdGhlIG5ldyBmZWF0dXJlIGJpdC4gV2UgcHJlc3VtZSB0aGF0Cj4gICB0 aG9zZSBndWVzdHMgd2lsbCBiZSBjb25mdXNlZCBpZiB0aGV5IGdldCB0d28gbmV0d29yayBkZXZp Y2VzIHdpdGgKPiAgIHRoZSBzYW1lIE1BQywgc28gdGhlIGlkZWEgaXMgdG8gbm90IHNob3cgdGhl bSB0aGUgdmZpby1oYW5kbGVkIGRldmljZQo+ICAgYXQgYWxsLgo+IC0gR3Vlc3RzIHRoYXQgbmVn b3RpYXRlIHRoZSBmZWF0dXJlIGJpdC4gV2Ugb25seSBrbm93IHBvc2l0aXZlbHkgdGhhdAo+ICAg dGhleSAoYSkga25vdyB0aGUgZmVhdHVyZSBiaXQgYW5kIChiKSBhcmUgcHJlcGFyZWQgdG8gaGFu ZGxlIHRoZQo+ICAgY29uc2VxdWVuY2VzIG9mIG5lZ290aWF0aW5nIGl0IGFmdGVyIHRoZXkgc2V0 IHRoZSBGRUFUVVJFU19PSyBiaXQuCj4gICBUaGlzIGlzIHRoZXJlZm9yZSB0aGUgZWFybGllc3Qg cG9pbnQgaW4gdGltZSB0aGF0IHRoZSB2ZmlvLWhhbmRsZWQKPiAgIGRldmljZSBzaG91bGQgYmUg dmlzaWJsZSBvciB1c2FibGUgZm9yIHRoZSBndWVzdC4KPgo+IE9uIFdlZCwgMTMgSnVuIDIwMTgg MTg6MDI6MDEgLTA3MDAKPiBTaXdlaSBMaXUgPGxvc2V3ZWlnaEBnbWFpbC5jb20+IHdyb3RlOgo+ Cj4+IE9uIFR1ZSwgSnVuIDEyLCAyMDE4IGF0IDU6MDggUE0sIFNhbXVkcmFsYSwgU3JpZGhhcgo+ PiA8c3JpZGhhci5zYW11ZHJhbGFAaW50ZWwuY29tPiB3cm90ZToKPj4gPiBPbiA2LzEyLzIwMTgg NDozNCBBTSwgTWljaGFlbCBTLiBUc2lya2luIHdyb3RlOgo+PiA+Pgo+PiA+PiBPbiBNb24sIEp1 biAxMSwgMjAxOCBhdCAxMDowMjo0NVBNIC0wNzAwLCBTYW11ZHJhbGEsIFNyaWRoYXIgd3JvdGU6 Cj4+ID4+Pgo+PiA+Pj4gT24gNi8xMS8yMDE4IDc6MTcgUE0sIE1pY2hhZWwgUy4gVHNpcmtpbiB3 cm90ZToKPj4gPj4+Pgo+PiA+Pj4+IE9uIFR1ZSwgSnVuIDEyLCAyMDE4IGF0IDA5OjU0OjQ0QU0g KzA4MDAsIEphc29uIFdhbmcgd3JvdGU6Cj4+ID4+Pj4+Cj4+ID4+Pj4+IE9uIDIwMTjlubQwNuac iDEy5pelIDAxOjI2LCBNaWNoYWVsIFMuIFRzaXJraW4gd3JvdGU6Cj4+ID4+Pj4+Pgo+PiA+Pj4+ Pj4gT24gTW9uLCBNYXkgMDcsIDIwMTggYXQgMDQ6MDk6NTRQTSAtMDcwMCwgU3JpZGhhciBTYW11 ZHJhbGEgd3JvdGU6Cj4+ID4+Pj4+Pj4KPj4gPj4+Pj4+PiBUaGlzIGZlYXR1cmUgYml0IGNhbiBi ZSB1c2VkIGJ5IGh5cGVydmlzb3IgdG8gaW5kaWNhdGUgdmlydGlvX25ldAo+PiA+Pj4+Pj4+IGRl dmljZSB0bwo+PiA+Pj4+Pj4+IGFjdCBhcyBhIHN0YW5kYnkgZm9yIGFub3RoZXIgZGV2aWNlIHdp dGggdGhlIHNhbWUgTUFDIGFkZHJlc3MuCj4+ID4+Pj4+Pj4KPj4gPj4+Pj4+PiBJIHRlc3RlZCB0 aGlzIHdpdGggYSBzbWFsbCBjaGFuZ2UgdG8gdGhlIHBhdGNoIHRvIG1hcmsgdGhlIFNUQU5EQlkK Pj4gPj4+Pj4+PiBmZWF0dXJlICd0cnVlJwo+PiA+Pj4+Pj4+IGJ5IGRlZmF1bHQgYXMgaSBhbSB1 c2luZyBsaWJ2aXJ0IHRvIHN0YXJ0IHRoZSBWTXMuCj4+ID4+Pj4+Pj4gSXMgdGhlcmUgYSB3YXkg dG8gcGFzcyB0aGUgbmV3bHkgYWRkZWQgZmVhdHVyZSBiaXQgJ3N0YW5kYnknIHRvIHFlbXUKPj4g Pj4+Pj4+PiB2aWEgbGlidmlydAo+PiA+Pj4+Pj4+IFhNTCBmaWxlPwo+PiA+Pj4+Pj4+Cj4+ID4+ Pj4+Pj4gU2lnbmVkLW9mZi1ieTogU3JpZGhhciBTYW11ZHJhbGEgPHNyaWRoYXIuc2FtdWRyYWxh QGludGVsLmNvbT4KPj4gPj4+Pj4+Cj4+ID4+Pj4+PiBTbyBJIGRvIG5vdCB0aGluayB3ZSBjYW4g Y29tbWl0IHRvIHRoaXMgaW50ZXJmYWNlOiB3ZQo+PiA+Pj4+Pj4gcmVhbGx5IG5lZWQgdG8gY29u dHJvbCB2aXNpYmlsaXR5IG9mIHRoZSBwcmltYXJ5IGRldmljZS4KPj4gPj4+Pj4KPj4gPj4+Pj4g VGhlIHByb2JsZW0gaXMgbGVnYWN5IGd1ZXN0IHdvbid0IHVzZSBwcmltYXJ5IGRldmljZSBhdCBh bGwgaWYgd2UgZG8KPj4gPj4+Pj4gdGhpcy4KPj4gPj4+Pgo+PiA+Pj4+IEFuZCB0aGF0J3MgYnkg ZGVzaWduIC0gSSB0aGluayBpdCdzIHRoZSBvbmx5IHdheSB0byBlbnN1cmUgdGhlCj4+ID4+Pj4g bGVnYWN5IGd1ZXN0IGlzbid0IGNvbmZ1c2VkLgo+PiA+Pj4KPj4gPj4+IFllcy4gSSB0aGluayBz by4gQnV0IGkgYW0gbm90IHN1cmUgaWYgUWVtdSBpcyB0aGUgcmlnaHQgcGxhY2UgdG8gY29udHJv bAo+PiA+Pj4gdGhlIHZpc2liaWxpdHkKPj4gPj4+IG9mIHRoZSBwcmltYXJ5IGRldmljZS4gVGhl IHByaW1hcnkgZGV2aWNlIG1heSBub3QgYmUgc3BlY2lmaWVkIGFzIGFuCj4+ID4+PiBhcmd1bWVu dCB0byBRZW11LiBJdAo+PiA+Pj4gbWF5IGJlIHBsdWdnZWQgaW4gbGF0ZXIuCj4+ID4+PiBUaGUg Y2xvdWQgc2VydmljZSBwcm92aWRlciBpcyBwcm92aWRpbmcgYSBmZWF0dXJlIHRoYXQgZW5hYmxl cyBsb3cKPj4gPj4+IGxhdGVuY3kgZGF0YXBhdGggYW5kIGxpdmUKPj4gPj4+IG1pZ3JhdGlvbiBj YXBhYmlsaXR5Lgo+PiA+Pj4gQSB0ZW5hbnQgY2FuIHVzZSB0aGlzIGZlYXR1cmUgb25seSBpZiBo ZSBpcyBydW5uaW5nIGEgVk0gdGhhdCBoYXMKPj4gPj4+IHZpcnRpby1uZXQgd2l0aCBmYWlsb3Zl ciBzdXBwb3J0Lgo+Cj4gU28sIGRvIHlvdSBrbm93IGZyb20gdGhlIG91dHNldCB0aGF0IHRoZXJl IHdpbGwgYmUgc3VjaCBhIGNvdXBsZWQKPiBkZXZpY2U/IEkuZS4sIGlzIGl0IGEgcHJvcGVydHkg b2YgdGhlIFZNIGRlZmluaXRpb24/Cj4KPiBDYW4gdGhlcmUgYmUgYSAncHJlcGFyZWQnIHZpcnRp by1uZXQgZGV2aWNlIHRoYXQgcHJlc2VudHMgdGhlIFNUQU5EQlkKPiBmZWF0dXJlIGV2ZW4gaWYg dGhlcmUgY3VycmVudGx5IGlzIG5vIHZmaW8taGFuZGxlZCBkZXZpY2UgYXZhaWxhYmxlIC0tCj4g YnV0IG1ha2luZyBpdCBwb3NzaWJsZSB0byBzaW1wbHkgaG90cGx1ZyB0aGF0IGRldmljZSBsYXRl cj8KPgo+IFNob3VsZCBpdCBiZSBwb3NzaWJsZSB0byBhZGQgYSB2aXJ0aW8vdmZpbyBwYWlyIGxh dGVyIG9uPwo+Cj4+ID4+Cj4+ID4+IFdlbGwgbGl2ZSBtaWdyYXRpb24gaXMgdGhlcmUgYWxyZWFk eS4gVGhlIG5ldyBmZWF0dXJlIGlzIGxvdyBsYXRlbmN5Cj4+ID4+IGRhdGEgcGF0aC4KPj4gPgo+ PiA+Cj4+ID4gd2UgZ2V0IGxpdmUgbWlncmF0aW9uIHdpdGgganVzdCB2aXJ0aW8uICBCdXQgSSBt ZWFudCBsaXZlIG1pZ3JhdGlvbiB3aXRoIFZGCj4+ID4gYXMKPj4gPiBwcmltYXJ5IGRldmljZS4K Pj4gPgo+PiA+Pgo+PiA+PiBBbmQgaXQncyB0aGUgZ3Vlc3QgdGhhdCBuZWVkcyBmYWlsb3ZlciBz dXBwb3J0IG5vdCB0aGUgVk0uCj4+ID4KPj4gPgo+PiA+IElzbid0IGd1ZXN0IGFuZCBWTSBzeW5v bnltb3VzPwo+Cj4gSSB0aGluayB3ZSBuZWVkIHRvIGJlIHJlYWxseSBjYXJlZnVsIHRvIG5vdCBt aXggdXAgdGhlIHR3bzogVGhlIFZNCj4gY29udGFpbnMgdGhlIGRlZmluaXRpb25zLCBidXQgaXQg aXMgdXAgdG8gdGhlIGd1ZXN0IGhvdyBpdCB1c2VzIHRoZW0uCj4KPj4gPgo+PiA+Cj4+ID4+Cj4+ ID4+Cj4+ID4+PiBJIHRoaW5rIFFlbXUgc2hvdWxkIGNoZWNrIGlmIGd1ZXN0IHZpcnRpby1uZXQg c3VwcG9ydHMgdGhpcyBmZWF0dXJlIGFuZAo+PiA+Pj4gcHJvdmlkZSBhIG1lY2hhbmlzbSBmb3IK Pj4gPj4+IGFuIHVwcGVyIGxheWVyIGluZGljYXRpbmcgaWYgdGhlIFNUQU5EQlkgZmVhdHVyZSBp cyBzdWNjZXNzZnVsbHkKPj4gPj4+IG5lZ290aWF0ZWQgb3Igbm90Lgo+PiA+Pj4gVGhlIHVwcGVy IGxheWVyIGNhbiB0aGVuIGRlY2lkZSBpZiBpdCBzaG91bGQgaG90IHBsdWcgYSBWRiB3aXRoIHRo ZSBzYW1lCj4+ID4+PiBNQUMgYW5kIG1hbmFnZSB0aGUgMiBsaW5rcy4KPj4gPj4+IElmIFZGIGlz IHN1Y2Nlc3NmdWxseSBob3QgcGx1Z2dlZCwgdmlydGlvLW5ldCBsaW5rIHNob3VsZCBiZSBkaXNh YmxlZC4KPj4gPj4KPj4gPj4gRGlkIHlvdSBldmVuIHRhbGsgdG8gdXBwZXIgbGF5ZXIgbWFuYWdl bWVudCBhYm91dCBpdD8KPj4gPj4gSnVzdCBsaXN0IHRoZSBzdGVwcyB0aGV5IG5lZWQgdG8gZG8g YW5kIHlvdSB3aWxsIHNlZQo+PiA+PiB0aGF0J3MgYSBsb3Qgb2YgbWFjaGluZXJ5IHRvIG1hbmFn ZSBieSB0aGUgdXBwZXIgbGF5ZXIuCj4+ID4+Cj4+ID4+IFdoYXQgZG8gd2UgZ2FpbiBpbiBmbGV4 aWJpbGl0eT8gQXMgZmFyIGFzIEkgY2FuIHNlZSB0aGUKPj4gPj4gb25seSBnYWluIGlzIHNvbWUg cmVzb3VyY2VzIHNhdmVkIGZvciBsZWdhY3kgVk1zLgo+PiA+Pgo+PiA+PiBUaGF0J3Mgbm90IGEg bG90IGFzIHRlbmFudCBvZiB0aGUgdXBwZXIgbGF5ZXIgcHJvYmFibHkgYWxyZWFkeSBoYXMKPj4g Pj4gYXQgbGVhc3QgYSBodW5jaCB0aGF0IGl0J3MgYSBuZXcgZ3Vlc3Qgb3RoZXJ3aXNlCj4+ID4+ IHdoeSBib3RoZXIgc3BlY2lmeWluZyB0aGUgZmVhdHVyZSBhdCBhbGwgLSB5b3UKPj4gPj4gc2F2 ZSBldmVuIG1vcmUgcmVzb3VyY2VzIHdpdGhvdXQgaXQuCj4+ID4+Cj4+ID4KPj4gPiBJIGFtIG5v dCBhbGwgdGhhdCBmYW1pbGlhciB3aXRoIGhvdyBRZW11IG1hbmFnZXMgbmV0d29yayBkZXZpY2Vz LiBJZiB3ZSBjYW4KPj4gPiBkbyBhbGwgdGhlCj4+ID4gcmVxdWlyZWQgbWFuYWdlbWVudCBvZiB0 aGUgcHJpbWFyeS9zdGFuZGJ5IGRldmljZXMgd2l0aGluIFFlbXUsIHRoYXQgaXMKPj4gPiBkZWZp bml0ZWx5IGEgYmV0dGVyCj4+ID4gYXBwcm9hY2ggd2l0aG91dCB1cHBlciBsYXllciBpbnZvbHZl bWVudC4KPj4KPj4gUmlnaHQuIEkgd291bGQgaW1hZ2luZSBpbiB0aGUgZXh0cmVtZSBjYXNlIHRo ZSB1cHBlciBsYXllciBkb2Vzbid0Cj4+IGhhdmUgdG8gYmUgaW52b2x2ZWQgYXQgYWxsIGlmIFFF TVUgbWFuYWdlcyBhbGwgaG90IHBsdWcvdW5wbHVnIGxvZ2ljLgo+PiBUaGUgbWFuYWdlbWVudCB0 b29sIGNhbiBzdXBwbHkgcGFzc3Rocm91Z2ggZGV2aWNlIGFuZCB2aXJ0aW8gd2l0aCB0aGUKPj4g c2FtZSBncm91cCBVVUlELCBRRU1VIGF1dG8tbWFuYWdlcyB0aGUgcHJlc2VuY2Ugb2YgdGhlIHBy aW1hcnksIGFuZAo+PiBob3QgcGx1ZyB0aGUgZGV2aWNlIGFzIG5lZWRlZCBiZWZvcmUgb3IgYWZ0 ZXIgdGhlIG1pZ3JhdGlvbi4KPgo+IEkgZG8gbm90IHJlYWxseSBzZWUgaG93IHlvdSBjYW4gbWFu YWdlIHRoYXQga2luZCBvZiBzdHVmZiBpbiBRRU1VIG9ubHkuCj4gSGF2ZSB5b3UgdGFsa2VkIHRv IHNvbWUgbGlidmlydCBmb2xrcz8gKEFuZCBJJ20gbm90IHN1cmUgd2hhdCB5b3UgcmVmZXIKPiB0 byB3aXRoICdncm91cCBVVUlEJz8pCj4KPiBBbHNvLCBJIHRoaW5rIHlvdSBuZWVkIHRvIG1ha2Ug YSBkaXN0aW5jdGlvbiBiZXR3ZWVuIGhvdHBsdWdnaW5nIGEKPiBkZXZpY2UgYW5kIG1ha2luZyBp dCB2aXNpYmxlIHRvIHRoZSBndWVzdC4gV2hhdCBkb2VzICdob3RwbHVnZ2luZycgbWVhbgo+IGhl cmU/IEFkZGluZyBpdCB0byB0aGUgVk0gZGVmaW5pdGlvbj8gV291bGQgaXQgYmUgZW5vdWdoIHRv IGhhdmUgdGhlCj4gdmZpby1iYXNlZCBkZXZpY2Ugbm90IG9wZXJhdGlvbmFsIHVudGlsIHRoZSB2 aXJ0aW8gZmVhdHVyZSBiaXQgaGFzIGJlZW4KPiBuZWdvdGlhdGVkPwo+Cj4gV2hhdCBoYXBwZW5z IGlmIHRoZSBndWVzdCBkb2VzIG5vdCB1c2UgdGhlIHZmaW8tYmFzZWQgZGV2aWNlIGFmdGVyIGl0 Cj4gaGFzIGJlZW4gbWFkZSBhdmFpbGFibGU/IFdpbGwgeW91IHN0aWxsIGRpc2FibGUgdGhlIHZp cnRpby1uZXQgbGluaz8KPiAoQWxsIHRoYXQgbGluayBoYW5kbGluZyBkZWZpbml0ZWx5IHNvdW5k cyBsaWtlIGEgdGFzayBmb3IgbGlidmlydCBvcgo+IHRoZSBsaWtlLikKPgo+IFJlZ2FyZGluZyBo b3QodW4pcGx1Z2dpbmcgZHVyaW5nIG1pZ3JhdGlvbiwgSSB0aGluayB5b3UgYWxzbyBuZWVkIHRv Cj4ga2VlcCBpbiBtaW5kIHRoYXQgZGlmZmVyZW50IGFyY2hpdGVjdHVyZXMvYnVzc2VzIGhhdmUg ZGlmZmVyZW50Cj4gc2VtYW50aWNzIHRoZXJlLiBTb21ldGhpbmcgdGhhdCB3b3JrcyBpZiB0aGVy ZSdzIGFuIHVucGx1ZyBoYW5kc2hha2UgbWF5Cj4gbm90IHdvcmsgb24gYSBwbGF0Zm9ybSB3aXRo IHN1cnByaXNlIHJlbW92YWwuCj4KPiBIYXZlIHlvdSBjb25zaWRlcmVkIGd1ZXN0IGFnZW50cz8g QWxsIG9mIHRoaXMgaXMgcHVuY2hpbmcgdGhyb3VnaAo+IHNldmVyYWwgbGF5ZXJzLCBhbmQgSSdt IG5vdCBzdXJlIGlmIHRoYXQgaXMgYSBnb29kIGlkZWEuCl9fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fClZpcnR1YWxpemF0aW9uIG1haWxpbmcgbGlzdApWaXJ0 dWFsaXphdGlvbkBsaXN0cy5saW51eC1mb3VuZGF0aW9uLm9yZwpodHRwczovL2xpc3RzLmxpbnV4 Zm91bmRhdGlvbi5vcmcvbWFpbG1hbi9saXN0aW5mby92aXJ0dWFsaXphdGlvbg== From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56529) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fTdzH-0007BF-UM for qemu-devel@nongnu.org; Thu, 14 Jun 2018 21:57:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fTdzF-0005aR-T9 for qemu-devel@nongnu.org; Thu, 14 Jun 2018 21:57:16 -0400 Received: from mail-it0-x244.google.com ([2607:f8b0:4001:c0b::244]:37999) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fTdzF-0005ZY-LO for qemu-devel@nongnu.org; Thu, 14 Jun 2018 21:57:13 -0400 Received: by mail-it0-x244.google.com with SMTP id v83-v6so871428itc.3 for ; Thu, 14 Jun 2018 18:57:13 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <20180614120231.0a72bd5f.cohuck@redhat.com> References: <1525734594-11134-1-git-send-email-sridhar.samudrala@intel.com> <20180611202207-mutt-send-email-mst@kernel.org> <20180612051432-mutt-send-email-mst@kernel.org> <23fc4aa4-ec41-d6e2-3354-10cbfc13b7ec@intel.com> <20180612142557-mutt-send-email-mst@kernel.org> <20180614120231.0a72bd5f.cohuck@redhat.com> From: Siwei Liu Date: Thu, 14 Jun 2018 18:57:11 -0700 Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [virtio-dev] Re: [PATCH] qemu: Introduce VIRTIO_NET_F_STANDBY feature bit to virtio_net List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Cornelia Huck Cc: "Samudrala, Sridhar" , Alexander Duyck , virtio-dev@lists.oasis-open.org, aaron.f.brown@intel.com, Jiri Pirko , "Michael S. Tsirkin" , Jakub Kicinski , Netdev , qemu-devel@nongnu.org, virtualization@lists.linux-foundation.org Thank you for sharing your thoughts, Cornelia. With questions below, I think you raised really good points, some of which I don't have answer yet and would also like to explore here. First off, I don't want to push the discussion to the extreme at this point, or sell anything about having QEMU manage everything automatically. Don't get me wrong, it's not there yet. Let's don't assume we are tied to a specific or concerte solution. I think the key for our discussion might be to define or refine the boundary between VM and guest, e.g. what each layer is expected to control and manage exactly. In my view, there might be possibly 3 different options to represent the failover device conceipt to QEMU and libvirt (or any upper layer software): a. Seperate device: in this model, virtio and passthough remains separate devices just as today. QEMU exposes the standby feature bit for virtio, and publish status/event around the negotiation process of this feature bit for libvirt to react upon. Since Libvirt has the pairing relationship itself, maybe through MAC address or something else, it can control the presence of primary by hot plugging or unplugging the passthrough device, although it has to work tightly with virtio's feature negotation process. Not just for migration but also various corner scenarios (driver/feature ok, device reset, reboot, legacy guest etc) along virtio's feature negotiation. b. Coupled device: in this model, virtio and passthough devices are weakly coupled using some group ID, i.e. QEMU match the passthough device for a standby virtio instance by comparing the group ID value present behind each device's bridge. Libvirt provides QEMU the group ID for both type of devices, and only deals with hot plug for migration, by checking some migration status exposed (e.g. the feature negotiation status on the virtio device) by QEMU. QEMU manages the visibility of the primary in guest along virtio's feature negotiation process. c. Fully combined device: in this model, virtio and passthough devices are viewed as a single VM interface altogther. QEMU not just controls the visibility of the primary in guest, but can also manage the exposure of the passthrough for migratability. It can be like that libvirt supplies the group ID to QEMU. Or libvirt does not even have to provide group ID for grouping the two devices, if just one single combined device is exposed by QEMU. In either case, QEMU manages all aspect of such internal construct, including virtio feature negotiation, presence of the primary, and live migration. It looks like to me that, in your opinion, you seem to prefer go with (a). While I'm actually okay with either (b) or (c). Do I understand your point correctly? The reason that I feel that (a) might not be ideal, just as Michael alluded to (quoting below), is that as management stack, it really doesn't need to care about the detailed process of feature negotiation (if we view the guest presence of the primary as part of feature negotiation at an extended level not just virtio). All it needs to be done is to hand in the required devices to QEMU and that's all. Why do we need to addd various hooks, events for whichever happens internally within the guest? '' Primary device is added with a special "primary-failover" flag. A virtual machine is then initialized with just a standby virtio device. Primary is not yet added. Later QEMU detects that guest driver device set DRIVER_OK. It then exposes the primary device to the guest, and triggers a device addition event (hot-plug event) for it. If QEMU detects guest driver removal, it initiates a hot-unplug sequence to remove the primary driver. In particular, if QEMU detects guest re-initialization (e.g. by detecting guest reset) it immediately removes the primary device. '' and, '' management just wants to give the primary to guest and later take it back, it really does not care about the details of the process, so I don't see what does pushing it up the stack buy you. So I don't think it *needs* to be done in libvirt. It probably can if you add a bunch of hooks so it knows whenever vm reboots, driver binds and unbinds from device, and can check that backup flag was set. If you are pushing for a setup like that please get a buy-in from libvirt maintainers or better write a patch. '' Let me know if my clarifications sound clear to you now. Thanks, -Siwei On Thu, Jun 14, 2018 at 3:02 AM, Cornelia Huck wrote: > I've been pointed to this discussion (which I had missed previously) > and I'm getting a headache. Let me first summarize how I understand how > this feature is supposed to work, then I'll respond to some individual > points. > > The basic idea is to enable guests to migrate seamlessly, while still > making it possible for them to use a passed-through device for more > performance etc. The means to do so is to hook a virtio-net device > together with a network device passed through via vfio. The > vfio-handled device is there for performance, the virtio device for > migratability. We have a new virtio feature bit for that which needs to > be negotiated for that 'combined' device to be available. We have to > consider two cases: > > - Older guests that do not support the new feature bit. We presume that > those guests will be confused if they get two network devices with > the same MAC, so the idea is to not show them the vfio-handled device > at all. > - Guests that negotiate the feature bit. We only know positively that > they (a) know the feature bit and (b) are prepared to handle the > consequences of negotiating it after they set the FEATURES_OK bit. > This is therefore the earliest point in time that the vfio-handled > device should be visible or usable for the guest. > > On Wed, 13 Jun 2018 18:02:01 -0700 > Siwei Liu wrote: > >> On Tue, Jun 12, 2018 at 5:08 PM, Samudrala, Sridhar >> wrote: >> > On 6/12/2018 4:34 AM, Michael S. Tsirkin wrote: >> >> >> >> On Mon, Jun 11, 2018 at 10:02:45PM -0700, Samudrala, Sridhar wrote: >> >>> >> >>> On 6/11/2018 7:17 PM, Michael S. Tsirkin wrote: >> >>>> >> >>>> On Tue, Jun 12, 2018 at 09:54:44AM +0800, Jason Wang wrote: >> >>>>> >> >>>>> On 2018=E5=B9=B406=E6=9C=8812=E6=97=A5 01:26, Michael S. Tsirkin w= rote: >> >>>>>> >> >>>>>> On Mon, May 07, 2018 at 04:09:54PM -0700, Sridhar Samudrala wrote= : >> >>>>>>> >> >>>>>>> This feature bit can be used by hypervisor to indicate virtio_ne= t >> >>>>>>> device to >> >>>>>>> act as a standby for another device with the same MAC address. >> >>>>>>> >> >>>>>>> I tested this with a small change to the patch to mark the STAND= BY >> >>>>>>> feature 'true' >> >>>>>>> by default as i am using libvirt to start the VMs. >> >>>>>>> Is there a way to pass the newly added feature bit 'standby' to = qemu >> >>>>>>> via libvirt >> >>>>>>> XML file? >> >>>>>>> >> >>>>>>> Signed-off-by: Sridhar Samudrala >> >>>>>> >> >>>>>> So I do not think we can commit to this interface: we >> >>>>>> really need to control visibility of the primary device. >> >>>>> >> >>>>> The problem is legacy guest won't use primary device at all if we = do >> >>>>> this. >> >>>> >> >>>> And that's by design - I think it's the only way to ensure the >> >>>> legacy guest isn't confused. >> >>> >> >>> Yes. I think so. But i am not sure if Qemu is the right place to con= trol >> >>> the visibility >> >>> of the primary device. The primary device may not be specified as an >> >>> argument to Qemu. It >> >>> may be plugged in later. >> >>> The cloud service provider is providing a feature that enables low >> >>> latency datapath and live >> >>> migration capability. >> >>> A tenant can use this feature only if he is running a VM that has >> >>> virtio-net with failover support. > > So, do you know from the outset that there will be such a coupled > device? I.e., is it a property of the VM definition? > > Can there be a 'prepared' virtio-net device that presents the STANDBY > feature even if there currently is no vfio-handled device available -- > but making it possible to simply hotplug that device later? > > Should it be possible to add a virtio/vfio pair later on? > >> >> >> >> Well live migration is there already. The new feature is low latency >> >> data path. >> > >> > >> > we get live migration with just virtio. But I meant live migration wi= th VF >> > as >> > primary device. >> > >> >> >> >> And it's the guest that needs failover support not the VM. >> > >> > >> > Isn't guest and VM synonymous? > > I think we need to be really careful to not mix up the two: The VM > contains the definitions, but it is up to the guest how it uses them. > >> > >> > >> >> >> >> >> >>> I think Qemu should check if guest virtio-net supports this feature = and >> >>> provide a mechanism for >> >>> an upper layer indicating if the STANDBY feature is successfully >> >>> negotiated or not. >> >>> The upper layer can then decide if it should hot plug a VF with the = same >> >>> MAC and manage the 2 links. >> >>> If VF is successfully hot plugged, virtio-net link should be disable= d. >> >> >> >> Did you even talk to upper layer management about it? >> >> Just list the steps they need to do and you will see >> >> that's a lot of machinery to manage by the upper layer. >> >> >> >> What do we gain in flexibility? As far as I can see the >> >> only gain is some resources saved for legacy VMs. >> >> >> >> That's not a lot as tenant of the upper layer probably already has >> >> at least a hunch that it's a new guest otherwise >> >> why bother specifying the feature at all - you >> >> save even more resources without it. >> >> >> > >> > I am not all that familiar with how Qemu manages network devices. If w= e can >> > do all the >> > required management of the primary/standby devices within Qemu, that i= s >> > definitely a better >> > approach without upper layer involvement. >> >> Right. I would imagine in the extreme case the upper layer doesn't >> have to be involved at all if QEMU manages all hot plug/unplug logic. >> The management tool can supply passthrough device and virtio with the >> same group UUID, QEMU auto-manages the presence of the primary, and >> hot plug the device as needed before or after the migration. > > I do not really see how you can manage that kind of stuff in QEMU only. > Have you talked to some libvirt folks? (And I'm not sure what you refer > to with 'group UUID'?) > > Also, I think you need to make a distinction between hotplugging a > device and making it visible to the guest. What does 'hotplugging' mean > here? Adding it to the VM definition? Would it be enough to have the > vfio-based device not operational until the virtio feature bit has been > negotiated? > > What happens if the guest does not use the vfio-based device after it > has been made available? Will you still disable the virtio-net link? > (All that link handling definitely sounds like a task for libvirt or > the like.) > > Regarding hot(un)plugging during migration, I think you also need to > keep in mind that different architectures/busses have different > semantics there. Something that works if there's an unplug handshake may > not work on a platform with surprise removal. > > Have you considered guest agents? All of this is punching through > several layers, and I'm not sure if that is a good idea. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: virtio-dev-return-4384-cohuck=redhat.com@lists.oasis-open.org Sender: List-Post: List-Help: List-Unsubscribe: List-Subscribe: Received: from lists.oasis-open.org (oasis-open.org [66.179.20.138]) by lists.oasis-open.org (Postfix) with ESMTP id 4918E581806A for ; Thu, 14 Jun 2018 18:57:24 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <20180614120231.0a72bd5f.cohuck@redhat.com> References: <1525734594-11134-1-git-send-email-sridhar.samudrala@intel.com> <20180611202207-mutt-send-email-mst@kernel.org> <20180612051432-mutt-send-email-mst@kernel.org> <23fc4aa4-ec41-d6e2-3354-10cbfc13b7ec@intel.com> <20180612142557-mutt-send-email-mst@kernel.org> <20180614120231.0a72bd5f.cohuck@redhat.com> From: Siwei Liu Date: Thu, 14 Jun 2018 18:57:11 -0700 Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [virtio-dev] Re: [Qemu-devel] [PATCH] qemu: Introduce VIRTIO_NET_F_STANDBY feature bit to virtio_net To: Cornelia Huck Cc: "Samudrala, Sridhar" , Alexander Duyck , virtio-dev@lists.oasis-open.org, aaron.f.brown@intel.com, Jiri Pirko , "Michael S. Tsirkin" , Jakub Kicinski , Netdev , qemu-devel@nongnu.org, virtualization@lists.linux-foundation.org List-ID: Thank you for sharing your thoughts, Cornelia. With questions below, I think you raised really good points, some of which I don't have answer yet and would also like to explore here. First off, I don't want to push the discussion to the extreme at this point, or sell anything about having QEMU manage everything automatically. Don't get me wrong, it's not there yet. Let's don't assume we are tied to a specific or concerte solution. I think the key for our discussion might be to define or refine the boundary between VM and guest, e.g. what each layer is expected to control and manage exactly. In my view, there might be possibly 3 different options to represent the failover device conceipt to QEMU and libvirt (or any upper layer software): a. Seperate device: in this model, virtio and passthough remains separate devices just as today. QEMU exposes the standby feature bit for virtio, and publish status/event around the negotiation process of this feature bit for libvirt to react upon. Since Libvirt has the pairing relationship itself, maybe through MAC address or something else, it can control the presence of primary by hot plugging or unplugging the passthrough device, although it has to work tightly with virtio's feature negotation process. Not just for migration but also various corner scenarios (driver/feature ok, device reset, reboot, legacy guest etc) along virtio's feature negotiation. b. Coupled device: in this model, virtio and passthough devices are weakly coupled using some group ID, i.e. QEMU match the passthough device for a standby virtio instance by comparing the group ID value present behind each device's bridge. Libvirt provides QEMU the group ID for both type of devices, and only deals with hot plug for migration, by checking some migration status exposed (e.g. the feature negotiation status on the virtio device) by QEMU. QEMU manages the visibility of the primary in guest along virtio's feature negotiation process. c. Fully combined device: in this model, virtio and passthough devices are viewed as a single VM interface altogther. QEMU not just controls the visibility of the primary in guest, but can also manage the exposure of the passthrough for migratability. It can be like that libvirt supplies the group ID to QEMU. Or libvirt does not even have to provide group ID for grouping the two devices, if just one single combined device is exposed by QEMU. In either case, QEMU manages all aspect of such internal construct, including virtio feature negotiation, presence of the primary, and live migration. It looks like to me that, in your opinion, you seem to prefer go with (a). While I'm actually okay with either (b) or (c). Do I understand your point correctly? The reason that I feel that (a) might not be ideal, just as Michael alluded to (quoting below), is that as management stack, it really doesn't need to care about the detailed process of feature negotiation (if we view the guest presence of the primary as part of feature negotiation at an extended level not just virtio). All it needs to be done is to hand in the required devices to QEMU and that's all. Why do we need to addd various hooks, events for whichever happens internally within the guest? '' Primary device is added with a special "primary-failover" flag. A virtual machine is then initialized with just a standby virtio device. Primary is not yet added. Later QEMU detects that guest driver device set DRIVER_OK. It then exposes the primary device to the guest, and triggers a device addition event (hot-plug event) for it. If QEMU detects guest driver removal, it initiates a hot-unplug sequence to remove the primary driver. In particular, if QEMU detects guest re-initialization (e.g. by detecting guest reset) it immediately removes the primary device. '' and, '' management just wants to give the primary to guest and later take it back, it really does not care about the details of the process, so I don't see what does pushing it up the stack buy you. So I don't think it *needs* to be done in libvirt. It probably can if you add a bunch of hooks so it knows whenever vm reboots, driver binds and unbinds from device, and can check that backup flag was set. If you are pushing for a setup like that please get a buy-in from libvirt maintainers or better write a patch. '' Let me know if my clarifications sound clear to you now. Thanks, -Siwei On Thu, Jun 14, 2018 at 3:02 AM, Cornelia Huck wrote: > I've been pointed to this discussion (which I had missed previously) > and I'm getting a headache. Let me first summarize how I understand how > this feature is supposed to work, then I'll respond to some individual > points. > > The basic idea is to enable guests to migrate seamlessly, while still > making it possible for them to use a passed-through device for more > performance etc. The means to do so is to hook a virtio-net device > together with a network device passed through via vfio. The > vfio-handled device is there for performance, the virtio device for > migratability. We have a new virtio feature bit for that which needs to > be negotiated for that 'combined' device to be available. We have to > consider two cases: > > - Older guests that do not support the new feature bit. We presume that > those guests will be confused if they get two network devices with > the same MAC, so the idea is to not show them the vfio-handled device > at all. > - Guests that negotiate the feature bit. We only know positively that > they (a) know the feature bit and (b) are prepared to handle the > consequences of negotiating it after they set the FEATURES_OK bit. > This is therefore the earliest point in time that the vfio-handled > device should be visible or usable for the guest. > > On Wed, 13 Jun 2018 18:02:01 -0700 > Siwei Liu wrote: > >> On Tue, Jun 12, 2018 at 5:08 PM, Samudrala, Sridhar >> wrote: >> > On 6/12/2018 4:34 AM, Michael S. Tsirkin wrote: >> >> >> >> On Mon, Jun 11, 2018 at 10:02:45PM -0700, Samudrala, Sridhar wrote: >> >>> >> >>> On 6/11/2018 7:17 PM, Michael S. Tsirkin wrote: >> >>>> >> >>>> On Tue, Jun 12, 2018 at 09:54:44AM +0800, Jason Wang wrote: >> >>>>> >> >>>>> On 2018=E5=B9=B406=E6=9C=8812=E6=97=A5 01:26, Michael S. Tsirkin w= rote: >> >>>>>> >> >>>>>> On Mon, May 07, 2018 at 04:09:54PM -0700, Sridhar Samudrala wrote= : >> >>>>>>> >> >>>>>>> This feature bit can be used by hypervisor to indicate virtio_ne= t >> >>>>>>> device to >> >>>>>>> act as a standby for another device with the same MAC address. >> >>>>>>> >> >>>>>>> I tested this with a small change to the patch to mark the STAND= BY >> >>>>>>> feature 'true' >> >>>>>>> by default as i am using libvirt to start the VMs. >> >>>>>>> Is there a way to pass the newly added feature bit 'standby' to = qemu >> >>>>>>> via libvirt >> >>>>>>> XML file? >> >>>>>>> >> >>>>>>> Signed-off-by: Sridhar Samudrala >> >>>>>> >> >>>>>> So I do not think we can commit to this interface: we >> >>>>>> really need to control visibility of the primary device. >> >>>>> >> >>>>> The problem is legacy guest won't use primary device at all if we = do >> >>>>> this. >> >>>> >> >>>> And that's by design - I think it's the only way to ensure the >> >>>> legacy guest isn't confused. >> >>> >> >>> Yes. I think so. But i am not sure if Qemu is the right place to con= trol >> >>> the visibility >> >>> of the primary device. The primary device may not be specified as an >> >>> argument to Qemu. It >> >>> may be plugged in later. >> >>> The cloud service provider is providing a feature that enables low >> >>> latency datapath and live >> >>> migration capability. >> >>> A tenant can use this feature only if he is running a VM that has >> >>> virtio-net with failover support. > > So, do you know from the outset that there will be such a coupled > device? I.e., is it a property of the VM definition? > > Can there be a 'prepared' virtio-net device that presents the STANDBY > feature even if there currently is no vfio-handled device available -- > but making it possible to simply hotplug that device later? > > Should it be possible to add a virtio/vfio pair later on? > >> >> >> >> Well live migration is there already. The new feature is low latency >> >> data path. >> > >> > >> > we get live migration with just virtio. But I meant live migration wi= th VF >> > as >> > primary device. >> > >> >> >> >> And it's the guest that needs failover support not the VM. >> > >> > >> > Isn't guest and VM synonymous? > > I think we need to be really careful to not mix up the two: The VM > contains the definitions, but it is up to the guest how it uses them. > >> > >> > >> >> >> >> >> >>> I think Qemu should check if guest virtio-net supports this feature = and >> >>> provide a mechanism for >> >>> an upper layer indicating if the STANDBY feature is successfully >> >>> negotiated or not. >> >>> The upper layer can then decide if it should hot plug a VF with the = same >> >>> MAC and manage the 2 links. >> >>> If VF is successfully hot plugged, virtio-net link should be disable= d. >> >> >> >> Did you even talk to upper layer management about it? >> >> Just list the steps they need to do and you will see >> >> that's a lot of machinery to manage by the upper layer. >> >> >> >> What do we gain in flexibility? As far as I can see the >> >> only gain is some resources saved for legacy VMs. >> >> >> >> That's not a lot as tenant of the upper layer probably already has >> >> at least a hunch that it's a new guest otherwise >> >> why bother specifying the feature at all - you >> >> save even more resources without it. >> >> >> > >> > I am not all that familiar with how Qemu manages network devices. If w= e can >> > do all the >> > required management of the primary/standby devices within Qemu, that i= s >> > definitely a better >> > approach without upper layer involvement. >> >> Right. I would imagine in the extreme case the upper layer doesn't >> have to be involved at all if QEMU manages all hot plug/unplug logic. >> The management tool can supply passthrough device and virtio with the >> same group UUID, QEMU auto-manages the presence of the primary, and >> hot plug the device as needed before or after the migration. > > I do not really see how you can manage that kind of stuff in QEMU only. > Have you talked to some libvirt folks? (And I'm not sure what you refer > to with 'group UUID'?) > > Also, I think you need to make a distinction between hotplugging a > device and making it visible to the guest. What does 'hotplugging' mean > here? Adding it to the VM definition? Would it be enough to have the > vfio-based device not operational until the virtio feature bit has been > negotiated? > > What happens if the guest does not use the vfio-based device after it > has been made available? Will you still disable the virtio-net link? > (All that link handling definitely sounds like a task for libvirt or > the like.) > > Regarding hot(un)plugging during migration, I think you also need to > keep in mind that different architectures/busses have different > semantics there. Something that works if there's an unplug handshake may > not work on a platform with surprise removal. > > Have you considered guest agents? All of this is punching through > several layers, and I'm not sure if that is a good idea. --------------------------------------------------------------------- To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org