From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7F23C10F0E for ; Tue, 9 Apr 2019 05:04:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A04F52084F for ; Tue, 9 Apr 2019 05:04:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726535AbfDIFEa (ORCPT ); Tue, 9 Apr 2019 01:04:30 -0400 Received: from mx2.suse.de ([195.135.220.15]:47690 "EHLO mx1.suse.de" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725781AbfDIFEa (ORCPT ); Tue, 9 Apr 2019 01:04:30 -0400 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id D1368AC89; Tue, 9 Apr 2019 05:04:27 +0000 (UTC) Subject: Re: [PATCH RFC 00/39] x86/KVM: Xen HVM guest support To: Joao Martins Cc: Paolo Bonzini , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Ankur Arora , Boris Ostrovsky , =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org, Stefano Stabellini , xen-devel@lists.xenproject.org References: <20190220201609.28290-1-joao.m.martins@oracle.com> <35051310-c497-8ad5-4434-1b8426a317d2@redhat.com> <8b1f4912-4f92-69ae-ae01-d899d5640572@oracle.com> <3ee91f33-2973-c2db-386f-afbf138081b4@redhat.com> <59676804-786d-3df8-7752-8e45dec6d65b@oracle.com> <94738323-ebdf-d58e-55b6-313e27c923b0@oracle.com> <585163c2-8dea-728d-7556-9cb3559f0eca@suse.com> <97808492-58ee-337f-c894-900b34b7b1a5@oracle.com> From: Juergen Gross Openpgp: preference=signencrypt Autocrypt: addr=jgross@suse.com; prefer-encrypt=mutual; keydata= mQENBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB AAG0H0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT6JATkEEwECACMFAlOMcK8CGwMH CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO RoVBYuiocc51872tRGywc03xaQydB+9R7BHPuQENBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAGJAR8EGAECAAkFAlOMcBYCGwwACgkQsN6d 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHf4kBrQQY AQgAIBYhBIUSZ3Lo9gSUpdCX97DendYovxMvBQJa3fDQAhsCAIEJELDendYovxMvdiAEGRYI AB0WIQRTLbB6QfY48x44uB6AXGG7T9hjvgUCWt3w0AAKCRCAXGG7T9hjvk2LAP99B/9FenK/ 1lfifxQmsoOrjbZtzCS6OKxPqOLHaY47BgEAqKKn36YAPpbk09d2GTVetoQJwiylx/Z9/mQI CUbQMg1pNQf9EjA1bNcMbnzJCgt0P9Q9wWCLwZa01SnQWFz8Z4HEaKldie+5bHBL5CzVBrLv 81tqX+/j95llpazzCXZW2sdNL3r8gXqrajSox7LR2rYDGdltAhQuISd2BHrbkQVEWD4hs7iV 1KQHe2uwXbKlguKPhk5ubZxqwsg/uIHw0qZDk+d0vxjTtO2JD5Jv/CeDgaBX4Emgp0NYs8IC UIyKXBtnzwiNv4cX9qKlz2Gyq9b+GdcLYZqMlIBjdCz0yJvgeb3WPNsCOanvbjelDhskx9gd 6YUUFFqgsLtrKpCNyy203a58g2WosU9k9H+LcheS37Ph2vMVTISMszW9W8gyORSgmw== Message-ID: <59deb041-2b5d-8451-32c7-644fe36e053b@suse.com> Date: Tue, 9 Apr 2019 07:04:26 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.5.1 MIME-Version: 1.0 In-Reply-To: <97808492-58ee-337f-c894-900b34b7b1a5@oracle.com> Content-Type: text/plain; charset=utf-8 Content-Language: de-DE Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 08/04/2019 19:31, Joao Martins wrote: > On 4/8/19 11:42 AM, Juergen Gross wrote: >> On 08/04/2019 12:36, Joao Martins wrote: >>> On 4/8/19 7:44 AM, Juergen Gross wrote: >>>> On 12/03/2019 18:14, Joao Martins wrote: >>>>> On 2/22/19 4:59 PM, Paolo Bonzini wrote: >>>>>> On 21/02/19 12:45, Joao Martins wrote: >>>>>>> On 2/20/19 9:09 PM, Paolo Bonzini wrote: >>>>>>>> On 20/02/19 21:15, Joao Martins wrote: >>>>>>>>> 2. PV Driver support (patches 17 - 39) >>>>>>>>> >>>>>>>>> We start by redirecting hypercalls from the backend to routines >>>>>>>>> which emulate the behaviour that PV backends expect i.e. grant >>>>>>>>> table and interdomain events. Next, we add support for late >>>>>>>>> initialization of xenbus, followed by implementing >>>>>>>>> frontend/backend communication mechanisms (i.e. grant tables and >>>>>>>>> interdomain event channels). Finally, introduce xen-shim.ko, >>>>>>>>> which will setup a limited Xen environment. This uses the added >>>>>>>>> functionality of Xen specific shared memory (grant tables) and >>>>>>>>> notifications (event channels). >>>>>>>> >>>>>>>> I am a bit worried by the last patches, they seem really brittle and >>>>>>>> prone to breakage. I don't know Xen well enough to understand if the >>>>>>>> lack of support for GNTMAP_host_map is fixable, but if not, you have to >>>>>>>> define a completely different hypercall. >>>>>>>> >>>>>>> I guess Ankur already answered this; so just to stack this on top of his comment. >>>>>>> >>>>>>> The xen_shim_domain() is only meant to handle the case where the backend >>>>>>> has/can-have full access to guest memory [i.e. netback and blkback would work >>>>>>> with similar assumptions as vhost?]. For the normal case, where a backend *in a >>>>>>> guest* maps and unmaps other guest memory, this is not applicable and these >>>>>>> changes don't affect that case. >>>>>>> >>>>>>> IOW, the PV backend here sits on the hypervisor, and the hypercalls aren't >>>>>>> actual hypercalls but rather invoking shim_hypercall(). The call chain would go >>>>>>> more or less like: >>>>>>> >>>>>>> >>>>>>> gnttab_map_refs(map_ops, pages) >>>>>>> HYPERVISOR_grant_table_op(GNTTABOP_map_grant_ref,...) >>>>>>> shim_hypercall() >>>>>>> shim_hcall_gntmap() >>>>>>> >>>>>>> Our reasoning was that given we are already in KVM, why mapping a page if the >>>>>>> user (i.e. the kernel PV backend) is himself? The lack of GNTMAP_host_map is how >>>>>>> the shim determines its user doesn't want to map the page. Also, there's another >>>>>>> issue where PV backends always need a struct page to reference the device >>>>>>> inflight data as Ankur pointed out. >>>>>> >>>>>> Ultimately it's up to the Xen people. It does make their API uglier, >>>>>> especially the in/out change for the parameter. If you can at least >>>>>> avoid that, it would alleviate my concerns quite a bit. >>>>> >>>>> In my view, we have two options overall: >>>>> >>>>> 1) Make it explicit, the changes the PV drivers we have to make in >>>>> order to support xen_shim_domain(). This could mean e.g. a) add a callback >>>>> argument to gnttab_map_refs() that is invoked for every page that gets looked up >>>>> successfully, and inside this callback the PV driver may update it's tracking >>>>> page. Here we no longer have this in/out parameter in gnttab_map_refs, and all >>>>> shim_domain specific bits would be a little more abstracted from Xen PV >>>>> backends. See netback example below the scissors mark. Or b) have sort of a >>>>> translate_gref() and put_gref() API that Xen PV drivers use which make it even >>>>> more explicit that there's no grant ops involved. The latter is more invasive. >>>>> >>>>> 2) The second option is to support guest grant mapping/unmapping [*] to allow >>>>> hosting PV backends inside the guest. This would remove the Xen changes in this >>>>> series completely. But it would require another guest being used >>>>> as netback/blkback/xenstored, and less performance than 1) (though, in theory, >>>>> it would be equivalent to what does Xen with grants/events). The only changes in >>>>> Linux Xen code is adding xenstored domain support, but that is useful on its own >>>>> outside the scope of this work. >>>>> >>>>> I think there's value on both; 1) is probably more familiar for KVM users >>>>> perhaps (as it is similar to what vhost does?) while 2) equates to implementing >>>>> Xen disagregation capabilities in KVM. >>>>> >>>>> Thoughts? Xen maintainers what's your take on this? >>>> >>>> What I'd like best would be a new handle (e.g. xenhost_t *) used as an >>>> abstraction layer for this kind of stuff. It should be passed to the >>>> backends and those would pass it on to low-level Xen drivers (xenbus, >>>> event channels, grant table, ...). >>>> >>> So if IIRC backends would use the xenhost layer to access grants or frames >>> referenced by grants, and that would grok into some of this. IOW, you would have >>> two implementors of xenhost: one for nested remote/local events+grants and >>> another for this "shim domain" ? >> >> As I'd need that for nested Xen I guess that would make it 3 variants. >> Probably the xen-shim variant would need more hooks, but that should be >> no problem. >> > I probably messed up in the short description but "nested remote/local > events+grants" was referring to nested Xen (FWIW remote meant L0 and local L1). > So maybe only 2 variants are needed? I need one xenhost variant for the "normal" case as today: talking to the single hypervisor (or in nested case: to the L1 hypervisor). Then I need a variant for the nested case talking to L0 hypervisor. And you need a variant talking to xen-shim. The first two variants can be active in the same system in case of nested Xen: the backends of L2 dom0 are talking to L1 hypervisor, while its frontends are talking with L0 hypervisor. > >>>> I was planning to do that (the xenhost_t * stuff) soon in order to add >>>> support for nested Xen using PV devices (you need two Xenstores for that >>>> as the nested dom0 is acting as Xen backend server, while using PV >>>> frontends for accessing the "real" world outside). >>>> >>>> The xenhost_t should be used for: >>>> >>>> - accessing Xenstore >>>> - issuing and receiving events >>>> - doing hypercalls >>>> - grant table operations >>>> >>> >>> In the text above, I sort of suggested a slice of this on 1.b) with a >>> translate_gref() and put_gref() API -- to get the page from a gref. This was >>> because of the flags|host_addr hurdle we depicted above wrt to using using grant >>> maps/unmaps. You think some of the xenhost layer would be ammenable to support >>> this case? >> >> I think so, yes. >> >>> >>>> So exactly the kind of stuff you want to do, too. >>>> >>> Cool idea! >> >> In the end you might make my life easier for nested Xen. :-) >> > Hehe :) > >> Do you want to have a try with that idea or should I do that? I might be >> able to start working on that in about a month. >> > Ankur (CC'ed) will give a shot at it, and should start a new thread on this > xenhost abstraction layer. Great, looking forward to it! Juergen From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5CD0C10F0E for ; Tue, 9 Apr 2019 05:04:56 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 925832084F for ; Tue, 9 Apr 2019 05:04:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 925832084F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hDiw0-0000NA-85; Tue, 09 Apr 2019 05:04:36 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hDivy-0000N5-K4 for xen-devel@lists.xenproject.org; Tue, 09 Apr 2019 05:04:34 +0000 X-Inumbo-ID: f3d7a3ee-5a84-11e9-bb7a-678c44e32ab6 Received: from mx1.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id f3d7a3ee-5a84-11e9-bb7a-678c44e32ab6; Tue, 09 Apr 2019 05:04:28 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id D1368AC89; Tue, 9 Apr 2019 05:04:27 +0000 (UTC) To: Joao Martins References: <20190220201609.28290-1-joao.m.martins@oracle.com> <35051310-c497-8ad5-4434-1b8426a317d2@redhat.com> <8b1f4912-4f92-69ae-ae01-d899d5640572@oracle.com> <3ee91f33-2973-c2db-386f-afbf138081b4@redhat.com> <59676804-786d-3df8-7752-8e45dec6d65b@oracle.com> <94738323-ebdf-d58e-55b6-313e27c923b0@oracle.com> <585163c2-8dea-728d-7556-9cb3559f0eca@suse.com> <97808492-58ee-337f-c894-900b34b7b1a5@oracle.com> From: Juergen Gross Openpgp: preference=signencrypt Autocrypt: addr=jgross@suse.com; prefer-encrypt=mutual; keydata= mQENBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB AAG0H0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT6JATkEEwECACMFAlOMcK8CGwMH CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO RoVBYuiocc51872tRGywc03xaQydB+9R7BHPuQENBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAGJAR8EGAECAAkFAlOMcBYCGwwACgkQsN6d 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHf4kBrQQY AQgAIBYhBIUSZ3Lo9gSUpdCX97DendYovxMvBQJa3fDQAhsCAIEJELDendYovxMvdiAEGRYI AB0WIQRTLbB6QfY48x44uB6AXGG7T9hjvgUCWt3w0AAKCRCAXGG7T9hjvk2LAP99B/9FenK/ 1lfifxQmsoOrjbZtzCS6OKxPqOLHaY47BgEAqKKn36YAPpbk09d2GTVetoQJwiylx/Z9/mQI CUbQMg1pNQf9EjA1bNcMbnzJCgt0P9Q9wWCLwZa01SnQWFz8Z4HEaKldie+5bHBL5CzVBrLv 81tqX+/j95llpazzCXZW2sdNL3r8gXqrajSox7LR2rYDGdltAhQuISd2BHrbkQVEWD4hs7iV 1KQHe2uwXbKlguKPhk5ubZxqwsg/uIHw0qZDk+d0vxjTtO2JD5Jv/CeDgaBX4Emgp0NYs8IC UIyKXBtnzwiNv4cX9qKlz2Gyq9b+GdcLYZqMlIBjdCz0yJvgeb3WPNsCOanvbjelDhskx9gd 6YUUFFqgsLtrKpCNyy203a58g2WosU9k9H+LcheS37Ph2vMVTISMszW9W8gyORSgmw== Message-ID: <59deb041-2b5d-8451-32c7-644fe36e053b@suse.com> Date: Tue, 9 Apr 2019 07:04:26 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.5.1 MIME-Version: 1.0 In-Reply-To: <97808492-58ee-337f-c894-900b34b7b1a5@oracle.com> Content-Language: de-DE Subject: Re: [Xen-devel] [PATCH RFC 00/39] x86/KVM: Xen HVM guest support X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , xen-devel@lists.xenproject.org, kvm@vger.kernel.org, =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= , x86@kernel.org, linux-kernel@vger.kernel.org, Ankur Arora , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , Paolo Bonzini , Boris Ostrovsky , Thomas Gleixner Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" T24gMDgvMDQvMjAxOSAxOTozMSwgSm9hbyBNYXJ0aW5zIHdyb3RlOgo+IE9uIDQvOC8xOSAxMTo0 MiBBTSwgSnVlcmdlbiBHcm9zcyB3cm90ZToKPj4gT24gMDgvMDQvMjAxOSAxMjozNiwgSm9hbyBN YXJ0aW5zIHdyb3RlOgo+Pj4gT24gNC84LzE5IDc6NDQgQU0sIEp1ZXJnZW4gR3Jvc3Mgd3JvdGU6 Cj4+Pj4gT24gMTIvMDMvMjAxOSAxODoxNCwgSm9hbyBNYXJ0aW5zIHdyb3RlOgo+Pj4+PiBPbiAy LzIyLzE5IDQ6NTkgUE0sIFBhb2xvIEJvbnppbmkgd3JvdGU6Cj4+Pj4+PiBPbiAyMS8wMi8xOSAx Mjo0NSwgSm9hbyBNYXJ0aW5zIHdyb3RlOgo+Pj4+Pj4+IE9uIDIvMjAvMTkgOTowOSBQTSwgUGFv bG8gQm9uemluaSB3cm90ZToKPj4+Pj4+Pj4gT24gMjAvMDIvMTkgMjE6MTUsIEpvYW8gTWFydGlu cyB3cm90ZToKPj4+Pj4+Pj4+ICAyLiBQViBEcml2ZXIgc3VwcG9ydCAocGF0Y2hlcyAxNyAtIDM5 KQo+Pj4+Pj4+Pj4KPj4+Pj4+Pj4+ICBXZSBzdGFydCBieSByZWRpcmVjdGluZyBoeXBlcmNhbGxz IGZyb20gdGhlIGJhY2tlbmQgdG8gcm91dGluZXMKPj4+Pj4+Pj4+ICB3aGljaCBlbXVsYXRlIHRo ZSBiZWhhdmlvdXIgdGhhdCBQViBiYWNrZW5kcyBleHBlY3QgaS5lLiBncmFudAo+Pj4+Pj4+Pj4g IHRhYmxlIGFuZCBpbnRlcmRvbWFpbiBldmVudHMuIE5leHQsIHdlIGFkZCBzdXBwb3J0IGZvciBs YXRlCj4+Pj4+Pj4+PiAgaW5pdGlhbGl6YXRpb24gb2YgeGVuYnVzLCBmb2xsb3dlZCBieSBpbXBs ZW1lbnRpbmcKPj4+Pj4+Pj4+ICBmcm9udGVuZC9iYWNrZW5kIGNvbW11bmljYXRpb24gbWVjaGFu aXNtcyAoaS5lLiBncmFudCB0YWJsZXMgYW5kCj4+Pj4+Pj4+PiAgaW50ZXJkb21haW4gZXZlbnQg Y2hhbm5lbHMpLiBGaW5hbGx5LCBpbnRyb2R1Y2UgeGVuLXNoaW0ua28sCj4+Pj4+Pj4+PiAgd2hp Y2ggd2lsbCBzZXR1cCBhIGxpbWl0ZWQgWGVuIGVudmlyb25tZW50LiBUaGlzIHVzZXMgdGhlIGFk ZGVkCj4+Pj4+Pj4+PiAgZnVuY3Rpb25hbGl0eSBvZiBYZW4gc3BlY2lmaWMgc2hhcmVkIG1lbW9y eSAoZ3JhbnQgdGFibGVzKSBhbmQKPj4+Pj4+Pj4+ICBub3RpZmljYXRpb25zIChldmVudCBjaGFu bmVscykuCj4+Pj4+Pj4+Cj4+Pj4+Pj4+IEkgYW0gYSBiaXQgd29ycmllZCBieSB0aGUgbGFzdCBw YXRjaGVzLCB0aGV5IHNlZW0gcmVhbGx5IGJyaXR0bGUgYW5kCj4+Pj4+Pj4+IHByb25lIHRvIGJy ZWFrYWdlLiAgSSBkb24ndCBrbm93IFhlbiB3ZWxsIGVub3VnaCB0byB1bmRlcnN0YW5kIGlmIHRo ZQo+Pj4+Pj4+PiBsYWNrIG9mIHN1cHBvcnQgZm9yIEdOVE1BUF9ob3N0X21hcCBpcyBmaXhhYmxl LCBidXQgaWYgbm90LCB5b3UgaGF2ZSB0bwo+Pj4+Pj4+PiBkZWZpbmUgYSBjb21wbGV0ZWx5IGRp ZmZlcmVudCBoeXBlcmNhbGwuCj4+Pj4+Pj4+Cj4+Pj4+Pj4gSSBndWVzcyBBbmt1ciBhbHJlYWR5 IGFuc3dlcmVkIHRoaXM7IHNvIGp1c3QgdG8gc3RhY2sgdGhpcyBvbiB0b3Agb2YgaGlzIGNvbW1l bnQuCj4+Pj4+Pj4KPj4+Pj4+PiBUaGUgeGVuX3NoaW1fZG9tYWluKCkgaXMgb25seSBtZWFudCB0 byBoYW5kbGUgdGhlIGNhc2Ugd2hlcmUgdGhlIGJhY2tlbmQKPj4+Pj4+PiBoYXMvY2FuLWhhdmUg ZnVsbCBhY2Nlc3MgdG8gZ3Vlc3QgbWVtb3J5IFtpLmUuIG5ldGJhY2sgYW5kIGJsa2JhY2sgd291 bGQgd29yawo+Pj4+Pj4+IHdpdGggc2ltaWxhciBhc3N1bXB0aW9ucyBhcyB2aG9zdD9dLiBGb3Ig dGhlIG5vcm1hbCBjYXNlLCB3aGVyZSBhIGJhY2tlbmQgKmluIGEKPj4+Pj4+PiBndWVzdCogbWFw cyBhbmQgdW5tYXBzIG90aGVyIGd1ZXN0IG1lbW9yeSwgdGhpcyBpcyBub3QgYXBwbGljYWJsZSBh bmQgdGhlc2UKPj4+Pj4+PiBjaGFuZ2VzIGRvbid0IGFmZmVjdCB0aGF0IGNhc2UuCj4+Pj4+Pj4K Pj4+Pj4+PiBJT1csIHRoZSBQViBiYWNrZW5kIGhlcmUgc2l0cyBvbiB0aGUgaHlwZXJ2aXNvciwg YW5kIHRoZSBoeXBlcmNhbGxzIGFyZW4ndAo+Pj4+Pj4+IGFjdHVhbCBoeXBlcmNhbGxzIGJ1dCBy YXRoZXIgaW52b2tpbmcgc2hpbV9oeXBlcmNhbGwoKS4gVGhlIGNhbGwgY2hhaW4gd291bGQgZ28K Pj4+Pj4+PiBtb3JlIG9yIGxlc3MgbGlrZToKPj4+Pj4+Pgo+Pj4+Pj4+IDxuZXRiYWNrfGJsa2Jh Y2t8c2NzaWJhY2s+Cj4+Pj4+Pj4gIGdudHRhYl9tYXBfcmVmcyhtYXBfb3BzLCBwYWdlcykKPj4+ Pj4+PiAgICBIWVBFUlZJU09SX2dyYW50X3RhYmxlX29wKEdOVFRBQk9QX21hcF9ncmFudF9yZWYs Li4uKQo+Pj4+Pj4+ICAgICAgc2hpbV9oeXBlcmNhbGwoKQo+Pj4+Pj4+ICAgICAgICBzaGltX2hj YWxsX2dudG1hcCgpCj4+Pj4+Pj4KPj4+Pj4+PiBPdXIgcmVhc29uaW5nIHdhcyB0aGF0IGdpdmVu IHdlIGFyZSBhbHJlYWR5IGluIEtWTSwgd2h5IG1hcHBpbmcgYSBwYWdlIGlmIHRoZQo+Pj4+Pj4+ IHVzZXIgKGkuZS4gdGhlIGtlcm5lbCBQViBiYWNrZW5kKSBpcyBoaW1zZWxmPyBUaGUgbGFjayBv ZiBHTlRNQVBfaG9zdF9tYXAgaXMgaG93Cj4+Pj4+Pj4gdGhlIHNoaW0gZGV0ZXJtaW5lcyBpdHMg dXNlciBkb2Vzbid0IHdhbnQgdG8gbWFwIHRoZSBwYWdlLiBBbHNvLCB0aGVyZSdzIGFub3RoZXIK Pj4+Pj4+PiBpc3N1ZSB3aGVyZSBQViBiYWNrZW5kcyBhbHdheXMgbmVlZCBhIHN0cnVjdCBwYWdl IHRvIHJlZmVyZW5jZSB0aGUgZGV2aWNlCj4+Pj4+Pj4gaW5mbGlnaHQgZGF0YSBhcyBBbmt1ciBw b2ludGVkIG91dC4KPj4+Pj4+Cj4+Pj4+PiBVbHRpbWF0ZWx5IGl0J3MgdXAgdG8gdGhlIFhlbiBw ZW9wbGUuICBJdCBkb2VzIG1ha2UgdGhlaXIgQVBJIHVnbGllciwKPj4+Pj4+IGVzcGVjaWFsbHkg dGhlIGluL291dCBjaGFuZ2UgZm9yIHRoZSBwYXJhbWV0ZXIuICBJZiB5b3UgY2FuIGF0IGxlYXN0 Cj4+Pj4+PiBhdm9pZCB0aGF0LCBpdCB3b3VsZCBhbGxldmlhdGUgbXkgY29uY2VybnMgcXVpdGUg YSBiaXQuCj4+Pj4+Cj4+Pj4+IEluIG15IHZpZXcsIHdlIGhhdmUgdHdvIG9wdGlvbnMgb3ZlcmFs bDoKPj4+Pj4KPj4+Pj4gMSkgTWFrZSBpdCBleHBsaWNpdCwgdGhlIGNoYW5nZXMgdGhlIFBWIGRy aXZlcnMgd2UgaGF2ZSB0byBtYWtlIGluCj4+Pj4+IG9yZGVyIHRvIHN1cHBvcnQgeGVuX3NoaW1f ZG9tYWluKCkuIFRoaXMgY291bGQgbWVhbiBlLmcuIGEpIGFkZCBhIGNhbGxiYWNrCj4+Pj4+IGFy Z3VtZW50IHRvIGdudHRhYl9tYXBfcmVmcygpIHRoYXQgaXMgaW52b2tlZCBmb3IgZXZlcnkgcGFn ZSB0aGF0IGdldHMgbG9va2VkIHVwCj4+Pj4+IHN1Y2Nlc3NmdWxseSwgYW5kIGluc2lkZSB0aGlz IGNhbGxiYWNrIHRoZSBQViBkcml2ZXIgbWF5IHVwZGF0ZSBpdCdzIHRyYWNraW5nCj4+Pj4+IHBh Z2UuIEhlcmUgd2Ugbm8gbG9uZ2VyIGhhdmUgdGhpcyBpbi9vdXQgcGFyYW1ldGVyIGluIGdudHRh Yl9tYXBfcmVmcywgYW5kIGFsbAo+Pj4+PiBzaGltX2RvbWFpbiBzcGVjaWZpYyBiaXRzIHdvdWxk IGJlIGEgbGl0dGxlIG1vcmUgYWJzdHJhY3RlZCBmcm9tIFhlbiBQVgo+Pj4+PiBiYWNrZW5kcy4g U2VlIG5ldGJhY2sgZXhhbXBsZSBiZWxvdyB0aGUgc2Npc3NvcnMgbWFyay4gT3IgYikgaGF2ZSBz b3J0IG9mIGEKPj4+Pj4gdHJhbnNsYXRlX2dyZWYoKSBhbmQgcHV0X2dyZWYoKSBBUEkgdGhhdCBY ZW4gUFYgZHJpdmVycyB1c2Ugd2hpY2ggbWFrZSBpdCBldmVuCj4+Pj4+IG1vcmUgZXhwbGljaXQg dGhhdCB0aGVyZSdzIG5vIGdyYW50IG9wcyBpbnZvbHZlZC4gVGhlIGxhdHRlciBpcyBtb3JlIGlu dmFzaXZlLgo+Pj4+Pgo+Pj4+PiAyKSBUaGUgc2Vjb25kIG9wdGlvbiBpcyB0byBzdXBwb3J0IGd1 ZXN0IGdyYW50IG1hcHBpbmcvdW5tYXBwaW5nIFsqXSB0byBhbGxvdwo+Pj4+PiBob3N0aW5nIFBW IGJhY2tlbmRzIGluc2lkZSB0aGUgZ3Vlc3QuIFRoaXMgd291bGQgcmVtb3ZlIHRoZSBYZW4gY2hh bmdlcyBpbiB0aGlzCj4+Pj4+IHNlcmllcyBjb21wbGV0ZWx5LiBCdXQgaXQgd291bGQgcmVxdWly ZSBhbm90aGVyIGd1ZXN0IGJlaW5nIHVzZWQKPj4+Pj4gYXMgbmV0YmFjay9ibGtiYWNrL3hlbnN0 b3JlZCwgYW5kIGxlc3MgcGVyZm9ybWFuY2UgdGhhbiAxKSAodGhvdWdoLCBpbiB0aGVvcnksCj4+ Pj4+IGl0IHdvdWxkIGJlIGVxdWl2YWxlbnQgdG8gd2hhdCBkb2VzIFhlbiB3aXRoIGdyYW50cy9l dmVudHMpLiBUaGUgb25seSBjaGFuZ2VzIGluCj4+Pj4+IExpbnV4IFhlbiBjb2RlIGlzIGFkZGlu ZyB4ZW5zdG9yZWQgZG9tYWluIHN1cHBvcnQsIGJ1dCB0aGF0IGlzIHVzZWZ1bCBvbiBpdHMgb3du Cj4+Pj4+IG91dHNpZGUgdGhlIHNjb3BlIG9mIHRoaXMgd29yay4KPj4+Pj4KPj4+Pj4gSSB0aGlu ayB0aGVyZSdzIHZhbHVlIG9uIGJvdGg7IDEpIGlzIHByb2JhYmx5IG1vcmUgZmFtaWxpYXIgZm9y IEtWTSB1c2Vycwo+Pj4+PiBwZXJoYXBzIChhcyBpdCBpcyBzaW1pbGFyIHRvIHdoYXQgdmhvc3Qg ZG9lcz8pIHdoaWxlICAyKSBlcXVhdGVzIHRvIGltcGxlbWVudGluZwo+Pj4+PiBYZW4gZGlzYWdy ZWdhdGlvbiBjYXBhYmlsaXRpZXMgaW4gS1ZNLgo+Pj4+Pgo+Pj4+PiBUaG91Z2h0cz8gWGVuIG1h aW50YWluZXJzIHdoYXQncyB5b3VyIHRha2Ugb24gdGhpcz8KPj4+Pgo+Pj4+IFdoYXQgSSdkIGxp a2UgYmVzdCB3b3VsZCBiZSBhIG5ldyBoYW5kbGUgKGUuZy4geGVuaG9zdF90ICopIHVzZWQgYXMg YW4KPj4+PiBhYnN0cmFjdGlvbiBsYXllciBmb3IgdGhpcyBraW5kIG9mIHN0dWZmLiBJdCBzaG91 bGQgYmUgcGFzc2VkIHRvIHRoZQo+Pj4+IGJhY2tlbmRzIGFuZCB0aG9zZSB3b3VsZCBwYXNzIGl0 IG9uIHRvIGxvdy1sZXZlbCBYZW4gZHJpdmVycyAoeGVuYnVzLAo+Pj4+IGV2ZW50IGNoYW5uZWxz LCBncmFudCB0YWJsZSwgLi4uKS4KPj4+Pgo+Pj4gU28gaWYgSUlSQyBiYWNrZW5kcyB3b3VsZCB1 c2UgdGhlIHhlbmhvc3QgbGF5ZXIgdG8gYWNjZXNzIGdyYW50cyBvciBmcmFtZXMKPj4+IHJlZmVy ZW5jZWQgYnkgZ3JhbnRzLCBhbmQgdGhhdCB3b3VsZCBncm9rIGludG8gc29tZSBvZiB0aGlzLiBJ T1csIHlvdSB3b3VsZCBoYXZlCj4+PiB0d28gaW1wbGVtZW50b3JzIG9mIHhlbmhvc3Q6IG9uZSBm b3IgbmVzdGVkIHJlbW90ZS9sb2NhbCBldmVudHMrZ3JhbnRzIGFuZAo+Pj4gYW5vdGhlciBmb3Ig dGhpcyAic2hpbSBkb21haW4iID8KPj4KPj4gQXMgSSdkIG5lZWQgdGhhdCBmb3IgbmVzdGVkIFhl biBJIGd1ZXNzIHRoYXQgd291bGQgbWFrZSBpdCAzIHZhcmlhbnRzLgo+PiBQcm9iYWJseSB0aGUg eGVuLXNoaW0gdmFyaWFudCB3b3VsZCBuZWVkIG1vcmUgaG9va3MsIGJ1dCB0aGF0IHNob3VsZCBi ZQo+PiBubyBwcm9ibGVtLgo+Pgo+IEkgcHJvYmFibHkgbWVzc2VkIHVwIGluIHRoZSBzaG9ydCBk ZXNjcmlwdGlvbiBidXQgIm5lc3RlZCByZW1vdGUvbG9jYWwKPiBldmVudHMrZ3JhbnRzIiB3YXMg cmVmZXJyaW5nIHRvIG5lc3RlZCBYZW4gKEZXSVcgcmVtb3RlIG1lYW50IEwwIGFuZCBsb2NhbCBM MSkuCj4gU28gbWF5YmUgb25seSAyIHZhcmlhbnRzIGFyZSBuZWVkZWQ/CgpJIG5lZWQgb25lIHhl bmhvc3QgdmFyaWFudCBmb3IgdGhlICJub3JtYWwiIGNhc2UgYXMgdG9kYXk6IHRhbGtpbmcgdG8K dGhlIHNpbmdsZSBoeXBlcnZpc29yIChvciBpbiBuZXN0ZWQgY2FzZTogdG8gdGhlIEwxIGh5cGVy dmlzb3IpLgoKVGhlbiBJIG5lZWQgYSB2YXJpYW50IGZvciB0aGUgbmVzdGVkIGNhc2UgdGFsa2lu ZyB0byBMMCBoeXBlcnZpc29yLgoKQW5kIHlvdSBuZWVkIGEgdmFyaWFudCB0YWxraW5nIHRvIHhl bi1zaGltLgoKVGhlIGZpcnN0IHR3byB2YXJpYW50cyBjYW4gYmUgYWN0aXZlIGluIHRoZSBzYW1l IHN5c3RlbSBpbiBjYXNlIG9mCm5lc3RlZCBYZW46IHRoZSBiYWNrZW5kcyBvZiBMMiBkb20wIGFy ZSB0YWxraW5nIHRvIEwxIGh5cGVydmlzb3IsCndoaWxlIGl0cyBmcm9udGVuZHMgYXJlIHRhbGtp bmcgd2l0aCBMMCBoeXBlcnZpc29yLgoKPiAKPj4+PiBJIHdhcyBwbGFubmluZyB0byBkbyB0aGF0 ICh0aGUgeGVuaG9zdF90ICogc3R1ZmYpIHNvb24gaW4gb3JkZXIgdG8gYWRkCj4+Pj4gc3VwcG9y dCBmb3IgbmVzdGVkIFhlbiB1c2luZyBQViBkZXZpY2VzICh5b3UgbmVlZCB0d28gWGVuc3RvcmVz IGZvciB0aGF0Cj4+Pj4gYXMgdGhlIG5lc3RlZCBkb20wIGlzIGFjdGluZyBhcyBYZW4gYmFja2Vu ZCBzZXJ2ZXIsIHdoaWxlIHVzaW5nIFBWCj4+Pj4gZnJvbnRlbmRzIGZvciBhY2Nlc3NpbmcgdGhl ICJyZWFsIiB3b3JsZCBvdXRzaWRlKS4KPj4+Pgo+Pj4+IFRoZSB4ZW5ob3N0X3Qgc2hvdWxkIGJl IHVzZWQgZm9yOgo+Pj4+Cj4+Pj4gLSBhY2Nlc3NpbmcgWGVuc3RvcmUKPj4+PiAtIGlzc3Vpbmcg YW5kIHJlY2VpdmluZyBldmVudHMKPj4+PiAtIGRvaW5nIGh5cGVyY2FsbHMKPj4+PiAtIGdyYW50 IHRhYmxlIG9wZXJhdGlvbnMKPj4+Pgo+Pj4KPj4+IEluIHRoZSB0ZXh0IGFib3ZlLCBJIHNvcnQg b2Ygc3VnZ2VzdGVkIGEgc2xpY2Ugb2YgdGhpcyBvbiAxLmIpIHdpdGggYQo+Pj4gdHJhbnNsYXRl X2dyZWYoKSBhbmQgcHV0X2dyZWYoKSBBUEkgLS0gdG8gZ2V0IHRoZSBwYWdlIGZyb20gYSBncmVm LiBUaGlzIHdhcwo+Pj4gYmVjYXVzZSBvZiB0aGUgZmxhZ3N8aG9zdF9hZGRyIGh1cmRsZSB3ZSBk ZXBpY3RlZCBhYm92ZSB3cnQgdG8gdXNpbmcgdXNpbmcgZ3JhbnQKPj4+IG1hcHMvdW5tYXBzLiBZ b3UgdGhpbmsgc29tZSBvZiB0aGUgeGVuaG9zdCBsYXllciB3b3VsZCBiZSBhbW1lbmFibGUgdG8g c3VwcG9ydAo+Pj4gdGhpcyBjYXNlPwo+Pgo+PiBJIHRoaW5rIHNvLCB5ZXMuCj4+Cj4+Pgo+Pj4+ IFNvIGV4YWN0bHkgdGhlIGtpbmQgb2Ygc3R1ZmYgeW91IHdhbnQgdG8gZG8sIHRvby4KPj4+Pgo+ Pj4gQ29vbCBpZGVhIQo+Pgo+PiBJbiB0aGUgZW5kIHlvdSBtaWdodCBtYWtlIG15IGxpZmUgZWFz aWVyIGZvciBuZXN0ZWQgWGVuLiA6LSkKPj4KPiBIZWhlIDopCj4gCj4+IERvIHlvdSB3YW50IHRv IGhhdmUgYSB0cnkgd2l0aCB0aGF0IGlkZWEgb3Igc2hvdWxkIEkgZG8gdGhhdD8gSSBtaWdodCBi ZQo+PiBhYmxlIHRvIHN0YXJ0IHdvcmtpbmcgb24gdGhhdCBpbiBhYm91dCBhIG1vbnRoLgo+Pgo+ IEFua3VyIChDQydlZCkgd2lsbCBnaXZlIGEgc2hvdCBhdCBpdCwgYW5kIHNob3VsZCBzdGFydCBh IG5ldyB0aHJlYWQgb24gdGhpcwo+IHhlbmhvc3QgYWJzdHJhY3Rpb24gbGF5ZXIuCgpHcmVhdCwg bG9va2luZyBmb3J3YXJkIHRvIGl0IQoKCkp1ZXJnZW4KCl9fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fClhlbi1kZXZlbCBtYWlsaW5nIGxpc3QKWGVuLWRldmVs QGxpc3RzLnhlbnByb2plY3Qub3JnCmh0dHBzOi8vbGlzdHMueGVucHJvamVjdC5vcmcvbWFpbG1h bi9saXN0aW5mby94ZW4tZGV2ZWw=