From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=0.3 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA77EC433E0 for ; Mon, 22 Feb 2021 18:53:35 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7EDD760232 for ; Mon, 22 Feb 2021 18:53:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7EDD760232 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.88348.166074 (Exim 4.92) (envelope-from ) id 1lEGKU-0006X8-58; Mon, 22 Feb 2021 18:53:10 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 88348.166074; Mon, 22 Feb 2021 18:53:10 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lEGKU-0006X1-22; Mon, 22 Feb 2021 18:53:10 +0000 Received: by outflank-mailman (input) for mailman id 88348; Mon, 22 Feb 2021 18:53:09 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lEGKT-0006Ww-At for xen-devel@lists.xenproject.org; Mon, 22 Feb 2021 18:53:09 +0000 Received: from mail-oi1-x233.google.com (unknown [2607:f8b0:4864:20::233]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 0d77ed82-a3aa-4b5b-9e2f-e3ad4407d926; Mon, 22 Feb 2021 18:53:08 +0000 (UTC) Received: by mail-oi1-x233.google.com with SMTP id f3so15004157oiw.13 for ; Mon, 22 Feb 2021 10:53:08 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0d77ed82-a3aa-4b5b-9e2f-e3ad4407d926 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=f1puqYGuQ0MULhYXk50f1Ja3v9r3HCUU4QrM/Mkm1gI=; b=f7q9AE0VqTXmXXz461I2PttTbNs3JNA/Jf4uQEHRhhjx0fZq23GF1nOT4PySEM81Bh 1JHgqnw8x9Jzrr6aThel9//jFsTG+KgJEGeuIg9wCYE83pgi0fP1ca1zh5uaMmbMnTZW M1CcgQP5Ls+b5h1DxH03SIeJPOFjfkGRa7HMx3JaNujHs9WSk5lIPOYwaHUBIjT9wjrH heXpSV5fnwGk8jt/iiea2fKNdePWCF3EmPCGmgUIaBTalmNErgqF6vY4oWtwa5y/j1Cx KtTtCK0qWKI6IIai1Te7n3HEPjJxHTR9u5vOG6/UIgO8e3WBRRJy3oVkbr20V1stMzHo ojFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=f1puqYGuQ0MULhYXk50f1Ja3v9r3HCUU4QrM/Mkm1gI=; b=sIk2kFRyJJcXujUWC7ZObbjV5OQQ5TDpN0KI+KDmYF0ADf6eAszkP4y9b9GAvRWfj1 dXYXYyMwvc+l7FOw6yeAgIvjafrj7vcOiHqaEgdiKfaq3pYvuYjnGR6l0KM4DhmNxoul 0rB2G24hcfaES7DYFGzWo68ZtLsyS3lDgqNRxxgURZcaP34Jl7j+yUx74+hN6ltkH5xC lLRqz89+8BOMExxsAGCEa1H0Xv8FpU0xtuguZj1vKoiQSdCtT74uXsuYIVk6IHhGETvU rSXjiskKr/UXvnpixHesabNZYrE7SOnykmGWkHGgQygX1kXDBsIFhNd3bpbvfvYbS1po VLnQ== X-Gm-Message-State: AOAM531ZQunZ1kNy9AKlsZU6iMvUwCxOYFOIEWEtK12kjqqO4Evh9sbP CX1jQTcfWkMmXDE854eGomNSDuR7ixFKvGXhsWIv9aioKTNZ X-Google-Smtp-Source: ABdhPJwZWWWvlTzS20caJnqhNysg3K9xiCXBWFkYWpvpdOUsDn5312T+Xf+kHOa0kIr+OoLW4JkEtDT1h2AK+fpyx+U= X-Received: by 2002:aca:5954:: with SMTP id n81mr16469325oib.25.1614019987627; Mon, 22 Feb 2021 10:53:07 -0800 (PST) MIME-Version: 1.0 References: <38cf0d39-da1d-5375-89dc-1668e26323a2@citrix.com> In-Reply-To: <38cf0d39-da1d-5375-89dc-1668e26323a2@citrix.com> From: Kevin Negy Date: Mon, 22 Feb 2021 13:52:31 -0500 Message-ID: Subject: Re: How does shadow page table work during migration? To: Andrew Cooper Cc: xen-devel@lists.xenproject.org Content-Type: multipart/alternative; boundary="00000000000005e3f605bbf150f0" --00000000000005e3f605bbf150f0 Content-Type: text/plain; charset="UTF-8" Hello again, Thank you for the helpful responses. I have several follow up questions. 1) > With Shadow, Xen has to do the combination of address spaces itself - > the shadow pagetables map guest virtual to host physical address. The shadow_blow_tables() call is "please recycle everything" which is used > to throw away all shadow pagetables, which in turn will cause the > shadows to be recreated from scratch as the guest continues to run. With shadowing enabled, given a guest virtual address, how does the hypervisor recreate the mapping to the host physical address (mfn) from the virtual address if the shadow page tables are empty (after a call to shadow_blow_tables, for instance)? I had been thinking of shadow page tables as the definitive mapping between guest pages and machine pages, but should I think of them as more of a TLB, which implies there's another way to get/recreate the mappings if there's no entry in the shadow table? 2) I'm trying to grasp the general steps of enabling shadowing and handling page faults. Is this correct? a) Normal PV - default shadowing is disabled, guest has its page tables in r/w mode or whatever mode is considered normal for guest page tables b) Shadowing is enabled - shadow memory pool allocated, all memory accesses must now go through shadow pages in CR3. Since no entries are in shadow tables, initial read and writes from the guest will result in page faults. c) As soon as the first guest memory access occurs, a mandatory page fault occurs because there is no mapping in the shadows. Xen does a guest page table walk for the address that caused the fault (va) and then marks all the guest page table pages along the walk as read only. d) Xen finds out the mfn of the guest va somehow (my first question) and adds the mapping of the va to the shadow page table. e) If the page fault was a write, the va is now marked as read/write but logged as dirty in the logdirty map. e) Now the next page fault to any of the page tables marked read-only in c) must have been caused by the guest writing to its tables, which can be reflected in the shadow page tables. 3) How do Xen/shadow page tables distinguish between two equivalent guest virtual addresses from different guest processes? I suppose when a guest OS tries to change page tables from one process to another, this will cause a page fault that Xen will trap and be able to infer that the current shadow page table should be swapped to a different one corresponding to the new guest process? Thank you so much, Kevin --00000000000005e3f605bbf150f0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hello again,

Thank you for the hel= pful responses. I have several follow up questions.

1)
With Shadow, Xen has to do the co= mbination of address spaces itself -
the shadow pagetables map guest virtual to host physical address.

The sha= dow_blow_tables() call is "please recycle everything" which is us= ed
to throw away all shadow pagetables, which in turn will cause the
shadow= s to be recreated from scratch as the guest continues to run.=C2=A0
With shadowing enabled, given a guest virtual address, how does th= e hypervisor recreate the mapping to the host physical address (mfn) from t= he virtual address if the shadow page tables are empty (after a call to sha= dow_blow_tables, for instance)? I had been thinking of shadow page tables a= s the definitive mapping between guest pages and machine pages, but should = I think of them as more of a TLB, which implies there's another way to = get/recreate the mappings if there's no entry in the shadow table?


2) I'm trying to grasp the general steps of e= nabling shadowing and handling page faults. Is this correct?
=C2=A0 =C2= =A0 a) Normal PV - default shadowing is disabled, guest has its page tables= in r/w mode or whatever mode is considered normal for guest page tables=C2=A0 =C2=A0 b) Shadowing is enabled - shadow memory pool allocated, all = memory accesses must now go through shadow pages in CR3. Since no entries a= re in shadow tables, initial read and writes from the guest will result in = page faults.
=C2=A0 =C2=A0 c) As soon as the first guest memory access o= ccurs, a mandatory page fault occurs because there is no mapping in the sha= dows. Xen does a guest page table walk for the address that caused the faul= t (va) and then marks all the guest page table pages along the walk as read= only.
=C2=A0 =C2=A0 d) Xen finds out the mfn of the guest va somehow (= my first question) and adds the mapping of the va to the shadow page table.=
=C2=A0 =C2=A0 e) If the page fault was a write, the va is now marked a= s read/write but logged as dirty in the logdirty map.
=C2=A0 =C2=A0 e) N= ow the next page fault to any of the page tables marked read-only in c) mus= t have been caused by the guest writing to its tables, which can be reflect= ed in the shadow page tables.
=C2=A0 =C2=A0

3) How do Xen/shadow= page tables distinguish between two equivalent guest virtual addresses fro= m different guest processes? I suppose when a guest OS tries to change page= tables from one process to another, this will cause a page fault that Xen = will trap and be able to infer that the current shadow page table should be= swapped to a different one corresponding to the new guest process?

Thank you so much,
Kevin
<= br>
--00000000000005e3f605bbf150f0--