From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C44A2C433F5 for ; Fri, 24 Sep 2021 20:22:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 65DF361100 for ; Fri, 24 Sep 2021 20:22:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 65DF361100 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id D48CA900002; Fri, 24 Sep 2021 16:22:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CF8E66B0072; Fri, 24 Sep 2021 16:22:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BE7F5900002; Fri, 24 Sep 2021 16:22:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0247.hostedemail.com [216.40.44.247]) by kanga.kvack.org (Postfix) with ESMTP id B26916B0071 for ; Fri, 24 Sep 2021 16:22:33 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 763D41802ED69 for ; Fri, 24 Sep 2021 20:22:33 +0000 (UTC) X-FDA: 78623589786.05.C82F71B Received: from mail-wr1-f49.google.com (mail-wr1-f49.google.com [209.85.221.49]) by imf12.hostedemail.com (Postfix) with ESMTP id 3391E10000A8 for ; Fri, 24 Sep 2021 20:22:33 +0000 (UTC) Received: by mail-wr1-f49.google.com with SMTP id w29so30784986wra.8 for ; Fri, 24 Sep 2021 13:22:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=P6UjX46EgO1JdvaA1S99BM8bVjWyLM+0KRRhsNVfqrc=; b=hNilYYi9DfGN1Dx6/MtN058s8ZQ30CNP0PCqpqvXCmeNiuOhFNazMI2RkNfZif6nAu rGIBi3W2uxP6WNpia+1cWs6TX03mE6nmeXC+3L1ns31tp1llFuebuAA/v7XkUejdwrwh V845bcchNr9ZOJjCFBOLaVkMIQCvhHcHNVA0BWSfr3cb4pSvlkXIrj0zYGbj6eBeE3Pt zGHYlYYrUxJJNdVUDPgyAaFeOV7d6ouoZgJXW5xwoMszebmsbXDdOg4LAWDzhss5Tnm2 r1TtraxcgioXJZ3Y+EpqAFcp2mmQi0NNeJNQ+tXpLR/y+DgFSipRMQNc/YZLV9BYxfj9 ye/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=P6UjX46EgO1JdvaA1S99BM8bVjWyLM+0KRRhsNVfqrc=; b=HsyBbFdolBELEbPM5hjFrev9xUKT6mS43/Fp//dAnHxG43bAm3BiUByrI5p2GzXqCx peIBWkulCwhMojD5FCDn5XqCJR0cUYppQ9b2LHKP/Vw+qkJaLl72D/ROaGrNn6XkGqhG jmltzYvbHiQ6atlKHK+D8L/LOoM9P2Ktddk1VjA1xvKqL60E9Q7Gq1l5hqWlAoGLqJfo 8NAeprSp5GO1SyswG/1iw0E9bjZbMOjr027hRn/H+D3KMow0puB7yVhAI2A52eZqFn8X AWXppUZqFVn9W1B55ylePe5n69SEvnAxMRXhM8dmbd1ecncxnfbCiXrebIOPCSi/ap+U HSTA== X-Gm-Message-State: AOAM530+cyry/Li/8qdixR6GaGIMSWengMBIwWqM7zlEI14NRFI7fNL2 5jeQJBqUuaTZB9SA+JhRwbN7kKVn284mO+gWRW1OZg== X-Google-Smtp-Source: ABdhPJwmZTsxjXTVscp7BYpW00OnOGQJJcYUcorRFaAk4dC1Hn0zxo5UNELpTtJfZBek6h0ROSX+WX77q3jllKbjarY= X-Received: by 2002:adf:f48e:: with SMTP id l14mr13662271wro.109.1632514951618; Fri, 24 Sep 2021 13:22:31 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Jue Wang Date: Fri, 24 Sep 2021 13:22:19 -0700 Message-ID: Subject: Re: [PATCH 1/3] userfaultfd/selftests: fix feature support detection To: Peter Xu Cc: James Houghton , Axel Rasmussen , Andrew Morton , Shuah Khan , Linux MM , Linuxkselftest , LKML Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 3391E10000A8 X-Stat-Signature: bzerbzpadjwb1txuczai585df3mt5hmn Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=hNilYYi9; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf12.hostedemail.com: domain of juew@google.com designates 209.85.221.49 as permitted sender) smtp.mailfrom=juew@google.com X-HE-Tag: 1632514953-564952 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Sep 24, 2021 at 1:09 PM Peter Xu wrote: > > On Wed, Sep 22, 2021 at 10:43:40PM -0700, Jue Wang wrote: > > [...] > > > > > Could I know what's the workaround? Normally if the workaround works solidly, > > > > then there's less need to introduce a kernel interface for that. Otherwise I'm > > > > glad to look into such a formal proposal. > > > > > > The workaround is, for the region that you want to zap, run through > > > this sequence of syscalls: mumap, mmap, and re-register with > > > userfaultfd if it was registered before. If we're using tmpfs, we can > > > use madvise(DONTNEED) instead, but this is kind of an abuse of the > > > API. I don't think there's a guarantee that the PTEs will get zapped, > > > but currently they will always get zapped if we're using tmpfs. I > > > really like the idea of adding a new madvise() mode that is guaranteed > > > to zap the PTEs. > > I see. > > > > > > > > > > > > > It's also useful for memory poisoning, I think, if the host > > > > > decides some page(s) are "bad" and wants to intercept any future guest > > > > > accesses to those page(s). > > > > > > > > Curious: isn't hwpoison information come from MCEs; or say, host kernel side? > > > > Then I thought the host kernel will have full control of it already. > > > > > > > > Or there's other way that the host can try to detect some pages are going to be > > > > rotten? So the userspace can do something before the kernel handles those > > > > exceptions? > > > > > > Here's a general idea of how we would like to use userfaultfd to support MPR: > > > > > > If a guest accesses a poisoned page for the first time, we will get an > > > MCE through the host kernel and send an MCE to the guest. The guest > > > will now no longer be able to access this page, and we have to enforce > > > this. After a live migration, the pages that were poisoned before > > > probably won't still be poisoned (from the host's perspective), so we > > > can't rely on the host kernel's MCE handling path. This is where > > > userfaultfd and this new madvise mode come in: we can just > > > madvise(MADV_ZAP) the poisoned page(s) on the target during a > > > migration. Now all accesses will be routed to the VMM and we can > > > inject an MCE. We don't *need* the new madvise mode, as we can also > > > use fallocate(PUNCH_HOLE) (works for tmpfs and hugetlbfs), but it > > > would be more convenient if we didn't have to use fallocate. > > > > > > Jue Wang can provide more context here, so I've cc'd him. There may be > > > some things I'm wrong about, so Jue feel free to correct me. > > > > > James is right. > > > > The page is marked PG_HWPoison in the source VM host's kernel. The need > > of intercepting guest accesses to it exist on the target VM host, where > > the same physical page is no longer poisoned. > > > > On the target host, the hypervisor needs to intercept all guest accesses > > to pages poisoned from the source VM host. > > Thanks for these information, James, Jue, Axel. I'm not familiar with memory > failures yet, so please bare with me with a few naive questions. > > So now I can undertand that hw-poisonsed pages on src host do not mean these > pages will be hw-poisoned on dest host too, but I may have missed the reason on > why dest host needs to trap it with pgtable removed. > > AFAIU after pages got hw-poisoned on src, and after vmm injects MCEs into the > guest, the guest shouldn't be accessing these pages any more, am I right? Then This is also our hope for the guest to behave but there is no guarantee on guest behavior. The goal here is to have the hypervisor provide consistent behavior aligned to native hardware, i.e., a guest page with "memory errors" stay persistently in that state no matter on source or target. > after migration completes, IIUC the guest shouldn't be accessing these pages > too. My current understanding is, instead of trapping these pages on dest, we > should just (somehow, which I have no real idea...) un-hw-poison these pages > after migration because these pages are very possibly normal pages there. When > there's real hw-poisoned pages reported on dst host, we should re-inject MCE > errors to guest with another set of pages. > > Could you tell me where did I miss? > > Thanks, > > -- > Peter Xu >