From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3566FC43381 for ; Mon, 11 Mar 2019 16:31:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0B09321738 for ; Mon, 11 Mar 2019 16:31:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727686AbfCKQbi (ORCPT ); Mon, 11 Mar 2019 12:31:38 -0400 Received: from szxga06-in.huawei.com ([45.249.212.32]:34232 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726675AbfCKQbi (ORCPT ); Mon, 11 Mar 2019 12:31:38 -0400 Received: from DGGEMS408-HUB.china.huawei.com (unknown [172.30.72.59]) by Forcepoint Email with ESMTP id 405269064B4B43CFC3F3; Tue, 12 Mar 2019 00:31:36 +0800 (CST) Received: from [127.0.0.1] (10.177.29.32) by DGGEMS408-HUB.china.huawei.com (10.3.19.208) with Microsoft SMTP Server id 14.3.408.0; Tue, 12 Mar 2019 00:31:29 +0800 To: , , , , , CC: , , , Wang Haibin , "yuzenghui@huawei.com" , , From: Zheng Xiang Subject: [RFC] Question about TLB flush while set Stage-2 huge pages Message-ID: <5f712cc6-0874-adbe-add6-46f5de24f36f@huawei.com> Date: Tue, 12 Mar 2019 00:31:10 +0800 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:64.0) Gecko/20100101 Thunderbird/64.0 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Language: en-US Content-Transfer-Encoding: 7bit X-Originating-IP: [10.177.29.32] X-CFilter-Loop: Reflected Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi all, While a page is merged into a transparent huge page, KVM will invalidate Stage-2 for the base address of the huge page and the whole of Stage-1. However, this just only invalidates the first page within the huge page and the other pages are not invalidated, see bellow: +---------------+--------------+ |abcde 2MB-Page | +---------------+--------------+ TLB before setting new pmd: +---------------+--------------+ | VA | PAGESIZE | +---------------+--------------+ | a | 4KB | +---------------+--------------+ | b | 4KB | +---------------+--------------+ | c | 4KB | +---------------+--------------+ | d | 4KB | +---------------+--------------+ TLB after setting new pmd: +---------------+--------------+ | VA | PAGESIZE | +---------------+--------------+ | a | 2MB | +---------------+--------------+ | b | 4KB | +---------------+--------------+ | c | 4KB | +---------------+--------------+ | d | 4KB | +---------------+--------------+ When VM access *b* address, it will hit the TLB and result in TLB conflict aborts or other potential exceptions. For example, we need to keep tracking of the VM memory dirty pages when VM is in live migration. KVM will set the memslot READONLY and split the huge pages. After live migration is canceled and abort, the pages will be merged into THP. The later access to these pages which are READONLY will cause level-3 Permission Fault until they are invalidated. So should we invalidate the tlb entries for all relative pages(e.g a,b,c,d), like __flush_tlb_range()? Or we can call __kvm_tlb_flush_vmid() to invalidate all tlb entries. -- Thanks, Xiang From mboxrd@z Thu Jan 1 00:00:00 1970 From: Zheng Xiang Subject: [RFC] Question about TLB flush while set Stage-2 huge pages Date: Tue, 12 Mar 2019 00:31:10 +0800 Message-ID: <5f712cc6-0874-adbe-add6-46f5de24f36f@huawei.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id BA4294A3FE for ; Mon, 11 Mar 2019 12:31:41 -0400 (EDT) Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9OYbyGkz6HbU for ; Mon, 11 Mar 2019 12:31:39 -0400 (EDT) Received: from huawei.com (szxga06-in.huawei.com [45.249.212.32]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id 82E554A321 for ; Mon, 11 Mar 2019 12:31:39 -0400 (EDT) Content-Language: en-US List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu To: christoffer.dall@arm.com, marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, suzuki.poulose@arm.com, james.morse@arm.com Cc: lishuo1@hisilicon.com, linux-kernel@vger.kernel.org, "yuzenghui@huawei.com" , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, lious.lilei@hisilicon.com List-Id: kvmarm@lists.cs.columbia.edu Hi all, While a page is merged into a transparent huge page, KVM will invalidate Stage-2 for the base address of the huge page and the whole of Stage-1. However, this just only invalidates the first page within the huge page and the other pages are not invalidated, see bellow: +---------------+--------------+ |abcde 2MB-Page | +---------------+--------------+ TLB before setting new pmd: +---------------+--------------+ | VA | PAGESIZE | +---------------+--------------+ | a | 4KB | +---------------+--------------+ | b | 4KB | +---------------+--------------+ | c | 4KB | +---------------+--------------+ | d | 4KB | +---------------+--------------+ TLB after setting new pmd: +---------------+--------------+ | VA | PAGESIZE | +---------------+--------------+ | a | 2MB | +---------------+--------------+ | b | 4KB | +---------------+--------------+ | c | 4KB | +---------------+--------------+ | d | 4KB | +---------------+--------------+ When VM access *b* address, it will hit the TLB and result in TLB conflict aborts or other potential exceptions. For example, we need to keep tracking of the VM memory dirty pages when VM is in live migration. KVM will set the memslot READONLY and split the huge pages. After live migration is canceled and abort, the pages will be merged into THP. The later access to these pages which are READONLY will cause level-3 Permission Fault until they are invalidated. So should we invalidate the tlb entries for all relative pages(e.g a,b,c,d), like __flush_tlb_range()? Or we can call __kvm_tlb_flush_vmid() to invalidate all tlb entries. -- Thanks, Xiang From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8EE0C43381 for ; Mon, 11 Mar 2019 16:31:55 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A846F21741 for ; Mon, 11 Mar 2019 16:31:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="WJGO3Xg+" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A846F21741 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Date:Message-ID:Subject: From:To:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=fY7Z/27GSlOuevcqixUTfJsMyxHG7Ym+cib99MX4UC0=; b=WJGO3Xg+TBltlt 8bEBInBpgjepnEVPcHREUMrrEcW+lcxU5sV82joEJWlRSr1gDAREgoQ5lRG2uq1l8A7Qml4ipt6Hk at/snwPABY+bsv6Qd0M9X+Y1Ecty46Qw/DtRduk0ehsBAHuM0Gauq2fHq9hHGVObqbI2kQDr9Ai8u ntKowwcritwRUqtlGFwTHNObnU3cNb2gZc2bNTiFUkgtoIINjsPtnD2hhhwSMqW2CCxUfNZ3A7gBq kMtWInuP0G+CcfNXAz1CjI/B+U12ckKG6PfdUgkPP/zk6M3s48rRgJ22Cw4/RN2QnY9QIbPAhuX4y FedCYZLtfGc+HLmyjPbg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1h3NqA-0007nX-G9; Mon, 11 Mar 2019 16:31:50 +0000 Received: from szxga06-in.huawei.com ([45.249.212.32] helo=huawei.com) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1h3Nq6-0007mq-Lh for linux-arm-kernel@lists.infradead.org; Mon, 11 Mar 2019 16:31:48 +0000 Received: from DGGEMS408-HUB.china.huawei.com (unknown [172.30.72.59]) by Forcepoint Email with ESMTP id 405269064B4B43CFC3F3; Tue, 12 Mar 2019 00:31:36 +0800 (CST) Received: from [127.0.0.1] (10.177.29.32) by DGGEMS408-HUB.china.huawei.com (10.3.19.208) with Microsoft SMTP Server id 14.3.408.0; Tue, 12 Mar 2019 00:31:29 +0800 To: , , , , , From: Zheng Xiang Subject: [RFC] Question about TLB flush while set Stage-2 huge pages Message-ID: <5f712cc6-0874-adbe-add6-46f5de24f36f@huawei.com> Date: Tue, 12 Mar 2019 00:31:10 +0800 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:64.0) Gecko/20100101 Thunderbird/64.0 MIME-Version: 1.0 Content-Language: en-US X-Originating-IP: [10.177.29.32] X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190311_093147_391529_B2F33C19 X-CRM114-Status: UNSURE ( 6.61 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lishuo1@hisilicon.com, linux-kernel@vger.kernel.org, "yuzenghui@huawei.com" , Wang Haibin , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, lious.lilei@hisilicon.com Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi all, While a page is merged into a transparent huge page, KVM will invalidate Stage-2 for the base address of the huge page and the whole of Stage-1. However, this just only invalidates the first page within the huge page and the other pages are not invalidated, see bellow: +---------------+--------------+ |abcde 2MB-Page | +---------------+--------------+ TLB before setting new pmd: +---------------+--------------+ | VA | PAGESIZE | +---------------+--------------+ | a | 4KB | +---------------+--------------+ | b | 4KB | +---------------+--------------+ | c | 4KB | +---------------+--------------+ | d | 4KB | +---------------+--------------+ TLB after setting new pmd: +---------------+--------------+ | VA | PAGESIZE | +---------------+--------------+ | a | 2MB | +---------------+--------------+ | b | 4KB | +---------------+--------------+ | c | 4KB | +---------------+--------------+ | d | 4KB | +---------------+--------------+ When VM access *b* address, it will hit the TLB and result in TLB conflict aborts or other potential exceptions. For example, we need to keep tracking of the VM memory dirty pages when VM is in live migration. KVM will set the memslot READONLY and split the huge pages. After live migration is canceled and abort, the pages will be merged into THP. The later access to these pages which are READONLY will cause level-3 Permission Fault until they are invalidated. So should we invalidate the tlb entries for all relative pages(e.g a,b,c,d), like __flush_tlb_range()? Or we can call __kvm_tlb_flush_vmid() to invalidate all tlb entries. -- Thanks, Xiang _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel