From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5ABBAC43381 for ; Thu, 28 Mar 2019 11:43:15 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2845920700 for ; Thu, 28 Mar 2019 11:43:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="erO320g0" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2845920700 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Content-Type: Content-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=gsnPwc8H1sihpNCJvoPMhrcxtAundyhBQaO9o0a3UFo=; b=erO320g0NtAB4o34GR0Fuxglz R6RNV9lmJ51+mKfndy0CPTvdhUS410Wb/axOvXdwrDi5Td3BvHJX1qxpLzGeD4w7FJTd2sX0LqtBz P3tbcNgKKMfW3EZiBXThnJ6E3FiVXA4PujzGlL9Frv4RaFdqO5ez2X8+kk5i4bzw9gyqnkuEISMmG vtLVvVPMzUURNwQxJcY2q3Au0s2zVwBCozu4sD69RsySJNpmMhbUoyPM3gRTGyVFOlG1Sb84Z3qIA InDSnIcGG8t8fPBMZ5RenEXL5WFj7L94a7LVFNa3JK6ODBH6/Qs62RoFvgkDNcifZHm5N2gfs5vc7 de2wcVi/g==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1h9TR8-0001sh-6S; Thu, 28 Mar 2019 11:43:10 +0000 Received: from mail-pl1-f195.google.com ([209.85.214.195]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1h9TR1-0001rO-RR for linux-arm-kernel@lists.infradead.org; Thu, 28 Mar 2019 11:43:07 +0000 Received: by mail-pl1-f195.google.com with SMTP id y6so4848170pll.13 for ; Thu, 28 Mar 2019 04:43:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=mX3ZNmbz3E6bL5fB7CSied1YiiuhskRa10DQGDO03lU=; b=jyOiC37bwj4ifmgq8qMXMicA7GfrK/elFEZ3MSIls/TGkAa7mcgVAAz/l+a256IRQG CYSGjL4gwHp+qEg5ZandKcFtalNaTTaPYLKK+aISXAiwF8y2kUwyuucMSUdSzcVWKXn3 Vupkim7AImDxJE9bZrsDmq4BSIRoHFUVVFp4uNMT7Fd/2bGLnWnCNVgw/sTMT7/P2o4G jdQgeFexy6MTpcFcp0+JT7CuBkOlcU7RZVG5XVfjMYaT92qJfdAAjH1/02/rwf7ukeLD zMnU1XF/bwiMr818z/7O4IkFYfdl3T6q2igGErVkReM0+vU3L9FeWyjHmEHrZPskiqab XKGA== X-Gm-Message-State: APjAAAU96a7mFJmN/txiWgZXIxYKTKHWgRcdSb/mv/H62nRsxZZA0+S0 H+pLnfxlLbXv2YaAMMR1AL94VQ== X-Google-Smtp-Source: APXvYqx2A/TMNggwE78YrfUH0fDr/fcDZP/WCS0MCJdq8x+BUtnpTpTABRNJYX9aZ5YVEP9gLm832w== X-Received: by 2002:a17:902:bb92:: with SMTP id m18mr25270650pls.316.1553773382483; Thu, 28 Mar 2019 04:43:02 -0700 (PDT) Received: from localhost.localdomain ([182.69.197.66]) by smtp.gmail.com with ESMTPSA id i24sm19437197pfo.85.2019.03.28.04.42.57 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 28 Mar 2019 04:43:01 -0700 (PDT) Subject: Re: [PATCH v3 1/3] arm64, vmcoreinfo : Append 'PTRS_PER_PGD' to vmcoreinfo To: James Morse References: <1553058574-18606-1-git-send-email-bhsharma@redhat.com> <1553058574-18606-2-git-send-email-bhsharma@redhat.com> <2757805b-61cb-8f4a-1917-0c57012f09df@arm.com> From: Bhupesh Sharma Message-ID: <58c6cda9-9fd4-3b3e-740a-7b9b80b1f634@redhat.com> Date: Thu, 28 Mar 2019 17:12:54 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: <2757805b-61cb-8f4a-1917-0c57012f09df@arm.com> Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190328_044303_888243_6F00FBCA X-CRM114-Status: GOOD ( 31.39 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Kazuhito Hagio , Steve Capper , kexec@lists.infradead.org, Will Deacon , linux-kernel@vger.kernel.org, Dave Anderson , bhupesh.linux@gmail.com, linux-arm-kernel@lists.infradead.org Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi James, Thanks for your review. Please see my comments inline: On 03/26/2019 10:06 PM, James Morse wrote: > Hi Bhupesh, > > On 20/03/2019 05:09, Bhupesh Sharma wrote: >> With ARMv8.2-LVA architecture extension availability, arm64 hardware >> which supports this extension can support a virtual address-space upto >> 52-bits. >> >> Since at the moment we enable the support of this extension in kernel >> via CONFIG flags, e.g. >> - User-space 52-bit LVA via CONFIG_ARM64_USER_VA_BITS_52 >> >> so, there is no clear mechanism in the user-space right now to >> determine these CONFIG flag values and hence determine the maximum >> virtual address space supported by the underlying kernel. >> >> User-space tools like 'makedumpfile' therefore are broken currently >> as they have no proper method to calculate the 'PTRS_PER_PGD' value >> which is required to perform a page table walk to determine the >> physical address of a corresponding virtual address found in >> kcore/vmcoreinfo. >> >> If one appends 'PTRS_PER_PGD' number to vmcoreinfo for arm64, >> it can be used in user-space to determine the maximum virtual address >> supported by underlying kernel. > > I don't think this really solves the problem, it feels fragile. > > I can see how vmcoreinfo tells you VA_BITS==48, PAGE_SIZE==64K and PTRS_PER_PGD=1024. > You can use this to work out that the top level page table size isn't consistent with a > 48bit VA, so 52bit VA must be in use... > > But wasn't your problem walking the kernel page tables? In particular the offset that we > apply because the tables were based on a 48bit VA shifted up in swapper_pg_dir. > > Where does the TTBR1_EL1 offset come from with this property? I assume makedumpfile > hard-codes it when it sees 52bit is in use ... somewhere. > We haven't solved the problem! But isn't the TTBR1_EL1 offset already appended by the kernel via e842dfb5a2d3 ("arm64: mm: Offset TTBR1 to allow 52-bit PTRS_PER_PGD") in case of kernel configuration where 52-bit userspace VAs are possible. I copy a snippet from the git log of the above from Steve, which explains the above: " In other words a 48-bit kernel virtual address will have a different pgd_index when using PTRS_PER_PGD = 64 and 1024. If, however, we note that: kva = 0xFFFF << 48 + lower (where lower[63:48] == 0b) and, PGDIR_SHIFT = 42 (as we are dealing with 64KB PAGE_SIZE) We can consider: (kva >> PGDIR_SHIFT) & (1024 - 1) - (kva >> PGDIR_SHIFT) & (64 - 1) = (0xFFFF << 6) & 0x3FF - (0xFFFF << 6) & 0x3F // "lower" cancels out = 0x3C0 In other words, one can switch PTRS_PER_PGD to the 52-bit value globally provided that they increment ttbr1_el1 by 0x3C0 * 8 = 0x1E00 bytes when running with 48-bit kernel VAs (TCR_EL1.T1SZ = 16). For kernel configuration where 52-bit userspace VAs are possible, this patch offsets ttbr1_el1 and sets PTRS_PER_PGD corresponding to the 52-bit value. " Accordingly we have the following assembler helper in 'arch/arm64/include/asm/assembler.h': .macro offset_ttbr1, ttbr #ifdef CONFIG_ARM64_52BIT_VA orr \ttbr, \ttbr, #TTBR1_BADDR_4852_OFFSET #endif .endm where: #ifdef CONFIG_ARM64_52BIT_VA /* Must be at least 64-byte aligned to prevent corruption of the TTBR */ #define TTBR1_BADDR_4852_OFFSET (((UL(1) << (52 - PGDIR_SHIFT)) - \ (UL(1) << (48 - PGDIR_SHIFT))) * 8) #endif And before any load TTBR1 operation in the kernel we offset ttbr1_el1, in case CONFIG_ARM64_52BIT_VA is true, for e.g in 'arch/arm64/kernel/head.S': ENTRY(__enable_mmu) <..snip..> offset_ttbr1 x1 msr ttbr1_el1, x1 // load TTBR1 <..snip..> So, the user-space (makedumpfile, for e.g.), just needs to know the PTRS_PER_PGD value and it can calculate the pgd_index for a virtual address using the following formula: pgd_index(vaddr) = (((vaddr) >> PGDIR_SHIFT) & (PTRS_PER_PGD - 1)); Note that the above computation holds true both for PTRS_PER_PGD = 64 (48-bit kernel with 48-bit User VA) and 1024 (48-bit with 52-bit User VA) cases. And these are the configurations for which we are trying to fix the user-space regressions reported (on arm64) recently. > Today __cpu_setup() sets T0SZ and T1SZ differently for 52bit VA, but in the future it > could set them the same, or different the other-way-round. > > Will makedumpfile using this value keep working once T1SZ is 52bit VA too? In this case > there would be no ttbr offset. > > If you need another vmcoreinfo flag once that happens, we've done something wrong here. I am currently experimenting with Steve's patches for 52-bit kernel VA () and will comment more on the same when I am able to get the user-space utilities like makedumpfile and kexec-tools to work with the same on both ARMv8 Fast Simulator model and older CPUs which don't support ARMv8.2 extensions. However, I think we should not hold up fixes for regressions already reported, because the 52-bit kernel VA changes probably still need some more rework. > (Not to mention what happens if the TTBR1_EL1 uses 52bit va, but TTBR0_EL1 doesn't) I am wondering if there are any real users of the above combination. So far, I have generally come across discussions where the following variations of the address spaces have been proposed/requested: - 48bit kernel VA + 48-bit User VA, - 48-bit kernel VA + 52-bit User VA, - 52-bit kernel VA + 52-bit User VA. Thanks, Bhupesh >> Suggested-by: Steve Capper > > (CC: +Steve) > > > Thanks, > > James > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel