From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07084C433E1 for ; Fri, 26 Jun 2020 22:31:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B2E572088E for ; Fri, 26 Jun 2020 22:31:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="SFzF4f30" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B2E572088E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0D4786B0008; Fri, 26 Jun 2020 18:31:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 00CC66B000A; Fri, 26 Jun 2020 18:31:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DEE456B000C; Fri, 26 Jun 2020 18:31:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0098.hostedemail.com [216.40.44.98]) by kanga.kvack.org (Postfix) with ESMTP id B53C86B000A for ; Fri, 26 Jun 2020 18:31:39 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 75E83180AD806 for ; Fri, 26 Jun 2020 22:31:39 +0000 (UTC) X-FDA: 76972811118.25.can12_1f0a3c826e59 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin25.hostedemail.com (Postfix) with ESMTP id 2EE231804E3A8 for ; Fri, 26 Jun 2020 22:31:39 +0000 (UTC) X-HE-Tag: can12_1f0a3c826e59 X-Filterd-Recvd-Size: 10599 Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by imf29.hostedemail.com (Postfix) with ESMTP for ; Fri, 26 Jun 2020 22:31:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593210697; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=6x/m22vLH3eEyRQboy/3NjalJn5cZPkkagnVGM9ijxA=; b=SFzF4f30Xvlzc4RsxWetLY4c6oI/xzipeEUWaPMNz9/Ti2ijNqpT5jOLxisu7HH/CM9BU6 9R4AT/pL9MHXuFDg/yaN99RZsjzaIFq19uA1/wJMVi+m21ikfYafnqd9riQKmhl/h59Ciy AN5ZiH7U5QNLAqEl9D/n9WctC7U4Vs4= Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-16-F5lltJ4LPKGnm0mnh42Ewg-1; Fri, 26 Jun 2020 18:31:35 -0400 X-MC-Unique: F5lltJ4LPKGnm0mnh42Ewg-1 Received: by mail-qt1-f197.google.com with SMTP id l19so7507477qtp.12 for ; Fri, 26 Jun 2020 15:31:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=6x/m22vLH3eEyRQboy/3NjalJn5cZPkkagnVGM9ijxA=; b=eVclRxsFrdX7Hs84JnZRL/taZbptermPE5VESByUy4rGioZix24IYlGGKn2JbjVsX3 hY+iRnq+LWB+63OwgHPNKNPc1e495MRD3Nz9xK52uGaRipChqfPyUBIXTZKWJTrDGj/Z JT+U9tMxOVpK/onDumHDCysh3fGARdCGOQk55tlreUeu+dn2a+c1+rwISM3pW22AZqK1 WlQ3gyUefTFlKX0U29lj3MDr+MKz3Oa5oIbp9Ieqk0jUo28lO9pyFNPORYlRqTl6d5S8 RVUDw4VWuRXy0tMw9EvVwoVwQCQyMzw/OPMDbU7K50XItVA7m7MO2JXnQLn6dPx01C68 akJA== X-Gm-Message-State: AOAM531gpqJwtOUyR/tIuLrv5lzRtiqbOMfBH96MEaZJHpGqCDqAUOfu zPa7TUIEiidJaTtmTSXz9HNVK4rQN/Kjee9p+hgb/BSgjhQFlPbGmIOaYZfuXndJr0gV94hF3hD 45d0R1/2q1s4= X-Received: by 2002:ac8:4d5d:: with SMTP id x29mr5223534qtv.358.1593210692634; Fri, 26 Jun 2020 15:31:32 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwroaD/HeLQ0iKgIfzQJszueUyajUkqJINM3Lp4dWWUtrmn1Cn0mhNqXhxIjin9TlGVWdKEEA== X-Received: by 2002:ac8:4d5d:: with SMTP id x29mr5223507qtv.358.1593210692342; Fri, 26 Jun 2020 15:31:32 -0700 (PDT) Received: from xz-x1.redhat.com ([2607:9880:19c0:32::2]) by smtp.gmail.com with ESMTPSA id f203sm9903311qke.135.2020.06.26.15.31.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Jun 2020 15:31:31 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Andrew Morton , Linus Torvalds , Gerald Schaefer , Andrea Arcangeli , Will Deacon , peterx@redhat.com, Michael Ellerman Subject: [PATCH 00/26] mm: Page fault accounting cleanups Date: Fri, 26 Jun 2020 18:31:04 -0400 Message-Id: <20200626223130.199227-1-peterx@redhat.com> X-Mailer: git-send-email 2.26.2 MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 2EE231804E3A8 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam03 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: v3: - comment fixes [Gerald] - check VM_FAULT_ERROR too [Gerald] - collect r-b =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D cover letter= for v2 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D This is v2 of the pf accounting cleanup series. It originates from Geral= d Schaefer's report on an issue a week ago regarding to incorrect page faul= t accountings for retried page fault after commit 4064b9827063 ("mm: allow VM_FAULT_RETRY for multiple times"): https://lore.kernel.org/lkml/20200610174811.44b94525@thinkpad/ This version used a better approach suggested by Linus so that we do acco= unting directly in handle_mm_fault(). Moreover, we'll cover some special accoun= ting too like gup or IOMMU fault requests on process page tables. The outcome= of this series is to keep all the pf accountings in handle_mm_fault() (besid= es PERF_COUNT_SW_PAGE_FAULTS, which is still done in per-arch #pf handlers). Since v2 changed quite a lot from v1, changelog is omitted, and I also di= dn't have a chance to pick up any r-b in previous version. I really appreciat= e anyone who has looked at v1. V1 for reference: https://lore.kernel.org/lkml/20200615221607.7764-1-peterx@redhat.com/ What this series did: - Correct page fault accounting: we do accounting for a page fault (no = matter whether it's from #PF handling, or gup, or anything else) only with t= he one that completed the fault. For example, page fault retries should not= be counted in page fault counters. Same to the perf events. - Unify definition of PERF_COUNT_SW_PAGE_FAULTS: currently this perf ev= ent is used in an adhoc way across different archs. Case (1): for many archs it's done at the entry of a page fault handl= er, so that it will also cover e.g. errornous faults. Case (2): for some other archs, it is only accounted when the page fa= ult is resolved successfully. Case (3): there're still quite some archs that have not enabled this = perf event. Since this series will touch merely all the archs, we unify this perf= event to always follow case (1), which is the one that makes most sense. A= nd since we moved the accounting into handle_mm_fault, the other two MAJ= /MIN perf events are well taken care of naturally. - Unify definition of "major faults": the definition of "major fault" i= s slightly changed when used in accounting (not VM_FAULT_MAJOR). More information in patch 1. - Always account the page fault onto the one that triggered the page fa= ult. This does not matter much for #PF handlings, but mostly for gup. Mor= e information on this in patch 25. Patchset layout: Patch 1: Introduced the accounting in handle_mm_fault(), not enabled. Patch 2-24: Enable the new accounting for arch #PF handlers one by one. Patch 25: Enable the new accounting for the rest outliers (gup, iommu,= etc.) Patch 26: Cleanup GUP task_struct pointer since it's not needed any mo= re For each of the patch that fixes a specific arch, I'm CCing the maintaine= rs and the arch list if there is. Besides, I only lightly tested this series on= x86. Please have a look, thanks. Peter Xu (26): mm: Do page fault accounting in handle_mm_fault mm/alpha: Use general page fault accounting mm/arc: Use general page fault accounting mm/arm: Use general page fault accounting mm/arm64: Use general page fault accounting mm/csky: Use general page fault accounting mm/hexagon: Use general page fault accounting mm/ia64: Use general page fault accounting mm/m68k: Use general page fault accounting mm/microblaze: Use general page fault accounting mm/mips: Use general page fault accounting mm/nds32: Use general page fault accounting mm/nios2: Use general page fault accounting mm/openrisc: Use general page fault accounting mm/parisc: Use general page fault accounting mm/powerpc: Use general page fault accounting mm/riscv: Use general page fault accounting mm/s390: Use general page fault accounting mm/sh: Use general page fault accounting mm/sparc32: Use general page fault accounting mm/sparc64: Use general page fault accounting mm/unicore32: Use general page fault accounting mm/x86: Use general page fault accounting mm/xtensa: Use general page fault accounting mm: Clean up the last pieces of page fault accountings mm/gup: Remove task_struct pointer for all gup code arch/alpha/mm/fault.c | 8 +- arch/arc/kernel/process.c | 2 +- arch/arc/mm/fault.c | 18 +--- arch/arm/mm/fault.c | 25 ++--- arch/arm64/mm/fault.c | 29 ++---- arch/csky/mm/fault.c | 13 +-- arch/hexagon/mm/vm_fault.c | 9 +- arch/ia64/mm/fault.c | 9 +- arch/m68k/mm/fault.c | 14 +-- arch/microblaze/mm/fault.c | 9 +- arch/mips/mm/fault.c | 14 +-- arch/nds32/mm/fault.c | 19 +--- arch/nios2/mm/fault.c | 14 +-- arch/openrisc/mm/fault.c | 9 +- arch/parisc/mm/fault.c | 8 +- arch/powerpc/mm/copro_fault.c | 7 +- arch/powerpc/mm/fault.c | 11 +- arch/riscv/mm/fault.c | 16 +-- arch/s390/kvm/interrupt.c | 2 +- arch/s390/kvm/kvm-s390.c | 2 +- arch/s390/kvm/priv.c | 8 +- arch/s390/mm/fault.c | 16 +-- arch/s390/mm/gmap.c | 4 +- arch/sh/mm/fault.c | 11 +- arch/sparc/mm/fault_32.c | 13 +-- arch/sparc/mm/fault_64.c | 11 +- arch/um/kernel/trap.c | 6 +- arch/unicore32/mm/fault.c | 14 +-- arch/x86/mm/fault.c | 17 +--- arch/xtensa/mm/fault.c | 15 +-- drivers/gpu/drm/i915/gem/i915_gem_userptr.c | 2 +- drivers/infiniband/core/umem_odp.c | 2 +- drivers/iommu/amd_iommu_v2.c | 2 +- drivers/iommu/intel-svm.c | 2 +- drivers/vfio/vfio_iommu_type1.c | 2 +- fs/exec.c | 2 +- include/linux/mm.h | 16 +-- kernel/events/uprobes.c | 6 +- kernel/futex.c | 2 +- mm/gup.c | 107 +++++++------------- mm/hmm.c | 3 +- mm/ksm.c | 3 +- mm/memory.c | 69 ++++++++++++- mm/process_vm_access.c | 2 +- security/tomoyo/domain.c | 2 +- virt/kvm/async_pf.c | 2 +- virt/kvm/kvm_main.c | 2 +- 47 files changed, 219 insertions(+), 360 deletions(-) --=20 2.26.2