From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.3 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2B66C43331 for ; Thu, 26 Mar 2020 14:56:57 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7A9A2206F6 for ; Thu, 26 Mar 2020 14:56:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=Mellanox.com header.i=@Mellanox.com header.b="Hieo8Ghi" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7A9A2206F6 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=mellanox.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E2BE76B000A; Thu, 26 Mar 2020 10:56:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DB5536B000C; Thu, 26 Mar 2020 10:56:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CA5A36B000D; Thu, 26 Mar 2020 10:56:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0114.hostedemail.com [216.40.44.114]) by kanga.kvack.org (Postfix) with ESMTP id AF7C66B000A for ; Thu, 26 Mar 2020 10:56:56 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id B7BEC5DF4 for ; Thu, 26 Mar 2020 14:56:56 +0000 (UTC) X-FDA: 76637815632.30.north41_59a810ff7f308 X-HE-Tag: north41_59a810ff7f308 X-Filterd-Recvd-Size: 8288 Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-eopbgr70083.outbound.protection.outlook.com [40.107.7.83]) by imf21.hostedemail.com (Postfix) with ESMTP for ; Thu, 26 Mar 2020 14:56:55 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=aT9T/WMctvUho8va1AfRZKySGKzD2NuN6Hxd3xIzsUzxyLFbAd4DEGWB11tAzH2iCtYlEhr1xk6km/pt6q1Z7nAzq4EuQn1TsEMFaMMEULWDF0BhtHeokcl0rylP5BK+H2VtMv1A6whngNaWsQPysxY4OfWh6TVZAzmtMjfpQeKyzrwe9cK2RkuR3Y7EF3bwEfnL6JN3KyBeZ7LzaK5/CLrOlAsw5iT0vTmrQ9QWXw5SakokbwVoySLt/hXZEmwTIXbEfxc3KoumFZ399hX12Ep5K2yvICfznIuAvnRYyVWCWy4bBCUp/68Vfr1SmhXX/a6BI043aSBrS7TTeCN0ag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pVcRr3vGcdf7o2oNzURLQQIKwYAH9QD5bknzAPWB4T8=; b=SnIn1fQLqUHQgxaISKrQKo5GYWVoXCQk2NtkzdQawiseCZNtrdqag3wsyc0O5d0ZNdTuSoAmvtISRUfjnzjBmvgeBNjf1VBcLBWczYtfJ/Z73C19JXCnku1ZPU8ZhtrDX43C/pXH00ar289peJXLhJ2S6UgW69HttSM/Qn8yOuroquuqclG46yFOUC0fRxSIQiBAiTz8qyUlf5bcRihDVg8PZV7n/Mbz7t70WdF2jB6+EuI3hYl1GSQ2krOuXxyxuwk1KObFgZjqimLhnQm9pH2megYcHxU0to5ML29cAeSaLtbCDDD3WR6v5x8zwLd7WjfZLAOG4WJ71xvMBtnmfA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pVcRr3vGcdf7o2oNzURLQQIKwYAH9QD5bknzAPWB4T8=; b=Hieo8GhiR/OQi839WQhOdx8uxobVlekYeRMXM6wn4L52/6kWbz5QdvIGjA3aZk/KTRXwDTka64FcIrFnQMvQxwtvgzjG7dn9xu0L1Y0e3Xt679fZ9tfMwM5rtmFxUeuB6i+wrcHCc0+Lt8HapLMZEqZ6UagnFSc40Fwxqki77NU= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=jgg@mellanox.com; Received: from VI1PR05MB4141.eurprd05.prod.outlook.com (52.133.14.15) by VI1PR05MB6144.eurprd05.prod.outlook.com (20.178.204.33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2856.18; Thu, 26 Mar 2020 14:56:53 +0000 Received: from VI1PR05MB4141.eurprd05.prod.outlook.com ([fe80::18d2:a9ea:519:add3]) by VI1PR05MB4141.eurprd05.prod.outlook.com ([fe80::18d2:a9ea:519:add3%7]) with mapi id 15.20.2856.019; Thu, 26 Mar 2020 14:56:53 +0000 Date: Thu, 26 Mar 2020 11:56:49 -0300 From: Jason Gunthorpe To: Qian Cai Cc: Linux Memory Management List , Michal Hocko , =?utf-8?B?SsOpcsO0bWU=?= Glisse , Christoph Hellwig Subject: Re: [PATCH v3] mm/mmu_notifier: prevent unpaired invalidate_start and invalidate_end Message-ID: <20200326145649.GL13183@mellanox.com> References: <20200211205252.GA10003@ziepe.ca> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) X-ClientProxiedBy: BL0PR02CA0071.namprd02.prod.outlook.com (2603:10b6:207:3d::48) To VI1PR05MB4141.eurprd05.prod.outlook.com (2603:10a6:803:44::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from mlx.ziepe.ca (142.68.57.212) by BL0PR02CA0071.namprd02.prod.outlook.com (2603:10b6:207:3d::48) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2835.21 via Frontend Transport; Thu, 26 Mar 2020 14:56:53 +0000 Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1jHTw9-0006Bd-UZ; Thu, 26 Mar 2020 11:56:49 -0300 X-Originating-IP: [142.68.57.212] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: dfd3317e-5d13-4f29-c613-08d7d195ebaf X-MS-TrafficTypeDiagnostic: VI1PR05MB6144: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-Forefront-PRVS: 0354B4BED2 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR05MB4141.eurprd05.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(10009020)(4636009)(366004)(396003)(136003)(39860400002)(346002)(376002)(86362001)(5660300002)(6916009)(26005)(2906002)(36756003)(186003)(1076003)(316002)(81166006)(478600001)(8936002)(2616005)(8676002)(81156014)(54906003)(66476007)(9786002)(53546011)(52116002)(9746002)(66556008)(4326008)(66946007)(33656002)(24400500001);DIR:OUT;SFP:1101; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: pz49/YwlxyINimRQOoZcGIakjMTPT6XjKM6rnOzZT99tKibff0PgCvyTliZ3VAaMtA0Ft4r+Bx2Gv6pJIg1wtAHAbHOpLKmZMGfZpSWJSAcr0ITWpi11zYTTuoYL9s6Pj12hjX9uAwLlCH7l8cIl90n5KnLMtIcWqZ3AoTpCoCVOW7ciC9QDyfS4sJaa/jQiLJeCYkEJXF35CejvK+elaYpnY0r3G1NAlyiaiI5nGW6YhnQrs/J4rYYd3qvu8v+9ek53PnMC4KaDQ9Fko2pLgJaRs2rjdCIOYMZgtyzDpD+GrJXR4DP7FLoRoEFYfFf3Rr8GDFi47jfy8KZtaTcc4B52H1Z7YJwWXo7KHejdwLFi/7eZmfnv1CJjgzR7leGZyOop1M/L+ytIF3h3fTYm9VAnxl4mSoXeHxYeH+OuS/j5FqxT5CtSqTIHhI99lP1GLZaI2PbLua5t4rLWQeTBQVgETNXaie1W65iLX90UZZqQYXP7ASs9xxXIbiiJC42Q X-MS-Exchange-AntiSpam-MessageData: 3FJOQUvjqDKjbi6FKZjRAJxcU2zo+tcZ8VK81iHeiO/v4VQimL2HZwo5NXNtkEcIG7MLHrq4drGmvYE/iHWYpx+pKkfPKOnAsJ/9eM1zwiLPqASXEw0i6F+y8zhK37E9jYGhj2b6RcbQ3DG2bLYVcg== X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: dfd3317e-5d13-4f29-c613-08d7d195ebaf X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Mar 2020 14:56:53.2021 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: wjQu/d7t7FCqNSBoo66sMWtCdHi/tH5+xTs8aMyLqkgjR3cMWKImGY0BKRzJvaOm3gIV1pKag+wLdPH/sz56zQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR05MB6144 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Mar 26, 2020 at 09:06:12AM -0400, Qian Cai wrote: > > > > On Feb 11, 2020, at 3:52 PM, Jason Gunthorpe wrote: > > > > Many users of the mmu_notifier invalidate_range callbacks maintain > > locking/counters/etc on a paired basis and have long expected that > > invalidate_range_start/end() are always paired. > > > > For instance kvm_mmu_notifier_invalidate_range_end() undoes > > kvm->mmu_notifier_count which was incremented during start(). > > > > The recent change to add non-blocking notifiers breaks this assumption > > when multiple notifiers are present in the list. When EAGAIN is returned > > from an invalidate_range_start() then no invalidate_range_ends() are > > called, even if the subscription's start had previously been called. > > > > Unfortunately, due to the RCU list traversal we can't reliably generate a > > subset of the linked list representing the notifiers already called to > > generate an invalidate_range_end() pairing. > > > > One case works correctly, if only one subscription requires > > invalidate_range_end() and it is the last entry in the hlist. In this > > case, when invalidate_range_start() returns -EAGAIN there will be nothing > > to unwind. > > > > Keep the notifier hlist sorted so that notifiers that require > > invalidate_range_end() are always last, and if two are added then disable > > non-blocking invalidation for the mm. > > > > A warning is printed for this case, if in future we determine this never > > happens then we can simply fail during registration when there are > > unsupported combinations of notifiers. > > This will generate a warning when running a simple qemu-kvm on arm64, > > qemu-kvm (37712) created two mmu_notifier's with > invalidate_range_end(): kvm_mmu_notifier_invalidate_range_end and > kvm_mmu_notifier_invalidate_range_end, non-blocking notifiers > disabled Thank you Qian, this is very valuable information It seems this solution will not work if kvm is registering multiple notifiers. I will take the patch out of linux-next Jason