From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D51E3C77B70 for ; Fri, 14 Apr 2023 05:39:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5DC1C6B0071; Fri, 14 Apr 2023 01:39:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 58C256B0072; Fri, 14 Apr 2023 01:39:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 453A9900002; Fri, 14 Apr 2023 01:39:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 33B956B0071 for ; Fri, 14 Apr 2023 01:39:17 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id E839612011A for ; Fri, 14 Apr 2023 05:39:16 +0000 (UTC) X-FDA: 80678893512.03.98CDF47 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by imf02.hostedemail.com (Postfix) with ESMTP id EB6C380008 for ; Fri, 14 Apr 2023 05:39:13 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=FFxxN33o; spf=pass (imf02.hostedemail.com: domain of ying.huang@intel.com designates 192.55.52.136 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681450754; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JJkio9vQElun2x7eTTkP6eyWBXXZQ1YYPZIv/EMoLx4=; b=kj7tP01Gr0iO3IV0OVgDRbGuGT47EQJybgi0BSXK8mnlpttBqZCGb5GNNfcNiTIEEoL+QA FwrdXZQWfBTApqPKXVKQ/trPWA3ksBnvjYXMHesSJmRxWAXCpv9yeVnRAcuIKBUwR95kUC O2oEVtieBEUZbjidyDK331vBKu2rAk4= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=FFxxN33o; spf=pass (imf02.hostedemail.com: domain of ying.huang@intel.com designates 192.55.52.136 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681450754; a=rsa-sha256; cv=none; b=Uy2kNOUscFCviTxjKqd6eQshFCnxbe08dS/ZMPGvkPLnelMcU4S/Jmm+Ff7qdZF753XYnC AToIQ1IZKlgCGeI6hvk7kmfXlf9Yov6FyAwNTrq0R5SXqhv94LSClZnMlUNVK17IzxuSKU ZR7n+8ec1q7CKrGK5fGhRKbzHqWMHdQ= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681450754; x=1712986754; h=from:to:cc:subject:references:date:in-reply-to: message-id:mime-version; bh=O1LRDFz3R3Vdf4KXd96xJLQIEE7mfukbxNtrhi8lDHE=; b=FFxxN33oUIGeo3apdh5opHx4fqq6xawEDngb9yYbZrVkgExkGpL3da76 EiwINKnZlLl6oa9N08yt4lW87UJEAbHo43ZzHj4yf6eeZdPyFkJmptvie 1NXjxzPhB6sRFa9kscZU3Pi6DNerWDIal4f4mvoWfwSPZbf8uN7euMtDo KYibDbVWaFy4/pjZLQolbAs3Dfz6WvE08Msdcm6fiNAZWUcCnQFDjnnAJ h+hDLevwiO18EHmzr1tXD4LgW+2ZtujQIA87zzttW5XCL4ICJJTGOLAFU bE3k+nDG0EWMVIyhYl12v2ikcsZvermad74xqSM8ECJHSYS+YzIzxhBnq g==; X-IronPort-AV: E=McAfee;i="6600,9927,10679"; a="324014979" X-IronPort-AV: E=Sophos;i="5.99,195,1677571200"; d="scan'208";a="324014979" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2023 22:39:12 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10679"; a="864074929" X-IronPort-AV: E=Sophos;i="5.99,195,1677571200"; d="scan'208";a="864074929" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2023 22:39:09 -0700 From: "Huang, Ying" To: Dragan Stancevic Cc: Gregory Price , lsf-pc@lists.linux-foundation.org, nil-migration@lists.linux.dev, linux-cxl@vger.kernel.org, linux-mm@kvack.org Subject: Re: [LSF/MM/BPF TOPIC] BoF VM live migration over CXL =?utf-8?Q?m?= =?utf-8?Q?emory=E2=80=8B?= References: <5d1156eb-02ae-a6cc-54bb-db3df3ca0597@stancevic.com> <87v8i22abl.fsf@yhuang6-desk2.ccr.corp.intel.com> <4a1e74f5-284f-7bfe-6a79-42b44909ca23@stancevic.com> Date: Fri, 14 Apr 2023 13:35:50 +0800 In-Reply-To: <4a1e74f5-284f-7bfe-6a79-42b44909ca23@stancevic.com> (Dragan Stancevic's message of "Thu, 13 Apr 2023 22:33:24 -0500") Message-ID: <87r0sn10w9.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: gwq1gkh8ge1fnh51ig8joqh9cqniorb9 X-Rspamd-Queue-Id: EB6C380008 X-HE-Tag: 1681450753-324604 X-HE-Meta: U2FsdGVkX18XireaFJwc8bR0jK/yhAw55Bd2Mx4p16BWVpYRWWfHOy7DaE5cllEe2jyO/GSLleI+jb9pP6uc6MgwrlypAvXTSMBHPyW9NFDRL0EEoZmyiY9krP7kW/0iGBmOfLZZGzmOTCe1qQh/rXmqwtEYuiF3oSiXphpcHmeOSoAKX4MhC0+sZqQhKTuUHjottI5MMU9KdgHmTSsnYlwTMQa0X5eaQR06clSaQDL/kjV/t07DUMjPROpcA0QP18iH6hZ+BVBOwPVCA8408Yqm+/0beVfKgUN+I/SQIKCMYC3ENSD+Ioqw34vVrV8JYxAYhQgPKXHi5n7amFDg5bpGZR7v0mlGeS2iHqyGC4cmoG/DrbwJGqyRS8J5gvYuZ8bEeUlNtUPOCkUJFxaJY1qzhYPqOklS77x2Erk6RdpCeTqdHV3o2rzVrQRGmo73QIBVOmeoj8V7zjpR2XOcWNFJgbJTzMqD/D69f8rJsekpg2k7GmtQ5TxEor8HX64A62MmW0m6U0UDKcv4WP2OFwhc9DMDY0CF/hr3qtcOX9ODa6/GXbp7oKCUstA/GKB7V4fj1DHHu0G+cM6NseHPp+m+sOjhKoIsg1NP932pm6iWFEu5/9VQhTKTtO+XIKl3tGkzv+JnAQ9JSpoe5X7LPD1oOpA3OtQd+aSHgjTSdH+NxgnwMaEVlF4Cqu1F7f0DLUo7EhkbGZKM7SL3kU6Uj/GQNyoJJu13+0YFadsetd4bYLrrFp0zYZhxzPA/RHrhlMEgUu2yJYauwsNJkjrnmmdmmwyYUx8LwxQ664JVhKYYCgyAPGhDQ4uGk/49Pbv1e23mWMlJc2ftEMssC3jouz2w/oMvPk+hF25KA669ZaTsA9rbiZql6oKMPlQNCVz1sc1zMbwJFVdF+J5WqIV+Ag0uo18p0UDPJSuadZGsMUDybzyR1fkiCo45dTNni5C/KylkASyYGqALl9YZ0HQ c2gUU+nt Hmfm07TWsVf6q53n+to62lBs+2kWiXEyWSHnP/evDx4TBxEZK1Jd4YR+oXFtEOg2Nl6HGwtm0JQP5eBhjQB7RCaFks/n52H+d0pRnPrZyrEm3Wt+5/YNzkknR4XDOwzsZU/R8oC1wQ0nHZL6+GEPm1AMEKZfZ37JzBSkHc/sxetW1D8jX29NPO7OdCm1ZecBImkCMOVdbfpM6uM/SAcS7aecSihx8PNbaPhT26S9HReTIebHCVqs3Ak+ffzwpQJL32pKI1EXpWt7gCXYfGBkcTK8xgKw5StbVhZgVEun/G01fpthQF1McPUNBoYNGcC3B/iZI4NQxABNUlJg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000015, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Dragan Stancevic writes: > On 4/11/23 01:37, Huang, Ying wrote: >> Gregory Price writes: >> [snip] >> >>> 2. During the migration process, the memory needs to be forced not to be >>> migrated to another node by other means (tiering software, swap, >>> etc). The obvious way of doing this would be to migrate and >>> temporarily pin the page... but going back to problem #1 we see that >>> ZONE_MOVABLE and Pinning are mutually exclusive. So that's >>> troublesome. >> Can we use memory policy (cpusets, mbind(), set_mempolicy(), etc.) >> to >> avoid move pages out of CXL.mem node? Now, there are gaps in tiering, >> but I think it is fixable. > > > Hmmm, I don't know about cpusets. For mbind, are you thinking > something along the lines of MPOL_MF_MOVE_ALL? I guess it does have > that deterministic placement, but this would have to be called from > the process itself. Unlike migrate_pages which takes a pid. You can still use migrate_pages(2). But after that, if you want to prevent the pages to be migrated out of CXL.mem, you can use some kind of memory policy, such as cpusets, mbind(), set_mempolicy(). Best Regards, Huang, Ying > Same for set_mempolicy, right? > > I mean I guess, if some of this needs to be added into qemu it's not > the end of the word... > > >> Best Regards, >> Huang, Ying >> [snip] >>