From: Dave Hansen <dave.hansen@intel.com> To: Jerome Glisse <jglisse@redhat.com> Cc: linux-mm@kvack.org, "Andrew Morton" <akpm@linux-foundation.org>, linux-kernel@vger.kernel.org, "Rafael J . Wysocki" <rafael@kernel.org>, "Matthew Wilcox" <willy@infradead.org>, "Ross Zwisler" <ross.zwisler@linux.intel.com>, "Keith Busch" <keith.busch@intel.com>, "Dan Williams" <dan.j.williams@intel.com>, "Haggai Eran" <haggaie@mellanox.com>, "Balbir Singh" <bsingharora@gmail.com>, "Aneesh Kumar K . V" <aneesh.kumar@linux.ibm.com>, "Benjamin Herrenschmidt" <benh@kernel.crashing.org>, "Felix Kuehling" <felix.kuehling@amd.com>, "Philip Yang" <Philip.Yang@amd.com>, "Christian König" <christian.koenig@amd.com>, "Paul Blinzer" <Paul.Blinzer@amd.com>, "Logan Gunthorpe" <logang@deltatee.com>, "John Hubbard" <jhubbard@nvidia.com>, "Ralph Campbell" <rcampbell@nvidia.com> Subject: Re: [RFC PATCH 00/14] Heterogeneous Memory System (HMS) and hbind() Date: Wed, 5 Dec 2018 09:27:09 -0800 [thread overview] Message-ID: <b3122fdf-02c3-2e9c-1da6-fb873b824d59@intel.com> (raw) In-Reply-To: <20181205021334.GB3045@redhat.com> On 12/4/18 6:13 PM, Jerome Glisse wrote: > On Tue, Dec 04, 2018 at 05:06:49PM -0800, Dave Hansen wrote: >> OK, but there are 1024*1024 matrix cells on a systems with 1024 >> proximity domains (ACPI term for NUMA node). So it sounds like you are >> proposing a million-directory approach. > > No, pseudo code: > struct list links; > > for (unsigned r = 0; r < nrows; r++) { > for (unsigned c = 0; c < ncolumns; c++) { > if (!link_find(links, hmat[r][c].bandwidth, > hmat[r][c].latency)) { > link = link_new(hmat[r][c].bandwidth, > hmat[r][c].latency); > // add initiator and target correspond to that row > // and columns to this new link > list_add(&link, links); > } > } > } > > So all cells that have same property are under the same link. OK, so the "link" here is like a cable. It's like saying, "we have a network and everything is connected with an ethernet cable that can do 1gbit/sec". But, what actually connects an initiator to a target? I assume we still need to know which link is used for each target/initiator pair. Where is that enumerated? I think this just means we need a million symlinks to a "link" instead of a million link directories. Still not great. > Note that userspace can parse all this once during its initialization > and create pools of target to use. It sounds like you're agreeing that there is too much data in this interface for applications to _regularly_ parse it. We need some central thing that parses it all and caches the results.
WARNING: multiple messages have this Message-ID (diff)
From: Dave Hansen <dave.hansen@intel.com> To: Jerome Glisse <jglisse@redhat.com> Cc: linux-mm@kvack.org, "Andrew Morton" <akpm@linux-foundation.org>, linux-kernel@vger.kernel.org, "Rafael J . Wysocki" <rafael@kernel.org>, "Matthew Wilcox" <willy@infradead.org>, "Ross Zwisler" <ross.zwisler@linux.intel.com>, "Keith Busch" <keith.busch@intel.com>, "Dan Williams" <dan.j.williams@intel.com>, "Haggai Eran" <haggaie@mellanox.com>, "Balbir Singh" <bsingharora@gmail.com>, "Aneesh Kumar K . V" <aneesh.kumar@linux.ibm.com>, "Benjamin Herrenschmidt" <benh@kernel.crashing.org>, "Felix Kuehling" <felix.kuehling@amd.com>, "Philip Yang" <Philip.Yang@amd.com>, "Christian König" <christian.koenig@amd.com>, "Paul Blinzer" <Paul.Blinzer@amd.com>, "Logan Gunthorpe" <logang@deltatee.com>, "John Hubbard" <jhubbard@nvidia.com>, "Ralph Campbell" <rcampbell@nvidia.com>, "Michal Hocko" <mhocko@kernel.org>, "Jonathan Cameron" <jonathan.cameron@huawei.com>, "Mark Hairgrove" <mhairgrove@nvidia.com>, "Vivek Kini" <vkini@nvidia.com>, "Mel Gorman" <mgorman@techsingularity.net>, "Dave Airlie" <airlied@redhat.com>, "Ben Skeggs" <bskeggs@redhat.com>, "Andrea Arcangeli" <aarcange@redhat.com>, "Rik van Riel" <riel@surriel.com>, "Ben Woodard" <woodard@redhat.com>, linux-acpi@vger.kernel.org Subject: Re: [RFC PATCH 00/14] Heterogeneous Memory System (HMS) and hbind() Date: Wed, 5 Dec 2018 09:27:09 -0800 [thread overview] Message-ID: <b3122fdf-02c3-2e9c-1da6-fb873b824d59@intel.com> (raw) In-Reply-To: <20181205021334.GB3045@redhat.com> On 12/4/18 6:13 PM, Jerome Glisse wrote: > On Tue, Dec 04, 2018 at 05:06:49PM -0800, Dave Hansen wrote: >> OK, but there are 1024*1024 matrix cells on a systems with 1024 >> proximity domains (ACPI term for NUMA node). So it sounds like you are >> proposing a million-directory approach. > > No, pseudo code: > struct list links; > > for (unsigned r = 0; r < nrows; r++) { > for (unsigned c = 0; c < ncolumns; c++) { > if (!link_find(links, hmat[r][c].bandwidth, > hmat[r][c].latency)) { > link = link_new(hmat[r][c].bandwidth, > hmat[r][c].latency); > // add initiator and target correspond to that row > // and columns to this new link > list_add(&link, links); > } > } > } > > So all cells that have same property are under the same link. OK, so the "link" here is like a cable. It's like saying, "we have a network and everything is connected with an ethernet cable that can do 1gbit/sec". But, what actually connects an initiator to a target? I assume we still need to know which link is used for each target/initiator pair. Where is that enumerated? I think this just means we need a million symlinks to a "link" instead of a million link directories. Still not great. > Note that userspace can parse all this once during its initialization > and create pools of target to use. It sounds like you're agreeing that there is too much data in this interface for applications to _regularly_ parse it. We need some central thing that parses it all and caches the results.
next prev parent reply other threads:[~2018-12-05 17:27 UTC|newest] Thread overview: 171+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-12-03 23:34 [RFC PATCH 00/14] Heterogeneous Memory System (HMS) and hbind() jglisse 2018-12-03 23:34 ` jglisse 2018-12-03 23:34 ` [RFC PATCH 01/14] mm/hms: heterogeneous memory system (sysfs infrastructure) jglisse 2018-12-03 23:34 ` [RFC PATCH 02/14] mm/hms: heterogenenous memory system (HMS) documentation jglisse 2018-12-04 17:06 ` Andi Kleen 2018-12-04 17:06 ` Andi Kleen 2018-12-04 18:24 ` Jerome Glisse 2018-12-04 18:24 ` Jerome Glisse 2018-12-04 18:31 ` Dan Williams 2018-12-04 18:57 ` Jerome Glisse 2018-12-04 18:57 ` Jerome Glisse 2018-12-04 19:11 ` Logan Gunthorpe 2018-12-04 19:22 ` Jerome Glisse 2018-12-04 19:22 ` Jerome Glisse 2018-12-04 19:41 ` Logan Gunthorpe 2018-12-04 20:13 ` Jerome Glisse 2018-12-04 20:13 ` Jerome Glisse 2018-12-04 20:30 ` Logan Gunthorpe 2018-12-04 20:59 ` Jerome Glisse 2018-12-04 20:59 ` Jerome Glisse 2018-12-04 21:19 ` Logan Gunthorpe 2018-12-04 21:51 ` Jerome Glisse 2018-12-04 21:51 ` Jerome Glisse 2018-12-04 22:16 ` Logan Gunthorpe 2018-12-04 23:56 ` Jerome Glisse 2018-12-04 23:56 ` Jerome Glisse 2018-12-05 1:15 ` Logan Gunthorpe 2018-12-05 2:31 ` Jerome Glisse 2018-12-05 2:31 ` Jerome Glisse 2018-12-05 17:41 ` Logan Gunthorpe 2018-12-05 18:07 ` Jerome Glisse 2018-12-05 18:07 ` Jerome Glisse 2018-12-05 18:20 ` Logan Gunthorpe 2018-12-05 18:33 ` Jerome Glisse 2018-12-05 18:33 ` Jerome Glisse 2018-12-05 18:48 ` Logan Gunthorpe 2018-12-05 18:55 ` Jerome Glisse 2018-12-05 18:55 ` Jerome Glisse 2018-12-05 19:10 ` Logan Gunthorpe 2018-12-05 22:58 ` Jerome Glisse 2018-12-05 22:58 ` Jerome Glisse 2018-12-05 23:09 ` Logan Gunthorpe 2018-12-05 23:20 ` Jerome Glisse 2018-12-05 23:20 ` Jerome Glisse 2018-12-05 23:23 ` Logan Gunthorpe 2018-12-05 23:27 ` Jerome Glisse 2018-12-06 0:08 ` Dan Williams 2018-12-05 2:34 ` Dan Williams 2018-12-05 2:37 ` Jerome Glisse 2018-12-05 2:37 ` Jerome Glisse 2018-12-05 17:25 ` Logan Gunthorpe 2018-12-05 18:01 ` Jerome Glisse 2018-12-05 18:01 ` Jerome Glisse 2018-12-04 20:14 ` Andi Kleen 2018-12-04 20:47 ` Logan Gunthorpe 2018-12-04 21:15 ` Jerome Glisse 2018-12-04 21:15 ` Jerome Glisse 2018-12-05 0:54 ` Kuehling, Felix 2018-12-04 19:19 ` Dan Williams 2018-12-04 19:32 ` Jerome Glisse 2018-12-04 19:32 ` Jerome Glisse 2018-12-04 20:12 ` Andi Kleen 2018-12-04 20:41 ` Jerome Glisse 2018-12-04 20:41 ` Jerome Glisse 2018-12-05 4:36 ` Aneesh Kumar K.V 2018-12-05 4:41 ` Jerome Glisse 2018-12-05 4:41 ` Jerome Glisse 2018-12-05 10:52 ` Mike Rapoport 2018-12-05 10:52 ` Mike Rapoport 2018-12-03 23:34 ` [RFC PATCH 03/14] mm/hms: add target memory to heterogeneous memory system infrastructure jglisse 2018-12-03 23:34 ` [RFC PATCH 04/14] mm/hms: add initiator " jglisse 2018-12-03 23:35 ` [RFC PATCH 05/14] mm/hms: add link " jglisse 2018-12-03 23:35 ` [RFC PATCH 06/14] mm/hms: add bridge " jglisse 2018-12-03 23:35 ` [RFC PATCH 07/14] mm/hms: register main memory with heterogenenous memory system jglisse 2018-12-03 23:35 ` [RFC PATCH 08/14] mm/hms: register main CPUs " jglisse 2018-12-03 23:35 ` [RFC PATCH 09/14] mm/hms: hbind() for heterogeneous memory system (aka mbind() for HMS) jglisse 2018-12-03 23:35 ` [RFC PATCH 10/14] mm/hbind: add heterogeneous memory policy tracking infrastructure jglisse 2018-12-03 23:35 ` [RFC PATCH 11/14] mm/hbind: add bind command to heterogeneous memory policy jglisse 2018-12-03 23:35 ` [RFC PATCH 12/14] mm/hbind: add migrate command to hbind() ioctl jglisse 2018-12-03 23:35 ` [RFC PATCH 13/14] drm/nouveau: register GPU under heterogeneous memory system jglisse 2018-12-03 23:35 ` [RFC PATCH 14/14] test/hms: tests for " jglisse 2018-12-04 7:44 ` [RFC PATCH 00/14] Heterogeneous Memory System (HMS) and hbind() Aneesh Kumar K.V 2018-12-04 7:44 ` Aneesh Kumar K.V 2018-12-04 14:44 ` Jerome Glisse 2018-12-04 14:44 ` Jerome Glisse 2018-12-04 14:44 ` Jerome Glisse 2018-12-04 18:02 ` Dave Hansen 2018-12-04 18:02 ` Dave Hansen 2018-12-04 18:49 ` Jerome Glisse 2018-12-04 18:49 ` Jerome Glisse 2018-12-04 18:49 ` Jerome Glisse 2018-12-04 18:54 ` Dave Hansen 2018-12-04 18:54 ` Dave Hansen 2018-12-04 19:11 ` Jerome Glisse 2018-12-04 19:11 ` Jerome Glisse 2018-12-04 19:11 ` Jerome Glisse 2018-12-04 21:37 ` Dave Hansen 2018-12-04 21:37 ` Dave Hansen 2018-12-04 21:57 ` Jerome Glisse 2018-12-04 21:57 ` Jerome Glisse 2018-12-04 21:57 ` Jerome Glisse 2018-12-04 23:58 ` Dave Hansen 2018-12-04 23:58 ` Dave Hansen 2018-12-05 0:29 ` Jerome Glisse 2018-12-05 0:29 ` Jerome Glisse 2018-12-05 0:29 ` Jerome Glisse 2018-12-05 1:22 ` Kuehling, Felix 2018-12-05 1:22 ` Kuehling, Felix 2018-12-05 1:22 ` Kuehling, Felix 2018-12-05 11:27 ` Aneesh Kumar K.V 2018-12-05 11:27 ` Aneesh Kumar K.V 2018-12-05 16:09 ` Jerome Glisse 2018-12-05 16:09 ` Jerome Glisse 2018-12-05 16:09 ` Jerome Glisse 2018-12-04 23:54 ` Dave Hansen 2018-12-04 23:54 ` Dave Hansen 2018-12-05 0:15 ` Jerome Glisse 2018-12-05 0:15 ` Jerome Glisse 2018-12-05 0:15 ` Jerome Glisse 2018-12-05 1:06 ` Dave Hansen 2018-12-05 1:06 ` Dave Hansen 2018-12-05 2:13 ` Jerome Glisse 2018-12-05 2:13 ` Jerome Glisse 2018-12-05 2:13 ` Jerome Glisse 2018-12-05 17:27 ` Dave Hansen [this message] 2018-12-05 17:27 ` Dave Hansen 2018-12-05 17:53 ` Jerome Glisse 2018-12-05 17:53 ` Jerome Glisse 2018-12-05 17:53 ` Jerome Glisse 2018-12-06 18:25 ` Dave Hansen 2018-12-06 18:25 ` Dave Hansen 2018-12-06 19:20 ` Jerome Glisse 2018-12-06 19:20 ` Jerome Glisse 2018-12-06 19:20 ` Jerome Glisse 2018-12-06 19:31 ` Dave Hansen 2018-12-06 19:31 ` Dave Hansen 2018-12-06 20:11 ` Logan Gunthorpe 2018-12-06 20:11 ` Logan Gunthorpe 2018-12-06 22:04 ` Dave Hansen 2018-12-06 22:04 ` Dave Hansen 2018-12-06 22:39 ` Jerome Glisse 2018-12-06 22:39 ` Jerome Glisse 2018-12-06 22:39 ` Jerome Glisse 2018-12-06 23:09 ` Dave Hansen 2018-12-06 23:09 ` Dave Hansen 2018-12-06 23:28 ` Logan Gunthorpe 2018-12-06 23:28 ` Logan Gunthorpe 2018-12-06 23:34 ` Dave Hansen 2018-12-06 23:34 ` Dave Hansen 2018-12-06 23:38 ` Dave Hansen 2018-12-06 23:38 ` Dave Hansen 2018-12-06 23:48 ` Logan Gunthorpe 2018-12-06 23:48 ` Logan Gunthorpe 2018-12-07 0:20 ` Jerome Glisse 2018-12-07 0:20 ` Jerome Glisse 2018-12-07 0:20 ` Jerome Glisse 2018-12-07 15:06 ` Jonathan Cameron 2018-12-07 15:06 ` Jonathan Cameron 2018-12-07 15:06 ` Jonathan Cameron 2018-12-07 19:37 ` Jerome Glisse 2018-12-07 19:37 ` Jerome Glisse 2018-12-07 19:37 ` Jerome Glisse 2018-12-07 0:15 ` Jerome Glisse 2018-12-07 0:15 ` Jerome Glisse 2018-12-07 0:15 ` Jerome Glisse 2018-12-06 20:27 ` Jerome Glisse 2018-12-06 20:27 ` Jerome Glisse 2018-12-06 20:27 ` Jerome Glisse 2018-12-06 21:46 ` Jerome Glisse 2018-12-06 21:46 ` Jerome Glisse 2018-12-06 21:46 ` Jerome Glisse
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=b3122fdf-02c3-2e9c-1da6-fb873b824d59@intel.com \ --to=dave.hansen@intel.com \ --cc=Paul.Blinzer@amd.com \ --cc=Philip.Yang@amd.com \ --cc=akpm@linux-foundation.org \ --cc=aneesh.kumar@linux.ibm.com \ --cc=benh@kernel.crashing.org \ --cc=bsingharora@gmail.com \ --cc=christian.koenig@amd.com \ --cc=dan.j.williams@intel.com \ --cc=felix.kuehling@amd.com \ --cc=haggaie@mellanox.com \ --cc=jglisse@redhat.com \ --cc=jhubbard@nvidia.com \ --cc=keith.busch@intel.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=logang@deltatee.com \ --cc=rafael@kernel.org \ --cc=rcampbell@nvidia.com \ --cc=ross.zwisler@linux.intel.com \ --cc=willy@infradead.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.