From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66F34C04EB8 for ; Tue, 4 Dec 2018 19:12:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3812E2082B for ; Tue, 4 Dec 2018 19:12:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3812E2082B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=deltatee.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726207AbeLDTMF (ORCPT ); Tue, 4 Dec 2018 14:12:05 -0500 Received: from ale.deltatee.com ([207.54.116.67]:53386 "EHLO ale.deltatee.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725797AbeLDTME (ORCPT ); Tue, 4 Dec 2018 14:12:04 -0500 Received: from guinness.priv.deltatee.com ([172.16.1.162]) by ale.deltatee.com with esmtp (Exim 4.89) (envelope-from ) id 1gUG6o-0007ax-4D; Tue, 04 Dec 2018 12:11:50 -0700 To: Jerome Glisse , Dan Williams Cc: Andi Kleen , Linux MM , Andrew Morton , Linux Kernel Mailing List , "Rafael J. Wysocki" , Ross Zwisler , Dave Hansen , Haggai Eran , balbirs@au1.ibm.com, "Aneesh Kumar K.V" , Benjamin Herrenschmidt , "Kuehling, Felix" , Philip.Yang@amd.com, "Koenig, Christian" , "Blinzer, Paul" , John Hubbard , rcampbell@nvidia.com References: <20181203233509.20671-1-jglisse@redhat.com> <20181203233509.20671-3-jglisse@redhat.com> <875zw98bm4.fsf@linux.intel.com> <20181204182421.GC2937@redhat.com> <20181204185725.GE2937@redhat.com> From: Logan Gunthorpe Message-ID: Date: Tue, 4 Dec 2018 12:11:42 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.3.0 MIME-Version: 1.0 In-Reply-To: <20181204185725.GE2937@redhat.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-CA Content-Transfer-Encoding: 7bit X-SA-Exim-Connect-IP: 172.16.1.162 X-SA-Exim-Rcpt-To: rcampbell@nvidia.com, jhubbard@nvidia.com, Paul.Blinzer@amd.com, christian.koenig@amd.com, Philip.Yang@amd.com, felix.kuehling@amd.com, benh@kernel.crashing.org, aneesh.kumar@linux.ibm.com, balbirs@au1.ibm.com, haggaie@mellanox.com, dave.hansen@intel.com, ross.zwisler@linux.intel.com, rafael@kernel.org, linux-kernel@vger.kernel.org, akpm@linux-foundation.org, linux-mm@kvack.org, ak@linux.intel.com, dan.j.williams@intel.com, jglisse@redhat.com X-SA-Exim-Mail-From: logang@deltatee.com Subject: Re: [RFC PATCH 02/14] mm/hms: heterogenenous memory system (HMS) documentation X-SA-Exim-Version: 4.2.1 (built Tue, 02 Aug 2016 21:08:31 +0000) X-SA-Exim-Scanned: Yes (on ale.deltatee.com) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2018-12-04 11:57 a.m., Jerome Glisse wrote: >> That sounds needlessly restrictive. Let the kernel arbitrate what >> memory an application gets, don't design a system where applications >> are hard coded to a memory type. Applications can hint, or optionally >> specify an override and the kernel can react accordingly. > > You do not want to randomly use non cache coherent memory inside your > application :) This is not gonna go well with C++ or atomic :) Yes they > are legitimate use case where application can decide to give up cache > coherency temporarily for a range of virtual address. But the application > needs to understand what it is doing and opt in to do that knowing full > well that. The version thing allows for scenario like. You do not have > to define a new version with every new type of memory. If your new memory > has all the properties of v1 than you expose it as v1 and old application > on the new platform will use your new memory type being non the wiser. I agree with Dan and the general idea that this version thing is really ugly. Define some standard attributes so the application can say "I want cache-coherent, high bandwidth memory". If there's some future new-memory attribute, then the application needs to know about it to request it. Also, in the same vein, I think it's wrong to have the API enumerate all the different memory available in the system. The API should simply allow userspace to say it wants memory that can be accessed by a set of initiators with a certain set of attributes and the bind call tries to fulfill that or fallback on system memory/hmm migration/whatever. Logan