Re: [RFC PATCH 0/5] madvise MADV_DOEXEC

From: David Hildenbrand <david@redhat.com>
To: Matthew Wilcox <willy@infradead.org>
Cc: Khalid Aziz <khalid.aziz@oracle.com>,
	"Longpeng (Mike,
	Cloud Infrastructure Service Product Dept.)"
	<longpeng2@huawei.com>,
	Steven Sistare <steven.sistare@oracle.com>,
	Anthony Yznaga <anthony.yznaga@oracle.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-mm@kvack.org" <linux-mm@kvack.org>,
	"Gonglei (Arei)" <arei.gonglei@huawei.com>
Subject: Re: [RFC PATCH 0/5] madvise MADV_DOEXEC
Date: Mon, 16 Aug 2021 15:24:38 +0200	[thread overview]
Message-ID: <88884f55-4991-11a9-d330-5d1ed9d5e688@redhat.com> (raw)
In-Reply-To: <YRpeHnP7QDNJRA8Y@casper.infradead.org>

On 16.08.21 14:46, Matthew Wilcox wrote:
> On Mon, Aug 16, 2021 at 02:20:43PM +0200, David Hildenbrand wrote:
>> On 16.08.21 14:07, Matthew Wilcox wrote:
>>> On Mon, Aug 16, 2021 at 10:02:22AM +0200, David Hildenbrand wrote:
>>>>> Mappings within this address range behave as if they were shared
>>>>> between threads, so a write to a MAP_PRIVATE mapping will create a
>>>>> page which is shared between all the sharers. The first process that
>>>>> declares an address range mshare'd can continue to map objects in the
>>>>> shared area. All other processes that want mshare'd access to this
>>>>> memory area can do so by calling mshare(). After this call, the
>>>>> address range given by mshare becomes a shared range in its address
>>>>> space. Anonymous mappings will be shared and not COWed.
>>>>
>>>> Did I understand correctly that you want to share actual page tables between
>>>> processes and consequently different MMs? That sounds like a very bad idea.
>>>
>>> That is the entire point.  Consider a machine with 10,000 instances
>>> of an application running (process model, not thread model).  If each
>>> application wants to map 1TB of RAM using 2MB pages, that's 4MB of page
>>> tables per process or 40GB of RAM for the whole machine.
>>
>> What speaks against 1 GB pages then?
> 
> Until recently, the CPUs only having 4 1GB TLB entries.  I'm sure we
> still have customers using that generation of CPUs.  2MB pages perform
> better than 1GB pages on the previous generation of hardware, and I
> haven't seen numbers for the next generation yet.

I read that somewhere else before, yet we have heavy 1 GiB page users, 
especially in the context of VMs and DPDK.

> 
>>> There's a reason hugetlbfs was enhanced to allow this page table sharing.
>>> I'm not a fan of the implementation as it gets some locks upside down,
>>> so this is an attempt to generalise the concept beyond hugetlbfs.
>>
>> Who do we account the page tables to? What are MADV_DONTNEED semantics? Who
>> cleans up the page tables? What happens during munmap? How does the rmap
>> even work? How to we actually synchronize page table walkers?
>>
>> See how hugetlbfs just doesn't raise these problems because we are sharing
>> pages and not page tables?
> 
> No, really, hugetlbfs shares page tables already.  You just didn't
> notice that yet.

So, it only works for hugetlbfs in case uffd is not in place (-> no 
per-process data in the page table) and we have an actual shared 
mappings. When unsharing, we zap the PUD entry, which will result in 
allocating a per-process page table on next fault.

I will rephrase my previous statement "hugetlbfs just doesn't raise 
these problems because we are special casing it all over the place 
already". For example, not allowing to swap such pages. Disallowing 
MADV_DONTNEED. Special hugetlbfs locking.

-- 
Thanks,

David / dhildenb