* How to mmap any address space using huge pages?
@ 2021-10-16 9:46 Adam Turowski
2021-10-16 15:50 ` Jason Gunthorpe
0 siblings, 1 reply; 6+ messages in thread
From: Adam Turowski @ 2021-10-16 9:46 UTC (permalink / raw)
To: linux-mm
Hello all,
I have a device and I need to mmap it into a user space using 1GB huge
pages. AFAIK I cannot use HugeTLB nor THP because they're backed by the
memory. The remap_pfn_range doesn't use huge pages neither. So I used a
huge_fault handler to set up the pud descriptor myself (arm64 here, so no
transparent puds) and that works. The problem is that the kernel warns
about a bad pud when the vma mapping is removed, the zap_pud_range function
doesn't accept block puds. The vunmap_pud_range function works with block
puds and checks for their presence and calls the pud_clear_huge function.
If I add this function call to the zap_pud_range function, the warnings are
gone.
Am I missing something? What is the proper way to mmap (and munmap) any
address space using huge pages (esp. 1GB pages)?
Thanks
--
Regards,
Adam Turowski
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: How to mmap any address space using huge pages?
2021-10-16 9:46 How to mmap any address space using huge pages? Adam Turowski
@ 2021-10-16 15:50 ` Jason Gunthorpe
2021-10-18 10:14 ` Adam Turowski
0 siblings, 1 reply; 6+ messages in thread
From: Jason Gunthorpe @ 2021-10-16 15:50 UTC (permalink / raw)
To: Adam Turowski; +Cc: linux-mm
On Sat, Oct 16, 2021 at 11:46:59AM +0200, Adam Turowski wrote:
> Hello all,
> I have a device and I need to mmap it into a user space using 1GB huge
> pages. AFAIK I cannot use HugeTLB nor THP because they're backed by the
> memory. The remap_pfn_range doesn't use huge pages neither. So I used a
> huge_fault handler to set up the pud descriptor myself (arm64 here, so no
> transparent puds) and that works. The problem is that the kernel warns
> about a bad pud when the vma mapping is removed, the zap_pud_range function
> doesn't accept block puds. The vunmap_pud_range function works with block
> puds and checks for their presence and calls the pud_clear_huge function.
> If I add this function call to the zap_pud_range function, the warnings are
> gone.
>
> Am I missing something? What is the proper way to mmap (and munmap) any
> address space using huge pages (esp. 1GB pages)?
AFAIK it can't be done today.
Certainly non-struct page memory cannot be installed in a PUD
If you create a ZONE_DEVICE struct page using memremap_pages() then
they can be inserted using vmf_insert_pfn_pud() - however that
requires arch support for pud_devmap() which only x86 does right now.
Jason
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: How to mmap any address space using huge pages?
2021-10-16 15:50 ` Jason Gunthorpe
@ 2021-10-18 10:14 ` Adam Turowski
0 siblings, 0 replies; 6+ messages in thread
From: Adam Turowski @ 2021-10-18 10:14 UTC (permalink / raw)
To: Jason Gunthorpe; +Cc: linux-mm
> On Sat, Oct 16, 2021 at 11:46:59AM +0200, Adam Turowski wrote:
> > Hello all,
> > I have a device and I need to mmap it into a user space using 1GB huge
> > pages. AFAIK I cannot use HugeTLB nor THP because they're backed by the
> > memory. The remap_pfn_range doesn't use huge pages neither. So I used a
> > huge_fault handler to set up the pud descriptor myself (arm64 here, so no
> > transparent puds) and that works. The problem is that the kernel warns
> > about a bad pud when the vma mapping is removed, the zap_pud_range function
> > doesn't accept block puds. The vunmap_pud_range function works with block
> > puds and checks for their presence and calls the pud_clear_huge function.
> > If I add this function call to the zap_pud_range function, the warnings are
> > gone.
> >
> > Am I missing something? What is the proper way to mmap (and munmap) any
> > address space using huge pages (esp. 1GB pages)?
>
> AFAIK it can't be done today.
>
> Certainly non-struct page memory cannot be installed in a PUD
>
> If you create a ZONE_DEVICE struct page using memremap_pages() then
> they can be inserted using vmf_insert_pfn_pud() - however that
> requires arch support for pud_devmap() which only x86 does right now.
>
OK, I can live with "bad pud" warnings for now.
Would it be a bad idea to make the zap_pud_range function to look like
vunmap_pud_range function by adding a pud_clear_huge(pud) call?
--
Regards,
Adam Turowski
^ permalink raw reply [flat|nested] 6+ messages in thread
* How to mmap any address space using huge pages?
@ 2021-10-15 13:07 Adam Turowski
2021-10-21 7:16 ` Chan Kim
0 siblings, 1 reply; 6+ messages in thread
From: Adam Turowski @ 2021-10-15 13:07 UTC (permalink / raw)
To: kernelnewbies
[-- Attachment #1.1: Type: text/plain, Size: 834 bytes --]
Hello all,
I have a device and I need to mmap it into a user space using 1GB huge
pages. AFAIK I cannot use HugeTLB nor THP because they're backed by the
memory. The remap_pfn_range doesn't use huge pages neither. So I used a
huge_fault handler to set up the pud descriptor myself (arm64 here, so no
transparent puds) and that works. The problem is that the kernel warns
about a bad pud when the vma mapping is removed, the zap_pud_range function
doesn't accept block puds. The vunmap_pud_range function works with block
puds and checks for their presence and calls the pud_clear_huge function.
If I add this function call to the zap_pud_range function, the warnings are
gone.
Am I missing something? What is the proper way to mmap (and munmap) any
address space using huge pages (esp. 1GB pages)?
Thanks
--
Regards,
Adam Turowski
[-- Attachment #1.2: Type: text/html, Size: 1073 bytes --]
[-- Attachment #2: Type: text/plain, Size: 170 bytes --]
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
^ permalink raw reply [flat|nested] 6+ messages in thread
* RE: How to mmap any address space using huge pages?
2021-10-15 13:07 Adam Turowski
@ 2021-10-21 7:16 ` Chan Kim
2021-10-21 7:22 ` Adam Turowski
0 siblings, 1 reply; 6+ messages in thread
From: Chan Kim @ 2021-10-21 7:16 UTC (permalink / raw)
To: 'Adam Turowski', kernelnewbies
[-- Attachment #1.1: Type: text/plain, Size: 1303 bytes --]
Hi, Adam
Did you tell the OS to reserve huge pages?
(in ubuntu, add vm.nr_hugepages = N in /etc/sysctl.conf or
Add hugepages=16, hugepagesz=256M in bootcommd)
Just in case you didn’t set it.
Thanks,
Chan Kim
From: Adam Turowski <adam.s.turowski@gmail.com>
Sent: Friday, October 15, 2021 10:08 PM
To: kernelnewbies@kernelnewbies.org
Subject: How to mmap any address space using huge pages?
Hello all,
I have a device and I need to mmap it into a user space using 1GB huge pages. AFAIK I cannot use HugeTLB nor THP because they're backed by the memory. The remap_pfn_range doesn't use huge pages neither. So I used a huge_fault handler to set up the pud descriptor myself (arm64 here, so no transparent puds) and that works. The problem is that the kernel warns about a bad pud when the vma mapping is removed, the zap_pud_range function doesn't accept block puds. The vunmap_pud_range function works with block puds and checks for their presence and calls the pud_clear_huge function. If I add this function call to the zap_pud_range function, the warnings are gone.
Am I missing something? What is the proper way to mmap (and munmap) any address space using huge pages (esp. 1GB pages)?
Thanks
--
Regards,
Adam Turowski
[-- Attachment #1.2: Type: text/html, Size: 5721 bytes --]
[-- Attachment #2: Type: text/plain, Size: 170 bytes --]
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-10-21 7:22 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-10-16 9:46 How to mmap any address space using huge pages? Adam Turowski
2021-10-16 15:50 ` Jason Gunthorpe
2021-10-18 10:14 ` Adam Turowski
-- strict thread matches above, loose matches on Subject: below --
2021-10-15 13:07 Adam Turowski
2021-10-21 7:16 ` Chan Kim
2021-10-21 7:22 ` Adam Turowski
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.