All of lore.kernel.org
 help / color / mirror / Atom feed
* How to mmap any address space using huge pages?
@ 2021-10-16  9:46 Adam Turowski
  2021-10-16 15:50 ` Jason Gunthorpe
  0 siblings, 1 reply; 6+ messages in thread
From: Adam Turowski @ 2021-10-16  9:46 UTC (permalink / raw)
  To: linux-mm

Hello all,
I have a device and I need to mmap it into a user space using 1GB huge
pages. AFAIK I cannot use HugeTLB nor THP because they're backed by the
memory. The remap_pfn_range doesn't use huge pages neither. So I used a
huge_fault handler to set up the pud descriptor myself (arm64 here, so no
transparent puds) and that works. The problem is that the kernel warns
about a bad pud when the vma mapping is removed, the zap_pud_range function
doesn't accept block puds. The vunmap_pud_range function works with block
puds and checks for their presence and calls the pud_clear_huge function.
If I add this function call to the zap_pud_range function, the warnings are
gone.

Am I missing something? What is the proper way to mmap (and munmap) any
address space using huge pages (esp. 1GB pages)?

Thanks

--
Regards,
Adam Turowski


^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: How to mmap any address space using huge pages?
  2021-10-16  9:46 How to mmap any address space using huge pages? Adam Turowski
@ 2021-10-16 15:50 ` Jason Gunthorpe
  2021-10-18 10:14   ` Adam Turowski
  0 siblings, 1 reply; 6+ messages in thread
From: Jason Gunthorpe @ 2021-10-16 15:50 UTC (permalink / raw)
  To: Adam Turowski; +Cc: linux-mm

On Sat, Oct 16, 2021 at 11:46:59AM +0200, Adam Turowski wrote:
> Hello all,
> I have a device and I need to mmap it into a user space using 1GB huge
> pages. AFAIK I cannot use HugeTLB nor THP because they're backed by the
> memory. The remap_pfn_range doesn't use huge pages neither. So I used a
> huge_fault handler to set up the pud descriptor myself (arm64 here, so no
> transparent puds) and that works. The problem is that the kernel warns
> about a bad pud when the vma mapping is removed, the zap_pud_range function
> doesn't accept block puds. The vunmap_pud_range function works with block
> puds and checks for their presence and calls the pud_clear_huge function.
> If I add this function call to the zap_pud_range function, the warnings are
> gone.
> 
> Am I missing something? What is the proper way to mmap (and munmap) any
> address space using huge pages (esp. 1GB pages)?

AFAIK it can't be done today.

Certainly non-struct page memory cannot be installed in a PUD

If you create a ZONE_DEVICE struct page using memremap_pages() then
they can be inserted using vmf_insert_pfn_pud() - however that
requires arch support for pud_devmap() which only x86 does right now.

Jason


^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: How to mmap any address space using huge pages?
  2021-10-16 15:50 ` Jason Gunthorpe
@ 2021-10-18 10:14   ` Adam Turowski
  0 siblings, 0 replies; 6+ messages in thread
From: Adam Turowski @ 2021-10-18 10:14 UTC (permalink / raw)
  To: Jason Gunthorpe; +Cc: linux-mm

> On Sat, Oct 16, 2021 at 11:46:59AM +0200, Adam Turowski wrote:
> > Hello all,
> > I have a device and I need to mmap it into a user space using 1GB huge
> > pages. AFAIK I cannot use HugeTLB nor THP because they're backed by the
> > memory. The remap_pfn_range doesn't use huge pages neither. So I used a
> > huge_fault handler to set up the pud descriptor myself (arm64 here, so no
> > transparent puds) and that works. The problem is that the kernel warns
> > about a bad pud when the vma mapping is removed, the zap_pud_range function
> > doesn't accept block puds. The vunmap_pud_range function works with block
> > puds and checks for their presence and calls the pud_clear_huge function.
> > If I add this function call to the zap_pud_range function, the warnings are
> > gone.
> >
> > Am I missing something? What is the proper way to mmap (and munmap) any
> > address space using huge pages (esp. 1GB pages)?
>
> AFAIK it can't be done today.
>
> Certainly non-struct page memory cannot be installed in a PUD
>
> If you create a ZONE_DEVICE struct page using memremap_pages() then
> they can be inserted using vmf_insert_pfn_pud() - however that
> requires arch support for pud_devmap() which only x86 does right now.
>

OK, I can live with "bad pud" warnings for now.
Would it be a bad idea to make the zap_pud_range function to look like
vunmap_pud_range function by adding a pud_clear_huge(pud) call?

--
Regards,
Adam Turowski


^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: How to mmap any address space using huge pages?
  2021-10-21  7:16 ` Chan Kim
@ 2021-10-21  7:22   ` Adam Turowski
  0 siblings, 0 replies; 6+ messages in thread
From: Adam Turowski @ 2021-10-21  7:22 UTC (permalink / raw)
  To: Chan Kim; +Cc: kernelnewbies

> Hi, Adam
>
> Did you tell the OS to reserve huge pages?
>
> (in ubuntu, add vm.nr_hugepages = N in /etc/sysctl.conf or
>
> Add hugepages=16, hugepagesz=256M in bootcommd)
>
> Just in case you didn’t set it.

That's not the case. The HugeTLB in my case is useless, because I
don't mmap the memory.

--
Regards,
Adam

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

^ permalink raw reply	[flat|nested] 6+ messages in thread
* RE: How to mmap any address space using huge pages?
  2021-10-15 13:07 Adam Turowski
@ 2021-10-21  7:16 ` Chan Kim
  2021-10-21  7:22   ` Adam Turowski
  0 siblings, 1 reply; 6+ messages in thread
From: Chan Kim @ 2021-10-21  7:16 UTC (permalink / raw)
  To: 'Adam Turowski', kernelnewbies


[-- Attachment #1.1: Type: text/plain, Size: 1303 bytes --]

Hi, Adam

Did you tell the OS to reserve huge pages?

(in ubuntu, add vm.nr_hugepages = N in /etc/sysctl.conf or

Add hugepages=16, hugepagesz=256M in bootcommd)

Just in case you didn’t set it.

Thanks,

Chan Kim

 

From: Adam Turowski <adam.s.turowski@gmail.com> 
Sent: Friday, October 15, 2021 10:08 PM
To: kernelnewbies@kernelnewbies.org
Subject: How to mmap any address space using huge pages?

 

Hello all,


I have a device and I need to mmap it into a user space using 1GB huge pages. AFAIK I cannot use HugeTLB nor THP because they're backed by the memory. The remap_pfn_range doesn't use huge pages neither. So I used a huge_fault handler to set up the pud descriptor myself (arm64 here, so no transparent puds) and that works. The problem is that the kernel warns about a bad pud when the vma mapping is removed, the zap_pud_range function doesn't accept block puds. The vunmap_pud_range function works with block puds and checks for their presence and calls the pud_clear_huge function. If I add this function call to the zap_pud_range function, the warnings are gone.

 

Am I missing something? What is the proper way to mmap (and munmap) any address space using huge pages (esp. 1GB pages)?

 

Thanks

 

--

Regards,

Adam Turowski

 


[-- Attachment #1.2: Type: text/html, Size: 5721 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

^ permalink raw reply	[flat|nested] 6+ messages in thread
* How to mmap any address space using huge pages?
@ 2021-10-15 13:07 Adam Turowski
  2021-10-21  7:16 ` Chan Kim
  0 siblings, 1 reply; 6+ messages in thread
From: Adam Turowski @ 2021-10-15 13:07 UTC (permalink / raw)
  To: kernelnewbies


[-- Attachment #1.1: Type: text/plain, Size: 834 bytes --]

Hello all,
I have a device and I need to mmap it into a user space using 1GB huge
pages. AFAIK I cannot use HugeTLB nor THP because they're backed by the
memory. The remap_pfn_range doesn't use huge pages neither. So I used a
huge_fault handler to set up the pud descriptor myself (arm64 here, so no
transparent puds) and that works. The problem is that the kernel warns
about a bad pud when the vma mapping is removed, the zap_pud_range function
doesn't accept block puds. The vunmap_pud_range function works with block
puds and checks for their presence and calls the pud_clear_huge function.
If I add this function call to the zap_pud_range function, the warnings are
gone.

Am I missing something? What is the proper way to mmap (and munmap) any
address space using huge pages (esp. 1GB pages)?

Thanks

--
Regards,
Adam Turowski

[-- Attachment #1.2: Type: text/html, Size: 1073 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

^ permalink raw reply	[flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-10-21  7:22 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-10-16  9:46 How to mmap any address space using huge pages? Adam Turowski
2021-10-16 15:50 ` Jason Gunthorpe
2021-10-18 10:14   ` Adam Turowski
  -- strict thread matches above, loose matches on Subject: below --
2021-10-15 13:07 Adam Turowski
2021-10-21  7:16 ` Chan Kim
2021-10-21  7:22   ` Adam Turowski

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.