* [next PATCH v4 1/3] mm: Rename __alloc_page_frag to page_frag_alloc and __free_page_frag to page_frag_free
2017-01-04 2:38 [next PATCH v4 0/3] Page fragment updates Alexander Duyck
@ 2017-01-04 2:39 ` Alexander Duyck
2017-01-04 2:41 ` [next PATCH v4 2/3] mm: Rename __page_frag functions to __page_frag_cache, drop order from drain Alexander Duyck
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Alexander Duyck @ 2017-01-04 2:39 UTC (permalink / raw)
To: intel-wired-lan, jeffrey.t.kirsher; +Cc: linux-mm, akpm, linux-kernel
From: Alexander Duyck <alexander.h.duyck@intel.com>
This patch renames the page frag functions to be more consistent with other
APIs. Specifically we place the name page_frag first in the name and then
have either an alloc or free call name that we append as the suffix. This
makes it a bit clearer in terms of naming.
In addition we drop the leading double underscores since we are technically
no longer a backing interface and instead the front end that is called from
the networking APIs.
Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
---
v2: Fixed a comparison between a void* and 0 due to copy/paste from free_pages
v3: Dropped changes to function and make this a rename patch only.
This fixes a small performance regression I saw in some tests.
v4: Added back page_frag_alloc which had leaked through to patch 2 after
I had tried reordering them in my own queue.
include/linux/gfp.h | 6 +++---
include/linux/skbuff.h | 2 +-
mm/page_alloc.c | 10 +++++-----
net/core/skbuff.c | 8 ++++----
4 files changed, 13 insertions(+), 13 deletions(-)
diff --git a/include/linux/gfp.h b/include/linux/gfp.h
index 4175dca4ac39..6238c74e0a01 100644
--- a/include/linux/gfp.h
+++ b/include/linux/gfp.h
@@ -508,9 +508,9 @@ extern struct page *alloc_pages_vma(gfp_t gfp_mask, int order,
struct page_frag_cache;
extern void __page_frag_drain(struct page *page, unsigned int order,
unsigned int count);
-extern void *__alloc_page_frag(struct page_frag_cache *nc,
- unsigned int fragsz, gfp_t gfp_mask);
-extern void __free_page_frag(void *addr);
+extern void *page_frag_alloc(struct page_frag_cache *nc,
+ unsigned int fragsz, gfp_t gfp_mask);
+extern void page_frag_free(void *addr);
#define __free_page(page) __free_pages((page), 0)
#define free_page(addr) free_pages((addr), 0)
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index b53c0cfd417e..a410715bbef8 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -2480,7 +2480,7 @@ static inline struct sk_buff *netdev_alloc_skb_ip_align(struct net_device *dev,
static inline void skb_free_frag(void *addr)
{
- __free_page_frag(addr);
+ page_frag_free(addr);
}
void *napi_alloc_frag(unsigned int fragsz);
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 2c6d5f64feca..9534e44308b2 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -3939,8 +3939,8 @@ void __page_frag_drain(struct page *page, unsigned int order,
}
EXPORT_SYMBOL(__page_frag_drain);
-void *__alloc_page_frag(struct page_frag_cache *nc,
- unsigned int fragsz, gfp_t gfp_mask)
+void *page_frag_alloc(struct page_frag_cache *nc,
+ unsigned int fragsz, gfp_t gfp_mask)
{
unsigned int size = PAGE_SIZE;
struct page *page;
@@ -3991,19 +3991,19 @@ void *__alloc_page_frag(struct page_frag_cache *nc,
return nc->va + offset;
}
-EXPORT_SYMBOL(__alloc_page_frag);
+EXPORT_SYMBOL(page_frag_alloc);
/*
* Frees a page fragment allocated out of either a compound or order 0 page.
*/
-void __free_page_frag(void *addr)
+void page_frag_free(void *addr)
{
struct page *page = virt_to_head_page(addr);
if (unlikely(put_page_testzero(page)))
__free_pages_ok(page, compound_order(page));
}
-EXPORT_SYMBOL(__free_page_frag);
+EXPORT_SYMBOL(page_frag_free);
static void *make_alloc_exact(unsigned long addr, unsigned int order,
size_t size)
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index 5a03730fbc1a..734c71468b01 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -369,7 +369,7 @@ static void *__netdev_alloc_frag(unsigned int fragsz, gfp_t gfp_mask)
local_irq_save(flags);
nc = this_cpu_ptr(&netdev_alloc_cache);
- data = __alloc_page_frag(nc, fragsz, gfp_mask);
+ data = page_frag_alloc(nc, fragsz, gfp_mask);
local_irq_restore(flags);
return data;
}
@@ -391,7 +391,7 @@ static void *__napi_alloc_frag(unsigned int fragsz, gfp_t gfp_mask)
{
struct napi_alloc_cache *nc = this_cpu_ptr(&napi_alloc_cache);
- return __alloc_page_frag(&nc->page, fragsz, gfp_mask);
+ return page_frag_alloc(&nc->page, fragsz, gfp_mask);
}
void *napi_alloc_frag(unsigned int fragsz)
@@ -441,7 +441,7 @@ struct sk_buff *__netdev_alloc_skb(struct net_device *dev, unsigned int len,
local_irq_save(flags);
nc = this_cpu_ptr(&netdev_alloc_cache);
- data = __alloc_page_frag(nc, len, gfp_mask);
+ data = page_frag_alloc(nc, len, gfp_mask);
pfmemalloc = nc->pfmemalloc;
local_irq_restore(flags);
@@ -505,7 +505,7 @@ struct sk_buff *__napi_alloc_skb(struct napi_struct *napi, unsigned int len,
if (sk_memalloc_socks())
gfp_mask |= __GFP_MEMALLOC;
- data = __alloc_page_frag(&nc->page, len, gfp_mask);
+ data = page_frag_alloc(&nc->page, len, gfp_mask);
if (unlikely(!data))
return NULL;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 6+ messages in thread* [next PATCH v4 2/3] mm: Rename __page_frag functions to __page_frag_cache, drop order from drain
2017-01-04 2:38 [next PATCH v4 0/3] Page fragment updates Alexander Duyck
2017-01-04 2:39 ` [next PATCH v4 1/3] mm: Rename __alloc_page_frag to page_frag_alloc and __free_page_frag to page_frag_free Alexander Duyck
@ 2017-01-04 2:41 ` Alexander Duyck
2017-01-04 2:42 ` [next PATCH v4 3/3] mm: Add documentation for page fragment APIs Alexander Duyck
2017-01-06 0:02 ` [next PATCH v4 0/3] Page fragment updates Andrew Morton
3 siblings, 0 replies; 6+ messages in thread
From: Alexander Duyck @ 2017-01-04 2:41 UTC (permalink / raw)
To: intel-wired-lan, jeffrey.t.kirsher; +Cc: linux-mm, akpm, linux-kernel
From: Alexander Duyck <alexander.h.duyck@intel.com>
This patch does two things.
First it goes through and renames the __page_frag prefixed functions to
__page_frag_cache so that we can be clear that we are draining or refilling
the cache, not the frags themselves.
Second we drop the order parameter from __page_frag_cache_drain since we
don't actually need to pass it since all fragments are either order 0 or
must be a compound page.
Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
---
v2: No change
v3: No change (in theory)
v4: Pulled out page_frag_alloc rename bits which had leaked through due to me
trying out reordering some patches in my own queue.
drivers/net/ethernet/intel/igb/igb_main.c | 6 +++---
include/linux/gfp.h | 3 +--
mm/page_alloc.c | 13 +++++++------
3 files changed, 11 insertions(+), 11 deletions(-)
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index 594604e09f8d..cb08900c9cf2 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -3964,8 +3964,8 @@ static void igb_clean_rx_ring(struct igb_ring *rx_ring)
PAGE_SIZE,
DMA_FROM_DEVICE,
DMA_ATTR_SKIP_CPU_SYNC);
- __page_frag_drain(buffer_info->page, 0,
- buffer_info->pagecnt_bias);
+ __page_frag_cache_drain(buffer_info->page,
+ buffer_info->pagecnt_bias);
buffer_info->page = NULL;
}
@@ -6993,7 +6993,7 @@ static struct sk_buff *igb_fetch_rx_buffer(struct igb_ring *rx_ring,
dma_unmap_page_attrs(rx_ring->dev, rx_buffer->dma,
PAGE_SIZE, DMA_FROM_DEVICE,
DMA_ATTR_SKIP_CPU_SYNC);
- __page_frag_drain(page, 0, rx_buffer->pagecnt_bias);
+ __page_frag_cache_drain(page, rx_buffer->pagecnt_bias);
}
/* clear contents of rx_buffer */
diff --git a/include/linux/gfp.h b/include/linux/gfp.h
index 6238c74e0a01..884080404e24 100644
--- a/include/linux/gfp.h
+++ b/include/linux/gfp.h
@@ -506,8 +506,7 @@ extern struct page *alloc_pages_vma(gfp_t gfp_mask, int order,
extern void free_hot_cold_page_list(struct list_head *list, bool cold);
struct page_frag_cache;
-extern void __page_frag_drain(struct page *page, unsigned int order,
- unsigned int count);
+extern void __page_frag_cache_drain(struct page *page, unsigned int count);
extern void *page_frag_alloc(struct page_frag_cache *nc,
unsigned int fragsz, gfp_t gfp_mask);
extern void page_frag_free(void *addr);
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 9534e44308b2..4b0541cd3699 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -3904,8 +3904,8 @@ void free_pages(unsigned long addr, unsigned int order)
* drivers to provide a backing region of memory for use as either an
* sk_buff->head, or to be used in the "frags" portion of skb_shared_info.
*/
-static struct page *__page_frag_refill(struct page_frag_cache *nc,
- gfp_t gfp_mask)
+static struct page *__page_frag_cache_refill(struct page_frag_cache *nc,
+ gfp_t gfp_mask)
{
struct page *page = NULL;
gfp_t gfp = gfp_mask;
@@ -3925,19 +3925,20 @@ static struct page *__page_frag_refill(struct page_frag_cache *nc,
return page;
}
-void __page_frag_drain(struct page *page, unsigned int order,
- unsigned int count)
+void __page_frag_cache_drain(struct page *page, unsigned int count)
{
VM_BUG_ON_PAGE(page_ref_count(page) == 0, page);
if (page_ref_sub_and_test(page, count)) {
+ unsigned int order = compound_order(page);
+
if (order == 0)
free_hot_cold_page(page, false);
else
__free_pages_ok(page, order);
}
}
-EXPORT_SYMBOL(__page_frag_drain);
+EXPORT_SYMBOL(__page_frag_cache_drain);
void *page_frag_alloc(struct page_frag_cache *nc,
unsigned int fragsz, gfp_t gfp_mask)
@@ -3948,7 +3949,7 @@ void *page_frag_alloc(struct page_frag_cache *nc,
if (unlikely(!nc->va)) {
refill:
- page = __page_frag_refill(nc, gfp_mask);
+ page = __page_frag_cache_refill(nc, gfp_mask);
if (!page)
return NULL;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 6+ messages in thread* [next PATCH v4 3/3] mm: Add documentation for page fragment APIs
2017-01-04 2:38 [next PATCH v4 0/3] Page fragment updates Alexander Duyck
2017-01-04 2:39 ` [next PATCH v4 1/3] mm: Rename __alloc_page_frag to page_frag_alloc and __free_page_frag to page_frag_free Alexander Duyck
2017-01-04 2:41 ` [next PATCH v4 2/3] mm: Rename __page_frag functions to __page_frag_cache, drop order from drain Alexander Duyck
@ 2017-01-04 2:42 ` Alexander Duyck
2017-01-06 0:02 ` [next PATCH v4 0/3] Page fragment updates Andrew Morton
3 siblings, 0 replies; 6+ messages in thread
From: Alexander Duyck @ 2017-01-04 2:42 UTC (permalink / raw)
To: intel-wired-lan, jeffrey.t.kirsher; +Cc: linux-mm, akpm, linux-kernel
From: Alexander Duyck <alexander.h.duyck@intel.com>
This is a first pass at trying to add documentation for the page_frag APIs.
They may still change over time but for now I thought I would try to get
these documented so that as more network drivers and stack calls make use
of them we have one central spot to document how they are meant to be used.
Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
---
v2,v3: No change
Documentation/vm/page_frags | 42 ++++++++++++++++++++++++++++++++++++++++++
1 file changed, 42 insertions(+)
create mode 100644 Documentation/vm/page_frags
diff --git a/Documentation/vm/page_frags b/Documentation/vm/page_frags
new file mode 100644
index 000000000000..a6714565dbf9
--- /dev/null
+++ b/Documentation/vm/page_frags
@@ -0,0 +1,42 @@
+Page fragments
+--------------
+
+A page fragment is an arbitrary-length arbitrary-offset area of memory
+which resides within a 0 or higher order compound page. Multiple
+fragments within that page are individually refcounted, in the page's
+reference counter.
+
+The page_frag functions, page_frag_alloc and page_frag_free, provide a
+simple allocation framework for page fragments. This is used by the
+network stack and network device drivers to provide a backing region of
+memory for use as either an sk_buff->head, or to be used in the "frags"
+portion of skb_shared_info.
+
+In order to make use of the page fragment APIs a backing page fragment
+cache is needed. This provides a central point for the fragment allocation
+and tracks allows multiple calls to make use of a cached page. The
+advantage to doing this is that multiple calls to get_page can be avoided
+which can be expensive at allocation time. However due to the nature of
+this caching it is required that any calls to the cache be protected by
+either a per-cpu limitation, or a per-cpu limitation and forcing interrupts
+to be disabled when executing the fragment allocation.
+
+The network stack uses two separate caches per CPU to handle fragment
+allocation. The netdev_alloc_cache is used by callers making use of the
+__netdev_alloc_frag and __netdev_alloc_skb calls. The napi_alloc_cache is
+used by callers of the __napi_alloc_frag and __napi_alloc_skb calls. The
+main difference between these two calls is the context in which they may be
+called. The "netdev" prefixed functions are usable in any context as these
+functions will disable interrupts, while the "napi" prefixed functions are
+only usable within the softirq context.
+
+Many network device drivers use a similar methodology for allocating page
+fragments, but the page fragments are cached at the ring or descriptor
+level. In order to enable these cases it is necessary to provide a generic
+way of tearing down a page cache. For this reason __page_frag_cache_drain
+was implemented. It allows for freeing multiple references from a single
+page via a single call. The advantage to doing this is that it allows for
+cleaning up the multiple references that were added to a page in order to
+avoid calling get_page per allocation.
+
+Alexander Duyck, Nov 29, 2016.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [next PATCH v4 0/3] Page fragment updates
2017-01-04 2:38 [next PATCH v4 0/3] Page fragment updates Alexander Duyck
` (2 preceding siblings ...)
2017-01-04 2:42 ` [next PATCH v4 3/3] mm: Add documentation for page fragment APIs Alexander Duyck
@ 2017-01-06 0:02 ` Andrew Morton
2017-01-06 0:07 ` Alexander Duyck
3 siblings, 1 reply; 6+ messages in thread
From: Andrew Morton @ 2017-01-06 0:02 UTC (permalink / raw)
To: Alexander Duyck
Cc: intel-wired-lan, jeffrey.t.kirsher, linux-mm, linux-kernel
On Tue, 03 Jan 2017 18:38:48 -0800 Alexander Duyck <alexander.duyck@gmail.com> wrote:
> This patch series takes care of a few cleanups for the page fragments API.
>
> First we do some renames so that things are much more consistent. First we
> move the page_frag_ portion of the name to the front of the functions
> names. Secondly we split out the cache specific functions from the other
> page fragment functions by adding the word "cache" to the name.
>
> Finally I added a bit of documentation that will hopefully help to explain
> some of this. I plan to revisit this later as we get things more ironed
> out in the near future with the changes planned for the DMA setup to
> support eXpress Data Path.
>
> ---
>
> v2: Fixed a comparison between a void* and 0 due to copy/paste from free_pages
> v3: Updated first rename patch so that it is just a rename and doesn't impact
> the actual functionality to avoid performance regression.
> v4: Fix mangling that occured due to a bad merge fix when patches 1 and 2
> were swapped and then swapped back.
>
> I'm submitting this to Intel Wired Lan and Jeff Kirsher's "next-queue" for
> acceptance as I have a series of other patches for igb that are blocked by
> by these patches since I had to rename the functionality fo draining extra
> references.
>
> This series was going to be accepted for mmotm back when it was v1, however
> since then I found a few minor issues that needed to be fixed.
>
> I am hoping to get an Acked-by from Andrew Morton for these patches and
> then have them submitted to David Miller as he has said he will accept them
> if I get the Acked-by. In the meantime if these can be applied to
> next-queue while waiting on that Acked-by then I can submit the other
> patches for igb and ixgbe for testing.
The patches look fine. How about I just scoot them straight into
mainline next week? I do that occasionally, just to simplify ongoing
development and these patches are safe enough.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [next PATCH v4 0/3] Page fragment updates
2017-01-06 0:02 ` [next PATCH v4 0/3] Page fragment updates Andrew Morton
@ 2017-01-06 0:07 ` Alexander Duyck
0 siblings, 0 replies; 6+ messages in thread
From: Alexander Duyck @ 2017-01-06 0:07 UTC (permalink / raw)
To: Andrew Morton; +Cc: intel-wired-lan, Jeff Kirsher, linux-mm, linux-kernel
On Thu, Jan 5, 2017 at 4:02 PM, Andrew Morton <akpm@linux-foundation.org> wrote:
> On Tue, 03 Jan 2017 18:38:48 -0800 Alexander Duyck <alexander.duyck@gmail.com> wrote:
>
>> This patch series takes care of a few cleanups for the page fragments API.
>>
>> First we do some renames so that things are much more consistent. First we
>> move the page_frag_ portion of the name to the front of the functions
>> names. Secondly we split out the cache specific functions from the other
>> page fragment functions by adding the word "cache" to the name.
>>
>> Finally I added a bit of documentation that will hopefully help to explain
>> some of this. I plan to revisit this later as we get things more ironed
>> out in the near future with the changes planned for the DMA setup to
>> support eXpress Data Path.
>>
>> ---
>>
>> v2: Fixed a comparison between a void* and 0 due to copy/paste from free_pages
>> v3: Updated first rename patch so that it is just a rename and doesn't impact
>> the actual functionality to avoid performance regression.
>> v4: Fix mangling that occured due to a bad merge fix when patches 1 and 2
>> were swapped and then swapped back.
>>
>> I'm submitting this to Intel Wired Lan and Jeff Kirsher's "next-queue" for
>> acceptance as I have a series of other patches for igb that are blocked by
>> by these patches since I had to rename the functionality fo draining extra
>> references.
>>
>> This series was going to be accepted for mmotm back when it was v1, however
>> since then I found a few minor issues that needed to be fixed.
>>
>> I am hoping to get an Acked-by from Andrew Morton for these patches and
>> then have them submitted to David Miller as he has said he will accept them
>> if I get the Acked-by. In the meantime if these can be applied to
>> next-queue while waiting on that Acked-by then I can submit the other
>> patches for igb and ixgbe for testing.
>
> The patches look fine. How about I just scoot them straight into
> mainline next week? I do that occasionally, just to simplify ongoing
> development and these patches are safe enough.
That should work for me.
Thanks.
- Alex
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread