* [net-next v4 01/11] page_pool: kconfig: Add flag for page pool stats
2022-02-04 0:09 [net-next v4 00/11] page_pool: Add page_pool stat counters Joe Damato
@ 2022-02-04 0:09 ` Joe Damato
2022-02-04 7:42 ` Ilias Apalodimas
2022-02-04 0:09 ` [net-next v4 02/11] page_pool: Add per-pool-per-cpu struct Joe Damato
` (10 subsequent siblings)
11 siblings, 1 reply; 20+ messages in thread
From: Joe Damato @ 2022-02-04 0:09 UTC (permalink / raw)
To: netdev, kuba, ilias.apalodimas, davem, hawk, saeed,
ttoukan.linux, brouer
Cc: Joe Damato
Control enabling / disabling page_pool_stats with a kernel config option.
Option is defaulted to N.
Signed-off-by: Joe Damato <jdamato@fastly.com>
---
net/Kconfig | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/net/Kconfig b/net/Kconfig
index 8a1f9d0..604b3eb 100644
--- a/net/Kconfig
+++ b/net/Kconfig
@@ -434,6 +434,18 @@ config NET_DEVLINK
config PAGE_POOL
bool
+config PAGE_POOL_STATS
+ default n
+ bool "Page pool stats"
+ depends on PAGE_POOL
+ help
+ Enable page pool statistics to track allocations. Stats are exported
+ to the file /proc/net/page_pool_stat. Users can examine these
+ stats to better understand how their drivers and the kernel's
+ page allocator, and the page pool interact with each other.
+
+ If unsure, say N.
+
config FAILOVER
tristate "Generic failover module"
help
--
2.7.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [net-next v4 01/11] page_pool: kconfig: Add flag for page pool stats
2022-02-04 0:09 ` [net-next v4 01/11] page_pool: kconfig: Add flag for page pool stats Joe Damato
@ 2022-02-04 7:42 ` Ilias Apalodimas
2022-02-04 17:59 ` Joe Damato
0 siblings, 1 reply; 20+ messages in thread
From: Ilias Apalodimas @ 2022-02-04 7:42 UTC (permalink / raw)
To: Joe Damato; +Cc: netdev, kuba, davem, hawk, saeed, ttoukan.linux, brouer
On Thu, Feb 03, 2022 at 04:09:23PM -0800, Joe Damato wrote:
> Control enabling / disabling page_pool_stats with a kernel config option.
> Option is defaulted to N.
>
> Signed-off-by: Joe Damato <jdamato@fastly.com>
> ---
> net/Kconfig | 12 ++++++++++++
> 1 file changed, 12 insertions(+)
>
> diff --git a/net/Kconfig b/net/Kconfig
> index 8a1f9d0..604b3eb 100644
> --- a/net/Kconfig
> +++ b/net/Kconfig
> @@ -434,6 +434,18 @@ config NET_DEVLINK
> config PAGE_POOL
> bool
>
> +config PAGE_POOL_STATS
> + default n
> + bool "Page pool stats"
> + depends on PAGE_POOL
> + help
> + Enable page pool statistics to track allocations. Stats are exported
> + to the file /proc/net/page_pool_stat. Users can examine these
There's no proc anymore
> + stats to better understand how their drivers and the kernel's
> + page allocator, and the page pool interact with each other.
> +
> + If unsure, say N.
> +
> config FAILOVER
> tristate "Generic failover module"
> help
> --
> 2.7.4
>
Regards
/Ilias
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [net-next v4 01/11] page_pool: kconfig: Add flag for page pool stats
2022-02-04 7:42 ` Ilias Apalodimas
@ 2022-02-04 17:59 ` Joe Damato
0 siblings, 0 replies; 20+ messages in thread
From: Joe Damato @ 2022-02-04 17:59 UTC (permalink / raw)
To: Ilias Apalodimas; +Cc: netdev, kuba, davem, hawk, saeed, ttoukan.linux, brouer
On Thu, Feb 3, 2022 at 11:42 PM Ilias Apalodimas
<ilias.apalodimas@linaro.org> wrote:
>
> On Thu, Feb 03, 2022 at 04:09:23PM -0800, Joe Damato wrote:
> > Control enabling / disabling page_pool_stats with a kernel config option.
> > Option is defaulted to N.
> >
> > Signed-off-by: Joe Damato <jdamato@fastly.com>
> > ---
> > net/Kconfig | 12 ++++++++++++
> > 1 file changed, 12 insertions(+)
> >
> > diff --git a/net/Kconfig b/net/Kconfig
> > index 8a1f9d0..604b3eb 100644
> > --- a/net/Kconfig
> > +++ b/net/Kconfig
> > @@ -434,6 +434,18 @@ config NET_DEVLINK
> > config PAGE_POOL
> > bool
> >
> > +config PAGE_POOL_STATS
> > + default n
> > + bool "Page pool stats"
> > + depends on PAGE_POOL
> > + help
> > + Enable page pool statistics to track allocations. Stats are exported
> > + to the file /proc/net/page_pool_stat. Users can examine these
>
> There's no proc anymore
Ah, yes. Thanks :) I've fixed this in my v5 branch.
^ permalink raw reply [flat|nested] 20+ messages in thread
* [net-next v4 02/11] page_pool: Add per-pool-per-cpu struct.
2022-02-04 0:09 [net-next v4 00/11] page_pool: Add page_pool stat counters Joe Damato
2022-02-04 0:09 ` [net-next v4 01/11] page_pool: kconfig: Add flag for page pool stats Joe Damato
@ 2022-02-04 0:09 ` Joe Damato
2022-02-04 0:09 ` [net-next v4 03/11] page_pool: Allocate and free stats structure Joe Damato
` (9 subsequent siblings)
11 siblings, 0 replies; 20+ messages in thread
From: Joe Damato @ 2022-02-04 0:09 UTC (permalink / raw)
To: netdev, kuba, ilias.apalodimas, davem, hawk, saeed,
ttoukan.linux, brouer
Cc: Joe Damato
A per pool per cpu struct is added. Empty for now as members will be added
next.
Signed-off-by: Joe Damato <jdamato@fastly.com>
---
include/net/page_pool.h | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/include/net/page_pool.h b/include/net/page_pool.h
index 79a8055..93e587d 100644
--- a/include/net/page_pool.h
+++ b/include/net/page_pool.h
@@ -135,7 +135,15 @@ struct page_pool {
refcount_t user_cnt;
u64 destroy_cnt;
+#ifdef CONFIG_PAGE_POOL_STATS
+ struct page_pool_stats __percpu *stats;
+#endif
+};
+
+#ifdef CONFIG_PAGE_POOL_STATS
+struct page_pool_stats {
};
+#endif
struct page *page_pool_alloc_pages(struct page_pool *pool, gfp_t gfp);
--
2.7.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [net-next v4 03/11] page_pool: Allocate and free stats structure
2022-02-04 0:09 [net-next v4 00/11] page_pool: Add page_pool stat counters Joe Damato
2022-02-04 0:09 ` [net-next v4 01/11] page_pool: kconfig: Add flag for page pool stats Joe Damato
2022-02-04 0:09 ` [net-next v4 02/11] page_pool: Add per-pool-per-cpu struct Joe Damato
@ 2022-02-04 0:09 ` Joe Damato
2022-02-04 0:09 ` [net-next v4 04/11] page_pool: Add macro for incrementing alloc stats Joe Damato
` (8 subsequent siblings)
11 siblings, 0 replies; 20+ messages in thread
From: Joe Damato @ 2022-02-04 0:09 UTC (permalink / raw)
To: netdev, kuba, ilias.apalodimas, davem, hawk, saeed,
ttoukan.linux, brouer
Cc: Joe Damato
This commit only alloc_percpu and free_percpu for the stats
structure.
Signed-off-by: Joe Damato <jdamato@fastly.com>
---
net/core/page_pool.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/net/core/page_pool.c b/net/core/page_pool.c
index bd62c01..5f822b0 100644
--- a/net/core/page_pool.c
+++ b/net/core/page_pool.c
@@ -73,6 +73,12 @@ static int page_pool_init(struct page_pool *pool,
pool->p.flags & PP_FLAG_PAGE_FRAG)
return -EINVAL;
+#ifdef CONFIG_PAGE_POOL_STATS
+ pool->stats = alloc_percpu(struct page_pool_stats);
+ if (!pool->stats)
+ return -ENOMEM;
+#endif
+
if (ptr_ring_init(&pool->ring, ring_qsize, GFP_KERNEL) < 0)
return -ENOMEM;
@@ -623,6 +629,9 @@ static void page_pool_free(struct page_pool *pool)
if (pool->p.flags & PP_FLAG_DMA_MAP)
put_device(pool->p.dev);
+#ifdef CONFIG_PAGE_POOL_STATS
+ free_percpu(pool->stats);
+#endif
kfree(pool);
}
--
2.7.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [net-next v4 04/11] page_pool: Add macro for incrementing alloc stats
2022-02-04 0:09 [net-next v4 00/11] page_pool: Add page_pool stat counters Joe Damato
` (2 preceding siblings ...)
2022-02-04 0:09 ` [net-next v4 03/11] page_pool: Allocate and free stats structure Joe Damato
@ 2022-02-04 0:09 ` Joe Damato
2022-02-04 0:09 ` [net-next v4 05/11] page_pool: Add fast path stat Joe Damato
` (7 subsequent siblings)
11 siblings, 0 replies; 20+ messages in thread
From: Joe Damato @ 2022-02-04 0:09 UTC (permalink / raw)
To: netdev, kuba, ilias.apalodimas, davem, hawk, saeed,
ttoukan.linux, brouer
Cc: Joe Damato
Add macro incrementing per pool per cpu stats.
Signed-off-by: Joe Damato <jdamato@fastly.com>
---
net/core/page_pool.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/net/core/page_pool.c b/net/core/page_pool.c
index 5f822b0..180e48b 100644
--- a/net/core/page_pool.c
+++ b/net/core/page_pool.c
@@ -26,6 +26,19 @@
#define BIAS_MAX LONG_MAX
+#ifdef CONFIG_PAGE_POOL_STATS
+/*
+ * this_cpu_inc_alloc_stat is intended to be used in softirq context
+ */
+#define this_cpu_inc_alloc_stat(pool, __stat) \
+ do { \
+ struct page_pool_stats __percpu *s = pool->stats; \
+ __this_cpu_inc(s->alloc.__stat); \
+ } while (0)
+#else
+#define this_cpu_inc_alloc_stat(pool, __stat)
+#endif
+
static int page_pool_init(struct page_pool *pool,
const struct page_pool_params *params)
{
--
2.7.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [net-next v4 05/11] page_pool: Add fast path stat
2022-02-04 0:09 [net-next v4 00/11] page_pool: Add page_pool stat counters Joe Damato
` (3 preceding siblings ...)
2022-02-04 0:09 ` [net-next v4 04/11] page_pool: Add macro for incrementing alloc stats Joe Damato
@ 2022-02-04 0:09 ` Joe Damato
2022-02-04 0:09 ` [net-next v4 06/11] page_pool: Add slow path order-0 stat Joe Damato
` (6 subsequent siblings)
11 siblings, 0 replies; 20+ messages in thread
From: Joe Damato @ 2022-02-04 0:09 UTC (permalink / raw)
To: netdev, kuba, ilias.apalodimas, davem, hawk, saeed,
ttoukan.linux, brouer
Cc: Joe Damato
Increment the pool's fast path allocation stat when this event occurs.
Signed-off-by: Joe Damato <jdamato@fastly.com>
---
include/net/page_pool.h | 3 +++
net/core/page_pool.c | 1 +
2 files changed, 4 insertions(+)
diff --git a/include/net/page_pool.h b/include/net/page_pool.h
index 93e587d..864a480 100644
--- a/include/net/page_pool.h
+++ b/include/net/page_pool.h
@@ -142,6 +142,9 @@ struct page_pool {
#ifdef CONFIG_PAGE_POOL_STATS
struct page_pool_stats {
+ struct {
+ u64 fast; /* fast path allocations */
+ } alloc;
};
#endif
diff --git a/net/core/page_pool.c b/net/core/page_pool.c
index 180e48b..b5bf41f 100644
--- a/net/core/page_pool.c
+++ b/net/core/page_pool.c
@@ -185,6 +185,7 @@ static struct page *__page_pool_get_cached(struct page_pool *pool)
if (likely(pool->alloc.count)) {
/* Fast-path */
page = pool->alloc.cache[--pool->alloc.count];
+ this_cpu_inc_alloc_stat(pool, fast);
} else {
page = page_pool_refill_alloc_cache(pool);
}
--
2.7.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [net-next v4 06/11] page_pool: Add slow path order-0 stat
2022-02-04 0:09 [net-next v4 00/11] page_pool: Add page_pool stat counters Joe Damato
` (4 preceding siblings ...)
2022-02-04 0:09 ` [net-next v4 05/11] page_pool: Add fast path stat Joe Damato
@ 2022-02-04 0:09 ` Joe Damato
2022-02-04 0:09 ` [net-next v4 07/11] page_pool: Add slow path high order alloc stat Joe Damato
` (5 subsequent siblings)
11 siblings, 0 replies; 20+ messages in thread
From: Joe Damato @ 2022-02-04 0:09 UTC (permalink / raw)
To: netdev, kuba, ilias.apalodimas, davem, hawk, saeed,
ttoukan.linux, brouer
Cc: Joe Damato
Increment slow path order-0 allocation stat when this event occurs.
Signed-off-by: Joe Damato <jdamato@fastly.com>
---
include/net/page_pool.h | 1 +
net/core/page_pool.c | 6 ++++--
2 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/include/net/page_pool.h b/include/net/page_pool.h
index 864a480..7d5f202 100644
--- a/include/net/page_pool.h
+++ b/include/net/page_pool.h
@@ -144,6 +144,7 @@ struct page_pool {
struct page_pool_stats {
struct {
u64 fast; /* fast path allocations */
+ u64 slow; /* slow-path order 0 allocations */
} alloc;
};
#endif
diff --git a/net/core/page_pool.c b/net/core/page_pool.c
index b5bf41f..264d8c9 100644
--- a/net/core/page_pool.c
+++ b/net/core/page_pool.c
@@ -313,10 +313,12 @@ static struct page *__page_pool_alloc_pages_slow(struct page_pool *pool,
}
/* Return last page */
- if (likely(pool->alloc.count > 0))
+ if (likely(pool->alloc.count > 0)) {
page = pool->alloc.cache[--pool->alloc.count];
- else
+ this_cpu_inc_alloc_stat(pool, slow);
+ } else {
page = NULL;
+ }
/* When page just alloc'ed is should/must have refcnt 1. */
return page;
--
2.7.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [net-next v4 07/11] page_pool: Add slow path high order alloc stat
2022-02-04 0:09 [net-next v4 00/11] page_pool: Add page_pool stat counters Joe Damato
` (5 preceding siblings ...)
2022-02-04 0:09 ` [net-next v4 06/11] page_pool: Add slow path order-0 stat Joe Damato
@ 2022-02-04 0:09 ` Joe Damato
2022-02-04 0:09 ` [net-next v4 08/11] page_pool: Add stat tracking empty ring Joe Damato
` (4 subsequent siblings)
11 siblings, 0 replies; 20+ messages in thread
From: Joe Damato @ 2022-02-04 0:09 UTC (permalink / raw)
To: netdev, kuba, ilias.apalodimas, davem, hawk, saeed,
ttoukan.linux, brouer
Cc: Joe Damato
Track high order allocations in the slow path which cause an interaction
with the buddy allocator.
Signed-off-by: Joe Damato <jdamato@fastly.com>
---
include/net/page_pool.h | 1 +
net/core/page_pool.c | 1 +
2 files changed, 2 insertions(+)
diff --git a/include/net/page_pool.h b/include/net/page_pool.h
index 7d5f202..0c4cb49 100644
--- a/include/net/page_pool.h
+++ b/include/net/page_pool.h
@@ -145,6 +145,7 @@ struct page_pool_stats {
struct {
u64 fast; /* fast path allocations */
u64 slow; /* slow-path order 0 allocations */
+ u64 slow_high_order; /* slow-path high order allocations */
} alloc;
};
#endif
diff --git a/net/core/page_pool.c b/net/core/page_pool.c
index 264d8c9..b7d0995 100644
--- a/net/core/page_pool.c
+++ b/net/core/page_pool.c
@@ -259,6 +259,7 @@ static struct page *__page_pool_alloc_page_order(struct page_pool *pool,
return NULL;
}
+ this_cpu_inc_alloc_stat(pool, slow_high_order);
page_pool_set_pp_info(pool, page);
/* Track how many pages are held 'in-flight' */
--
2.7.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [net-next v4 08/11] page_pool: Add stat tracking empty ring
2022-02-04 0:09 [net-next v4 00/11] page_pool: Add page_pool stat counters Joe Damato
` (6 preceding siblings ...)
2022-02-04 0:09 ` [net-next v4 07/11] page_pool: Add slow path high order alloc stat Joe Damato
@ 2022-02-04 0:09 ` Joe Damato
2022-02-04 0:09 ` [net-next v4 09/11] page_pool: Add stat tracking cache refill Joe Damato
` (3 subsequent siblings)
11 siblings, 0 replies; 20+ messages in thread
From: Joe Damato @ 2022-02-04 0:09 UTC (permalink / raw)
To: netdev, kuba, ilias.apalodimas, davem, hawk, saeed,
ttoukan.linux, brouer
Cc: Joe Damato
Add a stat tracking when the ptr ring is empty. When this occurs, the cache
could not be refilled and a slow path allocation was forced.
Signed-off-by: Joe Damato <jdamato@fastly.com>
---
include/net/page_pool.h | 3 +++
net/core/page_pool.c | 4 +++-
2 files changed, 6 insertions(+), 1 deletion(-)
diff --git a/include/net/page_pool.h b/include/net/page_pool.h
index 0c4cb49..b243480 100644
--- a/include/net/page_pool.h
+++ b/include/net/page_pool.h
@@ -146,6 +146,9 @@ struct page_pool_stats {
u64 fast; /* fast path allocations */
u64 slow; /* slow-path order 0 allocations */
u64 slow_high_order; /* slow-path high order allocations */
+ u64 empty; /* failed refills due to empty ptr ring, forcing
+ * slow path allocation
+ */
} alloc;
};
#endif
diff --git a/net/core/page_pool.c b/net/core/page_pool.c
index b7d0995..b25ded1 100644
--- a/net/core/page_pool.c
+++ b/net/core/page_pool.c
@@ -136,8 +136,10 @@ static struct page *page_pool_refill_alloc_cache(struct page_pool *pool)
int pref_nid; /* preferred NUMA node */
/* Quicker fallback, avoid locks when ring is empty */
- if (__ptr_ring_empty(r))
+ if (__ptr_ring_empty(r)) {
+ this_cpu_inc_alloc_stat(pool, empty);
return NULL;
+ }
/* Softirq guarantee CPU and thus NUMA node is stable. This,
* assumes CPU refilling driver RX-ring will also run RX-NAPI.
--
2.7.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [net-next v4 09/11] page_pool: Add stat tracking cache refill
2022-02-04 0:09 [net-next v4 00/11] page_pool: Add page_pool stat counters Joe Damato
` (7 preceding siblings ...)
2022-02-04 0:09 ` [net-next v4 08/11] page_pool: Add stat tracking empty ring Joe Damato
@ 2022-02-04 0:09 ` Joe Damato
2022-02-04 0:09 ` [net-next v4 10/11] page_pool: Add a stat tracking waived pages Joe Damato
` (2 subsequent siblings)
11 siblings, 0 replies; 20+ messages in thread
From: Joe Damato @ 2022-02-04 0:09 UTC (permalink / raw)
To: netdev, kuba, ilias.apalodimas, davem, hawk, saeed,
ttoukan.linux, brouer
Cc: Joe Damato
Add a stat tracking succesfull allocations which triggered a refill.
Signed-off-by: Joe Damato <jdamato@fastly.com>
---
include/net/page_pool.h | 1 +
net/core/page_pool.c | 4 +++-
2 files changed, 4 insertions(+), 1 deletion(-)
diff --git a/include/net/page_pool.h b/include/net/page_pool.h
index b243480..65cd0ca 100644
--- a/include/net/page_pool.h
+++ b/include/net/page_pool.h
@@ -149,6 +149,7 @@ struct page_pool_stats {
u64 empty; /* failed refills due to empty ptr ring, forcing
* slow path allocation
*/
+ u64 refill; /* allocations via successful refill */
} alloc;
};
#endif
diff --git a/net/core/page_pool.c b/net/core/page_pool.c
index b25ded1..4fe48ec 100644
--- a/net/core/page_pool.c
+++ b/net/core/page_pool.c
@@ -172,8 +172,10 @@ static struct page *page_pool_refill_alloc_cache(struct page_pool *pool)
} while (pool->alloc.count < PP_ALLOC_CACHE_REFILL);
/* Return last page */
- if (likely(pool->alloc.count > 0))
+ if (likely(pool->alloc.count > 0)) {
page = pool->alloc.cache[--pool->alloc.count];
+ this_cpu_inc_alloc_stat(pool, refill);
+ }
return page;
}
--
2.7.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [net-next v4 10/11] page_pool: Add a stat tracking waived pages
2022-02-04 0:09 [net-next v4 00/11] page_pool: Add page_pool stat counters Joe Damato
` (8 preceding siblings ...)
2022-02-04 0:09 ` [net-next v4 09/11] page_pool: Add stat tracking cache refill Joe Damato
@ 2022-02-04 0:09 ` Joe Damato
2022-02-04 7:44 ` Ilias Apalodimas
2022-02-04 0:09 ` [net-next v4 11/11] page_pool: Add function to batch and return stats Joe Damato
2022-02-04 13:58 ` [net-next v4 00/11] page_pool: Add page_pool stat counters Toke Høiland-Jørgensen
11 siblings, 1 reply; 20+ messages in thread
From: Joe Damato @ 2022-02-04 0:09 UTC (permalink / raw)
To: netdev, kuba, ilias.apalodimas, davem, hawk, saeed,
ttoukan.linux, brouer
Cc: Joe Damato
Track how often pages obtained from the ring cannot be added to the cache
because of a NUMA mismatch.
Signed-off-by: Joe Damato <jdamato@fastly.com>
---
include/net/page_pool.h | 1 +
net/core/page_pool.c | 1 +
2 files changed, 2 insertions(+)
diff --git a/include/net/page_pool.h b/include/net/page_pool.h
index 65cd0ca..bb87706 100644
--- a/include/net/page_pool.h
+++ b/include/net/page_pool.h
@@ -150,6 +150,7 @@ struct page_pool_stats {
* slow path allocation
*/
u64 refill; /* allocations via successful refill */
+ u64 waive; /* failed refills due to numa zone mismatch */
} alloc;
};
#endif
diff --git a/net/core/page_pool.c b/net/core/page_pool.c
index 4fe48ec..0bd084c 100644
--- a/net/core/page_pool.c
+++ b/net/core/page_pool.c
@@ -166,6 +166,7 @@ static struct page *page_pool_refill_alloc_cache(struct page_pool *pool)
* This limit stress on page buddy alloactor.
*/
page_pool_return_page(pool, page);
+ this_cpu_inc_alloc_stat(pool, waive);
page = NULL;
break;
}
--
2.7.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [net-next v4 10/11] page_pool: Add a stat tracking waived pages
2022-02-04 0:09 ` [net-next v4 10/11] page_pool: Add a stat tracking waived pages Joe Damato
@ 2022-02-04 7:44 ` Ilias Apalodimas
2022-02-04 17:59 ` Joe Damato
0 siblings, 1 reply; 20+ messages in thread
From: Ilias Apalodimas @ 2022-02-04 7:44 UTC (permalink / raw)
To: Joe Damato; +Cc: netdev, kuba, davem, hawk, saeed, ttoukan.linux, brouer
Hi Joe,
On Thu, Feb 03, 2022 at 04:09:32PM -0800, Joe Damato wrote:
> Track how often pages obtained from the ring cannot be added to the cache
> because of a NUMA mismatch.
>
> Signed-off-by: Joe Damato <jdamato@fastly.com>
> ---
> include/net/page_pool.h | 1 +
> net/core/page_pool.c | 1 +
> 2 files changed, 2 insertions(+)
>
> diff --git a/include/net/page_pool.h b/include/net/page_pool.h
> index 65cd0ca..bb87706 100644
> --- a/include/net/page_pool.h
> +++ b/include/net/page_pool.h
> @@ -150,6 +150,7 @@ struct page_pool_stats {
> * slow path allocation
> */
> u64 refill; /* allocations via successful refill */
> + u64 waive; /* failed refills due to numa zone mismatch */
> } alloc;
> };
> #endif
> diff --git a/net/core/page_pool.c b/net/core/page_pool.c
> index 4fe48ec..0bd084c 100644
> --- a/net/core/page_pool.c
> +++ b/net/core/page_pool.c
> @@ -166,6 +166,7 @@ static struct page *page_pool_refill_alloc_cache(struct page_pool *pool)
> * This limit stress on page buddy alloactor.
> */
> page_pool_return_page(pool, page);
> + this_cpu_inc_alloc_stat(pool, waive);
> page = NULL;
> break;
> }
> --
> 2.7.4
>
Personally i'd find it easier to read if patches 1-10 were squashed in a
single commit.
Regards
/Ilias
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [net-next v4 10/11] page_pool: Add a stat tracking waived pages
2022-02-04 7:44 ` Ilias Apalodimas
@ 2022-02-04 17:59 ` Joe Damato
0 siblings, 0 replies; 20+ messages in thread
From: Joe Damato @ 2022-02-04 17:59 UTC (permalink / raw)
To: Ilias Apalodimas; +Cc: netdev, kuba, davem, hawk, saeed, ttoukan.linux, brouer
On Thu, Feb 3, 2022 at 11:44 PM Ilias Apalodimas
<ilias.apalodimas@linaro.org> wrote:
>
> Hi Joe,
>
> On Thu, Feb 03, 2022 at 04:09:32PM -0800, Joe Damato wrote:
> > Track how often pages obtained from the ring cannot be added to the cache
> > because of a NUMA mismatch.
> >
> > Signed-off-by: Joe Damato <jdamato@fastly.com>
> > ---
> > include/net/page_pool.h | 1 +
> > net/core/page_pool.c | 1 +
> > 2 files changed, 2 insertions(+)
> >
> > diff --git a/include/net/page_pool.h b/include/net/page_pool.h
> > index 65cd0ca..bb87706 100644
> > --- a/include/net/page_pool.h
> > +++ b/include/net/page_pool.h
> > @@ -150,6 +150,7 @@ struct page_pool_stats {
> > * slow path allocation
> > */
> > u64 refill; /* allocations via successful refill */
> > + u64 waive; /* failed refills due to numa zone mismatch */
> > } alloc;
> > };
> > #endif
> > diff --git a/net/core/page_pool.c b/net/core/page_pool.c
> > index 4fe48ec..0bd084c 100644
> > --- a/net/core/page_pool.c
> > +++ b/net/core/page_pool.c
> > @@ -166,6 +166,7 @@ static struct page *page_pool_refill_alloc_cache(struct page_pool *pool)
> > * This limit stress on page buddy alloactor.
> > */
> > page_pool_return_page(pool, page);
> > + this_cpu_inc_alloc_stat(pool, waive);
> > page = NULL;
> > break;
> > }
> > --
> > 2.7.4
> >
>
> Personally i'd find it easier to read if patches 1-10 were squashed in a
> single commit.
Thanks for the feedback. I've squashed patches 1-10 to a single commit
in my v5 branch.
^ permalink raw reply [flat|nested] 20+ messages in thread
* [net-next v4 11/11] page_pool: Add function to batch and return stats
2022-02-04 0:09 [net-next v4 00/11] page_pool: Add page_pool stat counters Joe Damato
` (9 preceding siblings ...)
2022-02-04 0:09 ` [net-next v4 10/11] page_pool: Add a stat tracking waived pages Joe Damato
@ 2022-02-04 0:09 ` Joe Damato
2022-02-04 7:46 ` Ilias Apalodimas
2022-02-04 13:58 ` [net-next v4 00/11] page_pool: Add page_pool stat counters Toke Høiland-Jørgensen
11 siblings, 1 reply; 20+ messages in thread
From: Joe Damato @ 2022-02-04 0:09 UTC (permalink / raw)
To: netdev, kuba, ilias.apalodimas, davem, hawk, saeed,
ttoukan.linux, brouer
Cc: Joe Damato
Adds a function page_pool_get_stats which can be used by drivers to obtain
the batched stats for a specified page pool.
Signed-off-by: Joe Damato <jdamato@fastly.com>
---
include/net/page_pool.h | 9 +++++++++
net/core/page_pool.c | 25 +++++++++++++++++++++++++
2 files changed, 34 insertions(+)
diff --git a/include/net/page_pool.h b/include/net/page_pool.h
index bb87706..5257e46 100644
--- a/include/net/page_pool.h
+++ b/include/net/page_pool.h
@@ -153,6 +153,15 @@ struct page_pool_stats {
u64 waive; /* failed refills due to numa zone mismatch */
} alloc;
};
+
+/*
+ * Drivers that wish to harvest page pool stats and report them to users
+ * (perhaps via ethtool, debugfs, or another mechanism) can allocate a
+ * struct page_pool_stats and call page_pool_get_stats to get the batched pcpu
+ * stats.
+ */
+struct page_pool_stats *page_pool_get_stats(struct page_pool *pool,
+ struct page_pool_stats *stats);
#endif
struct page *page_pool_alloc_pages(struct page_pool *pool, gfp_t gfp);
diff --git a/net/core/page_pool.c b/net/core/page_pool.c
index 0bd084c..076593bb 100644
--- a/net/core/page_pool.c
+++ b/net/core/page_pool.c
@@ -35,6 +35,31 @@
struct page_pool_stats __percpu *s = pool->stats; \
__this_cpu_inc(s->alloc.__stat); \
} while (0)
+
+struct page_pool_stats *page_pool_get_stats(struct page_pool *pool,
+ struct page_pool_stats *stats)
+{
+ int cpu = 0;
+
+ if (!stats)
+ return NULL;
+
+ for_each_possible_cpu(cpu) {
+ const struct page_pool_stats *pcpu =
+ per_cpu_ptr(pool->stats, cpu);
+
+ stats->alloc.fast += pcpu->alloc.fast;
+ stats->alloc.slow += pcpu->alloc.slow;
+ stats->alloc.slow_high_order +=
+ pcpu->alloc.slow_high_order;
+ stats->alloc.empty += pcpu->alloc.empty;
+ stats->alloc.refill += pcpu->alloc.refill;
+ stats->alloc.waive += pcpu->alloc.waive;
+ }
+
+ return stats;
+}
+EXPORT_SYMBOL(page_pool_get_stats);
#else
#define this_cpu_inc_alloc_stat(pool, __stat)
#endif
--
2.7.4
^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [net-next v4 11/11] page_pool: Add function to batch and return stats
2022-02-04 0:09 ` [net-next v4 11/11] page_pool: Add function to batch and return stats Joe Damato
@ 2022-02-04 7:46 ` Ilias Apalodimas
2022-02-04 18:00 ` Joe Damato
0 siblings, 1 reply; 20+ messages in thread
From: Ilias Apalodimas @ 2022-02-04 7:46 UTC (permalink / raw)
To: Joe Damato; +Cc: netdev, kuba, davem, hawk, saeed, ttoukan.linux, brouer
On Thu, Feb 03, 2022 at 04:09:33PM -0800, Joe Damato wrote:
> Adds a function page_pool_get_stats which can be used by drivers to obtain
> the batched stats for a specified page pool.
>
> Signed-off-by: Joe Damato <jdamato@fastly.com>
> ---
> include/net/page_pool.h | 9 +++++++++
> net/core/page_pool.c | 25 +++++++++++++++++++++++++
> 2 files changed, 34 insertions(+)
>
> diff --git a/include/net/page_pool.h b/include/net/page_pool.h
> index bb87706..5257e46 100644
> --- a/include/net/page_pool.h
> +++ b/include/net/page_pool.h
> @@ -153,6 +153,15 @@ struct page_pool_stats {
> u64 waive; /* failed refills due to numa zone mismatch */
> } alloc;
> };
> +
> +/*
> + * Drivers that wish to harvest page pool stats and report them to users
> + * (perhaps via ethtool, debugfs, or another mechanism) can allocate a
> + * struct page_pool_stats and call page_pool_get_stats to get the batched pcpu
> + * stats.
> + */
> +struct page_pool_stats *page_pool_get_stats(struct page_pool *pool,
> + struct page_pool_stats *stats);
> #endif
>
> struct page *page_pool_alloc_pages(struct page_pool *pool, gfp_t gfp);
> diff --git a/net/core/page_pool.c b/net/core/page_pool.c
> index 0bd084c..076593bb 100644
> --- a/net/core/page_pool.c
> +++ b/net/core/page_pool.c
> @@ -35,6 +35,31 @@
> struct page_pool_stats __percpu *s = pool->stats; \
> __this_cpu_inc(s->alloc.__stat); \
> } while (0)
> +
> +struct page_pool_stats *page_pool_get_stats(struct page_pool *pool,
> + struct page_pool_stats *stats)
> +{
> + int cpu = 0;
> +
> + if (!stats)
> + return NULL;
> +
> + for_each_possible_cpu(cpu) {
> + const struct page_pool_stats *pcpu =
> + per_cpu_ptr(pool->stats, cpu);
> +
> + stats->alloc.fast += pcpu->alloc.fast;
> + stats->alloc.slow += pcpu->alloc.slow;
> + stats->alloc.slow_high_order +=
> + pcpu->alloc.slow_high_order;
> + stats->alloc.empty += pcpu->alloc.empty;
> + stats->alloc.refill += pcpu->alloc.refill;
> + stats->alloc.waive += pcpu->alloc.waive;
> + }
> +
> + return stats;
> +}
> +EXPORT_SYMBOL(page_pool_get_stats);
You don't really need to return a pointer here. Just make the return code a
bool
Regards
/Ilias
> #else
> #define this_cpu_inc_alloc_stat(pool, __stat)
> #endif
> --
> 2.7.4
>
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [net-next v4 11/11] page_pool: Add function to batch and return stats
2022-02-04 7:46 ` Ilias Apalodimas
@ 2022-02-04 18:00 ` Joe Damato
0 siblings, 0 replies; 20+ messages in thread
From: Joe Damato @ 2022-02-04 18:00 UTC (permalink / raw)
To: Ilias Apalodimas; +Cc: netdev, kuba, davem, hawk, saeed, ttoukan.linux, brouer
On Thu, Feb 3, 2022 at 11:46 PM Ilias Apalodimas
<ilias.apalodimas@linaro.org> wrote:
>
> On Thu, Feb 03, 2022 at 04:09:33PM -0800, Joe Damato wrote:
> > Adds a function page_pool_get_stats which can be used by drivers to obtain
> > the batched stats for a specified page pool.
> >
> > Signed-off-by: Joe Damato <jdamato@fastly.com>
> > ---
> > include/net/page_pool.h | 9 +++++++++
> > net/core/page_pool.c | 25 +++++++++++++++++++++++++
> > 2 files changed, 34 insertions(+)
> >
> > diff --git a/include/net/page_pool.h b/include/net/page_pool.h
> > index bb87706..5257e46 100644
> > --- a/include/net/page_pool.h
> > +++ b/include/net/page_pool.h
> > @@ -153,6 +153,15 @@ struct page_pool_stats {
> > u64 waive; /* failed refills due to numa zone mismatch */
> > } alloc;
> > };
> > +
> > +/*
> > + * Drivers that wish to harvest page pool stats and report them to users
> > + * (perhaps via ethtool, debugfs, or another mechanism) can allocate a
> > + * struct page_pool_stats and call page_pool_get_stats to get the batched pcpu
> > + * stats.
> > + */
> > +struct page_pool_stats *page_pool_get_stats(struct page_pool *pool,
> > + struct page_pool_stats *stats);
> > #endif
> >
> > struct page *page_pool_alloc_pages(struct page_pool *pool, gfp_t gfp);
> > diff --git a/net/core/page_pool.c b/net/core/page_pool.c
> > index 0bd084c..076593bb 100644
> > --- a/net/core/page_pool.c
> > +++ b/net/core/page_pool.c
> > @@ -35,6 +35,31 @@
> > struct page_pool_stats __percpu *s = pool->stats; \
> > __this_cpu_inc(s->alloc.__stat); \
> > } while (0)
> > +
> > +struct page_pool_stats *page_pool_get_stats(struct page_pool *pool,
> > + struct page_pool_stats *stats)
> > +{
> > + int cpu = 0;
> > +
> > + if (!stats)
> > + return NULL;
> > +
> > + for_each_possible_cpu(cpu) {
> > + const struct page_pool_stats *pcpu =
> > + per_cpu_ptr(pool->stats, cpu);
> > +
> > + stats->alloc.fast += pcpu->alloc.fast;
> > + stats->alloc.slow += pcpu->alloc.slow;
> > + stats->alloc.slow_high_order +=
> > + pcpu->alloc.slow_high_order;
> > + stats->alloc.empty += pcpu->alloc.empty;
> > + stats->alloc.refill += pcpu->alloc.refill;
> > + stats->alloc.waive += pcpu->alloc.waive;
> > + }
> > +
> > + return stats;
> > +}
> > +EXPORT_SYMBOL(page_pool_get_stats);
>
> You don't really need to return a pointer here. Just make the return code a
> bool
OK. Updated page_pool_get_stats to return a bool in my v5 branch.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [net-next v4 00/11] page_pool: Add page_pool stat counters
2022-02-04 0:09 [net-next v4 00/11] page_pool: Add page_pool stat counters Joe Damato
` (10 preceding siblings ...)
2022-02-04 0:09 ` [net-next v4 11/11] page_pool: Add function to batch and return stats Joe Damato
@ 2022-02-04 13:58 ` Toke Høiland-Jørgensen
2022-02-04 18:04 ` Joe Damato
11 siblings, 1 reply; 20+ messages in thread
From: Toke Høiland-Jørgensen @ 2022-02-04 13:58 UTC (permalink / raw)
To: Joe Damato, netdev, kuba, ilias.apalodimas, davem, hawk, saeed,
ttoukan.linux, brouer
Cc: Joe Damato
Joe Damato <jdamato@fastly.com> writes:
> Greetings:
>
> Welcome to v4.
>
> This revision changes stats to be per-cpu and per-pool after getting
> feedback from Ilias, Tariq, and Jesper.
FYI, Jesper is on holiday until the 12th, so it may be be a little while
until he gets back to you on this :)
-Toke
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [net-next v4 00/11] page_pool: Add page_pool stat counters
2022-02-04 13:58 ` [net-next v4 00/11] page_pool: Add page_pool stat counters Toke Høiland-Jørgensen
@ 2022-02-04 18:04 ` Joe Damato
0 siblings, 0 replies; 20+ messages in thread
From: Joe Damato @ 2022-02-04 18:04 UTC (permalink / raw)
To: Toke Høiland-Jørgensen
Cc: netdev, kuba, ilias.apalodimas, davem, hawk, saeed,
ttoukan.linux, brouer
On Fri, Feb 4, 2022 at 5:58 AM Toke Høiland-Jørgensen <toke@redhat.com> wrote:
>
> Joe Damato <jdamato@fastly.com> writes:
>
> > Greetings:
> >
> > Welcome to v4.
> >
> > This revision changes stats to be per-cpu and per-pool after getting
> > feedback from Ilias, Tariq, and Jesper.
>
> FYI, Jesper is on holiday until the 12th, so it may be be a little while
> until he gets back to you on this :)
Thanks for letting me know.
I've queued some changes per Ilias' feedback for the v5, but perhaps I
should hold off on submitting those until Jesper has returned and has
a chance to take a look at this revision.
Thanks,
Joe
^ permalink raw reply [flat|nested] 20+ messages in thread