From: Waiman Long <longman@redhat.com>
To: Qian Cai <cai@gmx.us>, akpm@linux-foundation.org, tglx@linutronix.de
Cc: yang.shi@linux.alibaba.com, arnd@arndb.de, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2] debugobjects: scale the static pool size
Date: Tue, 20 Nov 2018 15:54:57 -0500 [thread overview]
Message-ID: <4a4398f6-74a6-9e89-21bc-41c02edda90a@redhat.com> (raw)
In-Reply-To: <20181120201432.2163-1-cai@gmx.us>
On 11/20/2018 03:14 PM, Qian Cai wrote:
> The current value of the early boot static pool size is not big enough
> for systems with large number of CPUs with timer or/and workqueue
> objects selected. As the results, systems have 60+ CPUs with both timer
> and workqueue objects enabled could trigger "ODEBUG: Out of memory.
> ODEBUG disabled". Hence, fixed it by computing it according to
> CONFIG_NR_CPUS and CONFIG_DEBUG_OBJECTS_* options.
>
> Signed-off-by: Qian Cai <cai@gmx.us>
> ---
>
> Changes since v1:
> * Removed the CONFIG_NR_CPUS check since it is always defined.
> * Introduced a minimal default pool size to start with. Otherwise, the pool
> size would be too low for systems only with a small number of CPUs.
> * Adjusted the scale factors according to data.
>
> lib/debugobjects.c | 81 ++++++++++++++++++++++++++++++++++++++++++++--
> 1 file changed, 79 insertions(+), 2 deletions(-)
>
> diff --git a/lib/debugobjects.c b/lib/debugobjects.c
> index 70935ed91125..a79f02c29617 100644
> --- a/lib/debugobjects.c
> +++ b/lib/debugobjects.c
> @@ -23,9 +23,81 @@
> #define ODEBUG_HASH_BITS 14
> #define ODEBUG_HASH_SIZE (1 << ODEBUG_HASH_BITS)
>
> -#define ODEBUG_POOL_SIZE 1024
> +#define ODEBUG_DEFAULT_POOL 512
> #define ODEBUG_POOL_MIN_LEVEL 256
>
> +/*
> + * Some debug objects are allocated during the early boot. Enabling some
> + * options like timers or workqueue objects may increase the size required
> + * significantly with large number of CPUs. For example (as today, 20 Nov.
> + * 2018),
> + *
> + * No. CPUs x 2 (worker pool) objects:
> + *
> + * start_kernel
> + * workqueue_init_early
> + * init_worker_pool
> + * init_timer_key
> + * debug_object_init
> + *
> + * No. CPUs objects (CONFIG_HIGH_RES_TIMERS):
> + *
> + * sched_init
> + * hrtick_rq_init
> + * hrtimer_init
> + *
> + * CONFIG_DEBUG_OBJECTS_WORK:
> + * No. CPUs x 6 (workqueue) objects:
> + *
> + * workqueue_init_early
> + * alloc_workqueue
> + * __alloc_workqueue_key
> + * alloc_and_link_pwqs
> + * init_pwq
> + *
> + * Also, plus No. CPUs objects:
> + *
> + * perf_event_init
> + * __init_srcu_struct
> + * init_srcu_struct_fields
> + * init_srcu_struct_nodes
> + * __init_work
> + *
> + * Increase the number a bit more in case the implmentatins are changed in
> + * the future, as it is better to avoid OOM than spending a bit more kernel
> + * memory that will/can be freed.
> + */
> +#if defined(CONFIG_DEBUG_OBJECTS_TIMERS) && \
> +!defined(CONFIG_DEBUG_OBJECTS_WORK)
> +/*
> + * With all debug objects config options selected except workqueue, kernel
> + * reports,
> + * 64-CPU: ODEBUG: 466 of 466 active objects replaced
> + * 256-CPU: ODEBUG: 1810 of 1810 active objects replaced
> + */
> +#define ODEBUG_POOL_SIZE (ODEBUG_DEFAULT_POOL + CONFIG_NR_CPUS * 10)
> +
> +#elif defined(CONFIG_DEBUG_OBJECTS_WORK) && \
> +!defined(CONFIG_DEBUG_OBJECTS_TIMERS)
> +/*
> + * all the options except the timers:
> + * 64-CPU: ODEBUG: 652 of 652 active objects replaced
> + * 256-CPU: ODEBUG: 2572 of 2572 active objects replaced
> + */
> +#define ODEBUG_POOL_SIZE (ODEBUG_DEFAULT_POOL + CONFIG_NR_CPUS * 10)
> +
> +#elif defined(CONFIG_DEBUG_OBJECTS_WORK) && \
> +defined(CONFIG_DEBUG_OBJECTS_TIMERS)
> +/*
> + * all the options:
> + * 64-CPU: ODEBUG: 1114 of 1114 active objects replaced
> + * 256-CPU: ODEBUG: 4378 of 4378 active objects replaced
> + */
> +#define ODEBUG_POOL_SIZE (ODEBUG_DEFAULT_POOL + CONFIG_NR_CPUS * 20)
> +#else
> +#define ODEBUG_POOL_SIZE ODEBUG_DEFAULT_POOL
> +#endif
> +
> #define ODEBUG_CHUNK_SHIFT PAGE_SHIFT
> #define ODEBUG_CHUNK_SIZE (1 << ODEBUG_CHUNK_SHIFT)
> #define ODEBUG_CHUNK_MASK (~(ODEBUG_CHUNK_SIZE - 1))
> @@ -58,8 +130,13 @@ static int debug_objects_fixups __read_mostly;
> static int debug_objects_warnings __read_mostly;
> static int debug_objects_enabled __read_mostly
> = CONFIG_DEBUG_OBJECTS_ENABLE_DEFAULT;
> +/*
> + * This is only used after replaced static objects, so no need to scale it
> + * to use the early boot static pool size and it has already been scaled
> + * according to actual No. CPUs in the box within debug_objects_mem_init().
> + */
> static int debug_objects_pool_size __read_mostly
> - = ODEBUG_POOL_SIZE;
> + = ODEBUG_DEFAULT_POOL;
> static int debug_objects_pool_min_level __read_mostly
> = ODEBUG_POOL_MIN_LEVEL;
> static struct debug_obj_descr *descr_test __read_mostly;
The calculation for ODEBUG_POOL_SIZE is somewhat hard to read. Maybe you
can do something like
#ifdef CONFIG_DEBUG_OBJECTS_WORK
#define ODEBUG_WORK_PCPU_CNT 10
#else
#define ODEBUG_WORK_PCPU_CNT 0
#endif
#ifdef CONFIG_DEBUG_OBJECTS_TIMERS
#define ODEBUG_TIMERS_PCPU_CNT 10
#else
#define ODEBUG_TIMERS_PCPU_CNT 0
#endif
#define ODEBUG_POOL_SIZE (ODEBUG_DEFAULT_POOL + CONFIG_NR_CPUS * \
(ODEBUG_TIMERS_PCPU_CNT + ODEBUG_WORK_PCPU_CNT))
Cheers,
Longman
next prev parent reply other threads:[~2018-11-20 20:55 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-11-20 20:14 [PATCH v2] debugobjects: scale the static pool size Qian Cai
2018-11-20 20:54 ` Waiman Long [this message]
2018-11-20 20:56 ` Thomas Gleixner
2018-11-20 23:28 ` [PATCH v3] " Qian Cai
2018-11-20 23:38 ` Waiman Long
2018-11-20 23:54 ` Qian Cai
2018-11-20 23:58 ` Joe Perches
2018-11-21 2:11 ` [PATCH v4] " Qian Cai
2018-11-21 14:45 ` Waiman Long
2018-11-22 21:56 ` Thomas Gleixner
2018-11-23 4:31 ` [PATCH] debugobjects: call debug_objects_mem_init eariler Qian Cai
2018-11-23 4:59 ` Waiman Long
2018-11-23 21:46 ` Thomas Gleixner
2018-11-24 2:54 ` Qian Cai
2018-11-24 3:01 ` [PATCH v4] debugobjects: scale the static pool size Qian Cai
2018-11-25 20:42 ` Qian Cai
2018-11-26 1:31 ` Waiman Long
2018-11-26 4:52 ` Qian Cai
2018-11-26 9:45 ` Qian Cai
2018-11-26 10:50 ` Catalin Marinas
2018-11-25 20:48 ` Qian Cai
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4a4398f6-74a6-9e89-21bc-41c02edda90a@redhat.com \
--to=longman@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=arnd@arndb.de \
--cc=cai@gmx.us \
--cc=linux-kernel@vger.kernel.org \
--cc=tglx@linutronix.de \
--cc=yang.shi@linux.alibaba.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).