From: "Paul E. McKenney" <paulmck@linux.ibm.com>
To: Tejun Heo <tj@kernel.org>
Cc: Angel Shtilianov <angel.shtilianov@siteground.com>,
linux-mm@kvack.org, dennis@kernel.org, cl@linux.com,
jeyu@kernel.org
Subject: Re: Ipmi modules and linux-4.19.1
Date: Thu, 20 Dec 2018 08:03:13 -0800 [thread overview]
Message-ID: <20181220160313.GB4170@linux.ibm.com> (raw)
In-Reply-To: <20181220154217.GB2509588@devbig004.ftw2.facebook.com>
On Thu, Dec 20, 2018 at 07:42:17AM -0800, Tejun Heo wrote:
> Hello, Angel.
>
> (cc'ing Paul for SRCU)
>
> On Thu, Dec 20, 2018 at 09:55:10AM +0200, Angel Shtilianov wrote:
> > Hi everybody.
> > A couple of days I've decided to migrate several servers on
> > linux-4.19. What I've observed is that I have no /dev/ipmi. After
> > taking a look into the boot log I've found that ipmi modules are
> > complaining about percpu memory allocation failures:
> > https://pastebin.com/MCDssZzV
> ...
> > -#define PERCPU_DYNAMIC_RESERVE (28 << 10)
> > +#define PERCPU_DYNAMIC_RESERVE (28 << 11)
>
> So, you prolly just needed to bump this number. The reserved percpu
> area is used to accommodate static percpu variables used by modules.
> They are special because code generation assumes static symbols aren't
> too far from the program counter. The usual dynamic percpu area is
> way high up in vmalloc area, so if we put static percpu allocations
> there, they go out of range for module symbol relocations.
>
> The reserved area has some issues.
>
> 1. The area is not dynamically mapped, meaning that however much we
> reserve is hard allocated on boot for future module uses, so we
> don't can't increase it willy-nilly.
>
> 2. There is no mechanism to adjust the size dynamically. 28k is just
> a number I pulled out of my ass after looking at some common
> configs like a decade ago, so it being low now isn't too
> surprising. Provided that we can't make it run-time dynamic (and I
> can't think of a way to do that), the right thing to do would be
> sizing it during build with some buffer and allow it to be
> overridden boot time. This is definitely doable.
>
> BTW, ipmi's extra usage, 8k, is coming from the use of static SRCU.
> Paul, that's quite a bit of percpu memory to reserve statically.
> Would it be possible to make srcu_struct init dynamic so that it can
> use the normal percpu_alloc? That way, this problem can be completely
> side-stepped and it only occupies percpu memory which tends to be
> pretty expensive unless ipmi is actually initialized.
Yes, it is possible. Just do something like this:
struct srcu_struct my_srcu_struct;
And before the first use of my_srcu_struct, do this:
init_srcu_struct(&my_srcu_struct);
This will result in alloc_percpu() being invoked to allocate the
needed per-CPU space.
If my_srcu_struct is used in a module or some such, then to avoid memory
leaks, after the last use of my_srcu_struct, do this:
cleanup_srcu_struct(&my_srcu_struct);
There are several places in the kernel that take this approach.
Thanx, Paul
next prev parent reply other threads:[~2018-12-20 16:03 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-12-20 7:55 Ipmi modules and linux-4.19.1 Angel Shtilianov
2018-12-20 15:42 ` Tejun Heo
2018-12-20 16:00 ` Tejun Heo
2018-12-20 16:03 ` Paul E. McKenney [this message]
2018-12-20 16:04 ` Paul E. McKenney
2018-12-20 16:05 ` Tejun Heo
2018-12-20 16:22 ` Paul E. McKenney
2018-12-20 21:59 ` Corey Minyard
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20181220160313.GB4170@linux.ibm.com \
--to=paulmck@linux.ibm.com \
--cc=angel.shtilianov@siteground.com \
--cc=cl@linux.com \
--cc=dennis@kernel.org \
--cc=jeyu@kernel.org \
--cc=linux-mm@kvack.org \
--cc=tj@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).