From: Luis Chamberlain <mcgrof@kernel.org>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Lucas De Marchi <lucas.demarchi@intel.com>,
Petr Pavlu <petr.pavlu@suse.com>,
gregkh@linuxfoundation.org, rafael@kernel.org, song@kernel.org,
lucas.de.marchi@gmail.com, christophe.leroy@csgroup.eu,
peterz@infradead.org, rppt@kernel.org, dave@stgolabs.net,
willy@infradead.org, vbabka@suse.cz, mhocko@suse.com,
dave.hansen@linux.intel.com, colin.i.king@gmail.com,
jim.cromie@gmail.com, catalin.marinas@arm.com, jbaron@akamai.com,
rick.p.edgecombe@intel.com, yujie.liu@intel.com,
david@redhat.com, tglx@linutronix.de, hch@lst.de,
patches@lists.linux.dev, linux-modules@vger.kernel.org,
linux-mm@kvack.org, linux-kernel@vger.kernel.org,
pmladek@suse.com, prarit@redhat.com, lennart@poettering.net
Subject: Re: [PATCH 2/2] module: add support to avoid duplicates early on load
Date: Thu, 25 May 2023 15:02:34 -0700 [thread overview]
Message-ID: <ZG/a+nrt4/AAUi5z@bombadil.infradead.org> (raw)
In-Reply-To: <CAHk-=whu8Wh4JP1hrc80ZvGgVW4GV6hw1vwzSiwOo9-1=Y1dWw@mail.gmail.com>
On Thu, May 25, 2023 at 02:12:49PM -0700, Linus Torvalds wrote:
> On Thu, May 25, 2023 at 11:45 AM Lucas De Marchi
> <lucas.demarchi@intel.com> wrote:
> >
> > Are you willig to merge (a possibly improved version of) your patch
> > or the userspace change is still something that would be desired?
>
> I think a user space change should still be something that people
> should look at, particularly as the kernel side patch I'm willing to
> accept doesn't catch the "completely serial" cases, only the "trying
> to load at the same time that the same module is literally busy being
> loaded".
>
> But I've cleaned up my patch a bit, and while the cleaned-up version
> is rather larger as a patch (mainly because of just also re-organizing
> the finit_module() code to do all the 'struct file' prep), I'm
> actually pretty happy with this attached patch conceptually.
>
> In this form, it actually "makes sense" to me, rather than being just
> clearly a workaround. Also, unlike the previous patch, this doesn't
> actually make any changes to the basic kernel_read_file() set of
> functions, it's all done by the module loading code itself.
>
> Luis, would you mind testing this version on your load? It still won't
> actually handle the purely serial case, so there *will* be those
> spurious double module reads from different CPU's just doing the
> things serially, but the exclusive file access region has been
> extended to not just cover the actual file content reading, but to
> cover the whole "turn it into a a real module" part too.
>
> Also, this does *not* update some of the comments in the module
> loading. I changed finit_module to use "kernel_read_file()" instead of
> "kernel_read_file_from_fd()", since it actually now has to look up the
> file descriptor anyway. But the comments still talk about that
> "from_fd" thing.
>
> Anyway, this is back to "ENTIRELY UNTESTED" territory, in that I've
> compiled this, but haven't booted it. The changes look obvious, but
> hey, mistakes happen.
>
> And the commit message is just a place-holder. Obviously. I won't sign
> off on this or write more of a commit message until it has had some
> real testing.
With 255 vcpus:
Before:
vagrant@kmod ~ $ sudo systemd-analyze
Startup finished in 41.653s (kernel) + 44.305s (userspace) = 1min 25.958s
graphical.target reached after 44.178s in userspace.
root@kmod ~ # grep "Virtual mem wasted bytes"
/sys/kernel/debug/modules/stats
Virtual mem wasted bytes 1949006968
So ~1.8 GiB.
After:
root@kmod ~ # systemd-analyze
Startup finished in 35.872s (kernel) + 41.715s (userspace) = 1min 17.588s
graphical.target reached after 41.594s in userspace.
root@kmod ~ # cat /sys/kernel/debug/modules/stats
Mods ever loaded 66
Mods failed on kread 0
Mods failed on decompress 0
Mods failed on becoming 1
Mods failed on load 0
Total module size 11268096
Total mod text size 4149248
Failed kread bytes 0
Failed decompress bytes 0
Failed becoming bytes 474688
Failed kmod bytes 0
Virtual mem wasted bytes 474688
Average mod size 170729
Average mod text size 62868
Avg fail becoming bytes 474688
Duplicate failed modules:
Module-name How-many-times Reason
cryptd 1 Becoming
root@kmod ~ # du -b /lib/modules/6.3.0-next-20230505+/kernel/crypto/cryptd.ko
475409 /lib/modules/6.3.0-next-20230505+/kernel/crypto/cryptd.ko
So yeah definitely a pretty good improvement. Sometimes the system boots
without any duplicates at all, for some reason Vs the previous attempt.
Tested-by: Luis Chamberlain <mcgrof@kernel.org>
Luis
next prev parent reply other threads:[~2023-05-25 22:03 UTC|newest]
Thread overview: 53+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-05-24 21:36 [PATCH 0/2] module: avoid all memory pressure due to duplicates Luis Chamberlain
2023-05-24 21:36 ` [PATCH 1/2] fs/kernel_read_file: add support for duplicate detection Luis Chamberlain
2023-05-24 21:52 ` Linus Torvalds
2023-05-24 21:56 ` Linus Torvalds
2023-05-24 22:07 ` Luis Chamberlain
2023-05-25 4:00 ` Linus Torvalds
2023-05-25 18:08 ` Luis Chamberlain
2023-05-25 18:35 ` Luis Chamberlain
2023-05-25 18:50 ` Linus Torvalds
2023-05-25 19:32 ` Luis Chamberlain
2023-05-25 7:01 ` Christian Brauner
2023-05-24 21:36 ` [PATCH 2/2] module: add support to avoid duplicates early on load Luis Chamberlain
2023-05-25 11:40 ` Petr Pavlu
2023-05-25 16:07 ` Linus Torvalds
2023-05-25 16:42 ` Greg KH
2023-05-25 18:22 ` Luis Chamberlain
2023-05-25 17:52 ` Linus Torvalds
2023-05-25 18:45 ` Lucas De Marchi
2023-05-25 21:12 ` Linus Torvalds
2023-05-25 22:02 ` Luis Chamberlain [this message]
2023-05-26 1:39 ` Linus Torvalds
2023-05-29 8:58 ` Johan Hovold
2023-05-29 11:00 ` Linus Torvalds
2023-05-29 12:44 ` Johan Hovold
2023-05-29 15:18 ` Johan Hovold
2023-05-30 1:55 ` Linus Torvalds
2023-05-30 9:40 ` Johan Hovold
2023-06-05 12:25 ` Johan Hovold
2023-05-30 16:22 ` Luis Chamberlain
2023-05-30 17:16 ` Lucas De Marchi
2023-05-30 19:41 ` Luis Chamberlain
2023-05-30 22:17 ` Linus Torvalds
2023-05-31 5:30 ` Lucas De Marchi
2023-05-31 0:31 ` Luis Chamberlain
2023-05-31 7:51 ` David Hildenbrand
2023-05-31 16:57 ` Luis Chamberlain
2023-06-02 15:19 ` David Hildenbrand
2023-06-02 16:04 ` Luis Chamberlain
2023-06-05 11:26 ` David Hildenbrand
2023-06-05 15:17 ` Luis Chamberlain
2023-06-05 15:28 ` Luis Chamberlain
2023-06-28 18:52 ` Luis Chamberlain
2023-06-28 20:14 ` Linus Torvalds
2023-06-28 22:07 ` Linus Torvalds
2023-06-28 23:17 ` Linus Torvalds
2023-06-29 0:18 ` Luis Chamberlain
2023-06-02 16:06 ` Linus Torvalds
2023-06-02 16:37 ` David Hildenbrand
2023-05-30 22:45 ` Dan Williams
2023-06-04 14:26 ` Rudi Heitbaum
2023-05-29 17:47 ` Linus Torvalds
2023-05-30 10:01 ` Johan Hovold
2023-05-25 16:54 ` Lucas De Marchi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZG/a+nrt4/AAUi5z@bombadil.infradead.org \
--to=mcgrof@kernel.org \
--cc=catalin.marinas@arm.com \
--cc=christophe.leroy@csgroup.eu \
--cc=colin.i.king@gmail.com \
--cc=dave.hansen@linux.intel.com \
--cc=dave@stgolabs.net \
--cc=david@redhat.com \
--cc=gregkh@linuxfoundation.org \
--cc=hch@lst.de \
--cc=jbaron@akamai.com \
--cc=jim.cromie@gmail.com \
--cc=lennart@poettering.net \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-modules@vger.kernel.org \
--cc=lucas.de.marchi@gmail.com \
--cc=lucas.demarchi@intel.com \
--cc=mhocko@suse.com \
--cc=patches@lists.linux.dev \
--cc=peterz@infradead.org \
--cc=petr.pavlu@suse.com \
--cc=pmladek@suse.com \
--cc=prarit@redhat.com \
--cc=rafael@kernel.org \
--cc=rick.p.edgecombe@intel.com \
--cc=rppt@kernel.org \
--cc=song@kernel.org \
--cc=tglx@linutronix.de \
--cc=torvalds@linux-foundation.org \
--cc=vbabka@suse.cz \
--cc=willy@infradead.org \
--cc=yujie.liu@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).