From: Josh Poimboeuf <jpoimboe@redhat.com>
To: Petr Mladek <pmladek@suse.com>
Cc: Jessica Yu <jeyu@redhat.com>, Jiri Kosina <jikos@kernel.org>,
Miroslav Benes <mbenes@suse.cz>,
linux-kernel@vger.kernel.org, live-patching@vger.kernel.org,
Michael Ellerman <mpe@ellerman.id.au>,
Heiko Carstens <heiko.carstens@de.ibm.com>,
x86@kernel.org, linuxppc-dev@lists.ozlabs.org,
linux-s390@vger.kernel.org, Vojtech Pavlik <vojtech@suse.com>,
Jiri Slaby <jslaby@suse.cz>,
Chris J Arges <chris.j.arges@canonical.com>,
Andy Lutomirski <luto@kernel.org>, Ingo Molnar <mingo@kernel.org>,
Peter Zijlstra <peterz@infradead.org>
Subject: Re: [PATCH v3 13/15] livepatch: change to a per-task consistency model
Date: Wed, 11 Jan 2017 09:26:39 -0600 [thread overview]
Message-ID: <20170111152639.l6ehubjuys66if4l@treble> (raw)
In-Reply-To: <20170111151828.GJ20785@pathway.suse.cz>
On Wed, Jan 11, 2017 at 04:18:28PM +0100, Petr Mladek wrote:
> On Tue 2017-01-10 14:46:46, Josh Poimboeuf wrote:
> > On Tue, Jan 10, 2017 at 02:00:58PM +0100, Petr Mladek wrote:
> > > On Thu 2016-12-22 12:31:37, Josh Poimboeuf wrote:
> > > > On Thu, Dec 22, 2016 at 03:34:52PM +0100, Petr Mladek wrote:
> > > > > On Wed 2016-12-21 15:25:05, Josh Poimboeuf wrote:
> > > > > > On Tue, Dec 20, 2016 at 06:32:46PM +0100, Petr Mladek wrote:
> > > > > > > On Thu 2016-12-08 12:08:38, Josh Poimboeuf wrote:
> > > > > > > > + read_unlock(&tasklist_lock);
> > > > > > > > +
> > > > > > > > + /*
> > > > > > > > + * Ditto for the idle "swapper" tasks, though they never cross the
> > > > > > > > + * syscall barrier. Instead they switch over in cpu_idle_loop().
> > > > > > > > + */
> > > > > > > > + get_online_cpus();
> > > > > > > > + for_each_online_cpu(cpu)
> > > > > > > > + set_tsk_thread_flag(idle_task(cpu), TIF_PATCH_PENDING);
> > > > > > > > + put_online_cpus();
> > > > > > >
> > > > > > > Also this stage need to be somehow handled by CPU coming/going
> > > > > > > handlers.
> > > > > >
> > > > > > Here I think we could automatically switch any offline CPUs' idle tasks.
> > > > > > And something similar in klp_try_complete_transition().
> > > > >
> > > > > We still need to make sure to do not race with the cpu_up()/cpu_down()
> > > > > calls.
> > > >
> > > > Hm, maybe we'd need to call cpu_hotplug_disable() before switching the
> > > > offline idle tasks?
> > > >
> > > > > I would use here the trick with for_each_possible_cpu() and let
> > > > > the migration for the stack check.
> > > >
> > > > There are a few issues with that:
> > > >
> > > > 1) The idle task of a missing CPU doesn't *have* a stack, so it doesn't
> > > > make much sense to try to check it.
> > > >
> > > > 2) We can't rely *only* on the stack check, because not all arches have
> > > > it. The other way to migrate idle tasks is from the idle loop switch
> > > > point. But if the task's CPU is down, its idle loop isn't running so
> > > > it can't migrate.
> > > >
> > > > (Note this is currently a theoretical point: we currently don't allow
> > > > such arches to use the consistency model anyway because there's no
> > > > way for them to migrate kthreads.)
> > >
> > > Good points. My only concern is that the transaction might take a long
> > > or even forever. I am not sure if it is wise to disable cpu_hotplug
> > > for the entire transaction.
> > >
> > > A compromise might be to disable cpu hotplug only when the task
> > > state is manipulated a more complex way. Hmm, cpu_hotplug_disable()
> > > looks like a rather costly function. We should not call it in
> > > klp_try_complete_transition(). But we could do:
> > >
> > > 1. When the patch is being enabled, disable cpu hotplug,
> > > go through each_possible_cpu and setup the transaction
> > > only for CPUs that are online. Then we could enable
> > > the hotplug again.
> > >
> > > 2. Check only each_online_cpu in klp_try_complete_transition().
> > > If all tasks are migrated, disable cpu hotplug and re-check
> > > idle tasks on online CPUs. If any is not migrated, enable
> > > hotplug and return failure. Othewise, continue with
> > > completion of the transaction. [*]
> > >
> > > 3. In klp_complete_transition, update all tasks including
> > > the offline CPUs and enable cpu hotplug again.
> > >
> > > If the re-check in the 2nd step looks ugly, we could add some hotlug
> > > notifiers to make sure that enabled/disabled CPUs are in a reasonable
> > > state. We still should disable the hotplug in the 1st and 3rd step.
> > >
> > > BTW: There is a new API for the cpu hotplug callbacks. I was involved
> > > in one conversion. You might take inspiration in
> > > drivers/thermal/intel_powerclamp.c. See cpuhp_setup_state_nocalls()
> > > there.
> >
> > Backing up a bit, although I brought up cpu_hotplug_disable(), I think I
> > misunderstood the race you mentioned. I actually don't think
> > cpu_hotplug_disable() is necessary.
>
> Great backing! You made me to study the difference. If I get it
> correctly:
>
> + cpu_hotplug_disable() works like a writer lock. It gets
> exclusive access via cpu_hotplug_begin(). A side effect
> is that do_cpu_up() and do_cpu_down() do not wait. They
> return -EBUSY if hotplug is disabled.
>
> + get_online_cpus() is kind of reader lock. It makes sure
> that all the hotplug operations are finished and "softly"
> blocks other further operation. By "softly" I mean that
> the operations wait for the exclusive (write) access
> in cpu_hotplug_begin().
>
> IMHO, we really have to use get_online_cpus() and avoid the
> the "hard" blocking.
>
>
> > What do you think about something like the following:
>
> > In klp_start_transition:
> >
> > get_online_cpus();
> > for_each_possible_cpu(cpu)
> > set_tsk_thread_flag(idle_task(cpu), TIF_PATCH_PENDING);
> > put_online_cpus();
> >
> > In klp_try_complete_transition:
> >
> > get_online_cpus();
> > for_each_possible_cpu(cpu) {
> > task = idle_task(cpu);
> > if (cpu_online(cpu)) {
> > if (!klp_try_switch_task(task))
> > complete = false;
> > } else if (task->patch_state != klp_target_state) {
> > /* offline CPU idle tasks can be switched immediately */
> > clear_tsk_thread_flag(task, TIF_PATCH_PENDING);
> > task->patch_state = klp_target_state;
> > }
> > }
> > put_online_cpus();
>
> I like the idea. You are right that it is enough to always get/put
> CPUs only when a state of the per-CPU idle tasks are manipulated.
> In the meantime, we are safe because of the consistency model
> (clever ftrace handler).
>
> Note that we have to use for_each_possible_cpu() everywhere,
> e.g. in klp_init_transition(), klp_complete_transition().
> Otherwise, we might see an inconsistent state.
>
> For example, klp_ftrace_handler() might see KLP_UNDEFINED state
> if we do not set a valid one in klp_init_transition() and a CPU
> gets online.
Agreed! I will try to have v4 soon-ish.
--
Josh
next prev parent reply other threads:[~2017-01-11 15:26 UTC|newest]
Thread overview: 78+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-12-08 18:08 [PATCH v3 00/15] livepatch: hybrid consistency model Josh Poimboeuf
2016-12-08 18:08 ` [PATCH v3 01/15] stacktrace/x86: add function for detecting reliable stack traces Josh Poimboeuf
2016-12-16 13:07 ` Petr Mladek
2016-12-16 22:09 ` Josh Poimboeuf
2016-12-19 16:25 ` Miroslav Benes
2016-12-19 17:25 ` Josh Poimboeuf
2016-12-19 18:23 ` Miroslav Benes
2016-12-20 9:39 ` Petr Mladek
2016-12-20 21:21 ` Josh Poimboeuf
2016-12-08 18:08 ` [PATCH v3 02/15] x86/entry: define _TIF_ALLWORK_MASK flags explicitly Josh Poimboeuf
2016-12-16 14:17 ` Petr Mladek
2016-12-16 22:13 ` Josh Poimboeuf
2016-12-19 16:39 ` Miroslav Benes
2017-01-10 8:49 ` Kamalesh Babulal
2016-12-08 18:08 ` [PATCH v3 03/15] livepatch: temporary stubs for klp_patch_pending() and klp_update_patch_state() Josh Poimboeuf
2016-12-16 14:41 ` Petr Mladek
2016-12-16 22:15 ` Josh Poimboeuf
2016-12-08 18:08 ` [PATCH v3 04/15] livepatch/x86: add TIF_PATCH_PENDING thread flag Josh Poimboeuf
2016-12-08 18:27 ` Andy Lutomirski
2016-12-16 15:39 ` Petr Mladek
2016-12-21 13:54 ` Miroslav Benes
2017-01-11 7:06 ` Kamalesh Babulal
2016-12-08 18:08 ` [PATCH v3 05/15] livepatch/powerpc: " Josh Poimboeuf
2016-12-16 16:00 ` Petr Mladek
2016-12-21 14:30 ` Miroslav Benes
2017-01-10 8:29 ` Kamalesh Babulal
2016-12-08 18:08 ` [PATCH v3 06/15] livepatch/s390: reorganize TIF thread flag bits Josh Poimboeuf
2016-12-21 15:29 ` Miroslav Benes
2016-12-08 18:08 ` [PATCH v3 07/15] livepatch/s390: add TIF_PATCH_PENDING thread flag Josh Poimboeuf
2016-12-08 18:08 ` [PATCH v3 08/15] livepatch: separate enabled and patched states Josh Poimboeuf
2016-12-16 16:21 ` Petr Mladek
2016-12-23 12:54 ` Miroslav Benes
2017-01-10 9:10 ` Kamalesh Babulal
2016-12-08 18:08 ` [PATCH v3 09/15] livepatch: remove unnecessary object loaded check Josh Poimboeuf
2016-12-16 16:26 ` Petr Mladek
2016-12-23 12:58 ` Miroslav Benes
2017-01-10 9:14 ` Kamalesh Babulal
2016-12-08 18:08 ` [PATCH v3 10/15] livepatch: move patching functions into patch.c Josh Poimboeuf
2016-12-16 16:49 ` Petr Mladek
2016-12-23 13:06 ` Miroslav Benes
2017-01-10 9:15 ` Kamalesh Babulal
2016-12-08 18:08 ` [PATCH v3 11/15] livepatch: use kstrtobool() in enabled_store() Josh Poimboeuf
2016-12-16 16:55 ` Petr Mladek
2016-12-16 22:19 ` Josh Poimboeuf
2016-12-23 13:13 ` Miroslav Benes
2016-12-08 18:08 ` [PATCH v3 12/15] livepatch: store function sizes Josh Poimboeuf
2016-12-19 13:10 ` Petr Mladek
2016-12-23 13:40 ` Miroslav Benes
2017-01-11 10:09 ` Kamalesh Babulal
2016-12-08 18:08 ` [PATCH v3 13/15] livepatch: change to a per-task consistency model Josh Poimboeuf
2016-12-20 17:32 ` Petr Mladek
2016-12-21 21:25 ` Josh Poimboeuf
2016-12-22 14:34 ` Petr Mladek
2016-12-22 18:31 ` Josh Poimboeuf
2017-01-10 13:00 ` Petr Mladek
2017-01-10 20:46 ` Josh Poimboeuf
2017-01-11 15:18 ` Petr Mladek
2017-01-11 15:26 ` Josh Poimboeuf [this message]
2016-12-23 9:24 ` Miroslav Benes
2016-12-23 10:18 ` Petr Mladek
2017-01-06 20:07 ` Josh Poimboeuf
2017-01-10 10:40 ` Petr Mladek
2017-01-04 13:44 ` Miroslav Benes
2017-01-06 21:01 ` Josh Poimboeuf
2017-01-10 10:45 ` Miroslav Benes
2017-01-05 9:34 ` Miroslav Benes
2017-01-06 21:04 ` Josh Poimboeuf
2016-12-08 18:08 ` [PATCH v3 14/15] livepatch: add /proc/<pid>/patch_state Josh Poimboeuf
2016-12-21 11:20 ` Petr Mladek
2017-01-04 14:50 ` Miroslav Benes
2016-12-08 18:08 ` [PATCH v3 15/15] livepatch: allow removal of a disabled patch Josh Poimboeuf
2016-12-21 14:44 ` Petr Mladek
2017-01-04 14:57 ` Miroslav Benes
2017-01-06 21:04 ` Josh Poimboeuf
2016-12-10 5:46 ` [PATCH v3 00/15] livepatch: hybrid consistency model Balbir Singh
2016-12-10 17:17 ` Josh Poimboeuf
2016-12-11 2:08 ` Balbir Singh
2016-12-12 14:04 ` Josh Poimboeuf
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170111152639.l6ehubjuys66if4l@treble \
--to=jpoimboe@redhat.com \
--cc=chris.j.arges@canonical.com \
--cc=heiko.carstens@de.ibm.com \
--cc=jeyu@redhat.com \
--cc=jikos@kernel.org \
--cc=jslaby@suse.cz \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-s390@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=live-patching@vger.kernel.org \
--cc=luto@kernel.org \
--cc=mbenes@suse.cz \
--cc=mingo@kernel.org \
--cc=mpe@ellerman.id.au \
--cc=peterz@infradead.org \
--cc=pmladek@suse.com \
--cc=vojtech@suse.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).