All of lore.kernel.org
 help / color / mirror / Atom feed
From: Peter Oskolkov <posk@posk.io>
To: Thierry Delisle <tdelisle@uwaterloo.ca>
Cc: Peter Zijlstra <peterz@infradead.org>,
	Ingo Molnar <mingo@redhat.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	linux-api@vger.kernel.org, Paul Turner <pjt@google.com>,
	Ben Segall <bsegall@google.com>, Peter Oskolkov <posk@google.com>,
	Andrei Vagin <avagin@google.com>, Jann Horn <jannh@google.com>
Subject: Re: [PATCH 5/5 v0.6] sched/umcg: add Documentation/userspace-api/umcg.txt
Date: Tue, 12 Oct 2021 14:44:01 -0700	[thread overview]
Message-ID: <CAFTs51XksSW+YW2H2iZk+j0AY+NApr=aBBVgakyx55yX_5AA8A@mail.gmail.com> (raw)
In-Reply-To: <40c37212-ab15-01ac-f5c5-e3f53c9b8e4e@uwaterloo.ca>

Hi Thierry!

Again, it seems you are ascribing higher level semantics to lower
level constructs here, specifically to "idle workers list".

In this patchset, idle workers list is just a mechanism used by the
kernel to notify the userspace that previously blocked workers are now
available for scheduling. Maybe a better name would have been
"unblocked workers list". The true list of idle workers that the
userspace scheduler can schedule is maintained by the userspace; it
can be said that if a worker is on the kernel's idle workers list, it
is NOT on the userspace's idle workers list, and so workers on the
kernel's idle workers list are not yet fully "idle workers" that can
be scheduled.

On Tue, Oct 12, 2021 at 11:46 AM Thierry Delisle <tdelisle@uwaterloo.ca> wrote:
>
>  >> Just to be clear, sys_umcg_wait supports an operation that, when called
>  >> from a worker, puts the worker to sleep without triggering block
> detection
>  >> or context-switching back to the server?
>  >
>  > Potentially, yes - when a worker wants to yield (e.g. as part of a
>  > custom UMCG-aware mutex/condvar code), and calls into the userspace
>  > scheduler, it may be faster to skip the server wakeup (e.g. reassign
>  > the server to another sleeping worker and wake this worker). This is
>  > not a supported operation right now, but I see how it could be used to
>  > optimize some things in the future.
>  >
>  > Do you have any concerns here?
>
> To be honest, I did not realize this was a possibility until your previous
> email. I'm not sure I buy your example, it just sounds like worker to worker
> context-switching, but I could imagine "stop the world" cases or some "race
> to idle" policy using this feature.
>
> It seems to me the corresponding wake needs to know if it needs to enqueue
> the worker into the idle workers list or if it should just schedule the
> worker
> as it would a server.
>
> How does the wake know which to do?

If the worker is IDLE, and the userspace knows about it (i.e. the
worker is NOT on the kernel's idle workers list), the userspace either
can directly schedule the worker with a server (mark it RUNNING,
assign a server, etc.), or instruct the kernel to put the worker onto
the kernel's idle workers list so that it can later be picked up by a
userspace thread checking the kernel's idle workers list. This last
operation is provided for cases when for example a server wants to run
IDLE worker A and knows about IDLE worker B, but for some reason
cannot properly process worker B at the moment, so by putting worker B
back into the kernel's idle worker list this server delegates
processing worker B to another server in the future.

Both of these state transitions are explicitly covered in the documentation.

Again, you appear to be trying to equate kernel UMCG API with higher
level userspace scheduling notions/facilities, while in reality kernel
UMCG API is a low level facility that will be _used_ to construct the
higher level behaviors/semantics you are wondering about.

I suggest you wait until I post the userspace library to ask these
higher-level semantic questions.

Thanks,
Peter

[...]

      reply	other threads:[~2021-10-12 21:44 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-09-17 18:03 [PATCH 0/5 v0.6] sched/umcg: RFC UMCG patchset Peter Oskolkov
2021-09-17 18:03 ` [PATCH 1/5 v0.6] sched/umcg: add WF_CURRENT_CPU and externise ttwu Peter Oskolkov
2021-09-17 18:03 ` [PATCH 2/5 v0.6] sched/umcg: RFC: add userspace atomic helpers Peter Oskolkov
2021-09-19 18:25   ` Tao Zhou
2021-09-28 21:58   ` Thomas Gleixner
2021-09-28 23:07     ` Peter Oskolkov
2021-09-17 18:03 ` [PATCH 3/5 v0.6] sched/umcg: RFC: implement UMCG syscalls Peter Oskolkov
2021-09-19 18:25   ` Tao Zhou
2021-09-19 23:24   ` kernel test robot
2021-09-20  5:18   ` kernel test robot
2021-09-28  9:21   ` Thomas Gleixner
2021-09-28 17:26     ` Peter Oskolkov
2021-09-28 20:00       ` Thomas Gleixner
2021-09-17 18:03 ` [PATCH 4/5 v0.6] sched/umcg: add Documentation/userspace-api/umcg.rst Peter Oskolkov
2021-09-17 18:03 ` [PATCH 5/5 v0.6] sched/umcg: add Documentation/userspace-api/umcg.txt Peter Oskolkov
2021-09-22 18:39   ` Thierry Delisle
2021-10-11 22:45     ` Peter Oskolkov
2021-10-12 16:25       ` Thierry Delisle
2021-10-12 16:58         ` Peter Oskolkov
2021-10-12 18:46           ` Thierry Delisle
2021-10-12 21:44             ` Peter Oskolkov [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAFTs51XksSW+YW2H2iZk+j0AY+NApr=aBBVgakyx55yX_5AA8A@mail.gmail.com' \
    --to=posk@posk.io \
    --cc=avagin@google.com \
    --cc=bsegall@google.com \
    --cc=jannh@google.com \
    --cc=linux-api@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@redhat.com \
    --cc=peterz@infradead.org \
    --cc=pjt@google.com \
    --cc=posk@google.com \
    --cc=tdelisle@uwaterloo.ca \
    --cc=tglx@linutronix.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.