From: Daniel Jordan <daniel.m.jordan@oracle.com>
To: Randy Dunlap <rdunlap@infradead.org>
Cc: Daniel Jordan <daniel.m.jordan@oracle.com>,
linux-mm@kvack.org, kvm@vger.kernel.org,
linux-kernel@vger.kernel.org, aarcange@redhat.com,
aaron.lu@intel.com, akpm@linux-foundation.org,
alex.williamson@redhat.com, bsd@redhat.com,
darrick.wong@oracle.com, dave.hansen@linux.intel.com,
jgg@mellanox.com, jwadams@google.com, jiangshanlai@gmail.com,
mhocko@kernel.org, mike.kravetz@oracle.com,
Pavel.Tatashin@microsoft.com, prasad.singamsetty@oracle.com,
steven.sistare@oracle.com, tim.c.chen@intel.com, tj@kernel.org,
vbabka@suse.cz
Subject: Re: [RFC PATCH v4 01/13] ktask: add documentation
Date: Mon, 5 Nov 2018 18:27:47 -0800 [thread overview]
Message-ID: <20181106022747.dmtq24pvulcnv3lc@ca-dmjordan1.us.oracle.com> (raw)
In-Reply-To: <7693f8a2-e180-520a-0d07-cc3090d2139f@infradead.org>
On Mon, Nov 05, 2018 at 01:19:50PM -0800, Randy Dunlap wrote:
> On 11/5/18 8:55 AM, Daniel Jordan wrote:
>
> Hi,
>
> > +Resource Limits
> > +===============
> > +
> > +ktask has resource limits on the number of work items it sends to workqueue.
>
> to a workqueue.
> or: to workqueues.
Ok, I'll do "to workqueues" since ktask uses two internally (NUMA-aware and
non-NUMA-aware).
>
> > +In ktask, a workqueue item is a thread that runs chunks of the task until the
> > +task is finished.
> > +
> > +These limits support the different ways ktask uses workqueues:
> > + - ktask_run to run threads on the calling thread's node.
> > + - ktask_run_numa to run threads on the node(s) specified.
> > + - ktask_run_numa with nid=NUMA_NO_NODE to run threads on any node in the
> > + system.
> > +
> > +To support these different ways of queueing work while maintaining an efficient
> > +concurrency level, we need both system-wide and per-node limits on the number
>
> I would prefer to refer to ktask as ktask instead of "we", so
> s/we need/ktask needs/
Good idea, I'll change it.
> > +of threads. Without per-node limits, a node might become oversubscribed
> > +despite ktask staying within the system-wide limit, and without a system-wide
> > +limit, we can't properly account for work that can run on any node.
>
> s/we/ktask/
Ok.
> > +
> > +The system-wide limit is based on the total number of CPUs, and the per-node
> > +limit on the CPU count for each node. A per-node work item counts against the
> > +system-wide limit. Workqueue's max_active can't accommodate both types of
> > +limit, no matter how many workqueues are used, so ktask implements its own.
> > +
> > +If a per-node limit is reached, the work item is allowed to run anywhere on the
> > +machine to avoid overwhelming the node. If the global limit is also reached,
> > +ktask won't queue additional work items until we fall below the limit again.
>
> s/we fall/ktask falls/
> or s/we fall/it falls/
'ktask.' Will change.
> > +
> > +These limits apply only to workqueue items--that is, helper threads beyond the
> > +one starting the task. That way, one thread per task is always allowed to run.
>
>
> thanks.
Appreciate the feedback!
next prev parent reply other threads:[~2018-11-06 2:28 UTC|newest]
Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-11-05 16:55 [RFC PATCH v4 00/13] ktask: multithread CPU-intensive kernel work Daniel Jordan
2018-11-05 16:55 ` [RFC PATCH v4 01/13] ktask: add documentation Daniel Jordan
2018-11-05 21:19 ` Randy Dunlap
2018-11-06 2:27 ` Daniel Jordan [this message]
2018-11-06 8:49 ` Peter Zijlstra
2018-11-06 20:34 ` Daniel Jordan
2018-11-06 20:51 ` Jason Gunthorpe
2018-11-07 10:27 ` Peter Zijlstra
2018-11-07 20:21 ` Daniel Jordan
2018-11-07 10:35 ` Peter Zijlstra
2018-11-07 21:20 ` Daniel Jordan
2018-11-08 17:26 ` Jonathan Corbet
2018-11-08 19:15 ` Daniel Jordan
2018-11-08 19:24 ` Jonathan Corbet
2018-11-27 19:50 ` Pavel Machek
2018-11-28 16:56 ` Daniel Jordan
2018-11-05 16:55 ` [RFC PATCH v4 02/13] ktask: multithread CPU-intensive kernel work Daniel Jordan
2018-11-05 20:51 ` Randy Dunlap
2018-11-06 2:24 ` Daniel Jordan
2018-11-05 16:55 ` [RFC PATCH v4 03/13] ktask: add undo support Daniel Jordan
2018-11-05 16:55 ` [RFC PATCH v4 04/13] ktask: run helper threads at MAX_NICE Daniel Jordan
2018-11-05 16:55 ` [RFC PATCH v4 05/13] workqueue, ktask: renice helper threads to prevent starvation Daniel Jordan
2018-11-13 16:34 ` Tejun Heo
2018-11-19 16:45 ` Daniel Jordan
2018-11-20 16:33 ` Tejun Heo
2018-11-20 17:03 ` Daniel Jordan
2018-11-05 16:55 ` [RFC PATCH v4 06/13] vfio: parallelize vfio_pin_map_dma Daniel Jordan
2018-11-05 21:51 ` Alex Williamson
2018-11-06 2:42 ` Daniel Jordan
2018-11-05 16:55 ` [RFC PATCH v4 07/13] mm: change locked_vm's type from unsigned long to atomic_long_t Daniel Jordan
2018-11-05 16:55 ` [RFC PATCH v4 08/13] vfio: remove unnecessary mmap_sem writer acquisition around locked_vm Daniel Jordan
2018-11-05 16:55 ` [RFC PATCH v4 09/13] vfio: relieve mmap_sem reader cacheline bouncing by holding it longer Daniel Jordan
2018-11-05 16:55 ` [RFC PATCH v4 10/13] mm: enlarge type of offset argument in mem_map_offset and mem_map_next Daniel Jordan
2018-11-05 16:55 ` [RFC PATCH v4 11/13] mm: parallelize deferred struct page initialization within each node Daniel Jordan
2018-11-10 3:48 ` Elliott, Robert (Persistent Memory)
2018-11-12 16:54 ` Daniel Jordan
2018-11-12 22:15 ` Elliott, Robert (Persistent Memory)
2018-11-19 16:01 ` Daniel Jordan
2018-11-27 0:12 ` Elliott, Robert (Persistent Memory)
2018-11-27 20:23 ` Daniel Jordan
2018-11-19 16:29 ` Daniel Jordan
2018-11-05 16:55 ` [RFC PATCH v4 12/13] mm: parallelize clear_gigantic_page Daniel Jordan
2018-11-05 16:55 ` [RFC PATCH v4 13/13] hugetlbfs: parallelize hugetlbfs_fallocate with ktask Daniel Jordan
2018-11-05 17:29 ` [RFC PATCH v4 00/13] ktask: multithread CPU-intensive kernel work Michal Hocko
2018-11-06 1:29 ` Daniel Jordan
2018-11-06 9:21 ` Michal Hocko
2018-11-07 20:17 ` Daniel Jordan
2018-11-05 18:49 ` Zi Yan
2018-11-06 2:20 ` Daniel Jordan
2018-11-06 2:48 ` Zi Yan
2018-11-06 19:00 ` Daniel Jordan
2018-11-30 19:18 ` Tejun Heo
2018-12-01 0:13 ` Daniel Jordan
2018-12-03 16:16 ` Tejun Heo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20181106022747.dmtq24pvulcnv3lc@ca-dmjordan1.us.oracle.com \
--to=daniel.m.jordan@oracle.com \
--cc=Pavel.Tatashin@microsoft.com \
--cc=aarcange@redhat.com \
--cc=aaron.lu@intel.com \
--cc=akpm@linux-foundation.org \
--cc=alex.williamson@redhat.com \
--cc=bsd@redhat.com \
--cc=darrick.wong@oracle.com \
--cc=dave.hansen@linux.intel.com \
--cc=jgg@mellanox.com \
--cc=jiangshanlai@gmail.com \
--cc=jwadams@google.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@kernel.org \
--cc=mike.kravetz@oracle.com \
--cc=prasad.singamsetty@oracle.com \
--cc=rdunlap@infradead.org \
--cc=steven.sistare@oracle.com \
--cc=tim.c.chen@intel.com \
--cc=tj@kernel.org \
--cc=vbabka@suse.cz \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).