On Wed, Nov 25 2020, tj@kernel.org wrote:

> Hello,
>
> On Fri, Nov 20, 2020 at 10:23:44AM +1100, NeilBrown wrote:
>> On Mon, Nov 09 2020, tj@kernel.org wrote:
>> 
>> >                                                    Given that nothing on
>> > these types of workqueues can be latency sensitive
>> 
>> This caught my eye and it seems worth drilling in to.  There is no
>> mention of "latency" in workqueue.rst or workqueue.h.  But you seem to
>> be saying there is an undocumented assumption that latency-sensitive
>> work items much not be scheduled on CM-workqueues.
>> Is that correct?
>
> Yeah, correct. Because they're all sharing execution concurrency, the
> latency consistency is likely a lot worse.
>
>> NFS writes are latency sensitive to a degree as increased latency per
>> request will hurt overall throughput.  Does this mean that handling
>> write-completion in a CM-wq is a poor choice?
>> Would it be better to us WQ_HIGHPRI??  Is there any rule-of-thumb that
>> can be used to determine when WQ_HIGHPRI is appropriate?
>
> I don't think it'd need HIGHPRI but UNBOUND or CPU_INTENSIVE would make
> sense. I think the rule of the thumb is along the line of if you're worried
> about cpu consumption or latency, let the scheduler take care of it (ie. use
> unbound workqueues).

Thanks.
For nfsiod there are two contexts where it is used.

 In one context there is normally a thread waiting for the work item
 to complete.  It doesn't run the work in-line because the thread needs
 to abort if signaled, but the work needs to happen anyway so that the
 client and server remain in-sync.  In this case the fact that a
 application is waiting suggests that latency could be a problem.

 The other context is completing an async READ or WRITE.  I'm not sure
 if latency at this stage of the request will actually affect
 throughput, but we do need a WQ_MEM_RECLAIM wq for the WRITE at least.

 Keep both types of users on the same wq is simplest, so making it
  WQ_UNBOUND | WQ_MEM_RECLAIM
 is probably safest and would ensure that a cpu-intensive iput_final()
 doesn't interfere with other requests unduly.
 Quite a few other filesystems do use WQ_UNBOUND, often with
  WQ_MEM_RECLAIM, but it is not easy to do a like-for-like comparison.

 I might have a go at updating the workqueue documentation to provide
 some guidance on how to choose a workqueue and when certain flags are
 needed.

Thanks,
NeilBrown