dri-devel.lists.freedesktop.org archive mirror
 help / color / mirror / Atom feed
From: Kenny Ho <y2kenny@gmail.com>
To: Tejun Heo <tj@kernel.org>
Cc: "Kenny Ho" <Kenny.Ho@amd.com>,
	"Kuehling, Felix" <felix.kuehling@amd.com>,
	jsparks@cray.com, "amd-gfx list" <amd-gfx@lists.freedesktop.org>,
	lkaplan@cray.com, dri-devel <dri-devel@lists.freedesktop.org>,
	"Greathouse, Joseph" <joseph.greathouse@amd.com>,
	"Alex Deucher" <alexander.deucher@amd.com>,
	cgroups@vger.kernel.org,
	"Christian König" <christian.koenig@amd.com>
Subject: Re: [PATCH v2 00/11] new cgroup controller for gpu/drm subsystem
Date: Mon, 13 Apr 2020 16:17:14 -0400	[thread overview]
Message-ID: <CAOWid-dM=38faGOF9=-Pq=sxssaL+gm2umctyGVQWVx2etShyQ@mail.gmail.com> (raw)
In-Reply-To: <20200413191136.GI60335@mtj.duckdns.org>

(replying again in plain-text)

Hi Tejun,

Thanks for taking the time to reply.

Perhaps we can even narrow things down to just
gpu.weight/gpu.compute.weight as a start?  In this aspect, is the key
objection to the current implementation of gpu.compute.weight the
work-conserving bit?  This work-conserving requirement is probably
what I have missed for the last two years (and hence going in circle.)

If this is the case, can you clarify/confirm the followings?

1) Is resource scheduling goal of cgroup purely for the purpose of
throughput?  (at the expense of other scheduling goals such as
latency.)
2) If 1) is true, under what circumstances will the "Allocations"
resource distribution model (as defined in the cgroup-v2) be
acceptable?
3) If 1) is true, are things like cpuset from cgroup v1 no longer
acceptable going forward?

To be clear, while some have framed this (time sharing vs spatial
sharing) as a partisan issue, it is in fact a technical one.  I have
implemented the gpu cgroup support this way because we have a class of
users that value low latency/low jitter/predictability/synchronicity.
For example, they would like 4 tasks to share a GPU and they would
like the tasks to start and finish at the same time.

What is the rationale behind picking the Weight model over Allocations
as the first acceptable implementation?  Can't we have both
work-conserving and non-work-conserving ways of distributing GPU
resources?  If we can, why not allow non-work-conserving
implementation first, especially when we have users asking for such
functionality?

Regards,
Kenny

On Mon, Apr 13, 2020 at 3:11 PM Tejun Heo <tj@kernel.org> wrote:
>
> Hello, Kenny.
>
> On Tue, Mar 24, 2020 at 02:49:27PM -0400, Kenny Ho wrote:
> > Can you elaborate more on what are the missing pieces?
>
> Sorry about the long delay, but I think we've been going in circles for quite
> a while now. Let's try to make it really simple as the first step. How about
> something like the following?
>
> * gpu.weight (should it be gpu.compute.weight? idk) - A single number
>   per-device weight similar to io.weight, which distributes computation
>   resources in work-conserving way.
>
> * gpu.memory.high - A single number per-device on-device memory limit.
>
> The above two, if works well, should already be plenty useful. And my guess is
> that getting the above working well will be plenty challenging already even
> though it's already excluding work-conserving memory distribution. So, let's
> please do that as the first step and see what more would be needed from there.
>
> Thanks.
>
> --
> tejun
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

  parent reply	other threads:[~2020-04-13 20:17 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <lkaplan@cray.com; daniel@ffwll.ch; nirmoy.das@amd.com; damon.mcdougall@amd.com; juan.zuniga-anaya@amd.com; hannes@cmpxchg.org>
2020-02-26 19:01 ` [PATCH v2 00/11] new cgroup controller for gpu/drm subsystem Kenny Ho
2020-02-26 19:01   ` [PATCH v2 01/11] cgroup: Introduce cgroup for drm subsystem Kenny Ho
2020-02-26 19:01   ` [PATCH v2 02/11] drm, cgroup: Bind drm and cgroup subsystem Kenny Ho
2020-02-26 19:01   ` [PATCH v2 03/11] drm, cgroup: Initialize drmcg properties Kenny Ho
2020-02-26 19:01   ` [PATCH v2 04/11] drm, cgroup: Add total GEM buffer allocation stats Kenny Ho
2020-02-26 19:01   ` [PATCH v2 05/11] drm, cgroup: Add peak " Kenny Ho
2020-02-26 19:01   ` [PATCH v2 06/11] drm, cgroup: Add GEM buffer allocation count stats Kenny Ho
2020-02-26 19:01   ` [PATCH v2 07/11] drm, cgroup: Add total GEM buffer allocation limit Kenny Ho
2020-02-26 19:01   ` [PATCH v2 08/11] drm, cgroup: Add peak " Kenny Ho
2020-02-26 19:01   ` [PATCH v2 09/11] drm, cgroup: Add compute as gpu cgroup resource Kenny Ho
2020-02-26 19:01   ` [PATCH v2 10/11] drm, cgroup: add update trigger after limit change Kenny Ho
2020-02-26 19:01   ` [PATCH v2 11/11] drm/amdgpu: Integrate with DRM cgroup Kenny Ho
2020-03-17 16:03   ` [PATCH v2 00/11] new cgroup controller for gpu/drm subsystem Kenny Ho
2020-03-24 18:46     ` Tejun Heo
2020-03-24 18:49       ` Kenny Ho
2020-04-13 19:11         ` Tejun Heo
2020-04-13 20:12           ` Ho, Kenny
2020-04-13 20:17           ` Kenny Ho [this message]
2020-04-13 20:54             ` Tejun Heo
2020-04-13 21:40               ` Kenny Ho
2020-04-13 21:53                 ` Tejun Heo
2020-04-14 12:20           ` Daniel Vetter
2020-04-14 12:47             ` Kenny Ho
2020-04-14 12:52               ` Daniel Vetter
2020-04-14 13:14                 ` Kenny Ho
2020-04-14 13:26                   ` Daniel Vetter
2020-04-14 13:50                     ` Kenny Ho
2020-04-14 14:04                       ` Daniel Vetter
2020-04-14 14:29                         ` Kenny Ho
2020-04-14 15:01                           ` Daniel Vetter

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAOWid-dM=38faGOF9=-Pq=sxssaL+gm2umctyGVQWVx2etShyQ@mail.gmail.com' \
    --to=y2kenny@gmail.com \
    --cc=Kenny.Ho@amd.com \
    --cc=alexander.deucher@amd.com \
    --cc=amd-gfx@lists.freedesktop.org \
    --cc=cgroups@vger.kernel.org \
    --cc=christian.koenig@amd.com \
    --cc=dri-devel@lists.freedesktop.org \
    --cc=felix.kuehling@amd.com \
    --cc=joseph.greathouse@amd.com \
    --cc=jsparks@cray.com \
    --cc=lkaplan@cray.com \
    --cc=tj@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).