From: Johannes Weiner <hannes@cmpxchg.org> To: Brian Welty <brian.welty@intel.com> Cc: cgroups@vger.kernel.org, "Tejun Heo" <tj@kernel.org>, "Li Zefan" <lizefan@huawei.com>, linux-mm@kvack.org, "Michal Hocko" <mhocko@kernel.org>, "Vladimir Davydov" <vdavydov.dev@gmail.com>, dri-devel@lists.freedesktop.org, "David Airlie" <airlied@linux.ie>, "Daniel Vetter" <daniel@ffwll.ch>, intel-gfx@lists.freedesktop.org, "Jani Nikula" <jani.nikula@linux.intel.com>, "Joonas Lahtinen" <joonas.lahtinen@linux.intel.com>, "Rodrigo Vivi" <rodrigo.vivi@intel.com>, "Christian König" <christian.koenig@amd.com>, "Alex Deucher" <alexander.deucher@amd.com>, "ChunMing Zhou" <David1.Zhou@amd.com>, "Jérôme Glisse" <jglisse@redhat.com> Subject: Re: [RFC PATCH 0/5] cgroup support for GPU devices Date: Mon, 6 May 2019 11:16:13 -0400 [thread overview] Message-ID: <20190506151613.GB11505@cmpxchg.org> (raw) In-Reply-To: <20190501140438.9506-1-brian.welty@intel.com> On Wed, May 01, 2019 at 10:04:33AM -0400, Brian Welty wrote: > In containerized or virtualized environments, there is desire to have > controls in place for resources that can be consumed by users of a GPU > device. This RFC patch series proposes a framework for integrating > use of existing cgroup controllers into device drivers. > The i915 driver is updated in this series as our primary use case to > leverage this framework and to serve as an example for discussion. > > The patch series enables device drivers to use cgroups to control the > following resources within a GPU (or other accelerator device): > * control allocation of device memory (reuse of memcg) > and with future work, we could extend to: > * track and control share of GPU time (reuse of cpu/cpuacct) > * apply mask of allowed execution engines (reuse of cpusets) Please create a separate controller for your purposes. The memory controller is for traditional RAM. I don't see it having much in common with what you're trying to do, and it's barely reusing any of the memcg code. You can use the page_counter API directly. > Instead of introducing a new cgroup subsystem for GPU devices, a new > framework is proposed to allow devices to register with existing cgroup > controllers, which creates per-device cgroup_subsys_state within the > cgroup. This gives device drivers their own private cgroup controls > (such as memory limits or other parameters) to be applied to device > resources instead of host system resources. > Device drivers (GPU or other) are then able to reuse the existing cgroup > controls, instead of inventing similar ones. > > Per-device controls would be exposed in cgroup filesystem as: > mount/<cgroup_name>/<subsys_name>.devices/<dev_name>/<subsys_files> > such as (for example): > mount/<cgroup_name>/memory.devices/<dev_name>/memory.max > mount/<cgroup_name>/memory.devices/<dev_name>/memory.current > mount/<cgroup_name>/cpu.devices/<dev_name>/cpu.stat > mount/<cgroup_name>/cpu.devices/<dev_name>/cpu.weight Subdirectories for anything other than actual cgroups are a no-go. If you need a hierarchy, use dotted filenames: gpu.memory.max gpu.cycles.max etc. and look at Documentation/admin-guide/cgroup-v2.rst's 'Format' and 'Conventions', as well as how the io controller works, to see how multi-key / multi-device control files are implemented in cgroup2.
WARNING: multiple messages have this Message-ID (diff)
From: Johannes Weiner <hannes@cmpxchg.org> To: Brian Welty <brian.welty@intel.com> Cc: "ChunMing Zhou" <David1.Zhou@amd.com>, "Alex Deucher" <alexander.deucher@amd.com>, "David Airlie" <airlied@linux.ie>, intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, "Michal Hocko" <mhocko@kernel.org>, linux-mm@kvack.org, "Jérôme Glisse" <jglisse@redhat.com>, "Li Zefan" <lizefan@huawei.com>, "Vladimir Davydov" <vdavydov.dev@gmail.com>, "Tejun Heo" <tj@kernel.org>, cgroups@vger.kernel.org, "Christian König" <christian.koenig@amd.com> Subject: Re: [RFC PATCH 0/5] cgroup support for GPU devices Date: Mon, 6 May 2019 11:16:13 -0400 [thread overview] Message-ID: <20190506151613.GB11505@cmpxchg.org> (raw) In-Reply-To: <20190501140438.9506-1-brian.welty@intel.com> On Wed, May 01, 2019 at 10:04:33AM -0400, Brian Welty wrote: > In containerized or virtualized environments, there is desire to have > controls in place for resources that can be consumed by users of a GPU > device. This RFC patch series proposes a framework for integrating > use of existing cgroup controllers into device drivers. > The i915 driver is updated in this series as our primary use case to > leverage this framework and to serve as an example for discussion. > > The patch series enables device drivers to use cgroups to control the > following resources within a GPU (or other accelerator device): > * control allocation of device memory (reuse of memcg) > and with future work, we could extend to: > * track and control share of GPU time (reuse of cpu/cpuacct) > * apply mask of allowed execution engines (reuse of cpusets) Please create a separate controller for your purposes. The memory controller is for traditional RAM. I don't see it having much in common with what you're trying to do, and it's barely reusing any of the memcg code. You can use the page_counter API directly. > Instead of introducing a new cgroup subsystem for GPU devices, a new > framework is proposed to allow devices to register with existing cgroup > controllers, which creates per-device cgroup_subsys_state within the > cgroup. This gives device drivers their own private cgroup controls > (such as memory limits or other parameters) to be applied to device > resources instead of host system resources. > Device drivers (GPU or other) are then able to reuse the existing cgroup > controls, instead of inventing similar ones. > > Per-device controls would be exposed in cgroup filesystem as: > mount/<cgroup_name>/<subsys_name>.devices/<dev_name>/<subsys_files> > such as (for example): > mount/<cgroup_name>/memory.devices/<dev_name>/memory.max > mount/<cgroup_name>/memory.devices/<dev_name>/memory.current > mount/<cgroup_name>/cpu.devices/<dev_name>/cpu.stat > mount/<cgroup_name>/cpu.devices/<dev_name>/cpu.weight Subdirectories for anything other than actual cgroups are a no-go. If you need a hierarchy, use dotted filenames: gpu.memory.max gpu.cycles.max etc. and look at Documentation/admin-guide/cgroup-v2.rst's 'Format' and 'Conventions', as well as how the io controller works, to see how multi-key / multi-device control files are implemented in cgroup2. _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx
next prev parent reply other threads:[~2019-05-06 15:16 UTC|newest] Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-05-01 14:04 [RFC PATCH 0/5] cgroup support for GPU devices Brian Welty 2019-05-01 14:04 ` Brian Welty 2019-05-01 14:04 ` [RFC PATCH 1/5] cgroup: Add cgroup_subsys per-device registration framework Brian Welty 2019-05-01 14:04 ` Brian Welty 2019-05-01 14:04 ` [RFC PATCH 2/5] cgroup: Change kernfs_node for directories to store cgroup_subsys_state Brian Welty 2019-05-01 14:04 ` Brian Welty 2019-05-01 14:04 ` [RFC PATCH 3/5] memcg: Add per-device support to memory cgroup subsystem Brian Welty 2019-05-01 14:04 ` Brian Welty 2019-05-01 14:04 ` [RFC PATCH 4/5] drm: Add memory cgroup registration and DRIVER_CGROUPS feature bit Brian Welty 2019-05-01 14:04 ` Brian Welty 2019-05-01 14:04 ` [RFC PATCH 5/5] drm/i915: Use memory cgroup for enforcing device memory limit Brian Welty 2019-05-01 14:04 ` Brian Welty 2019-05-02 8:34 ` [RFC PATCH 0/5] cgroup support for GPU devices Leon Romanovsky 2019-05-02 8:34 ` Leon Romanovsky 2019-05-02 22:48 ` Kenny Ho 2019-05-02 22:48 ` Kenny Ho 2019-05-03 21:14 ` Welty, Brian 2019-05-03 21:14 ` Welty, Brian 2019-05-05 7:14 ` Leon Romanovsky 2019-05-05 7:14 ` Leon Romanovsky 2019-05-05 14:21 ` Kenny Ho 2019-05-05 14:21 ` Kenny Ho 2019-05-05 16:05 ` Leon Romanovsky 2019-05-05 16:05 ` Leon Romanovsky 2019-05-05 16:34 ` Kenny Ho 2019-05-05 16:34 ` Kenny Ho 2019-05-05 16:55 ` Leon Romanovsky 2019-05-05 16:55 ` Leon Romanovsky 2019-05-05 16:46 ` Chris Down 2019-05-05 16:46 ` Chris Down 2019-05-06 15:16 ` Johannes Weiner [this message] 2019-05-06 15:16 ` Johannes Weiner 2019-05-06 15:26 ` Tejun Heo 2019-05-06 15:26 ` Tejun Heo 2019-05-07 19:50 ` Welty, Brian 2019-05-07 19:50 ` Welty, Brian 2019-05-09 16:52 ` Tejun Heo 2019-05-09 16:52 ` Tejun Heo
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20190506151613.GB11505@cmpxchg.org \ --to=hannes@cmpxchg.org \ --cc=David1.Zhou@amd.com \ --cc=airlied@linux.ie \ --cc=alexander.deucher@amd.com \ --cc=brian.welty@intel.com \ --cc=cgroups@vger.kernel.org \ --cc=christian.koenig@amd.com \ --cc=daniel@ffwll.ch \ --cc=dri-devel@lists.freedesktop.org \ --cc=intel-gfx@lists.freedesktop.org \ --cc=jani.nikula@linux.intel.com \ --cc=jglisse@redhat.com \ --cc=joonas.lahtinen@linux.intel.com \ --cc=linux-mm@kvack.org \ --cc=lizefan@huawei.com \ --cc=mhocko@kernel.org \ --cc=rodrigo.vivi@intel.com \ --cc=tj@kernel.org \ --cc=vdavydov.dev@gmail.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.