From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.2 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17CA3C33CA9 for ; Mon, 13 Jan 2020 15:55:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 951D620678 for ; Mon, 13 Jan 2020 15:55:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=amdcloud.onmicrosoft.com header.i=@amdcloud.onmicrosoft.com header.b="lU/7Lpor" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 951D620678 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D70398E001A; Mon, 13 Jan 2020 10:55:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D20598E0012; Mon, 13 Jan 2020 10:55:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C0ED88E001A; Mon, 13 Jan 2020 10:55:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0018.hostedemail.com [216.40.44.18]) by kanga.kvack.org (Postfix) with ESMTP id A8A0A8E0012 for ; Mon, 13 Jan 2020 10:55:57 -0500 (EST) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 3A244824934B for ; Mon, 13 Jan 2020 15:55:57 +0000 (UTC) X-FDA: 76373061954.01.fork68_8b340ca7d2f5d X-HE-Tag: fork68_8b340ca7d2f5d X-Filterd-Recvd-Size: 11988 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2056.outbound.protection.outlook.com [40.107.92.56]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Mon, 13 Jan 2020 15:55:56 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=oQJsuS3JwhJTJnD5cwb9dHanWw6X3vUAce34N286teGuCqcC2D/mzbhf2OqJQ360qPcm9/CLxlFNrmgKq/IL/20A01jQ82qlwqg/h+L91cMuYcAzlzDczX3awSLhtJ+P8J/Kxku7w4bzCDmhJUb9w0WOyfigQtVXPkflxaKfAhmRsXTlppV1U6Ixcsp3KWSbiVX6Jz9ZsmQI81sR5wGLHt6VaEUs/Tb7BB72Fali91pMjZoolsFFtqcMmtEXCDL4/SfHHuyRGSNfqa2OERBZauXHz3v4euce/R2mcfjg/XI1CPE4lzyjRSoC+EUKbhjKXA/0/TgkrlhEjIBdfYG7dQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BLPWq8D42ESuKs+vGGWa65Rbhv86/dt79zfyvRTSz6Y=; b=XMbqnCNCY88EUoXAddxJT4zIC5gzFPkVRnJBv5QQ+ETg35u3VxAakXqYgmH9ScRliu34gT04xHR8PQwm+VKYY1FgDGqFuawsSKNqGQ9E+wlr7AF/Bwig0d+AYRAkOO8DlBQ9/bZ3piWW7odZq5IKClnM2TaQALnXcodSxfSRpzfrfb7XUl38aT/cgg7NPylXJ1muYN/h2oTSatsHl989rVkhtjdCUhgythGwR9fDVmz3t/G4AzKLVguFJcdmRHyvBFh215UF/7t8EPCShoEoVncNwYGxwsv6TLpdoJ1sIWlRDoWSPN05TWLPYm9a/o2/WnabQTZ5Ecf1CdhNv6NoXQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector2-amdcloud-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BLPWq8D42ESuKs+vGGWa65Rbhv86/dt79zfyvRTSz6Y=; b=lU/7LporyIr+LV/4mc8vwc6UYm2obwCekeBfq0J1o8VeQOPhwLwzb2uohu/zDpe51pyViWdKTxvaNfyYv8mtQAkFn048INPOvG3Xg9hJB9B1nRHRf26U9hFBaNrlNsG9K0Cn17plHc0pDU4FnmWcr219TRtLT+UicsXJ780meeY= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Christian.Koenig@amd.com; Received: from DM5PR12MB1705.namprd12.prod.outlook.com (10.175.88.22) by DM5PR12MB1852.namprd12.prod.outlook.com (10.175.87.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2623.10; Mon, 13 Jan 2020 15:55:53 +0000 Received: from DM5PR12MB1705.namprd12.prod.outlook.com ([fe80::8dde:b52a:d97a:e89]) by DM5PR12MB1705.namprd12.prod.outlook.com ([fe80::8dde:b52a:d97a:e89%2]) with mapi id 15.20.2623.015; Mon, 13 Jan 2020 15:55:53 +0000 Subject: Re: [PATCH RFC 3/3] drm/ttm: support memcg for ttm_tt To: Qiang Yu , linux-mm@kvack.org, cgroups@vger.kernel.org, dri-devel@lists.freedesktop.org Cc: Johannes Weiner , Michal Hocko , Andrew Morton , Tejun Heo , Huang Rui , David Airlie , Daniel Vetter , Kenny Ho References: <20200113153543.24957-1-qiang.yu@amd.com> <20200113153543.24957-4-qiang.yu@amd.com> From: =?UTF-8?Q?Christian_K=c3=b6nig?= Message-ID: Date: Mon, 13 Jan 2020 16:55:47 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.9.0 In-Reply-To: <20200113153543.24957-4-qiang.yu@amd.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-ClientProxiedBy: FR2P281CA0014.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:a::24) To DM5PR12MB1705.namprd12.prod.outlook.com (2603:10b6:3:10c::22) MIME-Version: 1.0 Received: from [IPv6:2a02:908:1252:fb60:be8a:bd56:1f94:86e7] (2a02:908:1252:fb60:be8a:bd56:1f94:86e7) by FR2P281CA0014.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:a::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2623.12 via Frontend Transport; Mon, 13 Jan 2020 15:55:51 +0000 X-Originating-IP: [2a02:908:1252:fb60:be8a:bd56:1f94:86e7] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 4d392e2d-cd36-4813-86da-08d7984111ab X-MS-TrafficTypeDiagnostic: DM5PR12MB1852:|DM5PR12MB1852: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-Forefront-PRVS: 028166BF91 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4636009)(136003)(39860400002)(346002)(376002)(396003)(366004)(199004)(189003)(186003)(36756003)(4326008)(16526019)(31686004)(2616005)(52116002)(86362001)(478600001)(81156014)(8676002)(81166006)(6666004)(316002)(66476007)(66556008)(5660300002)(66946007)(6486002)(54906003)(31696002)(2906002)(8936002);DIR:OUT;SFP:1101;SCL:1;SRVR:DM5PR12MB1852;H:DM5PR12MB1705.namprd12.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: UGmqeTqP/6TeRf5OMDx3KW/YJKFVSafKtO9HhW+J4c/LwgTFHRN5rO4ae76ICjD3F80HFm6rOztisnekadVt5HmUPblfQqS5ZoXNyjjfc+EkRcWzuvvjiMMJBoM/HwQ3EDj4xqhQNyghAkJmr9KdNR3Mrs8MK1m7jkDul4rFR+CgDtEa6bEL92O9gM2eE/uXJ2vNHowOVIOfCdff1DvPGpjdcpmXQldpVUH908yd8wbBGI/MeGU/MAFAcAccn8icj1aT64pTC7EohY6u08S6p+JuuqG8J7+ozBfDRmlJD5RWCJiTvdu9dEL8s6DQ136rAdW6ucK1+gzCOr3NB9gKImOHe/P582748ag4h1DXcO/dUATZdh/cPtz4VfDN9Ccr8BZgl8LukKFVOCztEEEFHhYcl1YCzdBJ8+tbt07r0REJyG6+Bc9RVPApQP+ycRTV X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4d392e2d-cd36-4813-86da-08d7984111ab X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jan 2020 15:55:53.6812 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: LmP0i33G4qVi4Jh025A59WEsfR2Cw/6lZkKZtIf3Xe+LnqcGFpkFsCrLvBq7XKBw X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB1852 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Am 13.01.20 um 16:35 schrieb Qiang Yu: > Charge TTM allocated system memory to memory cgroup which will > limit the memory usage of a group of processes. NAK to the whole approach. This belongs into the GEM or driver layer, but not into TTM. > The memory is always charged to the control group of task which > create this buffer object and when it's created. For example, > when a buffer is created by process A and exported to process B, > then process B populate this buffer, the memory is still charged > to process A's memcg; if a buffer is created by process A when in > memcg B, then A is moved to memcg C and populate this buffer, it > will charge memcg B. This is actually the most common use case for graphics application where the X server allocates most of the backing store. So we need a better handling than just accounting the memory to whoever allocated it first. Regards, Christian. > > Signed-off-by: Qiang Yu > --- > drivers/gpu/drm/ttm/ttm_bo.c | 10 ++++++++++ > drivers/gpu/drm/ttm/ttm_page_alloc.c | 18 +++++++++++++++++- > drivers/gpu/drm/ttm/ttm_tt.c | 3 +++ > include/drm/ttm/ttm_bo_api.h | 5 +++++ > include/drm/ttm/ttm_tt.h | 4 ++++ > 5 files changed, 39 insertions(+), 1 deletion(-) > > diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c > index 8d91b0428af1..4e64846ee523 100644 > --- a/drivers/gpu/drm/ttm/ttm_bo.c > +++ b/drivers/gpu/drm/ttm/ttm_bo.c > @@ -42,6 +42,7 @@ > #include > #include > #include > +#include > > static void ttm_bo_global_kobj_release(struct kobject *kobj); > > @@ -162,6 +163,10 @@ static void ttm_bo_release_list(struct kref *list_kref) > if (!ttm_bo_uses_embedded_gem_object(bo)) > dma_resv_fini(&bo->base._resv); > mutex_destroy(&bo->wu_mutex); > +#ifdef CONFIG_MEMCG > + if (bo->memcg) > + css_put(&bo->memcg->css); > +#endif > bo->destroy(bo); > ttm_mem_global_free(&ttm_mem_glob, acc_size); > } > @@ -1330,6 +1335,11 @@ int ttm_bo_init_reserved(struct ttm_bo_device *bdev, > } > atomic_inc(&ttm_bo_glob.bo_count); > > +#ifdef CONFIG_MEMCG > + if (bo->type == ttm_bo_type_device) > + bo->memcg = mem_cgroup_driver_get_from_current(); > +#endif > + > /* > * For ttm_bo_type_device buffers, allocate > * address space from the device. > diff --git a/drivers/gpu/drm/ttm/ttm_page_alloc.c b/drivers/gpu/drm/ttm/ttm_page_alloc.c > index b40a4678c296..ecd1831a1d38 100644 > --- a/drivers/gpu/drm/ttm/ttm_page_alloc.c > +++ b/drivers/gpu/drm/ttm/ttm_page_alloc.c > @@ -42,7 +42,7 @@ > #include /* for seq_printf */ > #include > #include > - > +#include > #include > > #include > @@ -1045,6 +1045,11 @@ ttm_pool_unpopulate_helper(struct ttm_tt *ttm, unsigned mem_count_update) > ttm_put_pages(ttm->pages, ttm->num_pages, ttm->page_flags, > ttm->caching_state); > ttm->state = tt_unpopulated; > + > +#ifdef CONFIG_MEMCG > + if (ttm->memcg) > + mem_cgroup_uncharge_drvmem(ttm->memcg, ttm->num_pages); > +#endif > } > > int ttm_pool_populate(struct ttm_tt *ttm, struct ttm_operation_ctx *ctx) > @@ -1059,6 +1064,17 @@ int ttm_pool_populate(struct ttm_tt *ttm, struct ttm_operation_ctx *ctx) > if (ttm_check_under_lowerlimit(mem_glob, ttm->num_pages, ctx)) > return -ENOMEM; > > +#ifdef CONFIG_MEMCG > + if (ttm->memcg) { > + gfp_t gfp_flags = GFP_USER; > + if (ttm->page_flags & TTM_PAGE_FLAG_NO_RETRY) > + gfp_flags |= __GFP_RETRY_MAYFAIL; > + ret = mem_cgroup_charge_drvmem(ttm->memcg, gfp_flags, ttm->num_pages); > + if (ret) > + return ret; > + } > +#endif > + > ret = ttm_get_pages(ttm->pages, ttm->num_pages, ttm->page_flags, > ttm->caching_state); > if (unlikely(ret != 0)) { > diff --git a/drivers/gpu/drm/ttm/ttm_tt.c b/drivers/gpu/drm/ttm/ttm_tt.c > index e0e9b4f69db6..1acb153084e1 100644 > --- a/drivers/gpu/drm/ttm/ttm_tt.c > +++ b/drivers/gpu/drm/ttm/ttm_tt.c > @@ -233,6 +233,9 @@ void ttm_tt_init_fields(struct ttm_tt *ttm, struct ttm_buffer_object *bo, > ttm->state = tt_unpopulated; > ttm->swap_storage = NULL; > ttm->sg = bo->sg; > +#ifdef CONFIG_MEMCG > + ttm->memcg = bo->memcg; > +#endif > } > > int ttm_tt_init(struct ttm_tt *ttm, struct ttm_buffer_object *bo, > diff --git a/include/drm/ttm/ttm_bo_api.h b/include/drm/ttm/ttm_bo_api.h > index 65e399d280f7..95a08e81a73e 100644 > --- a/include/drm/ttm/ttm_bo_api.h > +++ b/include/drm/ttm/ttm_bo_api.h > @@ -54,6 +54,8 @@ struct ttm_place; > > struct ttm_lru_bulk_move; > > +struct mem_cgroup; > + > /** > * struct ttm_bus_placement > * > @@ -180,6 +182,9 @@ struct ttm_buffer_object { > void (*destroy) (struct ttm_buffer_object *); > unsigned long num_pages; > size_t acc_size; > +#ifdef CONFIG_MEMCG > + struct mem_cgroup *memcg; > +#endif > > /** > * Members not needing protection. > diff --git a/include/drm/ttm/ttm_tt.h b/include/drm/ttm/ttm_tt.h > index c0e928abf592..10fb5a557b95 100644 > --- a/include/drm/ttm/ttm_tt.h > +++ b/include/drm/ttm/ttm_tt.h > @@ -33,6 +33,7 @@ struct ttm_tt; > struct ttm_mem_reg; > struct ttm_buffer_object; > struct ttm_operation_ctx; > +struct mem_cgroup; > > #define TTM_PAGE_FLAG_WRITE (1 << 3) > #define TTM_PAGE_FLAG_SWAPPED (1 << 4) > @@ -116,6 +117,9 @@ struct ttm_tt { > tt_unbound, > tt_unpopulated, > } state; > +#ifdef CONFIG_MEMCG > + struct mem_cgroup *memcg; > +#endif > }; > > /** From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 860EAC33CAF for ; Mon, 13 Jan 2020 15:55:59 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 59FF620678 for ; Mon, 13 Jan 2020 15:55:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amdcloud.onmicrosoft.com header.i=@amdcloud.onmicrosoft.com header.b="lU/7Lpor" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 59FF620678 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id AD7816E0FE; Mon, 13 Jan 2020 15:55:58 +0000 (UTC) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2066.outbound.protection.outlook.com [40.107.92.66]) by gabe.freedesktop.org (Postfix) with ESMTPS id 5F72D6E0FE for ; Mon, 13 Jan 2020 15:55:57 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=oQJsuS3JwhJTJnD5cwb9dHanWw6X3vUAce34N286teGuCqcC2D/mzbhf2OqJQ360qPcm9/CLxlFNrmgKq/IL/20A01jQ82qlwqg/h+L91cMuYcAzlzDczX3awSLhtJ+P8J/Kxku7w4bzCDmhJUb9w0WOyfigQtVXPkflxaKfAhmRsXTlppV1U6Ixcsp3KWSbiVX6Jz9ZsmQI81sR5wGLHt6VaEUs/Tb7BB72Fali91pMjZoolsFFtqcMmtEXCDL4/SfHHuyRGSNfqa2OERBZauXHz3v4euce/R2mcfjg/XI1CPE4lzyjRSoC+EUKbhjKXA/0/TgkrlhEjIBdfYG7dQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BLPWq8D42ESuKs+vGGWa65Rbhv86/dt79zfyvRTSz6Y=; b=XMbqnCNCY88EUoXAddxJT4zIC5gzFPkVRnJBv5QQ+ETg35u3VxAakXqYgmH9ScRliu34gT04xHR8PQwm+VKYY1FgDGqFuawsSKNqGQ9E+wlr7AF/Bwig0d+AYRAkOO8DlBQ9/bZ3piWW7odZq5IKClnM2TaQALnXcodSxfSRpzfrfb7XUl38aT/cgg7NPylXJ1muYN/h2oTSatsHl989rVkhtjdCUhgythGwR9fDVmz3t/G4AzKLVguFJcdmRHyvBFh215UF/7t8EPCShoEoVncNwYGxwsv6TLpdoJ1sIWlRDoWSPN05TWLPYm9a/o2/WnabQTZ5Ecf1CdhNv6NoXQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector2-amdcloud-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BLPWq8D42ESuKs+vGGWa65Rbhv86/dt79zfyvRTSz6Y=; b=lU/7LporyIr+LV/4mc8vwc6UYm2obwCekeBfq0J1o8VeQOPhwLwzb2uohu/zDpe51pyViWdKTxvaNfyYv8mtQAkFn048INPOvG3Xg9hJB9B1nRHRf26U9hFBaNrlNsG9K0Cn17plHc0pDU4FnmWcr219TRtLT+UicsXJ780meeY= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Christian.Koenig@amd.com; Received: from DM5PR12MB1705.namprd12.prod.outlook.com (10.175.88.22) by DM5PR12MB1852.namprd12.prod.outlook.com (10.175.87.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2623.10; Mon, 13 Jan 2020 15:55:53 +0000 Received: from DM5PR12MB1705.namprd12.prod.outlook.com ([fe80::8dde:b52a:d97a:e89]) by DM5PR12MB1705.namprd12.prod.outlook.com ([fe80::8dde:b52a:d97a:e89%2]) with mapi id 15.20.2623.015; Mon, 13 Jan 2020 15:55:53 +0000 Subject: Re: [PATCH RFC 3/3] drm/ttm: support memcg for ttm_tt To: Qiang Yu , linux-mm@kvack.org, cgroups@vger.kernel.org, dri-devel@lists.freedesktop.org References: <20200113153543.24957-1-qiang.yu@amd.com> <20200113153543.24957-4-qiang.yu@amd.com> From: =?UTF-8?Q?Christian_K=c3=b6nig?= Message-ID: Date: Mon, 13 Jan 2020 16:55:47 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.9.0 In-Reply-To: <20200113153543.24957-4-qiang.yu@amd.com> Content-Language: en-US X-ClientProxiedBy: FR2P281CA0014.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:a::24) To DM5PR12MB1705.namprd12.prod.outlook.com (2603:10b6:3:10c::22) MIME-Version: 1.0 Received: from [IPv6:2a02:908:1252:fb60:be8a:bd56:1f94:86e7] (2a02:908:1252:fb60:be8a:bd56:1f94:86e7) by FR2P281CA0014.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:a::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2623.12 via Frontend Transport; Mon, 13 Jan 2020 15:55:51 +0000 X-Originating-IP: [2a02:908:1252:fb60:be8a:bd56:1f94:86e7] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 4d392e2d-cd36-4813-86da-08d7984111ab X-MS-TrafficTypeDiagnostic: DM5PR12MB1852:|DM5PR12MB1852: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-Forefront-PRVS: 028166BF91 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4636009)(136003)(39860400002)(346002)(376002)(396003)(366004)(199004)(189003)(186003)(36756003)(4326008)(16526019)(31686004)(2616005)(52116002)(86362001)(478600001)(81156014)(8676002)(81166006)(6666004)(316002)(66476007)(66556008)(5660300002)(66946007)(6486002)(54906003)(31696002)(2906002)(8936002); DIR:OUT; SFP:1101; SCL:1; SRVR:DM5PR12MB1852; H:DM5PR12MB1705.namprd12.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: UGmqeTqP/6TeRf5OMDx3KW/YJKFVSafKtO9HhW+J4c/LwgTFHRN5rO4ae76ICjD3F80HFm6rOztisnekadVt5HmUPblfQqS5ZoXNyjjfc+EkRcWzuvvjiMMJBoM/HwQ3EDj4xqhQNyghAkJmr9KdNR3Mrs8MK1m7jkDul4rFR+CgDtEa6bEL92O9gM2eE/uXJ2vNHowOVIOfCdff1DvPGpjdcpmXQldpVUH908yd8wbBGI/MeGU/MAFAcAccn8icj1aT64pTC7EohY6u08S6p+JuuqG8J7+ozBfDRmlJD5RWCJiTvdu9dEL8s6DQ136rAdW6ucK1+gzCOr3NB9gKImOHe/P582748ag4h1DXcO/dUATZdh/cPtz4VfDN9Ccr8BZgl8LukKFVOCztEEEFHhYcl1YCzdBJ8+tbt07r0REJyG6+Bc9RVPApQP+ycRTV X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4d392e2d-cd36-4813-86da-08d7984111ab X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jan 2020 15:55:53.6812 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: LmP0i33G4qVi4Jh025A59WEsfR2Cw/6lZkKZtIf3Xe+LnqcGFpkFsCrLvBq7XKBw X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB1852 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: David Airlie , Kenny Ho , Michal Hocko , Huang Rui , Johannes Weiner , Tejun Heo , Andrew Morton Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Am 13.01.20 um 16:35 schrieb Qiang Yu: > Charge TTM allocated system memory to memory cgroup which will > limit the memory usage of a group of processes. NAK to the whole approach. This belongs into the GEM or driver layer, but not into TTM. > The memory is always charged to the control group of task which > create this buffer object and when it's created. For example, > when a buffer is created by process A and exported to process B, > then process B populate this buffer, the memory is still charged > to process A's memcg; if a buffer is created by process A when in > memcg B, then A is moved to memcg C and populate this buffer, it > will charge memcg B. This is actually the most common use case for graphics application where the X server allocates most of the backing store. So we need a better handling than just accounting the memory to whoever allocated it first. Regards, Christian. > > Signed-off-by: Qiang Yu > --- > drivers/gpu/drm/ttm/ttm_bo.c | 10 ++++++++++ > drivers/gpu/drm/ttm/ttm_page_alloc.c | 18 +++++++++++++++++- > drivers/gpu/drm/ttm/ttm_tt.c | 3 +++ > include/drm/ttm/ttm_bo_api.h | 5 +++++ > include/drm/ttm/ttm_tt.h | 4 ++++ > 5 files changed, 39 insertions(+), 1 deletion(-) > > diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c > index 8d91b0428af1..4e64846ee523 100644 > --- a/drivers/gpu/drm/ttm/ttm_bo.c > +++ b/drivers/gpu/drm/ttm/ttm_bo.c > @@ -42,6 +42,7 @@ > #include > #include > #include > +#include > > static void ttm_bo_global_kobj_release(struct kobject *kobj); > > @@ -162,6 +163,10 @@ static void ttm_bo_release_list(struct kref *list_kref) > if (!ttm_bo_uses_embedded_gem_object(bo)) > dma_resv_fini(&bo->base._resv); > mutex_destroy(&bo->wu_mutex); > +#ifdef CONFIG_MEMCG > + if (bo->memcg) > + css_put(&bo->memcg->css); > +#endif > bo->destroy(bo); > ttm_mem_global_free(&ttm_mem_glob, acc_size); > } > @@ -1330,6 +1335,11 @@ int ttm_bo_init_reserved(struct ttm_bo_device *bdev, > } > atomic_inc(&ttm_bo_glob.bo_count); > > +#ifdef CONFIG_MEMCG > + if (bo->type == ttm_bo_type_device) > + bo->memcg = mem_cgroup_driver_get_from_current(); > +#endif > + > /* > * For ttm_bo_type_device buffers, allocate > * address space from the device. > diff --git a/drivers/gpu/drm/ttm/ttm_page_alloc.c b/drivers/gpu/drm/ttm/ttm_page_alloc.c > index b40a4678c296..ecd1831a1d38 100644 > --- a/drivers/gpu/drm/ttm/ttm_page_alloc.c > +++ b/drivers/gpu/drm/ttm/ttm_page_alloc.c > @@ -42,7 +42,7 @@ > #include /* for seq_printf */ > #include > #include > - > +#include > #include > > #include > @@ -1045,6 +1045,11 @@ ttm_pool_unpopulate_helper(struct ttm_tt *ttm, unsigned mem_count_update) > ttm_put_pages(ttm->pages, ttm->num_pages, ttm->page_flags, > ttm->caching_state); > ttm->state = tt_unpopulated; > + > +#ifdef CONFIG_MEMCG > + if (ttm->memcg) > + mem_cgroup_uncharge_drvmem(ttm->memcg, ttm->num_pages); > +#endif > } > > int ttm_pool_populate(struct ttm_tt *ttm, struct ttm_operation_ctx *ctx) > @@ -1059,6 +1064,17 @@ int ttm_pool_populate(struct ttm_tt *ttm, struct ttm_operation_ctx *ctx) > if (ttm_check_under_lowerlimit(mem_glob, ttm->num_pages, ctx)) > return -ENOMEM; > > +#ifdef CONFIG_MEMCG > + if (ttm->memcg) { > + gfp_t gfp_flags = GFP_USER; > + if (ttm->page_flags & TTM_PAGE_FLAG_NO_RETRY) > + gfp_flags |= __GFP_RETRY_MAYFAIL; > + ret = mem_cgroup_charge_drvmem(ttm->memcg, gfp_flags, ttm->num_pages); > + if (ret) > + return ret; > + } > +#endif > + > ret = ttm_get_pages(ttm->pages, ttm->num_pages, ttm->page_flags, > ttm->caching_state); > if (unlikely(ret != 0)) { > diff --git a/drivers/gpu/drm/ttm/ttm_tt.c b/drivers/gpu/drm/ttm/ttm_tt.c > index e0e9b4f69db6..1acb153084e1 100644 > --- a/drivers/gpu/drm/ttm/ttm_tt.c > +++ b/drivers/gpu/drm/ttm/ttm_tt.c > @@ -233,6 +233,9 @@ void ttm_tt_init_fields(struct ttm_tt *ttm, struct ttm_buffer_object *bo, > ttm->state = tt_unpopulated; > ttm->swap_storage = NULL; > ttm->sg = bo->sg; > +#ifdef CONFIG_MEMCG > + ttm->memcg = bo->memcg; > +#endif > } > > int ttm_tt_init(struct ttm_tt *ttm, struct ttm_buffer_object *bo, > diff --git a/include/drm/ttm/ttm_bo_api.h b/include/drm/ttm/ttm_bo_api.h > index 65e399d280f7..95a08e81a73e 100644 > --- a/include/drm/ttm/ttm_bo_api.h > +++ b/include/drm/ttm/ttm_bo_api.h > @@ -54,6 +54,8 @@ struct ttm_place; > > struct ttm_lru_bulk_move; > > +struct mem_cgroup; > + > /** > * struct ttm_bus_placement > * > @@ -180,6 +182,9 @@ struct ttm_buffer_object { > void (*destroy) (struct ttm_buffer_object *); > unsigned long num_pages; > size_t acc_size; > +#ifdef CONFIG_MEMCG > + struct mem_cgroup *memcg; > +#endif > > /** > * Members not needing protection. > diff --git a/include/drm/ttm/ttm_tt.h b/include/drm/ttm/ttm_tt.h > index c0e928abf592..10fb5a557b95 100644 > --- a/include/drm/ttm/ttm_tt.h > +++ b/include/drm/ttm/ttm_tt.h > @@ -33,6 +33,7 @@ struct ttm_tt; > struct ttm_mem_reg; > struct ttm_buffer_object; > struct ttm_operation_ctx; > +struct mem_cgroup; > > #define TTM_PAGE_FLAG_WRITE (1 << 3) > #define TTM_PAGE_FLAG_SWAPPED (1 << 4) > @@ -116,6 +117,9 @@ struct ttm_tt { > tt_unbound, > tt_unpopulated, > } state; > +#ifdef CONFIG_MEMCG > + struct mem_cgroup *memcg; > +#endif > }; > > /** _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?UTF-8?Q?Christian_K=c3=b6nig?= Subject: Re: [PATCH RFC 3/3] drm/ttm: support memcg for ttm_tt Date: Mon, 13 Jan 2020 16:55:47 +0100 Message-ID: References: <20200113153543.24957-1-qiang.yu@amd.com> <20200113153543.24957-4-qiang.yu@amd.com> Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Return-path: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector2-amdcloud-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BLPWq8D42ESuKs+vGGWa65Rbhv86/dt79zfyvRTSz6Y=; b=lU/7LporyIr+LV/4mc8vwc6UYm2obwCekeBfq0J1o8VeQOPhwLwzb2uohu/zDpe51pyViWdKTxvaNfyYv8mtQAkFn048INPOvG3Xg9hJB9B1nRHRf26U9hFBaNrlNsG9K0Cn17plHc0pDU4FnmWcr219TRtLT+UicsXJ780meeY= In-Reply-To: <20200113153543.24957-4-qiang.yu-5C7GfCeVMHo@public.gmane.org> Content-Language: en-US Sender: cgroups-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org List-ID: Content-Type: text/plain; charset="us-ascii"; format="flowed" To: Qiang Yu , linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org, cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org Cc: Johannes Weiner , Michal Hocko , Andrew Morton , Tejun Heo , Huang Rui , David Airlie , Daniel Vetter , Kenny Ho Am 13.01.20 um 16:35 schrieb Qiang Yu: > Charge TTM allocated system memory to memory cgroup which will > limit the memory usage of a group of processes. NAK to the whole approach. This belongs into the GEM or driver layer, but not into TTM. > The memory is always charged to the control group of task which > create this buffer object and when it's created. For example, > when a buffer is created by process A and exported to process B, > then process B populate this buffer, the memory is still charged > to process A's memcg; if a buffer is created by process A when in > memcg B, then A is moved to memcg C and populate this buffer, it > will charge memcg B. This is actually the most common use case for graphics application where the X server allocates most of the backing store. So we need a better handling than just accounting the memory to whoever allocated it first. Regards, Christian. > > Signed-off-by: Qiang Yu > --- > drivers/gpu/drm/ttm/ttm_bo.c | 10 ++++++++++ > drivers/gpu/drm/ttm/ttm_page_alloc.c | 18 +++++++++++++++++- > drivers/gpu/drm/ttm/ttm_tt.c | 3 +++ > include/drm/ttm/ttm_bo_api.h | 5 +++++ > include/drm/ttm/ttm_tt.h | 4 ++++ > 5 files changed, 39 insertions(+), 1 deletion(-) > > diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c > index 8d91b0428af1..4e64846ee523 100644 > --- a/drivers/gpu/drm/ttm/ttm_bo.c > +++ b/drivers/gpu/drm/ttm/ttm_bo.c > @@ -42,6 +42,7 @@ > #include > #include > #include > +#include > > static void ttm_bo_global_kobj_release(struct kobject *kobj); > > @@ -162,6 +163,10 @@ static void ttm_bo_release_list(struct kref *list_kref) > if (!ttm_bo_uses_embedded_gem_object(bo)) > dma_resv_fini(&bo->base._resv); > mutex_destroy(&bo->wu_mutex); > +#ifdef CONFIG_MEMCG > + if (bo->memcg) > + css_put(&bo->memcg->css); > +#endif > bo->destroy(bo); > ttm_mem_global_free(&ttm_mem_glob, acc_size); > } > @@ -1330,6 +1335,11 @@ int ttm_bo_init_reserved(struct ttm_bo_device *bdev, > } > atomic_inc(&ttm_bo_glob.bo_count); > > +#ifdef CONFIG_MEMCG > + if (bo->type == ttm_bo_type_device) > + bo->memcg = mem_cgroup_driver_get_from_current(); > +#endif > + > /* > * For ttm_bo_type_device buffers, allocate > * address space from the device. > diff --git a/drivers/gpu/drm/ttm/ttm_page_alloc.c b/drivers/gpu/drm/ttm/ttm_page_alloc.c > index b40a4678c296..ecd1831a1d38 100644 > --- a/drivers/gpu/drm/ttm/ttm_page_alloc.c > +++ b/drivers/gpu/drm/ttm/ttm_page_alloc.c > @@ -42,7 +42,7 @@ > #include /* for seq_printf */ > #include > #include > - > +#include > #include > > #include > @@ -1045,6 +1045,11 @@ ttm_pool_unpopulate_helper(struct ttm_tt *ttm, unsigned mem_count_update) > ttm_put_pages(ttm->pages, ttm->num_pages, ttm->page_flags, > ttm->caching_state); > ttm->state = tt_unpopulated; > + > +#ifdef CONFIG_MEMCG > + if (ttm->memcg) > + mem_cgroup_uncharge_drvmem(ttm->memcg, ttm->num_pages); > +#endif > } > > int ttm_pool_populate(struct ttm_tt *ttm, struct ttm_operation_ctx *ctx) > @@ -1059,6 +1064,17 @@ int ttm_pool_populate(struct ttm_tt *ttm, struct ttm_operation_ctx *ctx) > if (ttm_check_under_lowerlimit(mem_glob, ttm->num_pages, ctx)) > return -ENOMEM; > > +#ifdef CONFIG_MEMCG > + if (ttm->memcg) { > + gfp_t gfp_flags = GFP_USER; > + if (ttm->page_flags & TTM_PAGE_FLAG_NO_RETRY) > + gfp_flags |= __GFP_RETRY_MAYFAIL; > + ret = mem_cgroup_charge_drvmem(ttm->memcg, gfp_flags, ttm->num_pages); > + if (ret) > + return ret; > + } > +#endif > + > ret = ttm_get_pages(ttm->pages, ttm->num_pages, ttm->page_flags, > ttm->caching_state); > if (unlikely(ret != 0)) { > diff --git a/drivers/gpu/drm/ttm/ttm_tt.c b/drivers/gpu/drm/ttm/ttm_tt.c > index e0e9b4f69db6..1acb153084e1 100644 > --- a/drivers/gpu/drm/ttm/ttm_tt.c > +++ b/drivers/gpu/drm/ttm/ttm_tt.c > @@ -233,6 +233,9 @@ void ttm_tt_init_fields(struct ttm_tt *ttm, struct ttm_buffer_object *bo, > ttm->state = tt_unpopulated; > ttm->swap_storage = NULL; > ttm->sg = bo->sg; > +#ifdef CONFIG_MEMCG > + ttm->memcg = bo->memcg; > +#endif > } > > int ttm_tt_init(struct ttm_tt *ttm, struct ttm_buffer_object *bo, > diff --git a/include/drm/ttm/ttm_bo_api.h b/include/drm/ttm/ttm_bo_api.h > index 65e399d280f7..95a08e81a73e 100644 > --- a/include/drm/ttm/ttm_bo_api.h > +++ b/include/drm/ttm/ttm_bo_api.h > @@ -54,6 +54,8 @@ struct ttm_place; > > struct ttm_lru_bulk_move; > > +struct mem_cgroup; > + > /** > * struct ttm_bus_placement > * > @@ -180,6 +182,9 @@ struct ttm_buffer_object { > void (*destroy) (struct ttm_buffer_object *); > unsigned long num_pages; > size_t acc_size; > +#ifdef CONFIG_MEMCG > + struct mem_cgroup *memcg; > +#endif > > /** > * Members not needing protection. > diff --git a/include/drm/ttm/ttm_tt.h b/include/drm/ttm/ttm_tt.h > index c0e928abf592..10fb5a557b95 100644 > --- a/include/drm/ttm/ttm_tt.h > +++ b/include/drm/ttm/ttm_tt.h > @@ -33,6 +33,7 @@ struct ttm_tt; > struct ttm_mem_reg; > struct ttm_buffer_object; > struct ttm_operation_ctx; > +struct mem_cgroup; > > #define TTM_PAGE_FLAG_WRITE (1 << 3) > #define TTM_PAGE_FLAG_SWAPPED (1 << 4) > @@ -116,6 +117,9 @@ struct ttm_tt { > tt_unbound, > tt_unpopulated, > } state; > +#ifdef CONFIG_MEMCG > + struct mem_cgroup *memcg; > +#endif > }; > > /**