From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1750890AbdDREJe (ORCPT ); Tue, 18 Apr 2017 00:09:34 -0400 Received: from mail-he1eur01on0102.outbound.protection.outlook.com ([104.47.0.102]:55392 "EHLO EUR01-HE1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1750707AbdDREJb (ORCPT ); Tue, 18 Apr 2017 00:09:31 -0400 Authentication-Results: huawei.com; dkim=none (message not signed) header.d=none;huawei.com; dmarc=none action=none header.from=virtuozzo.com; Date: Mon, 17 Apr 2017 21:09:10 -0700 From: Andrei Vagin To: Zefan Li CC: Tejun Heo , , LKML , Cgroups Subject: Re: cgroup: avoid attaching a cgroup root to two different superblocks Message-ID: <20170418040909.GA5546@outlook.office365.com> References: <58E7532B.4030505@huawei.com> <20170414232737.GC20350@outlook.office365.com> <20170414233233.GA8183@outlook.office365.com> <58F49BE2.8080200@huawei.com> MIME-Version: 1.0 Content-Type: text/plain; charset="koi8-r" Content-Disposition: inline In-Reply-To: <58F49BE2.8080200@huawei.com> User-Agent: Mutt/1.8.0 (2017-02-23) X-Originating-IP: [64.134.159.13] X-ClientProxiedBy: MWHPR10CA0023.namprd10.prod.outlook.com (10.172.48.33) To VI1PR0801MB1983.eurprd08.prod.outlook.com (10.173.74.16) X-MS-Office365-Filtering-Correlation-Id: 7cf3a6e5-6768-44f3-edad-08d48610b322 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001)(201703131423075)(201703031133081);SRVR:VI1PR0801MB1983; X-Microsoft-Exchange-Diagnostics: 1;VI1PR0801MB1983;3:wCXo0wNrc51fxP3RWefY2LIfCqbrqqLNAhXmWX/4R4vN+GktBvswFLzFoAyM7P2KCeDyc8u8kx6Yx4z+Yj23j8W4d4NjgKWU2Dpts4Ck2pplUu8f3KDNWR8flow3OjkoqTKqQb25ELalUKAB4M7ALmY+ulup2sikmd8bi+eiolS7yXJa1+ime4ujTCI8R1Ll947c3iEgU2v7o7aMKglNne6O6Oa0kdk66oKk7blZ/1aPwW+zBNVn4UTezt4obAfSWOinuQdLaNXHHX+yzm3MDTmJ9Vqi3RV31rDldlB4r03ARDLAOf/M3qsfKNljarLE9oyfwTlshtSgWEBwSmHjtQ==;25:7oeTBS++vSp2130WIOZwF+11o/guwTdiSYonCdoEfVCngYdo3GCTywLJswIO877WxfUvgy4DARMJCxOGqts69xIayxpfWt4p06DdELWNw9XROmSjP/Cu60he5fxrPRjPZWU4/k+ng+nt8eEFKfMVgE8KuoqS+FtcMKulu86yNgahvVkd7kxJsVoKU8hMCogozjvD8c6K3PEzEbFSdLZwSL1G6Ux+UR+r7Bco15qRzE/ewHgQZOAW3V5rH/DvTYp4SbQNkcVHlbUmLkItjfgtKtMZLMwXUF3KCjwkFYc99r+RDEqICHr0J0BuxEVH//03uMEOJFBrT2PTmUVwMeoHcCTLMXcu2czF5zKskpAU2BGH6cPYEc8REOHjM4oR03JoXIe1gxDkwAUKVfodzCxVUSdw8eXdLQ4VXeiH9mznkwM1N2A4EWtC+iJ/WRV54X1VHGI8GOhUrdBtKiF9uhXaCw== X-Microsoft-Exchange-Diagnostics: 1;VI1PR0801MB1983;31:qBYU9wtR27RzI1/maNcZp7MyiazQ/pelJ2lSomF4jlH6xcvi9fTaJa66eRuh1S22924M45nPwkwwy+P6a+khPBb+jE8PEIFH7kArduzrRo+CRQA8w1rkVnFAI/k6sculfpf9CzfOYIEjpMerEY10M1A/0Becvcz7OhBuSIOz75C+QOlIrWoII/lVmec7O3gTXid3j3fikwVHyIK6zd55hQ8QcrnkanQ25JT8XANUHVXiPSolKRgf3BbF5y908MWsBjHlYSdihem8GeJz4gtAkgSnoHcG0hoVm4ohcETCqDI=;20:Rzhx/FYuGFReB9uL1W/oFZI1is71AbPBaDtOWlsU3BnklWEPi04KXcgKHiMtcZnskUo0sG2ntXvpcMNmXQMhWl8N9sCjHLdo0C2Q/i8kFqs0tvDDpz5jLMLmKEhP6t9bHcFHpGEHj5R/XL73eCpYCvTD+1Wa8ZA65eq9PwXqwcgeKlmTMFdBaP4RwD1/YKajDpiVj3EhOi6GAT4OMM+v2p+WNt9C/9n27N6cadK7fBYt7UQg+s/bf3zxK2m2eJwqCCAR9r8miyhzjlmzJ3YOf45lr6IUiB8EMJNaFVmOq3yE3ZF2fVyOEJhzn6oH9zV9L1qtkJH8v4py4fGq26b2Nz5V+RGaVoPzPk4cNn9IXCg0SLAri79vlu7Y9jzOdOa9IbI2k+AFTo5G8V3TTr6iEsiikRiNL9nBa1zz9qB/Drk= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(9452136761055)(50582790962513)(211936372134217); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040450)(601004)(2401047)(5005006)(8121501046)(93006095)(93001095)(3002001)(10201501046)(6041248)(20161123564025)(20161123562025)(20161123560025)(201703131423075)(201702281528075)(201703061421075)(20161123555025)(6072148);SRVR:VI1PR0801MB1983;BCL:0;PCL:0;RULEID:;SRVR:VI1PR0801MB1983; X-Microsoft-Exchange-Diagnostics: 1;VI1PR0801MB1983;4:z3us3r6teQ+t4+IUpE1MwPf2+cfUsAFO0A7wlONaTegirv1VbWDmmIUW0A2oGUH0oIu1GEGnCpNTTVevV5pjaq9J6OrAp/AUUKTVKdQtVwYnHRjrdRKcCv80XTKCFOsLt5Rq7c3jG34v0Jra3hJT6C2qC6K5aZsEg/IS+62crcUDs+t8lPadXkA5sWl2IAzUEKOy3gdA1XQiL5TtI2LXPge6sijxRUNVvw6GkneyY5XZwReXjhmi9j+Jp0xNP++DB+0hDecBDe52l50/hGaNpmENBtUN6S45DLoBRZrEZSKjxZbYc1//Bb0PwiBdfu9SDhuSDSv541twk9DYU0JxEucjO4gmZUU5Eu/vl5j3GBcdseuyjiYx0MIyQ20sSyAkwuC8dX0OXbt8ihyYXzx4/dRUt0xMxxu+3XcjHvz+C7HepyxKoptAetaufbJkh8W/i3lhqCGsnRpNfC+8KwqWpPv9uaqXMjt0DZ6QwIM5utHtzVhfjHhxcxeiQA4UMj6E2YmX/IWXe4rDQGa4xA1L7qFaAXjE27nGqB8Kn5ziWyKdJUVom6sZ6eHftmZH4b0rElkBkPvXwiwiJwTkj1aEDI4pCGbwFgFdGp5Sa7mxOCcEd1VaXaqUKZL4IYu3nk3FSooCHELEqQWQ5u1CrXAmvwH1Etpjim4JDZka4SYot/xo4H3VeQI6LN/cmdEh997InfiuSfKr2US0/VQfM3sw6TdmC4z/Ohltq4bBy0uVk3YCSzTt0ZXYVtXWVv5B0gUR9OVqIT/Er4xppcBBX6pI4ODd3XCNqlEYyQy5xwrqMXMwRp4PjuTGHqRpkjaOCOGC9ACr9AfkxRdexmU7UWMIZw== X-Forefront-PRVS: 028166BF91 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6009001)(39450400003)(39830400002)(39400400002)(39410400002)(501624003)(24454002)(229853002)(47776003)(38730400002)(110136004)(93886004)(25786009)(53546009)(76176999)(4326008)(54356999)(305945005)(33656002)(2950100002)(50466002)(189998001)(50986999)(7736002)(6916009)(575784001)(53416004)(42186005)(6666003)(5890100001)(66066001)(55016002)(8676002)(83506001)(81166006)(2906002)(4001350100001)(3846002)(86362001)(54906002)(1076002)(6306002)(9686003)(6506006)(5660300001)(53936002)(6116002)(23686003)(6246003)(18370500001)(26326002)(21314002);DIR:OUT;SFP:1102;SCL:1;SRVR:VI1PR0801MB1983;H:outlook.office365.com;FPR:;SPF:None;MLV:nov;PTR:InfoNoRecords;LANG:en; X-Microsoft-Exchange-Diagnostics: =?koi8-r?Q?1;VI1PR0801MB1983;23:Jqda1rLcI03LwKSPzCJfmrOm/D8gJnNipwxna5N1+?= =?koi8-r?Q?DTIMdii0/bpe8vm1qePMDWZcf/8NoTyk/ao17D3CSuR+Et1Z1fFcz0sOEFy9+o?= =?koi8-r?Q?fbMWcAVi4FR5dekAby53AZT+awKQjqQY+kIlfp+57+gT6QLq9kuP7+I/tx5MCX?= =?koi8-r?Q?G93mlbIYRk3f6AgQpBmzPJJ2QvGqt/UBKEaEHKNNeO9KVtadOgUoJjDsdnMxpm?= =?koi8-r?Q?e+/6hlFZli+xqglf5uObdBRoEyBxlOdPF93/OC0F1QHR/XMRnI/LKDmdn9V862?= =?koi8-r?Q?ITGtsJtu8yHF7UxUA4gZCGSCYnbkn5aXZ5nOnn8rTuwJF8q5b/IypNZ5QajnKe?= =?koi8-r?Q?Src7HgEYoB9Y1lzZmnTFI3CKTUn9XfkV2P9bQJensCYmeSpE+Al/UjCzdGhm7a?= =?koi8-r?Q?dAdP0Dqqy2xPtxUGDZg9d1apDXtG4RRDEJ25FtNUZnkslrlPigYVoIpc30fzSw?= =?koi8-r?Q?L07RL2eQHT6+DgpWmtdWOptAqvgrKPjr5GTZMMj7Lp6Y88H6rBSSAe93E1WXfs?= =?koi8-r?Q?thv8mbhVJ9aV1RUS4ttd00+Irrlo1BnH37EIkOrg4IAPsVXiYWoDqK7nfHcjU0?= =?koi8-r?Q?+ayVNrXpxmQ8qDNlIb6iP/hlWyT5GWTRnxfCD0b9YNRZjB6fMGQAgCBaXhNSIM?= =?koi8-r?Q?61uSLaWu5KjlPrMeJzPGOGe/gyIhYNajl4W9tThoWCVAHqP4zqk2VDrlM1FHZr?= =?koi8-r?Q?pm1huCrtrYfue8xpSxOR5oUR41nTsNDgqv0VyftO8Tr1TxtKoJ72PD7cPz7vRJ?= =?koi8-r?Q?HG+GzW7ZfF8NNkqsefD2w6guQT+rkEXkudMoljZLRwX/yT/BhaXQrR4/HTstWu?= =?koi8-r?Q?Uw6s2gccqK8SgkJfTroGfxE6rX2xAlWfPnulJfVU1F8H/Ik0lf+Qty9JT3ozNy?= =?koi8-r?Q?sm9xusrKUTZg/Bmud57SFx2j1hGxAwEgQJ0UJLvRptacxU7RjAo/p3bKs0966f?= =?koi8-r?Q?KLDBJtvUXcRzO5nY2SBhy4ZzzlJtZJZMszyx/nZP5bRzBRC5mNBfiFYbTXLZL2?= =?koi8-r?Q?N19jahXkyxGom40CuKas0wtJfdgnjL1kY5vJDNi/3uFkNE2tz++lTS0GIEgDFX?= =?koi8-r?Q?aMLBWDfN3b1eEHdHD3aCZgBd6Qn9DexwthdISzj3nwjxiPs/06I8jtmrGjqaFe?= =?koi8-r?Q?54947V/3zZKsy3E6PqVwaZir6rmoqoMnPoGWvXjfgXi9EiwyNMvBZSMhQoNVef?= =?koi8-r?Q?qhIlWiebeMinJHxFEqAL3e9XEkoiIsPGQK7VcV/lI3cGND2fYqis3fbM72D8jl?= =?koi8-r?Q?Ac5FaG/7Z0h3ref1Lrh1lpQV/lmizpbm5F54nBkmuY=3D?= X-Microsoft-Exchange-Diagnostics: 1;VI1PR0801MB1983;6:xVlikbpZPUTV8nUPDvWPiUDahCqr34YgL0iEewYyENRsCRognqnQTiVbk/h975+7MizNINNmUZM6ejtS+R1In/+leEycmp+bIf3ufx/j//xLnQeEA6rGowQawkZVMtQluVyoYGoRxwliCT9UGCixc46Qr03H+4WVwqgg9urrG0CWsDIqz+GEqdIipf57KBJd7wnHgb7muSvjj1IsRzlS9ApvR/SbfvcZM3FHT8A7l5atR2Di8z5UQHeEB1CnzXyu7NLIB2tVPr/q+prz7LY6n2rF1TgYdXoKVKs/cAswuMnlIociy6Mv+ziFR+b4IX7aoF3HgQn4nGRNlgcLUhecWbX3rHLqSjivPnRDdkPjGla9SxxccaSogCu7z2pFTaAGXVz/LAZUyvCNRPagpltn2ENY7LYpVC/kpVBPx0IcNTIgLXfhwxrqKyH5CHPdCjS6spFsyA/uJtCV1YK/C2yUWg==;5:PAm41bu6JPDW3fOjJFOPSygguU/tb6rb6Sfk3Xqg/vJX82fezSbRQEpaLMwlv8d1PI5WZPq6EN5nkz1Q6ORF0rMgvJmyq5UVTR7g9zNBM/pSZT+VfeeuqQX4mBxJTnfP8GTI9ZDKUAmNE9NSibop3Q==;24:SqJgJh89Dcllst6yRbwkOhufTBXnOM0dAdO5poN4+AlppV5fl6BK5IPtGbJI0bUkh9B/f7jD0OpCARi10XQtDDDyNmaF0YdkmQUQX0OOZdo= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;VI1PR0801MB1983;7:J2MCradfduF97QXXcda3aEJbvnlgBFXUPtpZsiQuG0zZ1IqL749ArwUtuYVwRZhdFBemDF2JlUR4RHpan0PR4dmbVI6MjJ3Bseb0pNxCH6MEnJHw3k7bGLBV7btbthXWKhR1O5hznWmIMQ9K06Q3GtOTVpVOF0CKtOHw+P0ksQ/OJxi4DlPsx5ol34KHJiHNQ6hMEqrvoiIfsV96lg64aApCND7OxzDisGckc+Ybe97iiLrVNd7NKUHQat5PlCBuzMFiEbY4J3N88vBBcRlgqiee5a9/FRq2vt/+s4Qs1jbuLo6AkuzsOkgQiR/aRzZ8q3C7jT3JJ8oqWzuZsUzDAw==;20:E3GHl8gTJgpR/r/wA8pbMDiZRaSDg3Bucl5Jd+JH0bl9BhrDY0ZKvBemgSIsst0LrjZxZiDXDWlVQg9hGh0NfA9aScXtsSn6rLJpQ97fBeGrjbj9k0CIDYzHlVsNRuPeowrhcWrm1MBmPdRo/fLLZqpOqvQCRHmxaJo6mvwgppQ= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Apr 2017 04:09:22.9875 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0801MB1983 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Apr 17, 2017 at 06:41:38PM +0800, Zefan Li wrote: > On 2017/4/15 7:32, Andrei Vagin wrote: > > On Fri, Apr 14, 2017 at 04:27:37PM -0700, Andrei Vagin wrote: > >> Hello, > >> > >> One of our CRIU tests hangs with this patch. > >> > >> Steps to reproduce: > >> curl -o cgroupns.c https://gist.githubusercontent.com/avagin/f87c8a8bd2a0de9afcc74976327786bc/raw/5843701ef3679f50dd2427cf57a80871082eb28c/gistfile1.txt > >> gcc cgroupns.c -o cgroupns > >> ./cgroupns > >> ./cgroupns > > > > I've found a trivial reproducer: > > mkdir /tmp/xxx > > mount -t cgroup -o none,name=zdtmtst xxx /tmp/xxx > > mkdir /tmp/xxx/xxx > > umount /tmp/xxx > > mount -t cgroup -o none,name=zdtmtst xxx /tmp/xxx > > > > Now I remember why it didn't check NULL pointer... Could you try the following fix? > It also reverts my previous patch. I would appreciate if you run the full test suit, > to make sure it won't break anything. It works for me. Thanks. Tested-by: Andrei Vagin > > PS: Tejun, I found recently I can no longer receive your emails. Don't know why... > > ======= > > [PATCH] cgruop: avoid attaching a cgroup root to two different superblocks, take 2 > > Commit bfb0b80db5f9 is broken. Now we try to fix the race by delaying > the initialization of cgroup root refcnt until a superblock has been > allocated. > > Cc: stable@vger.kernel.org # 3.16+ > Reported-by: Dmitry Vyukov > Reported-by: Andrei Vagin > Signed-off-by: Zefan Li > --- > kernel/cgroup/cgroup-internal.h | 2 +- > kernel/cgroup/cgroup-v1.c | 18 ++++++++++++++++-- > kernel/cgroup/cgroup.c | 8 ++++---- > 3 files changed, 21 insertions(+), 7 deletions(-) > > diff --git a/kernel/cgroup/cgroup-internal.h b/kernel/cgroup/cgroup-internal.h > index 9203bfb..e470268 100644 > --- a/kernel/cgroup/cgroup-internal.h > +++ b/kernel/cgroup/cgroup-internal.h > @@ -163,7 +163,7 @@ int cgroup_path_ns_locked(struct cgroup *cgrp, char *buf, size_t buflen, > > void cgroup_free_root(struct cgroup_root *root); > void init_cgroup_root(struct cgroup_root *root, struct cgroup_sb_opts *opts); > -int cgroup_setup_root(struct cgroup_root *root, u16 ss_mask); > +int cgroup_setup_root(struct cgroup_root *root, u16 ss_mask, int ref_flags); > int rebind_subsystems(struct cgroup_root *dst_root, u16 ss_mask); > struct dentry *cgroup_do_mount(struct file_system_type *fs_type, int flags, > struct cgroup_root *root, unsigned long magic, > diff --git a/kernel/cgroup/cgroup-v1.c b/kernel/cgroup/cgroup-v1.c > index 12e19f0..6ca9b12 100644 > --- a/kernel/cgroup/cgroup-v1.c > +++ b/kernel/cgroup/cgroup-v1.c > @@ -1072,6 +1072,7 @@ struct dentry *cgroup1_mount(struct file_system_type *fs_type, int flags, > struct cgroup_subsys *ss; > struct dentry *dentry; > int i, ret; > + bool new_root = false; > > cgroup_lock_and_drain_offline(&cgrp_dfl_root.cgrp); > > @@ -1146,7 +1147,7 @@ struct dentry *cgroup1_mount(struct file_system_type *fs_type, int flags, > * path is super cold. Let's just sleep a bit and retry. > */ > pinned_sb = kernfs_pin_sb(root->kf_root, NULL); > - if (IS_ERR_OR_NULL(pinned_sb) || > + if (IS_ERR(pinned_sb) || > !percpu_ref_tryget_live(&root->cgrp.self.refcnt)) { > mutex_unlock(&cgroup_mutex); > if (!IS_ERR_OR_NULL(pinned_sb)) > @@ -1181,10 +1182,11 @@ struct dentry *cgroup1_mount(struct file_system_type *fs_type, int flags, > ret = -ENOMEM; > goto out_unlock; > } > + new_root = true; > > init_cgroup_root(root, &opts); > > - ret = cgroup_setup_root(root, opts.subsys_mask); > + ret = cgroup_setup_root(root, opts.subsys_mask, PERCPU_REF_INIT_DEAD); > if (ret) > cgroup_free_root(root); > > @@ -1201,6 +1203,18 @@ struct dentry *cgroup1_mount(struct file_system_type *fs_type, int flags, > CGROUP_SUPER_MAGIC, ns); > > /* > + * There's a race window after we release cgroup_mutex and before > + * allocating a superblock. Make sure a concurrent process won't > + * be able to re-use the root during this window by delaying the > + * initialization of root refcnt. > + */ > + if (new_root) { > + mutex_lock(&cgroup_mutex); > + percpu_ref_reinit(&root->cgrp.self.refcnt); > + mutex_unlock(&cgroup_mutex); > + } > + > + /* > * If @pinned_sb, we're reusing an existing root and holding an > * extra ref on its sb. Mount is complete. Put the extra ref. > */ > diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c > index 4885132..0f98010 100644 > --- a/kernel/cgroup/cgroup.c > +++ b/kernel/cgroup/cgroup.c > @@ -1640,7 +1640,7 @@ void init_cgroup_root(struct cgroup_root *root, struct cgroup_sb_opts *opts) > set_bit(CGRP_CPUSET_CLONE_CHILDREN, &root->cgrp.flags); > } > > -int cgroup_setup_root(struct cgroup_root *root, u16 ss_mask) > +int cgroup_setup_root(struct cgroup_root *root, u16 ss_mask, int ref_flags) > { > LIST_HEAD(tmp_links); > struct cgroup *root_cgrp = &root->cgrp; > @@ -1656,8 +1656,8 @@ int cgroup_setup_root(struct cgroup_root *root, u16 ss_mask) > root_cgrp->id = ret; > root_cgrp->ancestor_ids[0] = ret; > > - ret = percpu_ref_init(&root_cgrp->self.refcnt, css_release, 0, > - GFP_KERNEL); > + ret = percpu_ref_init(&root_cgrp->self.refcnt, css_release, > + ref_flags, GFP_KERNEL); > if (ret) > goto out; > > @@ -4512,7 +4512,7 @@ int __init cgroup_init(void) > hash_add(css_set_table, &init_css_set.hlist, > css_set_hash(init_css_set.subsys)); > > - BUG_ON(cgroup_setup_root(&cgrp_dfl_root, 0)); > + BUG_ON(cgroup_setup_root(&cgrp_dfl_root, 0, 0)); > > mutex_unlock(&cgroup_mutex); > > -- > 1.8.3.1 >