From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 15318C433ED for ; Fri, 21 May 2021 08:41:52 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9055861073 for ; Fri, 21 May 2021 08:41:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9055861073 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=l34jd7xF4dYT+y47bhnMojELQDqFhZkQWDKYGcI52/8=; b=TONUL9FrjXtCRtYw+AjOeSqqqW 0b9NJL4KnOLDPiZXXE7fI5lLWmgI8zdgUHi15hJPVOjA2K+tVZSgZ8EZGXyeGKR7VVk/6WYsxxIGt oJDJDe7MFsmdwMgOSmQNJcSBgnmvNXVw5X0b2F9sJR/4vKk3hJqqKmwDCtwWFXSqNXiM7z/W9dk+2 mcSA5CE3lqVoxlYUXKKzxDYpt9yLAHRlKknlWmF4RM6cZtdgYPGGXimw8g+GGYF2i66D43Wm4i5g4 /b5vv6LA98+WVk7DgMQcF943HsXj+0HftrnkjlPs7yRuA1kRLjMupFIqoPsVwmfCVONMWCvuulOLT H/V1/5CA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lk0h7-004TGZ-4W; Fri, 21 May 2021 08:39:45 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lk0h3-004TFH-VW for linux-arm-kernel@desiato.infradead.org; Fri, 21 May 2021 08:39:42 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:In-Reply-To:MIME-Version :References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=SIGUda3zFfWUzAh5vPjq/XAaI8wuK0BcxP3Vanw6sXM=; b=LoUQyD3jfek4m+mBrn6DX8aGun 2t0qAQJWUg2lGh4wO0JfIVgeiCriWHwin448FwIOiv1cwlRxTRrQ15G6HW/8CtRF6/04rvndxwYIZ uFad+/d/xktu7kVBJWrhrF1pSs9lX8pTwI1fC4iV48SakFwQM99p0oQeecHaBPfY3bE//t6UdN4fE 0qU3k5WKYp3+IFzlPeiuTBocQVLlcW069cp9iLo8lxWzwQXxICRRrQ6oWv0Y8NefA5mttnMc7ShyR NdTGzl2AOngwaSTAAr4n5vHNgTCKobCr6tqDTBii9qijp3ZXlOb4Bcg03F5P2yJrdP4FyjnvQra37 AAM9hKmg==; Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lk0h1-00GxEl-6B for linux-arm-kernel@lists.infradead.org; Fri, 21 May 2021 08:39:40 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621586378; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=SIGUda3zFfWUzAh5vPjq/XAaI8wuK0BcxP3Vanw6sXM=; b=dQL6fZ1+nBW5NF/17rEr4V+COyjV9q6kLTln/JHWqtamCnFTMy6sKyEV8/2I1OGsyAbvul x4ih/HI20AfusBaSnYdvNcULsyCTPOGoz7QstA41rApjRCRxm2gyZf4rXfwfTm7dluh2zA sDGqicOnNS7Q3qqSBSBq6xLz1VoB7pc= Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-206-P40L-ME7MIOKS0Ax4Egi9Q-1; Fri, 21 May 2021 04:39:36 -0400 X-MC-Unique: P40L-ME7MIOKS0Ax4Egi9Q-1 Received: by mail-ed1-f71.google.com with SMTP id h18-20020a05640250d2b029038cc3938914so11008374edb.17 for ; Fri, 21 May 2021 01:39:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=SIGUda3zFfWUzAh5vPjq/XAaI8wuK0BcxP3Vanw6sXM=; b=kCM5E8lTgNG0dIadhl73XffAZ5wZPuLC6vtpqGGxrTp+y0OAZ7O5Jt8BBn9Ugi5wj1 VnGY/WheI0zetH5lI/48uUzC5E/aKlbm9jYgPgy25+QuuBAoiI5g1EBl4WQhy6Nmz6Ep 2st+FWiQnFWIAUV7QQXowaZo9xpkinKuLXYmGJPeJ3VCUKKFZE2iNOildChkmKcmu7dA 7JB8gYNNAC1eE9NL4dfzu7a4ZdV/2N38D5Pnvo7qe6eDyqvg8o1R9sdsWQpJtwm3dCit p/H5U1Z9wt2NFdUsKYGK/mGgiGH6RQ5zWCGaTlFswqdNvhUGHIcwYQR37js+LD5R0BPx p6+A== X-Gm-Message-State: AOAM530aWTSUFpF/bV+ojJ4hvX5zdnbz2SW+o+THipyPH9C5MEljjcDx mWjf6nx9wwxFfCjPiErmFJIdnq7J32C8l0MFWOLRYTRI+GWHoB7HXFJywfwLP/PfCR0HXDhkN67 9jN9AxDVKOPgu2pAnmR9wlew1PBZ57ZWo/x4= X-Received: by 2002:a17:906:c352:: with SMTP id ci18mr9001142ejb.149.1621586375374; Fri, 21 May 2021 01:39:35 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxdd5WRAKk/AQzDdmZXxSEP43ND8VjF/XIffF9cLdHoFc07C/2r7v1/7V15PQS5O81Wo7ngWQ== X-Received: by 2002:a17:906:c352:: with SMTP id ci18mr9001125ejb.149.1621586375131; Fri, 21 May 2021 01:39:35 -0700 (PDT) Received: from localhost.localdomain ([151.29.18.58]) by smtp.gmail.com with ESMTPSA id d5sm3444716edt.49.2021.05.21.01.39.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 May 2021 01:39:34 -0700 (PDT) Date: Fri, 21 May 2021 10:39:32 +0200 From: Juri Lelli To: Quentin Perret Cc: Will Deacon , Daniel Bristot de Oliveira , linux-arm-kernel@lists.infradead.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Tejun Heo , Johannes Weiner , Ingo Molnar , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: Re: [PATCH v6 13/21] sched: Admit forcefully-affined tasks into SCHED_DEADLINE Message-ID: References: <20210518105951.GC7770@willie-the-truck> <20210520101640.GA10065@willie-the-truck> <20210520180138.GA10523@willie-the-truck> MIME-Version: 1.0 In-Reply-To: Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=juri.lelli@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210521_013939_329789_F0CE187A X-CRM114-Status: GOOD ( 42.87 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 21/05/21 08:15, Quentin Perret wrote: > On Friday 21 May 2021 at 07:25:51 (+0200), Juri Lelli wrote: > > On 20/05/21 19:01, Will Deacon wrote: > > > On Thu, May 20, 2021 at 02:38:55PM +0200, Daniel Bristot de Oliveira wrote: > > > > On 5/20/21 12:33 PM, Quentin Perret wrote: > > > > > On Thursday 20 May 2021 at 11:16:41 (+0100), Will Deacon wrote: > > > > >> Ok, thanks for the insight. In which case, I'll go with what we discussed: > > > > >> require admission control to be disabled for sched_setattr() but allow > > > > >> execve() to a 32-bit task from a 64-bit deadline task with a warning (this > > > > >> is probably similar to CPU hotplug?). > > > > > > > > > > Still not sure that we can let execve go through ... It will break AC > > > > > all the same, so it should probably fail as well if AC is on IMO > > > > > > > > > > > > > If the cpumask of the 32-bit task is != of the 64-bit task that is executing it, > > > > the admission control needs to be re-executed, and it could fail. So I see this > > > > operation equivalent to sched_setaffinity(). This will likely be true for future > > > > schedulers that will allow arbitrary affinities (AC should run on affinity > > > > change, and could fail). > > > > > > > > I would vote with Juri: "I'd go with fail hard if AC is on, let it > > > > pass if AC is off (supposedly the user knows what to do)," (also hope nobody > > > > complains until we add better support for affinity, and use this as a motivation > > > > to get back on this front). > > > > > > I can have a go at implementing it, but I don't think it's a great solution > > > and here's why: > > > > > > Failing an execve() is _very_ likely to be fatal to the application. It's > > > also very likely that the task calling execve() doesn't know whether the > > > program it's trying to execute is 32-bit or not. Consequently, if we go > > > with failing execve() then all that will happen is that people will disable > > > admission control altogether. > > Right, but only on these dumb 32bit asymmetric systems, and only if we > care about running 32bits deadline tasks -- which I seriously doubt for > the Android use-case. > > Note that running deadline tasks is also a privileged operation, it > can't be done by random apps. > > > > That has a negative impact on "pure" 64-bit > > > applications and so I think we end up with the tail wagging the dog because > > > admission control will be disabled for everybody just because there is a > > > handful of 32-bit programs which may get executed. I understand that it > > > also means that RT throttling would be disabled. > > > > Completely understand your perplexity. But how can the kernel still give > > guarantees to "pure" 64-bit applications if there are 32-bit > > applications around that essentially broke admission control when they > > were restricted to a subset of cores? > > > > > Allowing the execve() to continue with a warning is very similar to the > > > case in which all the 64-bit CPUs are hot-unplugged at the point of > > > execve(), and this is much closer to the illusion that this patch series > > > intends to provide. > > > > So, for hotplug we currently have a check that would make hotplug > > operations fail if removing a CPU would mean not enough bandwidth to run > > the currently admitted set of DEADLINE tasks. > > Aha, wasn't aware. Any pointers to that check for my education? Hotplug ends up calling dl_cpu_busy() (after the cpu being hotplugged out got removed), IIRC. So, if that fails the operation in undone. > > > So, personally speaking, I would prefer the behaviour where we refuse to > > > admit 32-bit tasks vioa sched_set_attr() if the root domain contains > > > 64-bit CPUs, but we _don't_ fail execve() of a 32-bit program from a > > > 64-bit deadline task. > > > > OK, this is interesting and I guess a very valid alternative. That would > > force users to create exclusive domains for 32-bit tasks, right? > > FWIW this is not practical at all for our use-cases, the implications of > splitting the system in independent root-domains are way too important > for us to be able to recommend that. Disabling AC, OTOH, sounds simple > enough. The RT throttling part is the only 'worrying' part, but even > that may not be the end of the world. Note that RT throttling (SCHED_{FIFO,RR}) is not handled by DEADLINE servers yet. Best, Juri _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel