From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 481DCC43460 for ; Fri, 21 May 2021 14:04:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 21C5B613C8 for ; Fri, 21 May 2021 14:04:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236555AbhEUOFx (ORCPT ); Fri, 21 May 2021 10:05:53 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:32627 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232587AbhEUOFw (ORCPT ); Fri, 21 May 2021 10:05:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621605865; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=0FWGnS1xTBJJ2Fo3anpEGnC99ELWCmAqg/0rfir/TOM=; b=S0WjffjOEhgRFANVKlMKrnN+y0GZ4FN4QZZeY92/YjzgqVKFf/b1L2OCHm0d93cJPolVZ+ XwcoKuwhPw7IgYH2dRoSxL/X4RRL///iFkbiM6+7NxBHpRxsYnMb1nRjDYG88EzE1i8qPj z4KFsmzxG9iFXv3dyrw5S2HECSHJB2k= Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-449-fEnEI5kjNqaz701bFFLNnw-1; Fri, 21 May 2021 10:04:23 -0400 X-MC-Unique: fEnEI5kjNqaz701bFFLNnw-1 Received: by mail-ed1-f71.google.com with SMTP id da10-20020a056402176ab029038f0fea1f51so3192388edb.13 for ; Fri, 21 May 2021 07:04:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=0FWGnS1xTBJJ2Fo3anpEGnC99ELWCmAqg/0rfir/TOM=; b=F4Lt1onQadAdqK3mkPkw8SByQF2D5jWlyNSxH70jYdMIj1Li7MQnQ8lHUQXFVtW+ua CBYCvoHyRk+XUuSc0DQtE7i6DFzXANJ/lXRJaL5mH8c5pGWBLBjEdzIazQ8ol5Gm40zo AwLw3o3oef1psbV8ea6ca+GuhQmrDA9mSpjoyWd/DKuilucB4NRv1l1WcIs352UvVcpn XB51jDrA7zgXfKZ6E7+E0Ptnq8PyFEXzSvnNdWifGKaJZyEa/dWDt9qkjBFxQTfglKSj 65YUxRe96EIpdmM3aZD13ZRMCH6Pzi/zIKVSuh+dXfSO8RmQwJ4ys+/AL1dRf6ix61Hr PS9A== X-Gm-Message-State: AOAM532RNbkAFiukm0BymOLHrsUk8gi9BnxPSumuxeUUTX0h0V7wrPGk wN9SxmkpKRZJm+nbzoEd0HRMbt84ddfe+BOS73iyZWo2Gc1gerCwM7SjBasaui4iZ2f89S3qH/z ItDDfkLYYuMwqvvVS0lEewog+ X-Received: by 2002:a17:906:7c4b:: with SMTP id g11mr10460873ejp.461.1621605862067; Fri, 21 May 2021 07:04:22 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxXqTbrJ0iKwqv77esCIvIhfq1WCbIdlw3aMc1GDJjhdYtzXix6uBQJwXOo+Ew6v5VsCgDyRg== X-Received: by 2002:a17:906:7c4b:: with SMTP id g11mr10460834ejp.461.1621605861789; Fri, 21 May 2021 07:04:21 -0700 (PDT) Received: from localhost.localdomain ([151.29.18.58]) by smtp.gmail.com with ESMTPSA id qo19sm3569598ejb.7.2021.05.21.07.04.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 May 2021 07:04:21 -0700 (PDT) Date: Fri, 21 May 2021 16:04:18 +0200 From: Juri Lelli To: Quentin Perret Cc: Dietmar Eggemann , Will Deacon , Daniel Bristot de Oliveira , linux-arm-kernel@lists.infradead.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Tejun Heo , Johannes Weiner , Ingo Molnar , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: Re: [PATCH v6 13/21] sched: Admit forcefully-affined tasks into SCHED_DEADLINE Message-ID: References: <20210520101640.GA10065@willie-the-truck> <20210520180138.GA10523@willie-the-truck> <20210521103724.GA11680@willie-the-truck> <3620bad5-2a27-0f9e-f1f0-70036997d33c@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 21/05/21 13:02, Quentin Perret wrote: ... > So I think Will has a point since, IIRC, the root domains get rebuilt > during hotplug. So you can imagine a case with a single root domain, but > CPUs 4-7 are offline. In this case, sched_setattr() will happily promote > a task to DL as long as its affinity mask is a superset of the rd span, > but things may get ugly when CPUs are plugged back in later on. > > This looks like an existing bug though. I just tried the following on a > system with 4 CPUs: > > // Create a task affined to CPU [0-2] > > while true; do echo "Hi" > /dev/null; done & > [1] 560 > > mypid=$! > > taskset -p 7 $mypid > pid 560's current affinity mask: f > pid 560's new affinity mask: 7 > > // Try to move it DL, this should fail because of the affinity > > chrt -d -T 5000000 -P 16666666 -p 0 $mypid > chrt: failed to set pid 560's policy: Operation not permitted > > // Offline CPU 3, so the rd now covers CPUs 0-2 only > > echo 0 > /sys/devices/system/cpu/cpu3/online > [ 400.843830] CPU3: shutdown > [ 400.844100] psci: CPU3 killed (polled 0 ms) > > // Try to admit the task again, which now succeeds > > chrt -d -T 5000000 -P 16666666 -p 0 $mypid > > // Plug CPU3 back online > > echo 1 > /sys/devices/system/cpu/cpu3/online > [ 408.819337] Detected PIPT I-cache on CPU3 > [ 408.819642] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000 > [ 408.820165] CPU3: Booted secondary processor 0x0000000003 [0x410fd083] > > I don't see any easy way to fix this w/o iterating over all deadline > tasks in the rd when hotplugging a CPU back on, and blocking the hotplug > operation if it'll cause affinity issues. Urgh. > Yeah this looks like a plain existing bug, joy. :) We fixed a few around AC lately, but I guess work wasn't complete. Thanks, Juri From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4E9CC433ED for ; Fri, 21 May 2021 14:06:46 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4858C6128A for ; Fri, 21 May 2021 14:06:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4858C6128A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=i9e540D4sNsFc8KTNsGi0wGK6TJrNUdrqO6mxDeZEx4=; b=kVW8oUhDrZHxjXf0kQZ1sKF13Y dWzPUjkL/uTNjNCz5XbGdNHBkWzL0vaJ2XzTDxyQUxe3TTZM83wAHAx9LloHjOvTGAowXuOerdUzd KP/qlgda1HdR627PUFcbTjMgUUscWCLkxq1h5g4FVFqvHTKZ+lf4eWv3RGn/1SRIeo2uYhEg4ltrq oxbR14YQCRwWnM2gEMwE8MZqoJGKC/kXFZt8Gvfd3/z9zrk0Fl3tDjrfhAbDkBjo7wKOMvowuX9P4 FlWsR0jRYoJO63gTd1cW49F4NSLiP6t2nqfvyIkyg+E9ZyMYK5/GP3g4RdwwHZ0FcdegnQE2GgDRS ArDXkddw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lk5lR-005hzh-8F; Fri, 21 May 2021 14:04:33 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lk5lP-005hzM-43 for linux-arm-kernel@desiato.infradead.org; Fri, 21 May 2021 14:04:31 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:In-Reply-To:MIME-Version :References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=0FWGnS1xTBJJ2Fo3anpEGnC99ELWCmAqg/0rfir/TOM=; b=AnrVF5YUp41Kd5SJoBN0MOYmFu 0HsPmBYoZgZWFlmrtr/AuEtWnJTwQK+yy7Qd8THFXm8aRA42xDCAqWpptpdcYMHXmW/zXkHSEPrhm VLN418YBAFxZz8LsHuEJrw3D0T7v014K44UkWyXgoe0ziglRkESgaXyGCeZpjI0x/+gbUfOHcEXFF DOL2K6uLHz2LjqjoOTVywS3MHktMVOeYNCasLe/kp2+1JcK/4lann75btOhV8YJq+keDu9rr8besn A+ZGEXIKeUQuOyXtipuTaPfe9Mq7BRtnsbshc/xVyA9uwxaQvFVN/MT+g5Dk8DxtE3z5Fx6qKGWTt +9Gl/1mw==; Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lk5lM-00HB1T-Dw for linux-arm-kernel@lists.infradead.org; Fri, 21 May 2021 14:04:29 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621605865; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=0FWGnS1xTBJJ2Fo3anpEGnC99ELWCmAqg/0rfir/TOM=; b=S0WjffjOEhgRFANVKlMKrnN+y0GZ4FN4QZZeY92/YjzgqVKFf/b1L2OCHm0d93cJPolVZ+ XwcoKuwhPw7IgYH2dRoSxL/X4RRL///iFkbiM6+7NxBHpRxsYnMb1nRjDYG88EzE1i8qPj z4KFsmzxG9iFXv3dyrw5S2HECSHJB2k= Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-576-v-pbXOH-PkG2n8GW3KoqSw-1; Fri, 21 May 2021 10:04:23 -0400 X-MC-Unique: v-pbXOH-PkG2n8GW3KoqSw-1 Received: by mail-ed1-f69.google.com with SMTP id b8-20020a05640202c8b029038f1782a77eso2918421edx.15 for ; Fri, 21 May 2021 07:04:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=0FWGnS1xTBJJ2Fo3anpEGnC99ELWCmAqg/0rfir/TOM=; b=YsARiilAh+ivebCqdvV06mm2api56AZ0LpwzyXAsgpagkHCnSkdNdKRuHP9OlV5Z45 w20EktRJ7V2hSTfxmOI+D747ftMzUGWwHRrayUwgl5a5kaieqEg3F2bM/NHP/7RQqW+t IC9lOqiykZqpW0c4ghicNvH8u0iPd1SRG5/xJCAktJHzfupZnJlwA2CVj4zFiv4EA/xX yFCzIWvn5M9pfiLOgjO88dFYtpeRR8kYmww4Ah7xq6O9prgz7MabAhVw6fIIA2YsArkQ WZ96qp0feTSUcaiQqvlMkPyInfKcrW/Y0vkZ3tHjdRbaddJ1WpkEHPi5E3m/p2Lwt2JL RqAA== X-Gm-Message-State: AOAM532tpXda8HoE/Kw4nnN7D496SvF8BPsK+rjqz56AOCza97hzIR04 BGvHY0P2YdHSyF8/KnW62UtisveAFn1995KrTgaUTMRoJHXU36VA7KHBCtOmDnd8fCgp8FUiA1o 04Q+Fpf5Oy8M+LVRDTJeDFIR13Dsikb/Fgto= X-Received: by 2002:a17:906:7c4b:: with SMTP id g11mr10460880ejp.461.1621605862069; Fri, 21 May 2021 07:04:22 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxXqTbrJ0iKwqv77esCIvIhfq1WCbIdlw3aMc1GDJjhdYtzXix6uBQJwXOo+Ew6v5VsCgDyRg== X-Received: by 2002:a17:906:7c4b:: with SMTP id g11mr10460834ejp.461.1621605861789; Fri, 21 May 2021 07:04:21 -0700 (PDT) Received: from localhost.localdomain ([151.29.18.58]) by smtp.gmail.com with ESMTPSA id qo19sm3569598ejb.7.2021.05.21.07.04.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 May 2021 07:04:21 -0700 (PDT) Date: Fri, 21 May 2021 16:04:18 +0200 From: Juri Lelli To: Quentin Perret Cc: Dietmar Eggemann , Will Deacon , Daniel Bristot de Oliveira , linux-arm-kernel@lists.infradead.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Tejun Heo , Johannes Weiner , Ingo Molnar , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: Re: [PATCH v6 13/21] sched: Admit forcefully-affined tasks into SCHED_DEADLINE Message-ID: References: <20210520101640.GA10065@willie-the-truck> <20210520180138.GA10523@willie-the-truck> <20210521103724.GA11680@willie-the-truck> <3620bad5-2a27-0f9e-f1f0-70036997d33c@arm.com> MIME-Version: 1.0 In-Reply-To: Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=juri.lelli@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210521_070428_596116_2FD21B98 X-CRM114-Status: GOOD ( 18.75 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 21/05/21 13:02, Quentin Perret wrote: ... > So I think Will has a point since, IIRC, the root domains get rebuilt > during hotplug. So you can imagine a case with a single root domain, but > CPUs 4-7 are offline. In this case, sched_setattr() will happily promote > a task to DL as long as its affinity mask is a superset of the rd span, > but things may get ugly when CPUs are plugged back in later on. > > This looks like an existing bug though. I just tried the following on a > system with 4 CPUs: > > // Create a task affined to CPU [0-2] > > while true; do echo "Hi" > /dev/null; done & > [1] 560 > > mypid=$! > > taskset -p 7 $mypid > pid 560's current affinity mask: f > pid 560's new affinity mask: 7 > > // Try to move it DL, this should fail because of the affinity > > chrt -d -T 5000000 -P 16666666 -p 0 $mypid > chrt: failed to set pid 560's policy: Operation not permitted > > // Offline CPU 3, so the rd now covers CPUs 0-2 only > > echo 0 > /sys/devices/system/cpu/cpu3/online > [ 400.843830] CPU3: shutdown > [ 400.844100] psci: CPU3 killed (polled 0 ms) > > // Try to admit the task again, which now succeeds > > chrt -d -T 5000000 -P 16666666 -p 0 $mypid > > // Plug CPU3 back online > > echo 1 > /sys/devices/system/cpu/cpu3/online > [ 408.819337] Detected PIPT I-cache on CPU3 > [ 408.819642] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000 > [ 408.820165] CPU3: Booted secondary processor 0x0000000003 [0x410fd083] > > I don't see any easy way to fix this w/o iterating over all deadline > tasks in the rd when hotplugging a CPU back on, and blocking the hotplug > operation if it'll cause affinity issues. Urgh. > Yeah this looks like a plain existing bug, joy. :) We fixed a few around AC lately, but I guess work wasn't complete. Thanks, Juri _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel