From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA338C3A5A1 for ; Thu, 22 Aug 2019 13:29:03 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3BAA7233FE for ; Thu, 22 Aug 2019 13:29:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3BAA7233FE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ellerman.id.au Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 46Dljl4dK7zDqK1 for ; Thu, 22 Aug 2019 23:28:59 +1000 (AEST) Received: from ozlabs.org (bilbo.ozlabs.org [IPv6:2401:3900:2:1::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 46DlGf5X9rzDrQ3 for ; Thu, 22 Aug 2019 23:08:58 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=ellerman.id.au Received: by ozlabs.org (Postfix, from userid 1034) id 46DlGd71RPz9sNk; Thu, 22 Aug 2019 23:08:57 +1000 (AEST) X-powerpc-patch-notification: thanks X-powerpc-patch-commit: a6717c01ddc259f6f73364779df058e2c67309f8 In-Reply-To: <20190802192926.19277-2-nathanl@linux.ibm.com> To: Nathan Lynch , linuxppc-dev@lists.ozlabs.org From: Michael Ellerman Subject: Re: [PATCH v2 1/3] powerpc/rtas: use device model APIs and serialization during LPM Message-Id: <46DlGd71RPz9sNk@ozlabs.org> Date: Thu, 22 Aug 2019 23:08:57 +1000 (AEST) X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Fri, 2019-08-02 at 19:29:24 UTC, Nathan Lynch wrote: > The LPAR migration implementation and userspace-initiated cpu hotplug > can interleave their executions like so: > > 1. Set cpu 7 offline via sysfs. > > 2. Begin a partition migration, whose implementation requires the OS > to ensure all present cpus are online; cpu 7 is onlined: > > rtas_ibm_suspend_me -> rtas_online_cpus_mask -> cpu_up > > This sets cpu 7 online in all respects except for the cpu's > corresponding struct device; dev->offline remains true. > > 3. Set cpu 7 online via sysfs. _cpu_up() determines that cpu 7 is > already online and returns success. The driver core (device_online) > sets dev->offline = false. > > 4. The migration completes and restores cpu 7 to offline state: > > rtas_ibm_suspend_me -> rtas_offline_cpus_mask -> cpu_down > > This leaves cpu7 in a state where the driver core considers the cpu > device online, but in all other respects it is offline and > unused. Attempts to online the cpu via sysfs appear to succeed but the > driver core actually does not pass the request to the lower-level > cpuhp support code. This makes the cpu unusable until the cpu device > is manually set offline and then online again via sysfs. > > Instead of directly calling cpu_up/cpu_down, the migration code should > use the higher-level device core APIs to maintain consistent state and > serialize operations. > > Fixes: 120496ac2d2d ("powerpc: Bring all threads online prior to migration/hibernation") > Signed-off-by: Nathan Lynch > Reviewed-by: Gautham R. Shenoy Series applied to powerpc next, thanks. https://git.kernel.org/powerpc/c/a6717c01ddc259f6f73364779df058e2c67309f8 cheers