From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3ABDDC433E0 for ; Tue, 21 Jul 2020 14:38:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 12DBC20714 for ; Tue, 21 Jul 2020 14:38:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=armlinux.org.uk header.i=@armlinux.org.uk header.b="oE2ciNzk" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726710AbgGUOiL (ORCPT ); Tue, 21 Jul 2020 10:38:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55672 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726412AbgGUOiK (ORCPT ); Tue, 21 Jul 2020 10:38:10 -0400 Received: from pandora.armlinux.org.uk (pandora.armlinux.org.uk [IPv6:2001:4d48:ad52:32c8:5054:ff:fe00:142]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 75E67C061794 for ; Tue, 21 Jul 2020 07:38:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=armlinux.org.uk; s=pandora-2019; h=Sender:Content-Type:MIME-Version: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=ti0HlU+Pgy/l5dQA5EeqxtMrHYv6NwKUn9x+3xhLNuE=; b=oE2ciNzkFCECT9IslsqB3+0GG iFDIwDkxV2UqKJAkouVvNXtsNWnrEELll5pirPUJ1Hb5vbonkT2qRDswhN7VkuBmkcvhAfW7eagWi F7dCOQeXM4znkcqBTyf/v4swq1Xh4YkJi2V51umdHEzIL/p735d+PZT58Ret4TH0ug3QcqennYFH5 s3ArmdmJTSXpaJjl/1u/qZxmS8O3umvj4NtkVCwppY6W6jngh1msVE0ScErqPCZ09iidyVsiDEMle pe3C4w0ZoWOlsLcSKWPOC4SzBcDbqHrY9Z+M0WUjtjTDFKsSykvfWcgvYbb42gVkPEkxRyJPAyqbe JlXv2a0Gg==; Received: from shell.armlinux.org.uk ([fd8f:7570:feb6:1:5054:ff:fe00:4ec]:42350) by pandora.armlinux.org.uk with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jxtP3-0004QT-Pn; Tue, 21 Jul 2020 15:37:57 +0100 Received: from linux by shell.armlinux.org.uk with local (Exim 4.92) (envelope-from ) id 1jxtP2-000641-Re; Tue, 21 Jul 2020 15:37:56 +0100 Date: Tue, 21 Jul 2020 15:37:56 +0100 From: Russell King - ARM Linux admin To: Andrew Lunn , Gregory Clement , Jason Cooper , Kishon Vijay Abraham I , Rob Herring , Sebastian Hesselbarth , Vinod Koul Cc: devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH v2 0/3] Fix Armada 38x mvneta lockups when switching speeds Message-ID: <20200721143756.GT1605@shell.armlinux.org.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.10.1 (2018-07-13) Sender: devicetree-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Hi, While testing phylink over the weekend, I found it was possible to cause the mvneta hardware to lockup in various weird and wonderful ways by switching the interface speed between 1G and 2.5G repeatedly. It didn't require a rapid switching, but one switch every few seconds. Symptoms included one or more of: - Timeout while trying to stop transmit (seen once) - 2500BASE-X link negotiation failure (fails to exchange link word.) - Detects lack of sync, but fails to flag 10ms of sync failure. - SyncOk bit randomly toggles. Once the hardware gets into a "bad" state, trying to recover it by using the mvneta GMAC port reset fails to resolve the issue. Disabling the port also fails to recover it. The only way to recover seemed to be via a reboot. Many solutions to solve this were tried in various combinations - while changing the COMPHY configuration: - putting the GMAC into reset - disabling the GMAC port - augmenting the COMPHY configuration to try to "cleanly" disable the COMPHY via phy_power_down() and reconfigure it via phy_power_up(), including resetting parts of the COMPHY and re-running the RX initialisation. None of that worked. It was then discovered from the u-boot sources that there is an undocumented register that has a lane-specific bit set at the end of COMPHY initialisation, once the loosely documented COMPHY setup has completed. Experimentation with that showed that if the lane specific bit is cleared before changing the COMPHY "GEN" configuration, and set afterwards, mvneta no longer locks up. Unfortunately, this undocumented register is not part of the COMPHY register set that we map - it is located in a region of "System Registers" which are shared between multiple different devices. Who should be responsible for mapping this register (mvneta or COMPHY) was considered; the register is only present on Armada 38x systems, and seemingly not on Armada 37x or Armada 37xx systems. It seems that it is a system-level register. The COMPHYs seem to be system specific, so let's make it part of the COMPHY. With no real information on this register, all we can do is guess about it's function and how to fit it into the system. .../bindings/phy/phy-armada38x-comphy.txt | 10 ++++- arch/arm/boot/dts/armada-38x.dtsi | 3 +- drivers/phy/marvell/phy-armada38x-comphy.c | 45 ++++++++++++++++++---- 3 files changed, 49 insertions(+), 9 deletions(-) -- RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last! From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B58FAC433DF for ; Tue, 21 Jul 2020 14:39:33 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 81E7D2065D for ; Tue, 21 Jul 2020 14:39:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="Cee2Lye9"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=armlinux.org.uk header.i=@armlinux.org.uk header.b="oE2ciNzk" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 81E7D2065D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=armlinux.org.uk Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:Message-ID:Subject:To:From:Date: Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender :Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=ZHhbQfw3JfdixBuqSYdGGhCicpHBOuBbGw975JbUM7Y=; b=Cee2Lye98G6I11w8VZa8GV7cwo /9HkAAODfwlMVO3rRN2B7980iZV4mWamAEW2Ww4bYNM3FVEKoDkBgPoWcWwDnpYX8d2dOrQ+vlsfL u4iWXths+m4pEvV2xsYj1g3Jr2JJunBtusXrj8KuOXBfAEU5t52M0apoIipjxs0JPZ7fgJ3hgBpiP ymzAr4ka45pQ/56ZalOqxi8HwQa+D/v+fVqlg7r5YNqf598nk7xm5glBLuVWnKoUagyY425CwSuNu /KBSF1kTKUxfOD3Kz07eL561hONhuqb7V2Onw2kXlpPmwT+gokYXlgaNTgKk2SZyziLQHxE3zQ0JP voDavkrQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jxtPD-0002ro-A6; Tue, 21 Jul 2020 14:38:07 +0000 Received: from pandora.armlinux.org.uk ([2001:4d48:ad52:32c8:5054:ff:fe00:142]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jxtPA-0002rO-Qv for linux-arm-kernel@lists.infradead.org; Tue, 21 Jul 2020 14:38:05 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=armlinux.org.uk; s=pandora-2019; h=Sender:Content-Type:MIME-Version: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=ti0HlU+Pgy/l5dQA5EeqxtMrHYv6NwKUn9x+3xhLNuE=; b=oE2ciNzkFCECT9IslsqB3+0GG iFDIwDkxV2UqKJAkouVvNXtsNWnrEELll5pirPUJ1Hb5vbonkT2qRDswhN7VkuBmkcvhAfW7eagWi F7dCOQeXM4znkcqBTyf/v4swq1Xh4YkJi2V51umdHEzIL/p735d+PZT58Ret4TH0ug3QcqennYFH5 s3ArmdmJTSXpaJjl/1u/qZxmS8O3umvj4NtkVCwppY6W6jngh1msVE0ScErqPCZ09iidyVsiDEMle pe3C4w0ZoWOlsLcSKWPOC4SzBcDbqHrY9Z+M0WUjtjTDFKsSykvfWcgvYbb42gVkPEkxRyJPAyqbe JlXv2a0Gg==; Received: from shell.armlinux.org.uk ([fd8f:7570:feb6:1:5054:ff:fe00:4ec]:42350) by pandora.armlinux.org.uk with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jxtP3-0004QT-Pn; Tue, 21 Jul 2020 15:37:57 +0100 Received: from linux by shell.armlinux.org.uk with local (Exim 4.92) (envelope-from ) id 1jxtP2-000641-Re; Tue, 21 Jul 2020 15:37:56 +0100 Date: Tue, 21 Jul 2020 15:37:56 +0100 From: Russell King - ARM Linux admin To: Andrew Lunn , Gregory Clement , Jason Cooper , Kishon Vijay Abraham I , Rob Herring , Sebastian Hesselbarth , Vinod Koul Subject: [PATCH v2 0/3] Fix Armada 38x mvneta lockups when switching speeds Message-ID: <20200721143756.GT1605@shell.armlinux.org.uk> MIME-Version: 1.0 Content-Disposition: inline User-Agent: Mutt/1.10.1 (2018-07-13) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200721_103804_911112_D81E5B19 X-CRM114-Status: GOOD ( 14.02 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi, While testing phylink over the weekend, I found it was possible to cause the mvneta hardware to lockup in various weird and wonderful ways by switching the interface speed between 1G and 2.5G repeatedly. It didn't require a rapid switching, but one switch every few seconds. Symptoms included one or more of: - Timeout while trying to stop transmit (seen once) - 2500BASE-X link negotiation failure (fails to exchange link word.) - Detects lack of sync, but fails to flag 10ms of sync failure. - SyncOk bit randomly toggles. Once the hardware gets into a "bad" state, trying to recover it by using the mvneta GMAC port reset fails to resolve the issue. Disabling the port also fails to recover it. The only way to recover seemed to be via a reboot. Many solutions to solve this were tried in various combinations - while changing the COMPHY configuration: - putting the GMAC into reset - disabling the GMAC port - augmenting the COMPHY configuration to try to "cleanly" disable the COMPHY via phy_power_down() and reconfigure it via phy_power_up(), including resetting parts of the COMPHY and re-running the RX initialisation. None of that worked. It was then discovered from the u-boot sources that there is an undocumented register that has a lane-specific bit set at the end of COMPHY initialisation, once the loosely documented COMPHY setup has completed. Experimentation with that showed that if the lane specific bit is cleared before changing the COMPHY "GEN" configuration, and set afterwards, mvneta no longer locks up. Unfortunately, this undocumented register is not part of the COMPHY register set that we map - it is located in a region of "System Registers" which are shared between multiple different devices. Who should be responsible for mapping this register (mvneta or COMPHY) was considered; the register is only present on Armada 38x systems, and seemingly not on Armada 37x or Armada 37xx systems. It seems that it is a system-level register. The COMPHYs seem to be system specific, so let's make it part of the COMPHY. With no real information on this register, all we can do is guess about it's function and how to fit it into the system. .../bindings/phy/phy-armada38x-comphy.txt | 10 ++++- arch/arm/boot/dts/armada-38x.dtsi | 3 +- drivers/phy/marvell/phy-armada38x-comphy.c | 45 ++++++++++++++++++---- 3 files changed, 49 insertions(+), 9 deletions(-) -- RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last! _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel