From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19995C47080 for ; Mon, 24 May 2021 08:37:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F0E806113B for ; Mon, 24 May 2021 08:37:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232457AbhEXIi7 (ORCPT ); Mon, 24 May 2021 04:38:59 -0400 Received: from labrats.qualcomm.com ([199.106.110.90]:8834 "EHLO labrats.qualcomm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232453AbhEXIi5 (ORCPT ); Mon, 24 May 2021 04:38:57 -0400 IronPort-SDR: TWa3Fhi63S60EDRXcnbhqJyFy8dhPfEshDMx4boyqFmZBRfzPijqwHNzwyMuNGUT4oo2r85BJD S8oBicywI3al7o/1oT79E6vCtuHZFoVWcMHaAVY/ShRH/7XNmMsDbv6J/Lq2hATtwvbvJWvD5d dWKc9rgJBH5ILrCHq670hnIzytXgq32XZ3EOEBiLQbcsrPW7jMTYhaSR40Ry6RUKu2omrS2ZAm CqZPmUrD+1ParrHaXFzyRA1DC1JnHW7vCRhA+0AEWF66ECJTJfbFiajZd5F1n5noriahUoH5kn EJY= X-IronPort-AV: E=Sophos;i="5.82,319,1613462400"; d="scan'208";a="29772330" Received: from unknown (HELO ironmsg03-sd.qualcomm.com) ([10.53.140.143]) by labrats.qualcomm.com with ESMTP; 24 May 2021 01:37:29 -0700 X-QCInternal: smtphost Received: from stor-presley.qualcomm.com ([192.168.140.85]) by ironmsg03-sd.qualcomm.com with ESMTP; 24 May 2021 01:37:28 -0700 Received: by stor-presley.qualcomm.com (Postfix, from userid 359480) id 9404621AD7; Mon, 24 May 2021 01:37:28 -0700 (PDT) From: Can Guo To: asutoshd@codeaurora.org, nguyenb@codeaurora.org, hongwus@codeaurora.org, linux-scsi@vger.kernel.org, kernel-team@android.com, cang@codeaurora.org Cc: Stanley Chu , Alim Akhtar , Avri Altman , "James E.J. Bottomley" , "Martin K. Petersen" , Matthias Brugger , Bean Huo , Jaegeuk Kim , Adrian Hunter , Kiwoong Kim , Satya Tangirala , "Gustavo A. R. Silva" , Caleb Connolly , linux-kernel@vger.kernel.org (open list), linux-arm-kernel@lists.infradead.org (moderated list:ARM/Mediatek SoC support), linux-mediatek@lists.infradead.org (moderated list:ARM/Mediatek SoC support) Subject: [PATCH v1 3/3] scsi: ufs: Utilize Transfer Request List Completion Notification Register Date: Mon, 24 May 2021 01:36:58 -0700 Message-Id: <1621845419-14194-4-git-send-email-cang@codeaurora.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1621845419-14194-1-git-send-email-cang@codeaurora.org> References: <1621845419-14194-1-git-send-email-cang@codeaurora.org> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org By reading the UTP Transfer Request List Completion Notification Register, which is added in UFSHCI Ver 3.0, SW can easily get the compeleted transfer requests. Thus, SW can get rid of host lock, which is used to synchronize the tr_doorbell and outstanding_reqs, on transfer requests dispatch and completion paths. This can further benefit random read/write performance. Cc: Stanley Chu Co-developed-by: Asutosh Das Signed-off-by: Asutosh Das Signed-off-by: Can Guo --- drivers/scsi/ufs/ufshcd.c | 52 +++++++++++++++++++++++++++++++++-------------- drivers/scsi/ufs/ufshcd.h | 5 +++++ drivers/scsi/ufs/ufshci.h | 1 + 3 files changed, 43 insertions(+), 15 deletions(-) diff --git a/drivers/scsi/ufs/ufshcd.c b/drivers/scsi/ufs/ufshcd.c index b9b5e61..2b7ad26 100644 --- a/drivers/scsi/ufs/ufshcd.c +++ b/drivers/scsi/ufs/ufshcd.c @@ -2106,7 +2106,6 @@ static inline void ufshcd_send_command(struct ufs_hba *hba, unsigned int task_tag) { struct ufshcd_lrb *lrbp = &hba->lrb[task_tag]; - unsigned long flags; lrbp->issue_time_stamp = ktime_get(); lrbp->compl_time_stamp = ktime_set(0, 0); @@ -2115,10 +2114,19 @@ void ufshcd_send_command(struct ufs_hba *hba, unsigned int task_tag) ufshcd_clk_scaling_start_busy(hba); if (unlikely(ufshcd_should_inform_monitor(hba, lrbp))) ufshcd_start_monitor(hba, lrbp); - spin_lock_irqsave(hba->host->host_lock, flags); - set_bit(task_tag, &hba->outstanding_reqs); - ufshcd_writel(hba, 1 << task_tag, REG_UTP_TRANSFER_REQ_DOOR_BELL); - spin_unlock_irqrestore(hba->host->host_lock, flags); + if (ufshcd_has_utrlcnr(hba)) { + set_bit(task_tag, &hba->outstanding_reqs); + ufshcd_writel(hba, 1 << task_tag, + REG_UTP_TRANSFER_REQ_DOOR_BELL); + } else { + unsigned long flags; + + spin_lock_irqsave(hba->host->host_lock, flags); + set_bit(task_tag, &hba->outstanding_reqs); + ufshcd_writel(hba, 1 << task_tag, + REG_UTP_TRANSFER_REQ_DOOR_BELL); + spin_unlock_irqrestore(hba->host->host_lock, flags); + } /* Make sure that doorbell is committed immediately */ wmb(); } @@ -5260,17 +5268,17 @@ static void __ufshcd_transfer_req_compl(struct ufs_hba *hba, } /** - * ufshcd_transfer_req_compl - handle SCSI and query command completion + * ufshcd_trc_handler - handle transfer requests completion * @hba: per adapter instance + * @use_utrlcnr: get completed requests from UTRLCNR * * Returns * IRQ_HANDLED - If interrupt is valid * IRQ_NONE - If invalid interrupt */ -static irqreturn_t ufshcd_transfer_req_compl(struct ufs_hba *hba) +static irqreturn_t ufshcd_trc_handler(struct ufs_hba *hba, bool use_utrlcnr) { - unsigned long completed_reqs, flags; - u32 tr_doorbell; + unsigned long completed_reqs = 0; /* Resetting interrupt aggregation counters first and reading the * DOOR_BELL afterward allows us to handle all the completed requests. @@ -5283,10 +5291,24 @@ static irqreturn_t ufshcd_transfer_req_compl(struct ufs_hba *hba) !(hba->quirks & UFSHCI_QUIRK_SKIP_RESET_INTR_AGGR)) ufshcd_reset_intr_aggr(hba); - spin_lock_irqsave(hba->host->host_lock, flags); - tr_doorbell = ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_DOOR_BELL); - completed_reqs = tr_doorbell ^ hba->outstanding_reqs; - spin_unlock_irqrestore(hba->host->host_lock, flags); + if (use_utrlcnr) { + u32 utrlcnr; + + utrlcnr = ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_LIST_COMPL); + if (utrlcnr) { + ufshcd_writel(hba, utrlcnr, + REG_UTP_TRANSFER_REQ_LIST_COMPL); + completed_reqs = utrlcnr; + } + } else { + unsigned long flags; + u32 tr_doorbell; + + spin_lock_irqsave(hba->host->host_lock, flags); + tr_doorbell = ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_DOOR_BELL); + completed_reqs = tr_doorbell ^ hba->outstanding_reqs; + spin_unlock_irqrestore(hba->host->host_lock, flags); + } if (completed_reqs) { __ufshcd_transfer_req_compl(hba, completed_reqs); @@ -5768,7 +5790,7 @@ static void ufshcd_exception_event_handler(struct work_struct *work) /* Complete requests that have door-bell cleared */ static void ufshcd_complete_requests(struct ufs_hba *hba) { - ufshcd_transfer_req_compl(hba); + ufshcd_trc_handler(hba, false); ufshcd_tmc_handler(hba); } @@ -6409,7 +6431,7 @@ static irqreturn_t ufshcd_sl_intr(struct ufs_hba *hba, u32 intr_status) retval |= ufshcd_tmc_handler(hba); if (intr_status & UTP_TRANSFER_REQ_COMPL) - retval |= ufshcd_transfer_req_compl(hba); + retval |= ufshcd_trc_handler(hba, ufshcd_has_utrlcnr(hba)); return retval; } diff --git a/drivers/scsi/ufs/ufshcd.h b/drivers/scsi/ufs/ufshcd.h index a70daf7..d5325e8 100644 --- a/drivers/scsi/ufs/ufshcd.h +++ b/drivers/scsi/ufs/ufshcd.h @@ -1159,6 +1159,11 @@ static inline u32 ufshcd_vops_get_ufs_hci_version(struct ufs_hba *hba) return ufshcd_readl(hba, REG_UFS_VERSION); } +static inline bool ufshcd_has_utrlcnr(struct ufs_hba *hba) +{ + return (hba->ufs_version >= ufshci_version(3, 0)); +} + static inline int ufshcd_vops_clk_scale_notify(struct ufs_hba *hba, bool up, enum ufs_notify_change_status status) { diff --git a/drivers/scsi/ufs/ufshci.h b/drivers/scsi/ufs/ufshci.h index de95be5..5affb1f 100644 --- a/drivers/scsi/ufs/ufshci.h +++ b/drivers/scsi/ufs/ufshci.h @@ -39,6 +39,7 @@ enum { REG_UTP_TRANSFER_REQ_DOOR_BELL = 0x58, REG_UTP_TRANSFER_REQ_LIST_CLEAR = 0x5C, REG_UTP_TRANSFER_REQ_LIST_RUN_STOP = 0x60, + REG_UTP_TRANSFER_REQ_LIST_COMPL = 0x64, REG_UTP_TASK_REQ_LIST_BASE_L = 0x70, REG_UTP_TASK_REQ_LIST_BASE_H = 0x74, REG_UTP_TASK_REQ_DOOR_BELL = 0x78, -- Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FDB3C2B9F7 for ; Mon, 24 May 2021 23:34:24 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F3BDB61209 for ; Mon, 24 May 2021 23:34:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F3BDB61209 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-mediatek-bounces+linux-mediatek=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=D7v6361X2IAk+4v2ILP8XjOupVznTeTZL1Te1urIRqk=; b=RjLSfW6l25CvCk fV8DsS2hbMFoRGzh1ZOGZAPyyBAXHxIxkC1Pr6yr/IDalC6WM/x6EE8WLhhP8fxE+t7VhQZATBch+ tJONpMbdwU1cA4GrPwLYaOZdIa2xs8nySEq9cD6pPsfTEYe1z0a9UU07ywRuzgkE2/bIu8X/zFzZl YHlbiQyU5UwGBrkdlyAfWEDMIIVlgWFHy3rMmPkYk1nxlOKk6YmdvU4iUWd4oNgh0iEQPisuxeS3+ PkmH1q3ziDbjI4HBfA9YE+jgtQR2ioByBbCjwvTDJ0O0WQ6GCWGcU6SyXFPSg1Fh9IJ9hHnpChsX/ Rby0hxjQo8G6mevAY7GA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1llK5O-002MMz-1Q; Mon, 24 May 2021 23:34:14 +0000 Received: from labrats.qualcomm.com ([199.106.110.90]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ll65Z-000m2m-71; Mon, 24 May 2021 08:37:30 +0000 IronPort-SDR: TWa3Fhi63S60EDRXcnbhqJyFy8dhPfEshDMx4boyqFmZBRfzPijqwHNzwyMuNGUT4oo2r85BJD S8oBicywI3al7o/1oT79E6vCtuHZFoVWcMHaAVY/ShRH/7XNmMsDbv6J/Lq2hATtwvbvJWvD5d dWKc9rgJBH5ILrCHq670hnIzytXgq32XZ3EOEBiLQbcsrPW7jMTYhaSR40Ry6RUKu2omrS2ZAm CqZPmUrD+1ParrHaXFzyRA1DC1JnHW7vCRhA+0AEWF66ECJTJfbFiajZd5F1n5noriahUoH5kn EJY= X-IronPort-AV: E=Sophos;i="5.82,319,1613462400"; d="scan'208";a="29772330" Received: from unknown (HELO ironmsg03-sd.qualcomm.com) ([10.53.140.143]) by labrats.qualcomm.com with ESMTP; 24 May 2021 01:37:29 -0700 X-QCInternal: smtphost Received: from stor-presley.qualcomm.com ([192.168.140.85]) by ironmsg03-sd.qualcomm.com with ESMTP; 24 May 2021 01:37:28 -0700 Received: by stor-presley.qualcomm.com (Postfix, from userid 359480) id 9404621AD7; Mon, 24 May 2021 01:37:28 -0700 (PDT) From: Can Guo To: asutoshd@codeaurora.org, nguyenb@codeaurora.org, hongwus@codeaurora.org, linux-scsi@vger.kernel.org, kernel-team@android.com, cang@codeaurora.org Cc: Stanley Chu , Alim Akhtar , Avri Altman , "James E.J. Bottomley" , "Martin K. Petersen" , Matthias Brugger , Bean Huo , Jaegeuk Kim , Adrian Hunter , Kiwoong Kim , Satya Tangirala , "Gustavo A. R. Silva" , Caleb Connolly , linux-kernel@vger.kernel.org (open list), linux-arm-kernel@lists.infradead.org (moderated list:ARM/Mediatek SoC support), linux-mediatek@lists.infradead.org (moderated list:ARM/Mediatek SoC support) Subject: [PATCH v1 3/3] scsi: ufs: Utilize Transfer Request List Completion Notification Register Date: Mon, 24 May 2021 01:36:58 -0700 Message-Id: <1621845419-14194-4-git-send-email-cang@codeaurora.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1621845419-14194-1-git-send-email-cang@codeaurora.org> References: <1621845419-14194-1-git-send-email-cang@codeaurora.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210524_013729_301579_F1C199BF X-CRM114-Status: GOOD ( 16.78 ) X-BeenThere: linux-mediatek@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "Linux-mediatek" Errors-To: linux-mediatek-bounces+linux-mediatek=archiver.kernel.org@lists.infradead.org By reading the UTP Transfer Request List Completion Notification Register, which is added in UFSHCI Ver 3.0, SW can easily get the compeleted transfer requests. Thus, SW can get rid of host lock, which is used to synchronize the tr_doorbell and outstanding_reqs, on transfer requests dispatch and completion paths. This can further benefit random read/write performance. Cc: Stanley Chu Co-developed-by: Asutosh Das Signed-off-by: Asutosh Das Signed-off-by: Can Guo --- drivers/scsi/ufs/ufshcd.c | 52 +++++++++++++++++++++++++++++++++-------------- drivers/scsi/ufs/ufshcd.h | 5 +++++ drivers/scsi/ufs/ufshci.h | 1 + 3 files changed, 43 insertions(+), 15 deletions(-) diff --git a/drivers/scsi/ufs/ufshcd.c b/drivers/scsi/ufs/ufshcd.c index b9b5e61..2b7ad26 100644 --- a/drivers/scsi/ufs/ufshcd.c +++ b/drivers/scsi/ufs/ufshcd.c @@ -2106,7 +2106,6 @@ static inline void ufshcd_send_command(struct ufs_hba *hba, unsigned int task_tag) { struct ufshcd_lrb *lrbp = &hba->lrb[task_tag]; - unsigned long flags; lrbp->issue_time_stamp = ktime_get(); lrbp->compl_time_stamp = ktime_set(0, 0); @@ -2115,10 +2114,19 @@ void ufshcd_send_command(struct ufs_hba *hba, unsigned int task_tag) ufshcd_clk_scaling_start_busy(hba); if (unlikely(ufshcd_should_inform_monitor(hba, lrbp))) ufshcd_start_monitor(hba, lrbp); - spin_lock_irqsave(hba->host->host_lock, flags); - set_bit(task_tag, &hba->outstanding_reqs); - ufshcd_writel(hba, 1 << task_tag, REG_UTP_TRANSFER_REQ_DOOR_BELL); - spin_unlock_irqrestore(hba->host->host_lock, flags); + if (ufshcd_has_utrlcnr(hba)) { + set_bit(task_tag, &hba->outstanding_reqs); + ufshcd_writel(hba, 1 << task_tag, + REG_UTP_TRANSFER_REQ_DOOR_BELL); + } else { + unsigned long flags; + + spin_lock_irqsave(hba->host->host_lock, flags); + set_bit(task_tag, &hba->outstanding_reqs); + ufshcd_writel(hba, 1 << task_tag, + REG_UTP_TRANSFER_REQ_DOOR_BELL); + spin_unlock_irqrestore(hba->host->host_lock, flags); + } /* Make sure that doorbell is committed immediately */ wmb(); } @@ -5260,17 +5268,17 @@ static void __ufshcd_transfer_req_compl(struct ufs_hba *hba, } /** - * ufshcd_transfer_req_compl - handle SCSI and query command completion + * ufshcd_trc_handler - handle transfer requests completion * @hba: per adapter instance + * @use_utrlcnr: get completed requests from UTRLCNR * * Returns * IRQ_HANDLED - If interrupt is valid * IRQ_NONE - If invalid interrupt */ -static irqreturn_t ufshcd_transfer_req_compl(struct ufs_hba *hba) +static irqreturn_t ufshcd_trc_handler(struct ufs_hba *hba, bool use_utrlcnr) { - unsigned long completed_reqs, flags; - u32 tr_doorbell; + unsigned long completed_reqs = 0; /* Resetting interrupt aggregation counters first and reading the * DOOR_BELL afterward allows us to handle all the completed requests. @@ -5283,10 +5291,24 @@ static irqreturn_t ufshcd_transfer_req_compl(struct ufs_hba *hba) !(hba->quirks & UFSHCI_QUIRK_SKIP_RESET_INTR_AGGR)) ufshcd_reset_intr_aggr(hba); - spin_lock_irqsave(hba->host->host_lock, flags); - tr_doorbell = ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_DOOR_BELL); - completed_reqs = tr_doorbell ^ hba->outstanding_reqs; - spin_unlock_irqrestore(hba->host->host_lock, flags); + if (use_utrlcnr) { + u32 utrlcnr; + + utrlcnr = ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_LIST_COMPL); + if (utrlcnr) { + ufshcd_writel(hba, utrlcnr, + REG_UTP_TRANSFER_REQ_LIST_COMPL); + completed_reqs = utrlcnr; + } + } else { + unsigned long flags; + u32 tr_doorbell; + + spin_lock_irqsave(hba->host->host_lock, flags); + tr_doorbell = ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_DOOR_BELL); + completed_reqs = tr_doorbell ^ hba->outstanding_reqs; + spin_unlock_irqrestore(hba->host->host_lock, flags); + } if (completed_reqs) { __ufshcd_transfer_req_compl(hba, completed_reqs); @@ -5768,7 +5790,7 @@ static void ufshcd_exception_event_handler(struct work_struct *work) /* Complete requests that have door-bell cleared */ static void ufshcd_complete_requests(struct ufs_hba *hba) { - ufshcd_transfer_req_compl(hba); + ufshcd_trc_handler(hba, false); ufshcd_tmc_handler(hba); } @@ -6409,7 +6431,7 @@ static irqreturn_t ufshcd_sl_intr(struct ufs_hba *hba, u32 intr_status) retval |= ufshcd_tmc_handler(hba); if (intr_status & UTP_TRANSFER_REQ_COMPL) - retval |= ufshcd_transfer_req_compl(hba); + retval |= ufshcd_trc_handler(hba, ufshcd_has_utrlcnr(hba)); return retval; } diff --git a/drivers/scsi/ufs/ufshcd.h b/drivers/scsi/ufs/ufshcd.h index a70daf7..d5325e8 100644 --- a/drivers/scsi/ufs/ufshcd.h +++ b/drivers/scsi/ufs/ufshcd.h @@ -1159,6 +1159,11 @@ static inline u32 ufshcd_vops_get_ufs_hci_version(struct ufs_hba *hba) return ufshcd_readl(hba, REG_UFS_VERSION); } +static inline bool ufshcd_has_utrlcnr(struct ufs_hba *hba) +{ + return (hba->ufs_version >= ufshci_version(3, 0)); +} + static inline int ufshcd_vops_clk_scale_notify(struct ufs_hba *hba, bool up, enum ufs_notify_change_status status) { diff --git a/drivers/scsi/ufs/ufshci.h b/drivers/scsi/ufs/ufshci.h index de95be5..5affb1f 100644 --- a/drivers/scsi/ufs/ufshci.h +++ b/drivers/scsi/ufs/ufshci.h @@ -39,6 +39,7 @@ enum { REG_UTP_TRANSFER_REQ_DOOR_BELL = 0x58, REG_UTP_TRANSFER_REQ_LIST_CLEAR = 0x5C, REG_UTP_TRANSFER_REQ_LIST_RUN_STOP = 0x60, + REG_UTP_TRANSFER_REQ_LIST_COMPL = 0x64, REG_UTP_TASK_REQ_LIST_BASE_L = 0x70, REG_UTP_TASK_REQ_LIST_BASE_H = 0x74, REG_UTP_TASK_REQ_DOOR_BELL = 0x78, -- Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project. _______________________________________________ Linux-mediatek mailing list Linux-mediatek@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-mediatek From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BA6EC2B9F7 for ; Mon, 24 May 2021 23:35:10 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 68DE56140B for ; Mon, 24 May 2021 23:35:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 68DE56140B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=y/GZBcxDxNDuDjv4tk7qsceKCbqyuaLLhnAShymE7Qk=; b=ypEAmYG8xW2EWn ILpjYssoa8Xn7Ai1usVWR2Qat+hAR+72PnpxSrb8oVXTFLFcdDIp3BTJF8eqyHUvhekPG7AFh1bOm jt1I6RqE6w0/uNcgd63yl8g0LDP/h7aeii0+YtmaYJgmK1pdH85zJhy35Ea/zTk2czU3e2xEeFMQf vpLNje8MFbtWkNHsTzuIsK9sbjtiWNc6nPUHMqcQBdKZDWY0RcjEqnIbd4EMiltdRJ98/URr+6uh/ SpPG9zeS45c8b5qpR16n9LT5OKd3yd3ct49165h7WfLNb718OAIoGv2JccEWeIbR2jwigD0eLciUz 2nWSAd+WouS5sr9CyI7g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1llK49-002M9k-Mq; Mon, 24 May 2021 23:32:58 +0000 Received: from labrats.qualcomm.com ([199.106.110.90]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ll65Z-000m2m-71; Mon, 24 May 2021 08:37:30 +0000 IronPort-SDR: TWa3Fhi63S60EDRXcnbhqJyFy8dhPfEshDMx4boyqFmZBRfzPijqwHNzwyMuNGUT4oo2r85BJD S8oBicywI3al7o/1oT79E6vCtuHZFoVWcMHaAVY/ShRH/7XNmMsDbv6J/Lq2hATtwvbvJWvD5d dWKc9rgJBH5ILrCHq670hnIzytXgq32XZ3EOEBiLQbcsrPW7jMTYhaSR40Ry6RUKu2omrS2ZAm CqZPmUrD+1ParrHaXFzyRA1DC1JnHW7vCRhA+0AEWF66ECJTJfbFiajZd5F1n5noriahUoH5kn EJY= X-IronPort-AV: E=Sophos;i="5.82,319,1613462400"; d="scan'208";a="29772330" Received: from unknown (HELO ironmsg03-sd.qualcomm.com) ([10.53.140.143]) by labrats.qualcomm.com with ESMTP; 24 May 2021 01:37:29 -0700 X-QCInternal: smtphost Received: from stor-presley.qualcomm.com ([192.168.140.85]) by ironmsg03-sd.qualcomm.com with ESMTP; 24 May 2021 01:37:28 -0700 Received: by stor-presley.qualcomm.com (Postfix, from userid 359480) id 9404621AD7; Mon, 24 May 2021 01:37:28 -0700 (PDT) From: Can Guo To: asutoshd@codeaurora.org, nguyenb@codeaurora.org, hongwus@codeaurora.org, linux-scsi@vger.kernel.org, kernel-team@android.com, cang@codeaurora.org Cc: Stanley Chu , Alim Akhtar , Avri Altman , "James E.J. Bottomley" , "Martin K. Petersen" , Matthias Brugger , Bean Huo , Jaegeuk Kim , Adrian Hunter , Kiwoong Kim , Satya Tangirala , "Gustavo A. R. Silva" , Caleb Connolly , linux-kernel@vger.kernel.org (open list), linux-arm-kernel@lists.infradead.org (moderated list:ARM/Mediatek SoC support), linux-mediatek@lists.infradead.org (moderated list:ARM/Mediatek SoC support) Subject: [PATCH v1 3/3] scsi: ufs: Utilize Transfer Request List Completion Notification Register Date: Mon, 24 May 2021 01:36:58 -0700 Message-Id: <1621845419-14194-4-git-send-email-cang@codeaurora.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1621845419-14194-1-git-send-email-cang@codeaurora.org> References: <1621845419-14194-1-git-send-email-cang@codeaurora.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210524_013729_301579_F1C199BF X-CRM114-Status: GOOD ( 16.78 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org By reading the UTP Transfer Request List Completion Notification Register, which is added in UFSHCI Ver 3.0, SW can easily get the compeleted transfer requests. Thus, SW can get rid of host lock, which is used to synchronize the tr_doorbell and outstanding_reqs, on transfer requests dispatch and completion paths. This can further benefit random read/write performance. Cc: Stanley Chu Co-developed-by: Asutosh Das Signed-off-by: Asutosh Das Signed-off-by: Can Guo --- drivers/scsi/ufs/ufshcd.c | 52 +++++++++++++++++++++++++++++++++-------------- drivers/scsi/ufs/ufshcd.h | 5 +++++ drivers/scsi/ufs/ufshci.h | 1 + 3 files changed, 43 insertions(+), 15 deletions(-) diff --git a/drivers/scsi/ufs/ufshcd.c b/drivers/scsi/ufs/ufshcd.c index b9b5e61..2b7ad26 100644 --- a/drivers/scsi/ufs/ufshcd.c +++ b/drivers/scsi/ufs/ufshcd.c @@ -2106,7 +2106,6 @@ static inline void ufshcd_send_command(struct ufs_hba *hba, unsigned int task_tag) { struct ufshcd_lrb *lrbp = &hba->lrb[task_tag]; - unsigned long flags; lrbp->issue_time_stamp = ktime_get(); lrbp->compl_time_stamp = ktime_set(0, 0); @@ -2115,10 +2114,19 @@ void ufshcd_send_command(struct ufs_hba *hba, unsigned int task_tag) ufshcd_clk_scaling_start_busy(hba); if (unlikely(ufshcd_should_inform_monitor(hba, lrbp))) ufshcd_start_monitor(hba, lrbp); - spin_lock_irqsave(hba->host->host_lock, flags); - set_bit(task_tag, &hba->outstanding_reqs); - ufshcd_writel(hba, 1 << task_tag, REG_UTP_TRANSFER_REQ_DOOR_BELL); - spin_unlock_irqrestore(hba->host->host_lock, flags); + if (ufshcd_has_utrlcnr(hba)) { + set_bit(task_tag, &hba->outstanding_reqs); + ufshcd_writel(hba, 1 << task_tag, + REG_UTP_TRANSFER_REQ_DOOR_BELL); + } else { + unsigned long flags; + + spin_lock_irqsave(hba->host->host_lock, flags); + set_bit(task_tag, &hba->outstanding_reqs); + ufshcd_writel(hba, 1 << task_tag, + REG_UTP_TRANSFER_REQ_DOOR_BELL); + spin_unlock_irqrestore(hba->host->host_lock, flags); + } /* Make sure that doorbell is committed immediately */ wmb(); } @@ -5260,17 +5268,17 @@ static void __ufshcd_transfer_req_compl(struct ufs_hba *hba, } /** - * ufshcd_transfer_req_compl - handle SCSI and query command completion + * ufshcd_trc_handler - handle transfer requests completion * @hba: per adapter instance + * @use_utrlcnr: get completed requests from UTRLCNR * * Returns * IRQ_HANDLED - If interrupt is valid * IRQ_NONE - If invalid interrupt */ -static irqreturn_t ufshcd_transfer_req_compl(struct ufs_hba *hba) +static irqreturn_t ufshcd_trc_handler(struct ufs_hba *hba, bool use_utrlcnr) { - unsigned long completed_reqs, flags; - u32 tr_doorbell; + unsigned long completed_reqs = 0; /* Resetting interrupt aggregation counters first and reading the * DOOR_BELL afterward allows us to handle all the completed requests. @@ -5283,10 +5291,24 @@ static irqreturn_t ufshcd_transfer_req_compl(struct ufs_hba *hba) !(hba->quirks & UFSHCI_QUIRK_SKIP_RESET_INTR_AGGR)) ufshcd_reset_intr_aggr(hba); - spin_lock_irqsave(hba->host->host_lock, flags); - tr_doorbell = ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_DOOR_BELL); - completed_reqs = tr_doorbell ^ hba->outstanding_reqs; - spin_unlock_irqrestore(hba->host->host_lock, flags); + if (use_utrlcnr) { + u32 utrlcnr; + + utrlcnr = ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_LIST_COMPL); + if (utrlcnr) { + ufshcd_writel(hba, utrlcnr, + REG_UTP_TRANSFER_REQ_LIST_COMPL); + completed_reqs = utrlcnr; + } + } else { + unsigned long flags; + u32 tr_doorbell; + + spin_lock_irqsave(hba->host->host_lock, flags); + tr_doorbell = ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_DOOR_BELL); + completed_reqs = tr_doorbell ^ hba->outstanding_reqs; + spin_unlock_irqrestore(hba->host->host_lock, flags); + } if (completed_reqs) { __ufshcd_transfer_req_compl(hba, completed_reqs); @@ -5768,7 +5790,7 @@ static void ufshcd_exception_event_handler(struct work_struct *work) /* Complete requests that have door-bell cleared */ static void ufshcd_complete_requests(struct ufs_hba *hba) { - ufshcd_transfer_req_compl(hba); + ufshcd_trc_handler(hba, false); ufshcd_tmc_handler(hba); } @@ -6409,7 +6431,7 @@ static irqreturn_t ufshcd_sl_intr(struct ufs_hba *hba, u32 intr_status) retval |= ufshcd_tmc_handler(hba); if (intr_status & UTP_TRANSFER_REQ_COMPL) - retval |= ufshcd_transfer_req_compl(hba); + retval |= ufshcd_trc_handler(hba, ufshcd_has_utrlcnr(hba)); return retval; } diff --git a/drivers/scsi/ufs/ufshcd.h b/drivers/scsi/ufs/ufshcd.h index a70daf7..d5325e8 100644 --- a/drivers/scsi/ufs/ufshcd.h +++ b/drivers/scsi/ufs/ufshcd.h @@ -1159,6 +1159,11 @@ static inline u32 ufshcd_vops_get_ufs_hci_version(struct ufs_hba *hba) return ufshcd_readl(hba, REG_UFS_VERSION); } +static inline bool ufshcd_has_utrlcnr(struct ufs_hba *hba) +{ + return (hba->ufs_version >= ufshci_version(3, 0)); +} + static inline int ufshcd_vops_clk_scale_notify(struct ufs_hba *hba, bool up, enum ufs_notify_change_status status) { diff --git a/drivers/scsi/ufs/ufshci.h b/drivers/scsi/ufs/ufshci.h index de95be5..5affb1f 100644 --- a/drivers/scsi/ufs/ufshci.h +++ b/drivers/scsi/ufs/ufshci.h @@ -39,6 +39,7 @@ enum { REG_UTP_TRANSFER_REQ_DOOR_BELL = 0x58, REG_UTP_TRANSFER_REQ_LIST_CLEAR = 0x5C, REG_UTP_TRANSFER_REQ_LIST_RUN_STOP = 0x60, + REG_UTP_TRANSFER_REQ_LIST_COMPL = 0x64, REG_UTP_TASK_REQ_LIST_BASE_L = 0x70, REG_UTP_TASK_REQ_LIST_BASE_H = 0x74, REG_UTP_TASK_REQ_DOOR_BELL = 0x78, -- Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel