From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753904AbcGSPpq (ORCPT ); Tue, 19 Jul 2016 11:45:46 -0400 Received: from mail-db5eur01on0102.outbound.protection.outlook.com ([104.47.2.102]:24391 "EHLO EUR01-DB5-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752366AbcGSPpn (ORCPT ); Tue, 19 Jul 2016 11:45:43 -0400 X-Greylist: delayed 27992 seconds by postgrey-1.27 at vger.kernel.org; Tue, 19 Jul 2016 11:45:42 EDT Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=bhuvanchandra.dv@toradex.com; From: Bhuvanchandra DV To: CC: , , , , , , , , , , Bhuvanchandra DV Subject: [PATCH RESEND v2 6/8] tty: serial: fsl_lpuart: Use scatter/gather DMA for Tx Date: Tue, 19 Jul 2016 13:13:08 +0530 Message-ID: <20160719074310.16872-7-bhuvanchandra.dv@toradex.com> X-Mailer: git-send-email 2.9.0 In-Reply-To: <20160719074310.16872-1-bhuvanchandra.dv@toradex.com> References: <20160719074310.16872-1-bhuvanchandra.dv@toradex.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [115.115.243.34] X-ClientProxiedBy: MA1PR01CA0045.INDPRD01.PROD.OUTLOOK.COM (10.164.116.145) To DB6PR0501MB2087.eurprd05.prod.outlook.com (10.168.9.139) X-MS-Office365-Filtering-Correlation-Id: 4597ab35-8f40-46bd-7aba-08d3afa892bf X-Microsoft-Exchange-Diagnostics: 1;DB6PR0501MB2087;2:3AnwPMGkRgJUxxe60xzhHLkqk1rncsxo9idFTwiyBuFJWsnIBEFAqwEW1xA3mwGk6NVcrdEuTuMoW0xviKEB/OY4lkCXUL8U4QywNqSjUbEG6598I6dyak74TPp5jKw0cCjsRo2BXgirPUHxWaT/s0pWPGtPqx3lK/63wmMPZDa21ep7OfCYzNc7+vu9oaz5;3:YEfxWtQlMOiX9Tqxac4RuUN15K9S7wGJazdzZ8eRAmmvX2/RQ5OA8YkASYN0HULhOxSaw+0ZVZ+T0J/nPWP6UUymvla5N5Qp1WVKI0jAfB0LNC7n+QhgKNiVV+x3MOpy X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:DB6PR0501MB2087; X-Microsoft-Exchange-Diagnostics: 1;DB6PR0501MB2087;25:3/wthp7X2UzkV9OA5r+lzfWTVNJ92JC6Q4Dr/rgCDEnUY/xYH73pp1ek11DLEmgym6K1dEYqI1oRR9JnvtPPHbFAeFZniEXoVALG0QNFRQvr7HbCYSxEttFQWlBkgSjG6dvpKEiBob9ia/wyknbZOYeFUS8sM3mkmTcP0DSuMygDXF1Q/E3TGWbMNt1AG3I34FKRJkb5avIdWn7r+2SBxbaTPYvzL25MAPkj1Bdo+xc2uIiG0PcpzDzi5bQBNa4yOeIHYh259XZx7y9CpW1Xmqhec9tCM6G6WbGT1+9reC2zUBng4WbGKzbpaWMyGunnywWPncTfIZ8XRwDzVURsIKbXy1Rp0NGLsZwbPi9aWwGBFZmJAlzjjDiQLYNGR1DYR4RF6qMY+/CP3bY4Ne69rvjnadqEwUvD3eIj9f5UdLAslEAchmUFiOnjlGBKYQmgh3iWHzqR0wh5fgWp3upb4pruSKAsH/EPloG2uFI3otgdx7unwhPGq4dYWTqVR/1uxejTGTfgHUENfSMH2rxpU/CKKXGKVcaOuANpRUDsROucekfVqv4cpbdCHO96DwjENfK03B4wImIP5S3KEb91tCWxYH6/p8W6qXIDhNuDprWSqadVrfOfBKgcSBgycYj/OoBIzSb6PkaZr1CzERgikFSHHjUimzSIirTjO0R6KFCNYmhZpGjvqsyE99t1xq803++7M3DkqKTSCrd3v2h5pw==;31:ZMjr6rG5KE9aoSXhPsH7p8XrAyF/pelco1XTv5+q0FmsBIeU38DU33yoq2AFQQKMCbOyOx4+myS9pRUXBNixLlixcqKCfTd3wnNbw0rjjGXqz9saKTintWIQZi+EZ09aKoAHAkwnTW7PosWM1BAosqqMEBBwzn04+YhsV4TNmlPCt3mqRtLWxjEwWG9z6lfwUVbC91Iei7sTUYlA7LHZpQ== X-Microsoft-Exchange-Diagnostics: 1;DB6PR0501MB2087;20:r+eeyLWIJCWjgFJja5TgSGZWTcTlZVmI9ql5508N2yXI4M3FtDBU33PfQXR9lltwobK1VqV6RgPAXm5ADEBqVMZjUvMa1Qy9CUTTvc6r+RUWl7UmvaPdYhlsJ5EXy8QwiQIBtZ037xKv5M3TEfxo9VAVWTH3H3wAUYjYfdW4App67Is6zJS4pGut/4D9mOO5o/Rw1AANdIXeORSJqWRsK2nu3Xhaj90SNllZIuWBLH87qRIdNXYjKowpekNtmmUZIe/BkxvePT3Oeihe3HVwaiH8xJrjObb/Rf0Srzd+zPUMX0ug3nN/XoQUNh2NzR6Ip1t0UYSW1H74j7vwZRU8zQ==;4:trMQgP7HNHbpdhubm9IAjk6egc+zs7+rq0PeqFvPT+t4JchbjXak4Q6FFvqM2VF6g7272AmgCy6/DSTmyFZtaR2MFuw6/r0uKbKzswE/rqVUO/3mFZxYyhqufsrNN09ThIV/SERcfqhmHb+eBONKOjAYUp1rGV3gKB6Mczr2FAy14NRih7WAvBFYc/DfqFvZImlg0kaHsB86+SkWKTRmqisEYF8AhnoaVdHVLZ/MvPRjUDCIUtsmSO2HrKdEjFCB7aREygugnpVU8D/43I2+Us4P/DPrSR84yl1NLWG0Fe5oT3AquH+8Uxve9MqIlo7Z1voWnRnwmg72Nu0rek46unBhAwFIvdLG5hxJvDsVBg9TFobx5yp0KSUI4U039Ff5 X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046);SRVR:DB6PR0501MB2087;BCL:0;PCL:0;RULEID:;SRVR:DB6PR0501MB2087; X-Forefront-PRVS: 000800954F X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6009001)(7916002)(199003)(189002)(22813001)(77096005)(36756003)(42186005)(69596002)(4001430100002)(19580395003)(86362001)(7736002)(19580405001)(47776003)(5003940100001)(2950100001)(92566002)(50466002)(66066001)(76176999)(50986999)(50226002)(4326007)(48376002)(6116002)(3846002)(1076002)(586003)(97736004)(305945005)(68736007)(189998001)(105586002)(8676002)(110136002)(107886002)(53416004)(101416001)(2906002)(2351001)(81156014)(81166006)(229853001)(106356001)(7846002)(33646002)(217873001);DIR:OUT;SFP:1102;SCL:1;SRVR:DB6PR0501MB2087;H:tdx-in-nb-0014.toradex.ext;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;DB6PR0501MB2087;23:0/h87TXEj8ViNH0DBploXfyzSjeI2MWRbaVTvpD?= =?us-ascii?Q?u2IraZblB6Ht1MvU9z4e+zPqRt5VLUq44cmDtWgvowr26dOLSp8rbzGymMVo?= =?us-ascii?Q?6uQNn+nEo+bGOa03rFlMRbnHw2WUHO9MVb4DP8Qi6aYlLcTJnQ8MO6p1BRZa?= =?us-ascii?Q?3awAt5qhthu9dQTOCmw5NLo4fxTeT4PCBh4iWZJFs/3wZUh0vFnJaZD0u2IJ?= =?us-ascii?Q?e3cmRN/sHFpv4FdFwD0iFwXFUkUCfY1yIB6rcLr6lBwZgp/8ks8zNNc4M4ri?= =?us-ascii?Q?ZNC63rOv+y4lN8U7HgXU+6+1kdfsZ0ky2sRViSJTSGDAZuvYOdAH9QEo42oA?= =?us-ascii?Q?ux1Efg8t1s2B/iMESFxI9gar7aiFbh5sUGNQKF4Ac4vWGunDa/anBa44oJxE?= =?us-ascii?Q?yU85lu+8lNh5xR3XJ07wFEjcyTWNKLLwu0dg/1txzdHjU4O4lTyRBpu6XTVZ?= =?us-ascii?Q?+BeokmmErEmMO/2+0kcH9mu2+5THbX8yxw7ZD/SBPg3wAg7Sf4JuO19wXljD?= =?us-ascii?Q?SAuqWt3gfNKOLezj7iUV2v9s5BeZx9jPL17GpA2hXCz5IeDR31713peulAdD?= =?us-ascii?Q?lNOhwfOeU5a/F3H74CwGgHlb8Y/DR5zhEDdxqC/C3Ctm6AukfLfV9dJSOANq?= =?us-ascii?Q?yhX1mIC6sEpWfr8PskNFJsSBiLIZQxHVOHkTcluZUDKp/bi8AIfzKyfygzMi?= =?us-ascii?Q?khirs4sosK+12EsGIldKi50x4ux24vjL5xzSrDFnqmJTNHwaSuDEEWW0bSiM?= =?us-ascii?Q?Uhqnx3zPesDztWS2uhDUnyQqZlkh8zt80HhkE+lLpGVyBfS3uXHN9uZWxCs+?= =?us-ascii?Q?Ia1li29uYfx/SjLvBf1dVT76qGAWjhezYbwBRtnYIIAxkZYcm8qMw/gnNyHg?= =?us-ascii?Q?k4sHtli3QnNG/ZJC7aa+RoSvSIDV14Xm8hXKjj7TQmAM+GacRI1L/7WowsME?= =?us-ascii?Q?ZwgIaZDFwH6GFZzBJn8y7fEXh5UH82KreYO9lZyTjDkaTVveIiAIVM9GEl9b?= =?us-ascii?Q?xDSz9bdAKLZN6EeyTns2k1GzyMHJ3ONZXNNQulgZBczah7P4EuYG8kLybDNj?= =?us-ascii?Q?6WGBNnQ44GWu60hwYPJwqC/SmFYJpUcYHjRk+02Z9rfDpU+2NamUIXjnliZK?= =?us-ascii?Q?ogvmxP9kTgegMdKdJsKHUhro4i5f51Qx4G7+8P0Lv6WyqyGh1gRue4J6VFoL?= =?us-ascii?Q?PzbNRSzpkq+mZ+bxwFYMhE/pL+q7+k1UL4JX8ISrVPBl6w2JD57LbM9yoow?= =?us-ascii?Q?=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1;DB6PR0501MB2087;6:aaG/i1OZjtfAn18aMeBYeRZZTVBst/UxxrYvweS+rH+taJb026fxS4MjAvamVLbq8UyNrC6K5ay9Z5c2iLNUXJEWIdiAIZenxRmdiMrCNYdXE4oz3czB10WhIvNRecbqSuIM8s6vzgROLttf8piI/UzR398v0OlQO7+5o0/breMrGRobklmPGE0iLfLdY9DPa+cCoo7F9ogWzTzpuO0OtNWs5eYWD5GvOdf0j2KPuGfNVjdDjqT4bo9XRtqQBnQ6ZQT297HKrjeRM+oKf6ZJTc5s8oLSosjNgtS8Ow3rYo8=;5:cw9KHFd/kCshM5KHwsrgA1uxpojuvkYmcRhfOyGkJ/GkbtaoE/kyGBEkBkt9QgNxzqp6njoLGaZRYAMbpAlg43Q9U0N9VQGBQA1H9Y4CXmgMIZDKMsZ3cpWjY1b+nxmxMmJKFPpdxsGlsiLNTRjFhA==;24:8Xqz7wlPnmaIyKXNjcRC5qbAX/ZJrDLNMKqfHfdEBPTuNeXls+m43HstMNTMmOS4YItrmsUumqH1P0SdShwFuQ7jo9dMEbKcUVEGdPY5Z7s=;7:EQbWqcHcipMBprHE4NibL2gBK5ULn1q/KJaS8lc4uol4RgfLxDI8FiU3+g5Oml+B/zfJqBbvDljilUFx/VofRAES1fhzbTNdREplKI3Ypv7UJGcuA2ae2PS1qsy7YPAPPI5neorJJmrwU03K5iuUWWX55PQZPUh94CvZXxbeyxSMPgDRFdfKCpm+4jkWbGPMXOjRMKScBOHD9q9r9PasW/JeldkUX8kym8iBpqCqtDklkbveIcLJVaBGnRaiNymo SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jul 2016 07:44:51.7281 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0501MB2087 X-Microsoft-Exchange-Diagnostics: 1;DB6PR0501MB2087;23:0r4g2Ge3rZjKN7dyJVynZ22Bk6z7iradvBcogu/kBJ4SJrmsg4jB1CwK75Ee36brlbRcG3Mbfg/246t5MKsejz9IPs8SvSlKK7An+8aR7PcYFhS+66w28ohr3cHCNQpZCLSJc5cthsyciVfonHmtRpOQDPHcQXONQu07LlPW4lIYscppVWY2Yi4jkGOxMLlO X-OriginatorOrg: toradex.com Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Drop PIO to DMA switching and use scatter/gather DMA for Tx path to improve performance. Some part of the code is borrowed from imx serial driver. Signed-off-by: Bhuvanchandra DV --- drivers/tty/serial/fsl_lpuart.c | 257 ++++++++++++++++++---------------------- 1 file changed, 113 insertions(+), 144 deletions(-) diff --git a/drivers/tty/serial/fsl_lpuart.c b/drivers/tty/serial/fsl_lpuart.c index 51d2b5a..27687d5 100644 --- a/drivers/tty/serial/fsl_lpuart.c +++ b/drivers/tty/serial/fsl_lpuart.c @@ -244,18 +244,18 @@ struct lpuart_port { struct dma_chan *dma_rx_chan; struct dma_async_tx_descriptor *dma_tx_desc; struct dma_async_tx_descriptor *dma_rx_desc; - dma_addr_t dma_tx_buf_bus; dma_cookie_t dma_tx_cookie; dma_cookie_t dma_rx_cookie; - unsigned char *dma_tx_buf_virt; unsigned int dma_tx_bytes; unsigned int dma_rx_bytes; - int dma_tx_in_progress; + bool dma_tx_in_progress; unsigned int dma_rx_timeout; struct timer_list lpuart_timer; - struct scatterlist rx_sgl; + struct scatterlist rx_sgl, tx_sgl[2]; struct circ_buf rx_ring; int rx_dma_rng_buf_len; + unsigned int dma_tx_nents; + wait_queue_head_t dma_wait; }; static const struct of_device_id lpuart_dt_ids[] = { @@ -316,103 +316,118 @@ static void lpuart32_stop_rx(struct uart_port *port) lpuart32_write(temp & ~UARTCTRL_RE, port->membase + UARTCTRL); } -static void lpuart_pio_tx(struct lpuart_port *sport) +static void lpuart_dma_tx(struct lpuart_port *sport) { struct circ_buf *xmit = &sport->port.state->xmit; - unsigned long flags; - - spin_lock_irqsave(&sport->port.lock, flags); + struct scatterlist *sgl = sport->tx_sgl; + struct device *dev = sport->port.dev; + int ret; - while (!uart_circ_empty(xmit) && - readb(sport->port.membase + UARTTCFIFO) < sport->txfifo_size) { - writeb(xmit->buf[xmit->tail], sport->port.membase + UARTDR); - xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1); - sport->port.icount.tx++; - } + if (sport->dma_tx_in_progress) + return; - if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS) - uart_write_wakeup(&sport->port); + sport->dma_tx_bytes = uart_circ_chars_pending(xmit); - if (uart_circ_empty(xmit)) - writeb(readb(sport->port.membase + UARTCR5) | UARTCR5_TDMAS, - sport->port.membase + UARTCR5); + if (xmit->tail < xmit->head) { + sport->dma_tx_nents = 1; + sg_init_one(sgl, xmit->buf + xmit->tail, sport->dma_tx_bytes); + } else { + sport->dma_tx_nents = 2; + sg_init_table(sgl, 2); + sg_set_buf(sgl, xmit->buf + xmit->tail, + UART_XMIT_SIZE - xmit->tail); + sg_set_buf(sgl + 1, xmit->buf, xmit->head); + } - spin_unlock_irqrestore(&sport->port.lock, flags); -} + ret = dma_map_sg(dev, sgl, sport->dma_tx_nents, DMA_TO_DEVICE); + if (!ret) { + dev_err(dev, "DMA mapping error for TX.\n"); + return; + } -static int lpuart_dma_tx(struct lpuart_port *sport, unsigned long count) -{ - struct circ_buf *xmit = &sport->port.state->xmit; - dma_addr_t tx_bus_addr; - - dma_sync_single_for_device(sport->port.dev, sport->dma_tx_buf_bus, - UART_XMIT_SIZE, DMA_TO_DEVICE); - sport->dma_tx_bytes = count & ~(sport->txfifo_size - 1); - tx_bus_addr = sport->dma_tx_buf_bus + xmit->tail; - sport->dma_tx_desc = dmaengine_prep_slave_single(sport->dma_tx_chan, - tx_bus_addr, sport->dma_tx_bytes, + sport->dma_tx_desc = dmaengine_prep_slave_sg(sport->dma_tx_chan, sgl, + sport->dma_tx_nents, DMA_MEM_TO_DEV, DMA_PREP_INTERRUPT); - if (!sport->dma_tx_desc) { - dev_err(sport->port.dev, "Not able to get desc for tx\n"); - return -EIO; + dma_unmap_sg(dev, sgl, sport->dma_tx_nents, DMA_TO_DEVICE); + dev_err(dev, "Cannot prepare TX slave DMA!\n"); + return; } sport->dma_tx_desc->callback = lpuart_dma_tx_complete; sport->dma_tx_desc->callback_param = sport; - sport->dma_tx_in_progress = 1; + sport->dma_tx_in_progress = true; sport->dma_tx_cookie = dmaengine_submit(sport->dma_tx_desc); dma_async_issue_pending(sport->dma_tx_chan); - return 0; -} - -static void lpuart_prepare_tx(struct lpuart_port *sport) -{ - struct circ_buf *xmit = &sport->port.state->xmit; - unsigned long count = CIRC_CNT_TO_END(xmit->head, - xmit->tail, UART_XMIT_SIZE); - - if (!count) - return; - - if (count < sport->txfifo_size) - writeb(readb(sport->port.membase + UARTCR5) & ~UARTCR5_TDMAS, - sport->port.membase + UARTCR5); - else { - writeb(readb(sport->port.membase + UARTCR5) | UARTCR5_TDMAS, - sport->port.membase + UARTCR5); - lpuart_dma_tx(sport, count); - } } static void lpuart_dma_tx_complete(void *arg) { struct lpuart_port *sport = arg; + struct scatterlist *sgl = &sport->tx_sgl[0]; struct circ_buf *xmit = &sport->port.state->xmit; unsigned long flags; - async_tx_ack(sport->dma_tx_desc); - spin_lock_irqsave(&sport->port.lock, flags); + dma_unmap_sg(sport->port.dev, sgl, sport->dma_tx_nents, DMA_TO_DEVICE); + xmit->tail = (xmit->tail + sport->dma_tx_bytes) & (UART_XMIT_SIZE - 1); - sport->dma_tx_in_progress = 0; + + sport->port.icount.tx += sport->dma_tx_bytes; + sport->dma_tx_in_progress = false; + spin_unlock_irqrestore(&sport->port.lock, flags); if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS) uart_write_wakeup(&sport->port); - lpuart_prepare_tx(sport); + if (waitqueue_active(&sport->dma_wait)) { + wake_up(&sport->dma_wait); + return; + } + + spin_lock_irqsave(&sport->port.lock, flags); + + if (!uart_circ_empty(xmit) && !uart_tx_stopped(&sport->port)) + lpuart_dma_tx(sport); spin_unlock_irqrestore(&sport->port.lock, flags); } +static int lpuart_dma_tx_request(struct uart_port *port) +{ + struct lpuart_port *sport = container_of(port, + struct lpuart_port, port); + struct dma_slave_config dma_tx_sconfig = {}; + int ret; + + dma_tx_sconfig.dst_addr = sport->port.mapbase + UARTDR; + dma_tx_sconfig.dst_addr_width = DMA_SLAVE_BUSWIDTH_1_BYTE; + dma_tx_sconfig.dst_maxburst = 1; + dma_tx_sconfig.direction = DMA_MEM_TO_DEV; + ret = dmaengine_slave_config(sport->dma_tx_chan, &dma_tx_sconfig); + + if (ret) { + dev_err(sport->port.dev, + "DMA slave config failed, err = %d\n", ret); + return ret; + } + + return 0; +} + static void lpuart_flush_buffer(struct uart_port *port) { struct lpuart_port *sport = container_of(port, struct lpuart_port, port); + if (sport->lpuart_dma_tx_use) { + if (sport->dma_tx_in_progress) { + dma_unmap_sg(sport->port.dev, &sport->tx_sgl[0], + sport->dma_tx_nents, DMA_TO_DEVICE); + sport->dma_tx_in_progress = false; + } dmaengine_terminate_all(sport->dma_tx_chan); - sport->dma_tx_in_progress = 0; } } @@ -469,8 +484,8 @@ static void lpuart_start_tx(struct uart_port *port) writeb(temp | UARTCR2_TIE, port->membase + UARTCR2); if (sport->lpuart_dma_tx_use) { - if (!uart_circ_empty(xmit) && !sport->dma_tx_in_progress) - lpuart_prepare_tx(sport); + if (!uart_circ_empty(xmit) && !uart_tx_stopped(port)) + lpuart_dma_tx(sport); } else { if (readb(port->membase + UARTSR1) & UARTSR1_TDRE) lpuart_transmit_buffer(sport); @@ -489,6 +504,29 @@ static void lpuart32_start_tx(struct uart_port *port) lpuart32_transmit_buffer(sport); } +/* return TIOCSER_TEMT when transmitter is not busy */ +static unsigned int lpuart_tx_empty(struct uart_port *port) +{ + struct lpuart_port *sport = container_of(port, + struct lpuart_port, port); + unsigned char sr1 = readb(port->membase + UARTSR1); + unsigned char sfifo = readb(port->membase + UARTSFIFO); + + if (sport->dma_tx_in_progress) + return 0; + + if (sr1 & UARTSR1_TC && sfifo & UARTSFIFO_TXEMPT) + return TIOCSER_TEMT; + + return 0; +} + +static unsigned int lpuart32_tx_empty(struct uart_port *port) +{ + return (lpuart32_read(port->membase + UARTSTAT) & UARTSTAT_TC) ? + TIOCSER_TEMT : 0; +} + static irqreturn_t lpuart_txint(int irq, void *dev_id) { struct lpuart_port *sport = dev_id; @@ -662,12 +700,8 @@ static irqreturn_t lpuart_int(int irq, void *dev_id) if (sts & UARTSR1_RDRF) lpuart_rxint(irq, dev_id); - if (sts & UARTSR1_TDRE) { - if (sport->lpuart_dma_tx_use) - lpuart_pio_tx(sport); - else - lpuart_txint(irq, dev_id); - } + if (sts & UARTSR1_TDRE) + lpuart_txint(irq, dev_id); return IRQ_HANDLED; } @@ -692,29 +726,6 @@ static irqreturn_t lpuart32_int(int irq, void *dev_id) return IRQ_HANDLED; } -/* return TIOCSER_TEMT when transmitter is not busy */ -static unsigned int lpuart_tx_empty(struct uart_port *port) -{ - struct lpuart_port *sport = container_of(port, - struct lpuart_port, port); - unsigned char sr1 = readb(port->membase + UARTSR1); - unsigned char sfifo = readb(port->membase + UARTSFIFO); - - if (sport->dma_tx_in_progress) - return 0; - - if (sr1 & UARTSR1_TC && sfifo & UARTSFIFO_TXEMPT) - return TIOCSER_TEMT; - - return 0; -} - -static unsigned int lpuart32_tx_empty(struct uart_port *port) -{ - return (lpuart32_read(port->membase + UARTSTAT) & UARTSTAT_TC) ? - TIOCSER_TEMT : 0; -} - static void lpuart_copy_rx_to_tty(struct lpuart_port *sport) { struct tty_port *port = &sport->port.state->port; @@ -890,18 +901,6 @@ static inline int lpuart_start_rx_dma(struct lpuart_port *sport) return 0; } -static void lpuart_dma_tx_free(struct uart_port *port) -{ - struct lpuart_port *sport = container_of(port, - struct lpuart_port, port); - - dma_unmap_single(sport->port.dev, sport->dma_tx_buf_bus, - UART_XMIT_SIZE, DMA_TO_DEVICE); - - sport->dma_tx_buf_bus = 0; - sport->dma_tx_buf_virt = NULL; -} - static void lpuart_dma_rx_free(struct uart_port *port) { struct lpuart_port *sport = container_of(port, @@ -1061,44 +1060,6 @@ static void lpuart32_setup_watermark(struct lpuart_port *sport) lpuart32_write(ctrl_saved, sport->port.membase + UARTCTRL); } -static int lpuart_dma_tx_request(struct uart_port *port) -{ - struct lpuart_port *sport = container_of(port, - struct lpuart_port, port); - struct dma_slave_config dma_tx_sconfig; - dma_addr_t dma_bus; - unsigned char *dma_buf; - int ret; - - dma_bus = dma_map_single(sport->dma_tx_chan->device->dev, - sport->port.state->xmit.buf, - UART_XMIT_SIZE, DMA_TO_DEVICE); - - if (dma_mapping_error(sport->dma_tx_chan->device->dev, dma_bus)) { - dev_err(sport->port.dev, "dma_map_single tx failed\n"); - return -ENOMEM; - } - - dma_buf = sport->port.state->xmit.buf; - dma_tx_sconfig.dst_addr = sport->port.mapbase + UARTDR; - dma_tx_sconfig.dst_addr_width = DMA_SLAVE_BUSWIDTH_1_BYTE; - dma_tx_sconfig.dst_maxburst = sport->txfifo_size; - dma_tx_sconfig.direction = DMA_MEM_TO_DEV; - ret = dmaengine_slave_config(sport->dma_tx_chan, &dma_tx_sconfig); - - if (ret < 0) { - dev_err(sport->port.dev, - "Dma slave config failed, err = %d\n", ret); - return ret; - } - - sport->dma_tx_buf_virt = dma_buf; - sport->dma_tx_buf_bus = dma_bus; - sport->dma_tx_in_progress = 0; - - return 0; -} - static void rx_dma_timer_init(struct lpuart_port *sport) { setup_timer(&sport->lpuart_timer, lpuart_timer_func, @@ -1151,6 +1112,7 @@ static int lpuart_startup(struct uart_port *port) } if (sport->dma_tx_chan && !lpuart_dma_tx_request(port)) { + init_waitqueue_head(&sport->dma_wait); sport->lpuart_dma_tx_use = true; temp = readb(port->membase + UARTCR5); writeb(temp | UARTCR5_TDMAS, port->membase + UARTCR5); @@ -1220,8 +1182,15 @@ static void lpuart_shutdown(struct uart_port *port) lpuart_dma_rx_free(&sport->port); } - if (sport->lpuart_dma_tx_use) - lpuart_dma_tx_free(&sport->port); + if (sport->lpuart_dma_tx_use) { + if (wait_event_interruptible(sport->dma_wait, + !sport->dma_tx_in_progress) != false) { + sport->dma_tx_in_progress = false; + dmaengine_terminate_all(sport->dma_tx_chan); + } + + lpuart_stop_tx(port); + } } static void lpuart32_shutdown(struct uart_port *port) -- 2.9.0 From mboxrd@z Thu Jan 1 00:00:00 1970 From: Bhuvanchandra DV Subject: [PATCH RESEND v2 6/8] tty: serial: fsl_lpuart: Use scatter/gather DMA for Tx Date: Tue, 19 Jul 2016 13:13:08 +0530 Message-ID: <20160719074310.16872-7-bhuvanchandra.dv@toradex.com> References: <20160719074310.16872-1-bhuvanchandra.dv@toradex.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20160719074310.16872-1-bhuvanchandra.dv@toradex.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=m.gmane.org@lists.infradead.org To: gregkh@linuxfoundation.org Cc: linux-serial@vger.kernel.org, mturquette@baylibre.com, sboyd@codeaurora.org, linux-kernel@vger.kernel.org, stefan@agner.ch, kernel@pengutronix.de, jslaby@suse.com, shawnguo@kernel.org, linux-clk@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Bhuvanchandra DV List-Id: linux-serial@vger.kernel.org Drop PIO to DMA switching and use scatter/gather DMA for Tx path to improve performance. Some part of the code is borrowed from imx serial driver. Signed-off-by: Bhuvanchandra DV --- drivers/tty/serial/fsl_lpuart.c | 257 ++++++++++++++++++---------------------- 1 file changed, 113 insertions(+), 144 deletions(-) diff --git a/drivers/tty/serial/fsl_lpuart.c b/drivers/tty/serial/fsl_lpuart.c index 51d2b5a..27687d5 100644 --- a/drivers/tty/serial/fsl_lpuart.c +++ b/drivers/tty/serial/fsl_lpuart.c @@ -244,18 +244,18 @@ struct lpuart_port { struct dma_chan *dma_rx_chan; struct dma_async_tx_descriptor *dma_tx_desc; struct dma_async_tx_descriptor *dma_rx_desc; - dma_addr_t dma_tx_buf_bus; dma_cookie_t dma_tx_cookie; dma_cookie_t dma_rx_cookie; - unsigned char *dma_tx_buf_virt; unsigned int dma_tx_bytes; unsigned int dma_rx_bytes; - int dma_tx_in_progress; + bool dma_tx_in_progress; unsigned int dma_rx_timeout; struct timer_list lpuart_timer; - struct scatterlist rx_sgl; + struct scatterlist rx_sgl, tx_sgl[2]; struct circ_buf rx_ring; int rx_dma_rng_buf_len; + unsigned int dma_tx_nents; + wait_queue_head_t dma_wait; }; static const struct of_device_id lpuart_dt_ids[] = { @@ -316,103 +316,118 @@ static void lpuart32_stop_rx(struct uart_port *port) lpuart32_write(temp & ~UARTCTRL_RE, port->membase + UARTCTRL); } -static void lpuart_pio_tx(struct lpuart_port *sport) +static void lpuart_dma_tx(struct lpuart_port *sport) { struct circ_buf *xmit = &sport->port.state->xmit; - unsigned long flags; - - spin_lock_irqsave(&sport->port.lock, flags); + struct scatterlist *sgl = sport->tx_sgl; + struct device *dev = sport->port.dev; + int ret; - while (!uart_circ_empty(xmit) && - readb(sport->port.membase + UARTTCFIFO) < sport->txfifo_size) { - writeb(xmit->buf[xmit->tail], sport->port.membase + UARTDR); - xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1); - sport->port.icount.tx++; - } + if (sport->dma_tx_in_progress) + return; - if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS) - uart_write_wakeup(&sport->port); + sport->dma_tx_bytes = uart_circ_chars_pending(xmit); - if (uart_circ_empty(xmit)) - writeb(readb(sport->port.membase + UARTCR5) | UARTCR5_TDMAS, - sport->port.membase + UARTCR5); + if (xmit->tail < xmit->head) { + sport->dma_tx_nents = 1; + sg_init_one(sgl, xmit->buf + xmit->tail, sport->dma_tx_bytes); + } else { + sport->dma_tx_nents = 2; + sg_init_table(sgl, 2); + sg_set_buf(sgl, xmit->buf + xmit->tail, + UART_XMIT_SIZE - xmit->tail); + sg_set_buf(sgl + 1, xmit->buf, xmit->head); + } - spin_unlock_irqrestore(&sport->port.lock, flags); -} + ret = dma_map_sg(dev, sgl, sport->dma_tx_nents, DMA_TO_DEVICE); + if (!ret) { + dev_err(dev, "DMA mapping error for TX.\n"); + return; + } -static int lpuart_dma_tx(struct lpuart_port *sport, unsigned long count) -{ - struct circ_buf *xmit = &sport->port.state->xmit; - dma_addr_t tx_bus_addr; - - dma_sync_single_for_device(sport->port.dev, sport->dma_tx_buf_bus, - UART_XMIT_SIZE, DMA_TO_DEVICE); - sport->dma_tx_bytes = count & ~(sport->txfifo_size - 1); - tx_bus_addr = sport->dma_tx_buf_bus + xmit->tail; - sport->dma_tx_desc = dmaengine_prep_slave_single(sport->dma_tx_chan, - tx_bus_addr, sport->dma_tx_bytes, + sport->dma_tx_desc = dmaengine_prep_slave_sg(sport->dma_tx_chan, sgl, + sport->dma_tx_nents, DMA_MEM_TO_DEV, DMA_PREP_INTERRUPT); - if (!sport->dma_tx_desc) { - dev_err(sport->port.dev, "Not able to get desc for tx\n"); - return -EIO; + dma_unmap_sg(dev, sgl, sport->dma_tx_nents, DMA_TO_DEVICE); + dev_err(dev, "Cannot prepare TX slave DMA!\n"); + return; } sport->dma_tx_desc->callback = lpuart_dma_tx_complete; sport->dma_tx_desc->callback_param = sport; - sport->dma_tx_in_progress = 1; + sport->dma_tx_in_progress = true; sport->dma_tx_cookie = dmaengine_submit(sport->dma_tx_desc); dma_async_issue_pending(sport->dma_tx_chan); - return 0; -} - -static void lpuart_prepare_tx(struct lpuart_port *sport) -{ - struct circ_buf *xmit = &sport->port.state->xmit; - unsigned long count = CIRC_CNT_TO_END(xmit->head, - xmit->tail, UART_XMIT_SIZE); - - if (!count) - return; - - if (count < sport->txfifo_size) - writeb(readb(sport->port.membase + UARTCR5) & ~UARTCR5_TDMAS, - sport->port.membase + UARTCR5); - else { - writeb(readb(sport->port.membase + UARTCR5) | UARTCR5_TDMAS, - sport->port.membase + UARTCR5); - lpuart_dma_tx(sport, count); - } } static void lpuart_dma_tx_complete(void *arg) { struct lpuart_port *sport = arg; + struct scatterlist *sgl = &sport->tx_sgl[0]; struct circ_buf *xmit = &sport->port.state->xmit; unsigned long flags; - async_tx_ack(sport->dma_tx_desc); - spin_lock_irqsave(&sport->port.lock, flags); + dma_unmap_sg(sport->port.dev, sgl, sport->dma_tx_nents, DMA_TO_DEVICE); + xmit->tail = (xmit->tail + sport->dma_tx_bytes) & (UART_XMIT_SIZE - 1); - sport->dma_tx_in_progress = 0; + + sport->port.icount.tx += sport->dma_tx_bytes; + sport->dma_tx_in_progress = false; + spin_unlock_irqrestore(&sport->port.lock, flags); if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS) uart_write_wakeup(&sport->port); - lpuart_prepare_tx(sport); + if (waitqueue_active(&sport->dma_wait)) { + wake_up(&sport->dma_wait); + return; + } + + spin_lock_irqsave(&sport->port.lock, flags); + + if (!uart_circ_empty(xmit) && !uart_tx_stopped(&sport->port)) + lpuart_dma_tx(sport); spin_unlock_irqrestore(&sport->port.lock, flags); } +static int lpuart_dma_tx_request(struct uart_port *port) +{ + struct lpuart_port *sport = container_of(port, + struct lpuart_port, port); + struct dma_slave_config dma_tx_sconfig = {}; + int ret; + + dma_tx_sconfig.dst_addr = sport->port.mapbase + UARTDR; + dma_tx_sconfig.dst_addr_width = DMA_SLAVE_BUSWIDTH_1_BYTE; + dma_tx_sconfig.dst_maxburst = 1; + dma_tx_sconfig.direction = DMA_MEM_TO_DEV; + ret = dmaengine_slave_config(sport->dma_tx_chan, &dma_tx_sconfig); + + if (ret) { + dev_err(sport->port.dev, + "DMA slave config failed, err = %d\n", ret); + return ret; + } + + return 0; +} + static void lpuart_flush_buffer(struct uart_port *port) { struct lpuart_port *sport = container_of(port, struct lpuart_port, port); + if (sport->lpuart_dma_tx_use) { + if (sport->dma_tx_in_progress) { + dma_unmap_sg(sport->port.dev, &sport->tx_sgl[0], + sport->dma_tx_nents, DMA_TO_DEVICE); + sport->dma_tx_in_progress = false; + } dmaengine_terminate_all(sport->dma_tx_chan); - sport->dma_tx_in_progress = 0; } } @@ -469,8 +484,8 @@ static void lpuart_start_tx(struct uart_port *port) writeb(temp | UARTCR2_TIE, port->membase + UARTCR2); if (sport->lpuart_dma_tx_use) { - if (!uart_circ_empty(xmit) && !sport->dma_tx_in_progress) - lpuart_prepare_tx(sport); + if (!uart_circ_empty(xmit) && !uart_tx_stopped(port)) + lpuart_dma_tx(sport); } else { if (readb(port->membase + UARTSR1) & UARTSR1_TDRE) lpuart_transmit_buffer(sport); @@ -489,6 +504,29 @@ static void lpuart32_start_tx(struct uart_port *port) lpuart32_transmit_buffer(sport); } +/* return TIOCSER_TEMT when transmitter is not busy */ +static unsigned int lpuart_tx_empty(struct uart_port *port) +{ + struct lpuart_port *sport = container_of(port, + struct lpuart_port, port); + unsigned char sr1 = readb(port->membase + UARTSR1); + unsigned char sfifo = readb(port->membase + UARTSFIFO); + + if (sport->dma_tx_in_progress) + return 0; + + if (sr1 & UARTSR1_TC && sfifo & UARTSFIFO_TXEMPT) + return TIOCSER_TEMT; + + return 0; +} + +static unsigned int lpuart32_tx_empty(struct uart_port *port) +{ + return (lpuart32_read(port->membase + UARTSTAT) & UARTSTAT_TC) ? + TIOCSER_TEMT : 0; +} + static irqreturn_t lpuart_txint(int irq, void *dev_id) { struct lpuart_port *sport = dev_id; @@ -662,12 +700,8 @@ static irqreturn_t lpuart_int(int irq, void *dev_id) if (sts & UARTSR1_RDRF) lpuart_rxint(irq, dev_id); - if (sts & UARTSR1_TDRE) { - if (sport->lpuart_dma_tx_use) - lpuart_pio_tx(sport); - else - lpuart_txint(irq, dev_id); - } + if (sts & UARTSR1_TDRE) + lpuart_txint(irq, dev_id); return IRQ_HANDLED; } @@ -692,29 +726,6 @@ static irqreturn_t lpuart32_int(int irq, void *dev_id) return IRQ_HANDLED; } -/* return TIOCSER_TEMT when transmitter is not busy */ -static unsigned int lpuart_tx_empty(struct uart_port *port) -{ - struct lpuart_port *sport = container_of(port, - struct lpuart_port, port); - unsigned char sr1 = readb(port->membase + UARTSR1); - unsigned char sfifo = readb(port->membase + UARTSFIFO); - - if (sport->dma_tx_in_progress) - return 0; - - if (sr1 & UARTSR1_TC && sfifo & UARTSFIFO_TXEMPT) - return TIOCSER_TEMT; - - return 0; -} - -static unsigned int lpuart32_tx_empty(struct uart_port *port) -{ - return (lpuart32_read(port->membase + UARTSTAT) & UARTSTAT_TC) ? - TIOCSER_TEMT : 0; -} - static void lpuart_copy_rx_to_tty(struct lpuart_port *sport) { struct tty_port *port = &sport->port.state->port; @@ -890,18 +901,6 @@ static inline int lpuart_start_rx_dma(struct lpuart_port *sport) return 0; } -static void lpuart_dma_tx_free(struct uart_port *port) -{ - struct lpuart_port *sport = container_of(port, - struct lpuart_port, port); - - dma_unmap_single(sport->port.dev, sport->dma_tx_buf_bus, - UART_XMIT_SIZE, DMA_TO_DEVICE); - - sport->dma_tx_buf_bus = 0; - sport->dma_tx_buf_virt = NULL; -} - static void lpuart_dma_rx_free(struct uart_port *port) { struct lpuart_port *sport = container_of(port, @@ -1061,44 +1060,6 @@ static void lpuart32_setup_watermark(struct lpuart_port *sport) lpuart32_write(ctrl_saved, sport->port.membase + UARTCTRL); } -static int lpuart_dma_tx_request(struct uart_port *port) -{ - struct lpuart_port *sport = container_of(port, - struct lpuart_port, port); - struct dma_slave_config dma_tx_sconfig; - dma_addr_t dma_bus; - unsigned char *dma_buf; - int ret; - - dma_bus = dma_map_single(sport->dma_tx_chan->device->dev, - sport->port.state->xmit.buf, - UART_XMIT_SIZE, DMA_TO_DEVICE); - - if (dma_mapping_error(sport->dma_tx_chan->device->dev, dma_bus)) { - dev_err(sport->port.dev, "dma_map_single tx failed\n"); - return -ENOMEM; - } - - dma_buf = sport->port.state->xmit.buf; - dma_tx_sconfig.dst_addr = sport->port.mapbase + UARTDR; - dma_tx_sconfig.dst_addr_width = DMA_SLAVE_BUSWIDTH_1_BYTE; - dma_tx_sconfig.dst_maxburst = sport->txfifo_size; - dma_tx_sconfig.direction = DMA_MEM_TO_DEV; - ret = dmaengine_slave_config(sport->dma_tx_chan, &dma_tx_sconfig); - - if (ret < 0) { - dev_err(sport->port.dev, - "Dma slave config failed, err = %d\n", ret); - return ret; - } - - sport->dma_tx_buf_virt = dma_buf; - sport->dma_tx_buf_bus = dma_bus; - sport->dma_tx_in_progress = 0; - - return 0; -} - static void rx_dma_timer_init(struct lpuart_port *sport) { setup_timer(&sport->lpuart_timer, lpuart_timer_func, @@ -1151,6 +1112,7 @@ static int lpuart_startup(struct uart_port *port) } if (sport->dma_tx_chan && !lpuart_dma_tx_request(port)) { + init_waitqueue_head(&sport->dma_wait); sport->lpuart_dma_tx_use = true; temp = readb(port->membase + UARTCR5); writeb(temp | UARTCR5_TDMAS, port->membase + UARTCR5); @@ -1220,8 +1182,15 @@ static void lpuart_shutdown(struct uart_port *port) lpuart_dma_rx_free(&sport->port); } - if (sport->lpuart_dma_tx_use) - lpuart_dma_tx_free(&sport->port); + if (sport->lpuart_dma_tx_use) { + if (wait_event_interruptible(sport->dma_wait, + !sport->dma_tx_in_progress) != false) { + sport->dma_tx_in_progress = false; + dmaengine_terminate_all(sport->dma_tx_chan); + } + + lpuart_stop_tx(port); + } } static void lpuart32_shutdown(struct uart_port *port) -- 2.9.0 From mboxrd@z Thu Jan 1 00:00:00 1970 From: bhuvanchandra.dv@toradex.com (Bhuvanchandra DV) Date: Tue, 19 Jul 2016 13:13:08 +0530 Subject: [PATCH RESEND v2 6/8] tty: serial: fsl_lpuart: Use scatter/gather DMA for Tx In-Reply-To: <20160719074310.16872-1-bhuvanchandra.dv@toradex.com> References: <20160719074310.16872-1-bhuvanchandra.dv@toradex.com> Message-ID: <20160719074310.16872-7-bhuvanchandra.dv@toradex.com> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org Drop PIO to DMA switching and use scatter/gather DMA for Tx path to improve performance. Some part of the code is borrowed from imx serial driver. Signed-off-by: Bhuvanchandra DV --- drivers/tty/serial/fsl_lpuart.c | 257 ++++++++++++++++++---------------------- 1 file changed, 113 insertions(+), 144 deletions(-) diff --git a/drivers/tty/serial/fsl_lpuart.c b/drivers/tty/serial/fsl_lpuart.c index 51d2b5a..27687d5 100644 --- a/drivers/tty/serial/fsl_lpuart.c +++ b/drivers/tty/serial/fsl_lpuart.c @@ -244,18 +244,18 @@ struct lpuart_port { struct dma_chan *dma_rx_chan; struct dma_async_tx_descriptor *dma_tx_desc; struct dma_async_tx_descriptor *dma_rx_desc; - dma_addr_t dma_tx_buf_bus; dma_cookie_t dma_tx_cookie; dma_cookie_t dma_rx_cookie; - unsigned char *dma_tx_buf_virt; unsigned int dma_tx_bytes; unsigned int dma_rx_bytes; - int dma_tx_in_progress; + bool dma_tx_in_progress; unsigned int dma_rx_timeout; struct timer_list lpuart_timer; - struct scatterlist rx_sgl; + struct scatterlist rx_sgl, tx_sgl[2]; struct circ_buf rx_ring; int rx_dma_rng_buf_len; + unsigned int dma_tx_nents; + wait_queue_head_t dma_wait; }; static const struct of_device_id lpuart_dt_ids[] = { @@ -316,103 +316,118 @@ static void lpuart32_stop_rx(struct uart_port *port) lpuart32_write(temp & ~UARTCTRL_RE, port->membase + UARTCTRL); } -static void lpuart_pio_tx(struct lpuart_port *sport) +static void lpuart_dma_tx(struct lpuart_port *sport) { struct circ_buf *xmit = &sport->port.state->xmit; - unsigned long flags; - - spin_lock_irqsave(&sport->port.lock, flags); + struct scatterlist *sgl = sport->tx_sgl; + struct device *dev = sport->port.dev; + int ret; - while (!uart_circ_empty(xmit) && - readb(sport->port.membase + UARTTCFIFO) < sport->txfifo_size) { - writeb(xmit->buf[xmit->tail], sport->port.membase + UARTDR); - xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1); - sport->port.icount.tx++; - } + if (sport->dma_tx_in_progress) + return; - if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS) - uart_write_wakeup(&sport->port); + sport->dma_tx_bytes = uart_circ_chars_pending(xmit); - if (uart_circ_empty(xmit)) - writeb(readb(sport->port.membase + UARTCR5) | UARTCR5_TDMAS, - sport->port.membase + UARTCR5); + if (xmit->tail < xmit->head) { + sport->dma_tx_nents = 1; + sg_init_one(sgl, xmit->buf + xmit->tail, sport->dma_tx_bytes); + } else { + sport->dma_tx_nents = 2; + sg_init_table(sgl, 2); + sg_set_buf(sgl, xmit->buf + xmit->tail, + UART_XMIT_SIZE - xmit->tail); + sg_set_buf(sgl + 1, xmit->buf, xmit->head); + } - spin_unlock_irqrestore(&sport->port.lock, flags); -} + ret = dma_map_sg(dev, sgl, sport->dma_tx_nents, DMA_TO_DEVICE); + if (!ret) { + dev_err(dev, "DMA mapping error for TX.\n"); + return; + } -static int lpuart_dma_tx(struct lpuart_port *sport, unsigned long count) -{ - struct circ_buf *xmit = &sport->port.state->xmit; - dma_addr_t tx_bus_addr; - - dma_sync_single_for_device(sport->port.dev, sport->dma_tx_buf_bus, - UART_XMIT_SIZE, DMA_TO_DEVICE); - sport->dma_tx_bytes = count & ~(sport->txfifo_size - 1); - tx_bus_addr = sport->dma_tx_buf_bus + xmit->tail; - sport->dma_tx_desc = dmaengine_prep_slave_single(sport->dma_tx_chan, - tx_bus_addr, sport->dma_tx_bytes, + sport->dma_tx_desc = dmaengine_prep_slave_sg(sport->dma_tx_chan, sgl, + sport->dma_tx_nents, DMA_MEM_TO_DEV, DMA_PREP_INTERRUPT); - if (!sport->dma_tx_desc) { - dev_err(sport->port.dev, "Not able to get desc for tx\n"); - return -EIO; + dma_unmap_sg(dev, sgl, sport->dma_tx_nents, DMA_TO_DEVICE); + dev_err(dev, "Cannot prepare TX slave DMA!\n"); + return; } sport->dma_tx_desc->callback = lpuart_dma_tx_complete; sport->dma_tx_desc->callback_param = sport; - sport->dma_tx_in_progress = 1; + sport->dma_tx_in_progress = true; sport->dma_tx_cookie = dmaengine_submit(sport->dma_tx_desc); dma_async_issue_pending(sport->dma_tx_chan); - return 0; -} - -static void lpuart_prepare_tx(struct lpuart_port *sport) -{ - struct circ_buf *xmit = &sport->port.state->xmit; - unsigned long count = CIRC_CNT_TO_END(xmit->head, - xmit->tail, UART_XMIT_SIZE); - - if (!count) - return; - - if (count < sport->txfifo_size) - writeb(readb(sport->port.membase + UARTCR5) & ~UARTCR5_TDMAS, - sport->port.membase + UARTCR5); - else { - writeb(readb(sport->port.membase + UARTCR5) | UARTCR5_TDMAS, - sport->port.membase + UARTCR5); - lpuart_dma_tx(sport, count); - } } static void lpuart_dma_tx_complete(void *arg) { struct lpuart_port *sport = arg; + struct scatterlist *sgl = &sport->tx_sgl[0]; struct circ_buf *xmit = &sport->port.state->xmit; unsigned long flags; - async_tx_ack(sport->dma_tx_desc); - spin_lock_irqsave(&sport->port.lock, flags); + dma_unmap_sg(sport->port.dev, sgl, sport->dma_tx_nents, DMA_TO_DEVICE); + xmit->tail = (xmit->tail + sport->dma_tx_bytes) & (UART_XMIT_SIZE - 1); - sport->dma_tx_in_progress = 0; + + sport->port.icount.tx += sport->dma_tx_bytes; + sport->dma_tx_in_progress = false; + spin_unlock_irqrestore(&sport->port.lock, flags); if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS) uart_write_wakeup(&sport->port); - lpuart_prepare_tx(sport); + if (waitqueue_active(&sport->dma_wait)) { + wake_up(&sport->dma_wait); + return; + } + + spin_lock_irqsave(&sport->port.lock, flags); + + if (!uart_circ_empty(xmit) && !uart_tx_stopped(&sport->port)) + lpuart_dma_tx(sport); spin_unlock_irqrestore(&sport->port.lock, flags); } +static int lpuart_dma_tx_request(struct uart_port *port) +{ + struct lpuart_port *sport = container_of(port, + struct lpuart_port, port); + struct dma_slave_config dma_tx_sconfig = {}; + int ret; + + dma_tx_sconfig.dst_addr = sport->port.mapbase + UARTDR; + dma_tx_sconfig.dst_addr_width = DMA_SLAVE_BUSWIDTH_1_BYTE; + dma_tx_sconfig.dst_maxburst = 1; + dma_tx_sconfig.direction = DMA_MEM_TO_DEV; + ret = dmaengine_slave_config(sport->dma_tx_chan, &dma_tx_sconfig); + + if (ret) { + dev_err(sport->port.dev, + "DMA slave config failed, err = %d\n", ret); + return ret; + } + + return 0; +} + static void lpuart_flush_buffer(struct uart_port *port) { struct lpuart_port *sport = container_of(port, struct lpuart_port, port); + if (sport->lpuart_dma_tx_use) { + if (sport->dma_tx_in_progress) { + dma_unmap_sg(sport->port.dev, &sport->tx_sgl[0], + sport->dma_tx_nents, DMA_TO_DEVICE); + sport->dma_tx_in_progress = false; + } dmaengine_terminate_all(sport->dma_tx_chan); - sport->dma_tx_in_progress = 0; } } @@ -469,8 +484,8 @@ static void lpuart_start_tx(struct uart_port *port) writeb(temp | UARTCR2_TIE, port->membase + UARTCR2); if (sport->lpuart_dma_tx_use) { - if (!uart_circ_empty(xmit) && !sport->dma_tx_in_progress) - lpuart_prepare_tx(sport); + if (!uart_circ_empty(xmit) && !uart_tx_stopped(port)) + lpuart_dma_tx(sport); } else { if (readb(port->membase + UARTSR1) & UARTSR1_TDRE) lpuart_transmit_buffer(sport); @@ -489,6 +504,29 @@ static void lpuart32_start_tx(struct uart_port *port) lpuart32_transmit_buffer(sport); } +/* return TIOCSER_TEMT when transmitter is not busy */ +static unsigned int lpuart_tx_empty(struct uart_port *port) +{ + struct lpuart_port *sport = container_of(port, + struct lpuart_port, port); + unsigned char sr1 = readb(port->membase + UARTSR1); + unsigned char sfifo = readb(port->membase + UARTSFIFO); + + if (sport->dma_tx_in_progress) + return 0; + + if (sr1 & UARTSR1_TC && sfifo & UARTSFIFO_TXEMPT) + return TIOCSER_TEMT; + + return 0; +} + +static unsigned int lpuart32_tx_empty(struct uart_port *port) +{ + return (lpuart32_read(port->membase + UARTSTAT) & UARTSTAT_TC) ? + TIOCSER_TEMT : 0; +} + static irqreturn_t lpuart_txint(int irq, void *dev_id) { struct lpuart_port *sport = dev_id; @@ -662,12 +700,8 @@ static irqreturn_t lpuart_int(int irq, void *dev_id) if (sts & UARTSR1_RDRF) lpuart_rxint(irq, dev_id); - if (sts & UARTSR1_TDRE) { - if (sport->lpuart_dma_tx_use) - lpuart_pio_tx(sport); - else - lpuart_txint(irq, dev_id); - } + if (sts & UARTSR1_TDRE) + lpuart_txint(irq, dev_id); return IRQ_HANDLED; } @@ -692,29 +726,6 @@ static irqreturn_t lpuart32_int(int irq, void *dev_id) return IRQ_HANDLED; } -/* return TIOCSER_TEMT when transmitter is not busy */ -static unsigned int lpuart_tx_empty(struct uart_port *port) -{ - struct lpuart_port *sport = container_of(port, - struct lpuart_port, port); - unsigned char sr1 = readb(port->membase + UARTSR1); - unsigned char sfifo = readb(port->membase + UARTSFIFO); - - if (sport->dma_tx_in_progress) - return 0; - - if (sr1 & UARTSR1_TC && sfifo & UARTSFIFO_TXEMPT) - return TIOCSER_TEMT; - - return 0; -} - -static unsigned int lpuart32_tx_empty(struct uart_port *port) -{ - return (lpuart32_read(port->membase + UARTSTAT) & UARTSTAT_TC) ? - TIOCSER_TEMT : 0; -} - static void lpuart_copy_rx_to_tty(struct lpuart_port *sport) { struct tty_port *port = &sport->port.state->port; @@ -890,18 +901,6 @@ static inline int lpuart_start_rx_dma(struct lpuart_port *sport) return 0; } -static void lpuart_dma_tx_free(struct uart_port *port) -{ - struct lpuart_port *sport = container_of(port, - struct lpuart_port, port); - - dma_unmap_single(sport->port.dev, sport->dma_tx_buf_bus, - UART_XMIT_SIZE, DMA_TO_DEVICE); - - sport->dma_tx_buf_bus = 0; - sport->dma_tx_buf_virt = NULL; -} - static void lpuart_dma_rx_free(struct uart_port *port) { struct lpuart_port *sport = container_of(port, @@ -1061,44 +1060,6 @@ static void lpuart32_setup_watermark(struct lpuart_port *sport) lpuart32_write(ctrl_saved, sport->port.membase + UARTCTRL); } -static int lpuart_dma_tx_request(struct uart_port *port) -{ - struct lpuart_port *sport = container_of(port, - struct lpuart_port, port); - struct dma_slave_config dma_tx_sconfig; - dma_addr_t dma_bus; - unsigned char *dma_buf; - int ret; - - dma_bus = dma_map_single(sport->dma_tx_chan->device->dev, - sport->port.state->xmit.buf, - UART_XMIT_SIZE, DMA_TO_DEVICE); - - if (dma_mapping_error(sport->dma_tx_chan->device->dev, dma_bus)) { - dev_err(sport->port.dev, "dma_map_single tx failed\n"); - return -ENOMEM; - } - - dma_buf = sport->port.state->xmit.buf; - dma_tx_sconfig.dst_addr = sport->port.mapbase + UARTDR; - dma_tx_sconfig.dst_addr_width = DMA_SLAVE_BUSWIDTH_1_BYTE; - dma_tx_sconfig.dst_maxburst = sport->txfifo_size; - dma_tx_sconfig.direction = DMA_MEM_TO_DEV; - ret = dmaengine_slave_config(sport->dma_tx_chan, &dma_tx_sconfig); - - if (ret < 0) { - dev_err(sport->port.dev, - "Dma slave config failed, err = %d\n", ret); - return ret; - } - - sport->dma_tx_buf_virt = dma_buf; - sport->dma_tx_buf_bus = dma_bus; - sport->dma_tx_in_progress = 0; - - return 0; -} - static void rx_dma_timer_init(struct lpuart_port *sport) { setup_timer(&sport->lpuart_timer, lpuart_timer_func, @@ -1151,6 +1112,7 @@ static int lpuart_startup(struct uart_port *port) } if (sport->dma_tx_chan && !lpuart_dma_tx_request(port)) { + init_waitqueue_head(&sport->dma_wait); sport->lpuart_dma_tx_use = true; temp = readb(port->membase + UARTCR5); writeb(temp | UARTCR5_TDMAS, port->membase + UARTCR5); @@ -1220,8 +1182,15 @@ static void lpuart_shutdown(struct uart_port *port) lpuart_dma_rx_free(&sport->port); } - if (sport->lpuart_dma_tx_use) - lpuart_dma_tx_free(&sport->port); + if (sport->lpuart_dma_tx_use) { + if (wait_event_interruptible(sport->dma_wait, + !sport->dma_tx_in_progress) != false) { + sport->dma_tx_in_progress = false; + dmaengine_terminate_all(sport->dma_tx_chan); + } + + lpuart_stop_tx(port); + } } static void lpuart32_shutdown(struct uart_port *port) -- 2.9.0