linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Jon Hunter <jonathanh@nvidia.com>
To: "Dmitry Osipenko" <digetx@gmail.com>,
	"Laxman Dewangan" <ldewangan@nvidia.com>,
	"Vinod Koul" <vkoul@kernel.org>,
	"Dan Williams" <dan.j.williams@intel.com>,
	"Thierry Reding" <thierry.reding@gmail.com>,
	"Michał Mirosław" <mirq-linux@rere.qmqm.pl>
Cc: <dmaengine@vger.kernel.org>, <linux-tegra@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v4 01/14] dmaengine: tegra-apb: Fix use-after-free
Date: Wed, 29 Jan 2020 10:42:16 +0000	[thread overview]
Message-ID: <7c1c0672-4e6d-ae49-d041-2838199f0fcd@nvidia.com> (raw)
In-Reply-To: <9ca67d17-3bb0-abb4-9d95-1057d0828ed2@gmail.com>


On 29/01/2020 00:12, Dmitry Osipenko wrote:
> 28.01.2020 17:51, Dmitry Osipenko пишет:
>> 28.01.2020 17:02, Jon Hunter пишет:
>>>
>>> On 16/01/2020 20:10, Dmitry Osipenko wrote:
>>>> 15.01.2020 12:00, Jon Hunter пишет:
>>>>>
>>>>> On 14/01/2020 20:33, Dmitry Osipenko wrote:
>>>>>> 14.01.2020 18:09, Jon Hunter пишет:
>>>>>>>
>>>>>>> On 12/01/2020 17:29, Dmitry Osipenko wrote:
>>>>>>>> I was doing some experiments with I2C and noticed that Tegra APB DMA
>>>>>>>> driver crashes sometime after I2C DMA transfer termination. The crash
>>>>>>>> happens because tegra_dma_terminate_all() bails out immediately if pending
>>>>>>>> list is empty, thus it doesn't release the half-completed descriptors
>>>>>>>> which are getting re-used before ISR tasklet kicks-in.
>>>>>>>
>>>>>>> Can you elaborate a bit more on how these are getting re-used? What is
>>>>>>> the sequence of events which results in the panic? I believe that this
>>>>>>> was also reported in the past [0] and so I don't doubt there is an issue
>>>>>>> here, but would like to completely understand this.
>>>>>>>
>>>>>>> Thanks!
>>>>>>> Jon
>>>>>>>
>>>>>>> [0] https://lore.kernel.org/patchwork/patch/675349/
>>>>>>>
>>>>>>
>>>>>> In my case it happens in the touchscreen driver during of the
>>>>>> touchscreen's interrupt handling (in a threaded IRQ handler) + CPU is
>>>>>> under load and there is other interrupts activity. So what happens here
>>>>>> is that the TS driver issues one I2C transfer, which fails with
>>>>>> (apparently bogus) timeout (because DMA descriptor is completed and
>>>>>> removed from the pending list, but tasklet not executed yet), and then
>>>>>> TS immediately issues another I2C transfer that re-uses the
>>>>>> yet-incompleted descriptor. That's my understanding.
>>>>>
>>>>> OK, but what is the exact sequence that it allowing it to re-use the
>>>>> incompleted descriptor?
>>>>
>>>>    TDMA driver                      DMA Client
>>>>
>>>> 1.
>>>>                                     dmaengine_prep()
>>>>
>>>> 2.
>>>>    tegra_dma_desc_get()
>>>>    dma_desc = kzalloc()
>>>>    ...
>>>>    tegra_dma_prep_slave_sg()
>>>>    INIT_LIST_HEAD(&dma_desc->tx_list);
>>>>    INIT_LIST_HEAD(&dma_desc->cb_node);
>>>>    list_add_tail(sgreq->node,
>>>>                  dma_desc->tx_list)
>>>>
>>>> 3.
>>>>                                     dma_async_issue_pending()
>>>>
>>>> 4.
>>>>    tegra_dma_tx_submit()
>>>>    list_splice_tail_init(dma_desc->tx_list,
>>>>                          tdc->pending_sg_req)
>>>>
>>>> 5.
>>>>    tegra_dma_isr()
>>>>    ...
>>>>    handle_once_dma_done()
>>>>    ...
>>>>    sgreq = list_first_entry(tdc->pending_sg_req)
>>>>    list_del(sgreq->node);
>>>>    ...
>>>>    list_add_tail(dma_desc->cb_node,
>>>>                  tdc->cb_desc);
>>>>    list_add_tail(dma_desc->node,
>>>>                  tdc->free_dma_desc);
>>>
>>> Isn't this the problem here, that we have placed this on the free list
>>> before we are actually done?
>>>
>>> It seems to me that there could still be a potential race condition
>>> between the ISR and the tasklet running.
>>
>> Yes, this should be addressed by the patch #3 "dmaengine: tegra-apb:
>> Prevent race conditions of tasklet vs free list".
> 
> correction (to avoid confusion): it's actually patch #5, my bad

Ah OK great! Could be worth making that patch #1 or #2 in the series as
theses are somewhat related.

Cheers
Jon

-- 
nvpublic

  reply	other threads:[~2020-01-29 10:42 UTC|newest]

Thread overview: 46+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-01-12 17:29 [PATCH v4 00/14] NVIDIA Tegra APB DMA driver fixes and improvements Dmitry Osipenko
2020-01-12 17:29 ` [PATCH v4 01/14] dmaengine: tegra-apb: Fix use-after-free Dmitry Osipenko
2020-01-14 15:09   ` Jon Hunter
2020-01-14 20:33     ` Dmitry Osipenko
2020-01-15  9:00       ` Jon Hunter
2020-01-16 20:10         ` Dmitry Osipenko
2020-01-28 14:02           ` Jon Hunter
2020-01-28 14:51             ` Dmitry Osipenko
2020-01-29  0:12               ` Dmitry Osipenko
2020-01-29 10:42                 ` Jon Hunter [this message]
2020-01-12 17:29 ` [PATCH v4 02/14] dmaengine: tegra-apb: Implement synchronization callback Dmitry Osipenko
2020-01-14 15:15   ` Jon Hunter
2020-01-14 21:02     ` Dmitry Osipenko
2020-01-15  9:18       ` Jon Hunter
2020-01-15 10:25         ` Jon Hunter
2020-01-12 17:29 ` [PATCH v4 03/14] dmaengine: tegra-apb: Prevent race conditions on channel's freeing Dmitry Osipenko
2020-01-14 15:16   ` Jon Hunter
2020-01-12 17:29 ` [PATCH v4 04/14] dmaengine: tegra-apb: Clean up tasklet releasing Dmitry Osipenko
2020-01-14 15:36   ` Jon Hunter
2020-01-12 17:29 ` [PATCH v4 05/14] dmaengine: tegra-apb: Prevent race conditions of tasklet vs free list Dmitry Osipenko
2020-01-14 15:43   ` Jon Hunter
2020-01-12 17:29 ` [PATCH v4 06/14] dmaengine: tegra-apb: Use devm_platform_ioremap_resource Dmitry Osipenko
2020-01-14 15:44   ` Jon Hunter
2020-01-12 17:29 ` [PATCH v4 07/14] dmaengine: tegra-apb: Use devm_request_irq Dmitry Osipenko
2020-01-14 15:44   ` Jon Hunter
2020-01-12 17:30 ` [PATCH v4 08/14] dmaengine: tegra-apb: Fix coding style problems Dmitry Osipenko
2020-01-15  9:49   ` Jon Hunter
2020-01-16 17:37     ` Dmitry Osipenko
2020-01-28 14:05       ` Jon Hunter
2020-01-28 15:01         ` Dmitry Osipenko
2020-01-12 17:30 ` [PATCH v4 09/14] dmaengine: tegra-apb: Clean up runtime PM teardown Dmitry Osipenko
2020-01-15  9:57   ` Jon Hunter
2020-01-16 17:18     ` Dmitry Osipenko
2020-01-12 17:30 ` [PATCH v4 10/14] dmaengine: tegra-apb: Keep clock enabled only during of DMA transfer Dmitry Osipenko
2020-01-15 10:08   ` Jon Hunter
2020-01-16 17:01     ` Dmitry Osipenko
2020-01-12 17:30 ` [PATCH v4 11/14] dmaengine: tegra-apb: Clean up suspend-resume Dmitry Osipenko
2020-01-21 21:23   ` Dmitry Osipenko
2020-01-28 14:10     ` Jon Hunter
2020-01-28 14:53       ` Dmitry Osipenko
2020-01-12 17:30 ` [PATCH v4 12/14] dmaengine: tegra-apb: Add missing of_dma_controller_free Dmitry Osipenko
2020-01-15 10:10   ` Jon Hunter
2020-01-12 17:30 ` [PATCH v4 13/14] dmaengine: tegra-apb: Allow to compile as a loadable kernel module Dmitry Osipenko
2020-01-15 10:10   ` Jon Hunter
2020-01-12 17:30 ` [PATCH v4 14/14] dmaengine: tegra-apb: Remove MODULE_ALIAS Dmitry Osipenko
2020-01-15 10:11   ` Jon Hunter

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=7c1c0672-4e6d-ae49-d041-2838199f0fcd@nvidia.com \
    --to=jonathanh@nvidia.com \
    --cc=dan.j.williams@intel.com \
    --cc=digetx@gmail.com \
    --cc=dmaengine@vger.kernel.org \
    --cc=ldewangan@nvidia.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-tegra@vger.kernel.org \
    --cc=mirq-linux@rere.qmqm.pl \
    --cc=thierry.reding@gmail.com \
    --cc=vkoul@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).