Async DMA operations with Xenomai 3.1

* Async DMA operations with Xenomai 3.1
@ 2021-07-07 17:19 Sylvan Williams
  2021-07-08  7:55 ` Philippe Gerum
  0 siblings, 1 reply; 2+ messages in thread
From: Sylvan Williams @ 2021-07-07 17:19 UTC (permalink / raw)
  To: xenomai

Hi everyone,

I have some questions about which approach I should take to allow OOB execution of DMA completion handlers when running kernel 4.14.110 + IPipe + Xenomai 3.1.

The system is a Zynq (dual ARM Cortex A-9), and I have a UDD driver which registers a hardware GPIO interrupt that’s incoming from the FPGA. The UDD driver also performs mmap’ing of some memory regions that are mapped to DMA, and those regions are submitted to the dmaengine for transfer into FPGA block RAM inside the interrupt handler of the UDD driver. The DMA controller also has an interrupt which is tied into the GIC to signal the completion of an operation

This works, for a while, but eventually I end up with a system hang that I’m (guessing) is due to deadlock between the head and root stages for calling the completion handler inside the xilinx-dma driver.

I stumbled upon the EVL project and Dovetail just last week, and it appears Dovetail addresses this issue with the introduction of the IRQF_OOB flag. I’m in the process of attempting to add the same functionality to IPipe, but now I’m looking for advice for the best approach since I feel like I’m about to get myself into a pretty intense code tangle.

So my question is:
Should I continue with this approach? or
Scrap it, and create an RTDM version of the xilinx DMA driver?
Is there an easier approach I haven’t thought of?

For more context, below is a snippet of the DMA transaction which is called from the udd_device.handler. Again, I believe the problem lies in the fact dmaengine_submit() and dma_async_issue_pending are called from OOB context, while the completion handler inside the xilinx DMA driver needs to run on receipt of the hardware interrupt from the DMA controller.

Also, this is a closed system, so the solution doesn’t necessarily have/need to fit into the kernel in an upstreamable fashion, if that makes sense.

There’s also the possibility that I’m completely incorrect in my theory, so no harm in pointing that out if it’s the case.

Any advice will be much appreciated!

Thanks,

Sylvan

static int dma_tx(fermi_tx_buf_t buf)
{
    int err;
    struct dma_async_tx_descriptor* txd;
    dma_addr_t dma_handle_src;
    dma_addr_t dma_handle_dest;
    size_t transaction_len;
    int active_bank;

    /* The MSB of the BRAM address bus is connected to the EMIO interface of the processor.
       Reading the value will indicate which half of the BRAM buffer is currently being written
       to the CV DAC array. Transfers need to occur into the _inactive_ buffer. Read the value
       add set the destination address accordingly
    */
    active_bank = gpio_get_value(zynq_emio_active_bank);

    /* MSB high - transfer to low buffer */
    if (active_bank == true)
    {
        dma_handle_dest = (dma_addr_t)soundengine_params_dest_addr;
    }
    /* MSB low - transfer to high buffer */
    else
    {
        dma_handle_dest = (dma_addr_t)(soundengine_params_dest_addr + soundengine_buf_len);
    }

    /* Set DDR RAM source address and transaction length depending on the value passed through the ioctl */
    if (buf == BUF_A)
    {
        transaction_len = fermi_platform_dev->tx.transaction_len;
        dma_handle_src = dma_map_single(fermi_platform_dev->tx.chan->device->dev, fermi_platform_dev->tx.dma_buf_src, transaction_len, DMA_TO_DEVICE);
    }
    else if (buf == BUF_B)
    {
        transaction_len = fermi_platform_dev->tx.transaction_len;
        dma_handle_src = dma_map_single(fermi_platform_dev->tx.chan->device->dev,
                                                         (void*)((uint8_t*)fermi_platform_dev->tx.dma_buf_src + soundengine_buf_len), transaction_len, DMA_TO_DEVICE);
    }
    else
    {
        dev_err(&fermi_platform_dev->pdev->dev, "Cyclic TX DMA - unknown argument!\n");
        goto fail;
    }

    fermi_platform_dev->tx.transaction_addr = dma_handle_src;
    fermi_platform_dev->tx.transaction_len = transaction_len;
    reinit_completion(&fermi_platform_dev->tx.cmd_complete);

    txd = dmaengine_prep_dma_memcpy(fermi_platform_dev->tx.chan, dma_handle_dest, dma_handle_src, transaction_len, DMA_PREP_INTERRUPT | DMA_CTRL_ACK);

    if (!txd)
    {
        dev_err(&fermi_platform_dev->pdev->dev, "Unable to prepare cyclic TX DMA transaction!\n");
        //dmaengine_terminate_all(fermi_platform_dev->tx.chan);
        err = -ENOMEM;
        goto fail;
    }

    txd->callback = tx_dma_callback;
    txd->callback_param = &fermi_platform_dev->tx;

    dmaengine_submit(txd);
    dma_async_issue_pending(fermi_platform_dev->tx.chan);

    return 0;

fail:
    return err;
}

^ permalink raw reply	[flat|nested] 2+ messages in thread