All of lore.kernel.org
 help / color / mirror / Atom feed
From: Frank Li <frank.li@nxp.com>
To: Will Deacon <will@kernel.org>
Cc: Shenwei Wang <shenwei.wang@nxp.com>, Han Xu <han.xu@nxp.com>,
	Nitin Garg <nitin.garg@nxp.com>,
	Jason Liu <jason.hui.liu@nxp.com>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>, Zhi Li <lznuaa@gmail.com>
Subject: The problem about arm64: io: Relax implicit barriers in default I/O accessors
Date: Mon, 14 Jun 2021 22:41:38 +0000	[thread overview]
Message-ID: <AS8PR04MB850017FD8F07864BB98C07D488319@AS8PR04MB8500.eurprd04.prod.outlook.com> (raw)

Will Deacon:

	Our a test case is failure at 8QM platform(arm64).  USB transfer failure if run with GPU stress test.
	I found it related with your below change. 
	
commit 22ec71615d824f4f11d38d0e55a88d8956b7e45f
Author: Will Deacon <will@kernel.org>
Date:   Fri Jun 7 15:48:58 2019 +0100

    arm64: io: Relax implicit barriers in default I/O accessors

    The arm64 implementation of the default I/O accessors requires barrier
    instructions to satisfy the memory ordering requirements documented in
    memory-barriers.txt [1], which are largely derived from the behaviour of
    I/O accesses on x86.
 
drivers/usb/host/xhci-ring.c

static void giveback_first_trb(struct xhci_hcd *xhci, int slot_id,
                unsigned int ep_index, unsigned int stream_id, int start_cycle,
                struct xhci_generic_trb *start_trb)
{
        /*
         * Pass all the TRBs to the hardware at once and make sure this write
         * isn't reordered.
         */
        wmb();
        if (start_cycle)
                start_trb->field[3] |= cpu_to_le32(start_cycle);
        else
                start_trb->field[3] &= cpu_to_le32(~TRB_CYCLE);
        xhci_ring_ep_doorbell(xhci, slot_id, ep_index, stream_id);
}

	If I added wmb() before xhci_ring_ep_doorbell, the problem gone.  Writel include io_wmb, which map into dma_wmb(). 
	
	1. write ddr
	2. writel
		2a. io_wmb(),   dmb(oshst)
		2b, write usb register
	3. usb dma read ddr.

	
	Internal bus fabric only guarantee the order for the same AXID.  1 write ddr may be slow.  USB register get data before 1 because GPU occupy ddr now.  So USB DMA start read from ddr and get old dma descriptor data and find not ready yet, then missed door bell. 

	If do 2-3 times doorbell, problem also gone.

	So I think dmb(oshst) is not enough for writel. 

       A writeX() by the CPU to the peripheral will first wait for the
        completion of all prior CPU writes to memory. For example, this ensures
        that writes by the CPU to an outbound DMA buffer allocated by
        dma_alloc_coherent() will be visible to a DMA engine when the CPU writes
        to its MMIO control register to trigger the transfer.

	
Best regards
Frank Li

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

             reply	other threads:[~2021-06-14 22:44 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-06-14 22:41 Frank Li [this message]
2021-06-16 16:27 ` Frank Li
2021-06-16 16:29   ` Frank Li
2021-06-16 18:40 ` Catalin Marinas
2021-06-16 18:55   ` Will Deacon
     [not found] <AS8PR04MB850004639EE6CE9432BBF13E880F9@AS8PR04MB8500.eurprd04.prod.outlook.com>
     [not found] ` <CAHrpEqRsp2_bt=p5JgS5F-2F_LCwgT+VX7mSENzpEYTQiW1tjg@mail.gmail.com>
2021-06-17  9:27   ` Catalin Marinas
2021-06-17 17:25     ` Will Deacon
2021-06-17 17:41       ` Will Deacon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=AS8PR04MB850017FD8F07864BB98C07D488319@AS8PR04MB8500.eurprd04.prod.outlook.com \
    --to=frank.li@nxp.com \
    --cc=han.xu@nxp.com \
    --cc=jason.hui.liu@nxp.com \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=lznuaa@gmail.com \
    --cc=nitin.garg@nxp.com \
    --cc=shenwei.wang@nxp.com \
    --cc=will@kernel.org \
    --subject='Re: The problem about arm64: io: Relax implicit barriers in default I/O accessors' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.