[U-Boot] [mips, usb xhci] Question / advice needed: cacheable adresses as a device output buffers?

[U-Boot] [mips, usb xhci] Question / advice needed: cacheable adresses as a device output buffers?

[Yuri Frolov]

Is it possible to use *cached* addresses as device's output buffer?
E.g, is it a correct thing, to read an uImage from USB pen to cacheable addresses in order to boot from these cached area?

U-Boot 2014.10-00051-g8cb056b-dirty / SDK  (May 15 2018 - 17:11:35)

CPU:   MIPS32 P5600 @ 1200 MHz (Rev 1.0)
FPU:   Present
Cores: 2 (running on CPU0)
Timer: 600 MHz
ECC:   L1 L2 (80800ff0)
PLLs:  CPU: 1200MHz  SATA: 600MHz  ETH:1250MHz
        PCIE:1200MHz  DDR3: 400MHz  AXI: 600MHz
Board: Baikal-T1 BFK3
        Watchdog enabled
I2C:   ready
DRAM:  Rank = 1 highmem = 1792 MiB lowmem = 128 MiB
NVRAM: ready
Rev:   3.1
MIPS:  SIMD ready
MIPS:  Write Merge enable
MIPS:  MAAR[0]: 0x00010000-0x07ffffff speculate
MIPS:  secondary cache 1024kB, 8-way, linesize 32 bytes.
In:    serial
Out:   serial
Err:   serial
RomID: 20ba181000002360907301001500102604164781
Net:   dwmac.bf05e000, dwmac.bf060000

First, read a file to cacheable adresses:

  # usb start; fatload usb 0:1 81000000 uimage.nfs
(Re)start USB...
USB0:   Register 1000140 NbrPorts 1
Starting the controller
USB XHCI 1.00
scanning bus 0 for devices... 2 USB Device(s) found
        scanning usb for storage devices... 1 Storage Device(s) found
reading uimage.nfs
4630013 bytes read in 2532 ms (1.7 MiB/s)
BAIKAL # md 81000000
81000000: 00000000 ffffffff 00000000 ffffffff    ................
81000010: 00000000 ffffffff 00000000 ffffffff    ................
81000020: 00000000 ffffffff 00000000 ffffffff    ................
81000030: 00000000 ffffffff 00000000 ffffffff    ................
81000040: 00000000 ffffffff 00000000 ffffffff    ................
81000050: 00000000 ffffffff 00000000 ffffffff    ................
81000060: 00000000 ffffffff 00000000 ffffffff    ................
81000070: 00000000 ffffffff 00000000 ffffffff    ................
81000080: 00000000 ffffffff 00000000 ffffffff    ................
81000090: 00000000 ffffffff 00000000 ffffffff    ................
810000a0: 00000000 ffffffff 00000000 ffffffff    ................
810000b0: 00000000 ffffffff 00000000 ffffffff    ................
810000c0: 00000000 ffffffff 00000000 ffffffff    ................
810000d0: 00000000 ffffffff 00000000 ffffffff    ................
810000e0: 00000000 ffffffff 00000000 ffffffff    ................
810000f0: 00000000 ffffffff 00000000 ffffffff    ................

Then, to the uncacheable area:

  # usb start; fatload usb 0:1 a1000000 uimage.nfs
(Re)start USB...
USB0:   Register 1000140 NbrPorts 1
Starting the controller
USB XHCI 1.00
scanning bus 0 for devices... 2 USB Device(s) found
        scanning usb for storage devices... 1 Storage Device(s) found
reading uimage.nfs
4630013 bytes read in 2531 ms (1.7 MiB/s)

BAIKAL # md a1000000
a1000000: 56190527 1697a1a2 0107d65a bda54600    '..V....Z....F..
a1000010: 00001080 d0618780 1644ad13 01020505    ......a...D.....
a1000020: 756e694c 2e342d78 31312e34 66622d38    Linux-4.4.118-bf
a1000030: 302d336b 33333335 6361672d 66623237    k3-05333-gac72bf
a1000040: 00088b1f 00000000 7dec0302 655c7c7d    ...........}}|\e
a1000050: ce6fef9d eda6499c 69da4f40 3300be99    ..o..I.. at O.i...3
a1000060: 691a49ed 880701c0 9261ca32 495d80be    .I.i....2.a...]I
a1000070: dd04505f 9794931d d5dbdc5d 72c4a2b2    _P......]......r
a1000080: 7418a648 5a2c44cc d243745d a6430294    H..t.D,Z]tC...C.
a1000090: 0af17960 74540b12 dd5652f5 12baeecb    `y....Tt.RV.....
a10000a0: 7c50575d 0a5655e9 fbf739b4 9273ce7d    ]WP|.UV..9..}.s.
a10000b0: 7ad69a69 ee3fdef7 e4f9f247 f39e7333    i..z..?.G...3s..
a10000c0: f7e7bf3c cf7fdefc f1bc88db 1bdfc6f7    <...............
a10000d0: 7faa7f7f 454b9f9b f45f9fab c9665b1e    ......KE.._..[f.
a10000e0: ae9dcaee e3cb8d95 3395f895 653f2f62    ...........3b/?e
a10000f0: d6e9dc8e f2f4b9f0 bc069153 d4f75d2f    ........S.../]..

and, exactly after loading/reading from uncached addresses:

# md 81000000
81000000: 00000000 ffffffff 00000000 ffffffff    ................
81000010: 00000000 ffffffff 00000000 ffffffff    ................
81000020: 00000000 ffffffff 00000000 ffffffff    ................
81000030: 00000000 ffffffff 00000000 ffffffff    ................
81000040: 00000000 ffffffff 00000000 ffffffff    ................
81000050: 00000000 ffffffff 00000000 ffffffff    ................
81000060: 00000000 ffffffff 00000000 ffffffff    ................
81000070: 00000000 ffffffff 00000000 ffffffff    ................
81000080: 00000000 ffffffff 00000000 ffffffff    ................
81000090: 00000000 ffffffff 00000000 ffffffff    ................
810000a0: 7c50575d 0a5655e9 fbf739b4 9273ce7d    ]WP|.UV..9..}.s.
810000b0: 7ad69a69 ee3fdef7 e4f9f247 f39e7333    i..z..?.G...3s..
810000c0: 00000000 ffffffff 00000000 ffffffff    ................
810000d0: 00000000 ffffffff 00000000 ffffffff    ................
810000e0: 00000000 ffffffff 00000000 ffffffff    ................
810000f0: 00000000 ffffffff 00000000 ffffffff    ................

 From the config for the board:

#define CONFIG_SYS_SDRAM_BASE           0x80000000
/* Only one bank on lower 512 MB can be reached without MMU */
#define CONFIG_SYS_SDRAM_SIZE           0x08000000

As far, as I can see, usb xhci is prepared to deal with uncached memory buffers:

common/usb_storage.c:

static int usb_stor_BBB_transport(ccb *srb, struct us_data *us)
{
	/* lines were deleted */
	...
	/* DATA phase + error handling */
	result = usb_bulk_msg(us->pusb_dev, pipe, srb->pdata, srb->datalen,
                               &data_actlen, USB_CNTL_TIMEOUT * 5);
         flush_cache(srb->pdata, srb->datalen); // this is the line I've added; it's not a part of official u-boot tree
	
	/* lines were deleted */
}

usb_bulk_msg -> ... -> xhci_bulk_tx (drivers/usb/host/xhci-ring.c) contains the line:

	/* flush the buffer before use */
         xhci_flush_cache((uint32_t)buffer, length);

xhci_flush_cache is:
	flush_dcache_range(addr & ~(CACHELINE_SIZE - 1),
                                 ALIGN(addr + len, CACHELINE_SIZE));