Hi,

I’m trying to enlarge buffers I use in ntb_tool by using cma:

$ dmesg | grep cma

[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-4.14.17 root=UUID=71fb5fd6-0242-444e-8e17-d31d65051f61 ro crashkernel=auto rhgb quiet cma=32G hugepages=100000 intel_pstate=disable

[    0.000000] cma: Reserved 32768 MiB at 0x000000f87fc00000

[    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-4.14.17 root=UUID=71fb5fd6-0242-444e-8e17-d31d65051f61 ro crashkernel=auto rhgb quiet cma=32G hugepages=100000 intel_pstate=disable

[    0.000000] Memory: 6057140K/1073389456K available (10252K kernel code, 2231K rwdata, 3476K rodata, 2252K init, 3040K bss, 17104596K reserved, 33554432K cma-reserved)

grep -i cma /proc/meminfo

CmaTotal:       33554432 kB

CmaFree:        33501788 kB

After loading drivers I allocate memory (uses dma_alloc_coherent):

echo 20971520 > /sys/kernel/debug/ntb_tool/0000\:3b\:00.1/peer_trans1

I get this stack:

[  313.224544] switchtec 0000:3b:00.1: swiotlb buffer is full (sz: 20971520 bytes)

[  313.224546] swiotlb: coherent allocation failed for device 0000:3b:00.1 size=20971520

[  313.224550] Call Trace:

[  313.224560]  dump_stack+0x5c/0x83

[  313.224567]  swiotlb_alloc_coherent+0x14c/0x160

[  313.224572]  tool_peer_mw_trans_write+0x1b2/0x320 [ntb_tool]

[  313.224578]  ? perf_trace_mm_compaction_migratepages+0x5a/0x120

[  313.224584]  full_proxy_write+0x5a/0xa0

[  313.224589]  __vfs_write+0x33/0x170

[  313.224594]  ? handle_mm_fault+0xc4/0x1d0

[  313.224599]  ? __audit_syscall_entry+0xaf/0x100

[  313.224601]  vfs_write+0xad/0x1a0

[  313.224603]  SyS_write+0x52/0xc0

[  313.224608]  do_syscall_64+0x61/0x1a0

[  313.224614]  entry_SYSCALL64_slow_path+0x25/0x25

[  313.224616] RIP: 0033:0x7f79ac50e840

[  313.224617] RSP: 002b:00007ffc8e89afe8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001

[  313.224619] RAX: ffffffffffffffda RBX: 0000000000000009 RCX: 00007f79ac50e840

[  313.224619] RDX: 0000000000000009 RSI: 00007f79ace2d000 RDI: 0000000000000001

[  313.224620] RBP: 00007f79ace2d000 R08: 000000000000000a R09: 00007f79ace27740

[  313.224621] R10: 00007ffc8e89ad90 R11: 0000000000000246 R12: 00007f79ac7e2400

[  313.224622] R13: 0000000000000009 R14: 0000000000000001 R15: 0000000000000000

Any help please?

--

Best regards, Boris