[PATCH rdma-next 1/2] IB/rxe: Fix kernel panic from skb destructor

[PATCH rdma-next 1/2] IB/rxe: Fix kernel panic from skb destructor

[Leon Romanovsky]

From: Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

In the time between rxe_send has finished and skb destructor
called, the QP's ref count might be 0, leading to a possible
QP destruction. This will lead to a kernel panic when the destructor
dereferences the QP.

The operation of incrementing QP ref count at rxe_send and decrementing
from skb destructor will prevent this crash.

BUG: unable to handle kernel NULL pointer dereference at 000000000000072c
IP: [<ffffffffa05df765>] rxe_skb_tx_dtor+0x15/0x50 [rdma_rxe]
PGD 0 [16240.211178]
Oops: 0002 [#1] SMP
CPU: 3 PID: 0 Comm: swapper/3 Tainted: G           OE   4.9.0-mlnx #1
Hardware name: Red Hat KVM, BIOS Bochs 01/01/2011
task: ffff88042d6b1480 task.stack: ffffc90001904000
RIP: 0010:[<ffffffffa05df765>]  [<ffffffffa05df765>] rxe_skb_tx_dtor+0x15/0x50 [rdma_rxe]
RSP: 0018:ffff88043fcc3df0  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff880429684700 RCX: ffff88042d248200
RDX: 00000000ffffffff RSI: 00000000fffffe01 RDI: ffff880429684700
RBP: ffff88043fcc3e00 R08: ffff88043fcda240 R09: 00000000ff2d1de6
R10: 0000000000000000 R11: 00000000f49cf6fe R12: ffff880429684700
R13: ffffffff81893f96 R14: ffffffff817d66f0 R15: ffff880427f74200
FS:  0000000000000000(0000) GS:ffff88043fcc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000000000072c CR3: 000000041d3df000 CR4: 00000000000006e0
Stack:
 ffffffff817b29cf ffff880429684700 ffff88043fcc3e18 ffffffff817b42c2
 ffff880429684700 ffff88043fcc3e40 ffffffff817b4332 ffff880429684700
 ffff880427f74238 ffff880427f74228 ffff88043fcc3e58 ffffffff81893f96
Call Trace:
 <IRQ> [16240.336345]  [<ffffffff817b29cf>] ? skb_release_head_state+0x4f/0xb0
 [<ffffffff817b42c2>] skb_release_all+0x12/0x30
 [<ffffffff817b4332>] kfree_skb+0x32/0x90
 [<ffffffff81893f96>] ndisc_error_report+0x36/0x40
 [<ffffffff817d4de1>] neigh_invalidate+0x81/0xf0
 [<ffffffff817d68f7>] neigh_timer_handler+0x207/0x2b0
 [<ffffffff81109295>] call_timer_fn+0x35/0x120
 [<ffffffff81109db7>] run_timer_softirq+0x1d7/0x460
 [<ffffffff8106155e>] ? kvm_sched_clock_read+0x1e/0x30
 [<ffffffff810366b9>] ? sched_clock+0x9/0x10
 [<ffffffff810cfed2>] ? sched_clock_cpu+0x72/0xa0
 [<ffffffff818dd537>] __do_softirq+0xd7/0x289
 [<ffffffff810a6c95>] irq_exit+0xb5/0xc0
 [<ffffffff818dd372>] smp_apic_timer_interrupt+0x42/0x50
 [<ffffffff818dc682>] apic_timer_interrupt+0x82/0x90
 <EOI> [16240.395776]  [<ffffffff818da156>] ? native_safe_halt+0x6/0x10
 [<ffffffff818d9e6e>] default_idle+0x1e/0xd0
 [<ffffffff8103797f>] arch_cpu_idle+0xf/0x20
 [<ffffffff818da2c5>] default_idle_call+0x35/0x40
 [<ffffffff810e3eb5>] cpu_startup_entry+0x185/0x210
 [<ffffffff81050433>] start_secondary+0x103/0x130
RIP  [<ffffffffa05df765>] rxe_skb_tx_dtor+0x15/0x50 [rdma_rxe]

Fixes: 8700e3e7c485 ("Soft RoCE driver")
Signed-off-by: Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Moni Shoua <monis-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
---
 drivers/infiniband/sw/rxe/rxe_net.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rxe/rxe_net.c
index c3a140ed4df2..08f3f90d2912 100644
--- a/drivers/infiniband/sw/rxe/rxe_net.c
+++ b/drivers/infiniband/sw/rxe/rxe_net.c
@@ -441,6 +441,8 @@ static void rxe_skb_tx_dtor(struct sk_buff *skb)
 	if (unlikely(qp->need_req_skb &&
 		     skb_out < RXE_INFLIGHT_SKBS_PER_QP_LOW))
 		rxe_run_task(&qp->req.task, 1);
+
+	rxe_drop_ref(qp);
 }

 int rxe_send(struct rxe_dev *rxe, struct rxe_pkt_info *pkt, struct sk_buff *skb)
@@ -473,6 +475,7 @@ int rxe_send(struct rxe_dev *rxe, struct rxe_pkt_info *pkt, struct sk_buff *skb)
 		return -EAGAIN;
 	}

+	rxe_add_ref(pkt->qp);
 	atomic_inc(&pkt->qp->skb_out);
 	kfree_skb(skb);

--
2.13.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-next 2/2] IB/rxe: Set dma_mask and coherent_dma_mask

[Leon Romanovsky]

From: yonatanc <yonatanc-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

The RXE coupled with dummy device causes to the kernel panic attached
below.  The panic happens when ib_register_device tries to set dma_mask
by accessing a NULLed parent device.

The RXE does not actually use DMA, so we can set the dma_mask
to architecture value.

[16240.199689] RIP: 0010:ib_register_device+0x468/0x5a0 [ib_core]
[16240.205289] RSP: 0018:ffffc9000220fc10 EFLAGS: 00010246
[16240.209909] RAX: 0000000000000024 RBX: ffff880220d1a2a8 RCX: 0000000000000000
[16240.212244] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000009
[16240.214385] RBP: ffffc9000220fcb0 R08: 0000000000000000 R09: 000000000000023f
[16240.254465] R10: 0000000000000007 R11: 0000000000000000 R12: 0000000000000000
[16240.259467] R13: 0000000000000000 R14: 0000000000000000 R15: ffff880220d1a2a8
[16240.263314] FS:  00007fd8ecca0740(0000) GS:ffff8802364c0000(0000) knlGS:0000000000000000
[16240.267292] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16240.273503] CR2: 0000000000000218 CR3: 00000002253ba000 CR4: 00000000000006e0
[16240.277066] Call Trace:
[16240.281836]  ? __kmalloc+0x26f/0x280
[16240.286596]  rxe_register_device+0x297/0x300 [rdma_rxe]
[16240.291377]  rxe_add+0x535/0x5b0 [rdma_rxe]
[16240.297586]  rxe_net_add+0x3e/0xc0 [rdma_rxe]
[16240.302375]  rxe_param_set_add+0x65/0x144 [rdma_rxe]
[16240.307769]  param_attr_store+0x68/0xd0
[16240.311640]  module_attr_store+0x1d/0x30
[16240.316421]  sysfs_kf_write+0x3a/0x50
[16240.317802]  kernfs_fop_write+0xff/0x180
[16240.322989]  __vfs_write+0x37/0x140
[16240.328164]  ? handle_mm_fault+0xce/0x240
[16240.333340]  vfs_write+0xb2/0x1b0
[16240.335013]  SyS_write+0x55/0xc0
[16240.340632]  entry_SYSCALL_64_fastpath+0x1a/0xa9

Fixes: 8700e3e7c485 ("Soft RoCE driver")
Signed-off-by: Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Moni Shoua <monis-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
---
 drivers/infiniband/sw/rxe/rxe_verbs.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c
index 83d709e74dfb..70fd060e30a7 100644
--- a/drivers/infiniband/sw/rxe/rxe_verbs.c
+++ b/drivers/infiniband/sw/rxe/rxe_verbs.c
@@ -1245,6 +1245,8 @@ int rxe_register_device(struct rxe_dev *rxe)
 	addrconf_addr_eui48((unsigned char *)&dev->node_guid,
 			    rxe->ndev->dev_addr);
 	dev->dev.dma_ops = &dma_virt_ops;
+	dma_coerce_mask_and_coherent(&dev->dev,
+				     dma_get_required_mask(dev->dev.parent));

 	dev->uverbs_abi_ver = RXE_UVERBS_ABI_VERSION;
 	dev->uverbs_cmd_mask = BIT_ULL(IB_USER_VERBS_CMD_GET_CONTEXT)
--
2.13.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-next 1/2] IB/rxe: Fix kernel panic from skb destructor

[Johannes Thumshirn]

On Thu, Jun 22, 2017 at 05:09:59PM +0300, Leon Romanovsky wrote:
> From: Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> 
> In the time between rxe_send has finished and skb destructor
> called, the QP's ref count might be 0, leading to a possible
> QP destruction. This will lead to a kernel panic when the destructor
> dereferences the QP.
> 
> The operation of incrementing QP ref count at rxe_send and decrementing
> from skb destructor will prevent this crash.
> 
> BUG: unable to handle kernel NULL pointer dereference at 000000000000072c
> IP: [<ffffffffa05df765>] rxe_skb_tx_dtor+0x15/0x50 [rdma_rxe]
> PGD 0 [16240.211178]
> Oops: 0002 [#1] SMP
> CPU: 3 PID: 0 Comm: swapper/3 Tainted: G           OE   4.9.0-mlnx #1
> Hardware name: Red Hat KVM, BIOS Bochs 01/01/2011
> task: ffff88042d6b1480 task.stack: ffffc90001904000
> RIP: 0010:[<ffffffffa05df765>]  [<ffffffffa05df765>] rxe_skb_tx_dtor+0x15/0x50 [rdma_rxe]
> RSP: 0018:ffff88043fcc3df0  EFLAGS: 00010246
> RAX: 0000000000000000 RBX: ffff880429684700 RCX: ffff88042d248200
> RDX: 00000000ffffffff RSI: 00000000fffffe01 RDI: ffff880429684700
> RBP: ffff88043fcc3e00 R08: ffff88043fcda240 R09: 00000000ff2d1de6
> R10: 0000000000000000 R11: 00000000f49cf6fe R12: ffff880429684700
> R13: ffffffff81893f96 R14: ffffffff817d66f0 R15: ffff880427f74200
> FS:  0000000000000000(0000) GS:ffff88043fcc0000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 000000000000072c CR3: 000000041d3df000 CR4: 00000000000006e0
> Stack:
>  ffffffff817b29cf ffff880429684700 ffff88043fcc3e18 ffffffff817b42c2
>  ffff880429684700 ffff88043fcc3e40 ffffffff817b4332 ffff880429684700
>  ffff880427f74238 ffff880427f74228 ffff88043fcc3e58 ffffffff81893f96
> Call Trace:
>  <IRQ> [16240.336345]  [<ffffffff817b29cf>] ? skb_release_head_state+0x4f/0xb0
>  [<ffffffff817b42c2>] skb_release_all+0x12/0x30
>  [<ffffffff817b4332>] kfree_skb+0x32/0x90
>  [<ffffffff81893f96>] ndisc_error_report+0x36/0x40
>  [<ffffffff817d4de1>] neigh_invalidate+0x81/0xf0
>  [<ffffffff817d68f7>] neigh_timer_handler+0x207/0x2b0
>  [<ffffffff81109295>] call_timer_fn+0x35/0x120
>  [<ffffffff81109db7>] run_timer_softirq+0x1d7/0x460
>  [<ffffffff8106155e>] ? kvm_sched_clock_read+0x1e/0x30
>  [<ffffffff810366b9>] ? sched_clock+0x9/0x10
>  [<ffffffff810cfed2>] ? sched_clock_cpu+0x72/0xa0
>  [<ffffffff818dd537>] __do_softirq+0xd7/0x289
>  [<ffffffff810a6c95>] irq_exit+0xb5/0xc0
>  [<ffffffff818dd372>] smp_apic_timer_interrupt+0x42/0x50
>  [<ffffffff818dc682>] apic_timer_interrupt+0x82/0x90
>  <EOI> [16240.395776]  [<ffffffff818da156>] ? native_safe_halt+0x6/0x10
>  [<ffffffff818d9e6e>] default_idle+0x1e/0xd0
>  [<ffffffff8103797f>] arch_cpu_idle+0xf/0x20
>  [<ffffffff818da2c5>] default_idle_call+0x35/0x40
>  [<ffffffff810e3eb5>] cpu_startup_entry+0x185/0x210
>  [<ffffffff81050433>] start_secondary+0x103/0x130
> RIP  [<ffffffffa05df765>] rxe_skb_tx_dtor+0x15/0x50 [rdma_rxe]
> 
> Fixes: 8700e3e7c485 ("Soft RoCE driver")
> Signed-off-by: Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> Reviewed-by: Moni Shoua <monis-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> Signed-off-by: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
> ---

Reviewed-by: Johannes Thumshirn <jthumshirn-l3A5Bk7waGM@public.gmane.org>
-- 
Johannes Thumshirn                                          Storage
jthumshirn-l3A5Bk7waGM@public.gmane.org                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-next 2/2] IB/rxe: Set dma_mask and coherent_dma_mask

[Johannes Thumshirn]

On Thu, Jun 22, 2017 at 05:10:00PM +0300, Leon Romanovsky wrote:
> From: yonatanc <yonatanc-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> 
> The RXE coupled with dummy device causes to the kernel panic attached
> below.  The panic happens when ib_register_device tries to set dma_mask
> by accessing a NULLed parent device.
> 
> The RXE does not actually use DMA, so we can set the dma_mask
> to architecture value.
> 
> [16240.199689] RIP: 0010:ib_register_device+0x468/0x5a0 [ib_core]
> [16240.205289] RSP: 0018:ffffc9000220fc10 EFLAGS: 00010246
> [16240.209909] RAX: 0000000000000024 RBX: ffff880220d1a2a8 RCX: 0000000000000000
> [16240.212244] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000009
> [16240.214385] RBP: ffffc9000220fcb0 R08: 0000000000000000 R09: 000000000000023f
> [16240.254465] R10: 0000000000000007 R11: 0000000000000000 R12: 0000000000000000
> [16240.259467] R13: 0000000000000000 R14: 0000000000000000 R15: ffff880220d1a2a8
> [16240.263314] FS:  00007fd8ecca0740(0000) GS:ffff8802364c0000(0000) knlGS:0000000000000000
> [16240.267292] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [16240.273503] CR2: 0000000000000218 CR3: 00000002253ba000 CR4: 00000000000006e0
> [16240.277066] Call Trace:
> [16240.281836]  ? __kmalloc+0x26f/0x280
> [16240.286596]  rxe_register_device+0x297/0x300 [rdma_rxe]
> [16240.291377]  rxe_add+0x535/0x5b0 [rdma_rxe]
> [16240.297586]  rxe_net_add+0x3e/0xc0 [rdma_rxe]
> [16240.302375]  rxe_param_set_add+0x65/0x144 [rdma_rxe]
> [16240.307769]  param_attr_store+0x68/0xd0
> [16240.311640]  module_attr_store+0x1d/0x30
> [16240.316421]  sysfs_kf_write+0x3a/0x50
> [16240.317802]  kernfs_fop_write+0xff/0x180
> [16240.322989]  __vfs_write+0x37/0x140
> [16240.328164]  ? handle_mm_fault+0xce/0x240
> [16240.333340]  vfs_write+0xb2/0x1b0
> [16240.335013]  SyS_write+0x55/0xc0
> [16240.340632]  entry_SYSCALL_64_fastpath+0x1a/0xa9
> 
> Fixes: 8700e3e7c485 ("Soft RoCE driver")
> Signed-off-by: Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> Reviewed-by: Moni Shoua <monis-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> Signed-off-by: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
> ---

Reviewed-by: Johannes Thumshirn <jthumshirn-l3A5Bk7waGM@public.gmane.org>
-- 
Johannes Thumshirn                                          Storage
jthumshirn-l3A5Bk7waGM@public.gmane.org                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-next 2/2] IB/rxe: Set dma_mask and coherent_dma_mask

[Yuval Shaia]

Hi Yonatan and Leon,
In one of my servers i got kernel oops also in ib_register_device when
using dummy device (macvtap) with rxe so was blindly hoping this patch
would solve it but is is not.

Crash is in alloc_name somewhere in the "list_for_each_entry" loop, i think
first line of it.

Anyway, steps that i'm doing are:
$ ip link add link eth0 name macvtap3 type macvtap mode bridge
$ modprobe ib_core ib_umad rdma_ucm ib_uverbs rdma_rxe
$ echo eth0 > /sys/module/rdma_rxe/parameters/add
$ echo macvtap3 > /sys/module/rdma_rxe/parameters/add
At this point the system crash.

I'm using 4.12.0-rc6.
This is 100% reproduced.
Interesting thing is that i'm unable to reproduce it on my workstation.

See the below kernel oops:

BUG: unable to handle kernel paging request at ffffffffa073b6db
[159135.410160] IP: report_bug+0x87/0x110
[159135.454889] PGD 1c0c067
[159135.454890] P4D 1c0c067
[159135.486112] PUD 1c0d063
[159135.517334] PMD c381c5067
[159135.548554] PTE 8000000c42ec7161
[159135.581852]
[159135.640138] Oops: 0003 [#1] SMP
[159135.678635] Modules linked in: crc32_generic(E) crc32_pclmul(E) rdma_rxe(E) udp_tunnel(E) ip6_udp_tunnel(E) ib_ipoib(E) rdma_ucm(E) ib_ucm(E) ib_uverbs(E) ib_umad(E) rdma_cm(E) ib_cm(E) iw_cm(E) mlx4_ib(E) ib_core(E) mlx4_en(E) mlx4_core(E) rds_tcp(E) rds(E) xt_REDIRECT(E) nf_nat_redirect(E) xt_nat(E) iptable_nat(E) nf_conntrack_ipv4(E) nf_defrag_ipv4(E) nf_nat_ipv4(E) nf_nat(E) nf_conntrack(E) iptable_filter(E) ip_tables(E) kvm_intel(E) kvm(E) irqbypass(E) macvtap(E) tap(E) macvlan(E) rpcsec_gss_krb5(E) auth_rpcgss(E) nfsv4(E) nfs(E) fscache(E) lockd(E) grace(E) sunrpc(E) bnx2fc(E) cnic(E) uio(E) fcoe(E) libfcoe(E) libfc(E) 8021q(E) scsi_transport_fc(E) mrp(E) garp(E) stp(E) llc(E) configfs(E) iTCO_wdt(E) iTCO_vendor_support(E) pcspkr(E) ipmi_ssif(E) ipmi_si(E) ipmi_msghandler(E) i2c_
 i801(E) lpc_ich(E)
[159136.531229]  mfd_core(E) ioatdma(E) i7core_edac(E) sg(E) acpi_cpufreq(E) igb(E) dca(E) i2c_algo_bit(E) i2c_core(E) ext4(E) mbcache(E) fscrypto(E) jbd2(E) sd_mod(E) ahci(E) libahci(E) ipv6(E) crc_ccitt(E) ptp(E) pps_core(E) megaraid_sas(E) dm_mirror(E) dm_region_hash(E) dm_log(E) dm_mod(E) [last unloaded: mlx4_core]
[159136.866231] CPU: 3 PID: 3533 Comm: bash Tainted: G            E   4.12.0-rc6.master.20170625.ol6.x86_64 #1
[159136.982848] Hardware name: Oracle Corporation SUN FIRE X4170 M2 SERVER        /ASSY,MOTHERBOARD,X4170, BIOS 08140109 12/10/2014
[159137.121282] task: ffff881843225200 task.stack: ffffc9000e160000
[159137.193126] RIP: 0010:report_bug+0x87/0x110
[159137.244188] RSP: 0018:ffffc9000e163938 EFLAGS: 00010202
[159137.307715] RAX: 0000000000000001 RBX: ffffffffa071d4e1 RCX: 0000000000000907
[159137.394202] RDX: ffffffffa073b6d1 RSI: 0000000000000000 RDI: ffffffffa071d4e1
[159137.480687] RBP: ffffc9000e163958 R08: ffffffffa073cf80 R09: ffffc9000e163908
[159137.567175] R10: ffffc9000e1638d8 R11: 00000000000008c4 R12: 000000000000015a
[159137.653660] R13: ffffffffa07376f8 R14: ffffc9000e163ac8 R15: ffff881843225200
[159137.740147] FS:  00007fe99f793700(0000) GS:ffff880c4fac0000(0000) knlGS:0000000000000000
[159137.838065] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[159137.907828] CR2: ffffffffa073b6db CR3: 000000184199c000 CR4: 00000000000006e0
[159137.994315] Call Trace:
[159138.024611]  fixup_bug+0x2e/0x50
[159138.064251]  do_trap+0x13f/0x190
[159138.103893]  do_error_trap+0xbd/0x100
[159138.148745]  ? ib_register_device+0x391/0x3a0 [ib_core]
[159138.212282]  ? kmalloc_order_trace+0x34/0xc0
[159138.264392]  ? __kmalloc+0x1cd/0x1e0
[159138.308185]  ? ttwu_do_activate+0x87/0xa0
[159138.357174]  do_invalid_op+0x20/0x30
[159138.400968]  invalid_op+0x1e/0x30
[159138.441654] RIP: 0010:ib_register_device+0x391/0x3a0 [ib_core]
[159138.512454] RSP: 0018:ffffc9000e163b78 EFLAGS: 00010246
[159138.575987] RAX: 0000000000000000 RBX: ffff880c3f8ed000 RCX: 0000000000000000
[159138.662473] RDX: ffffffffa03d3050 RSI: 0000000000000000 RDI: ffff880c3f8ed000
[159138.748959] RBP: ffffc9000e163be8 R08: ffff880c4fadf0e0 R09: ffff880c42cfa360
[159138.835446] R10: ffffc9000e163718 R11: 0000000000000000 R12: 00000000000005dc
[159138.921933] R13: 0000000000000009 R14: ffff8818431f38e0 R15: ffff881843af1a60
[159139.008430]  rxe_register_device+0x315/0x3a0 [rdma_rxe]
[159139.071963]  rxe_add+0x64/0x70 [rdma_rxe]
[159139.120950]  ? dev_get_by_name_rcu+0x76/0xa0
[159139.173054]  rxe_net_add+0x45/0xd0 [rdma_rxe]
[159139.226193]  ? _raw_spin_unlock_bh+0x1e/0x20
[159139.278299]  rxe_param_set_add+0xb5/0x1b0 [rdma_rxe]
[159139.338718]  ? path_to_nameidata+0x40/0x60
[159139.388752]  param_attr_store+0x64/0x90
[159139.435659]  module_attr_store+0x25/0x30
[159139.483610]  sysfs_kf_write+0x3e/0x40
[159139.528441]  kernfs_fop_write+0x113/0x1b0
[159139.577430]  __vfs_write+0x38/0xe0
[159139.619147]  ? filp_close+0x65/0x90
[159139.661906]  ? __getnstimeofday64+0x45/0xe0
[159139.712974]  ? do_dup2+0x99/0xe0
[159139.752614]  ? __sb_start_write+0x5e/0xc0
[159139.801602]  vfs_write+0xc1/0x130
[159139.842280]  ? __fdget+0x13/0x20
[159139.881916]  SyS_write+0x56/0xc0
[159139.921557]  do_syscall_64+0x7a/0x230
[159139.966390]  ? do_page_fault+0x37/0x90
[159140.012261]  entry_SYSCALL64_slow_path+0x25/0x25

Yuval

On Thu, Jun 22, 2017 at 05:10:00PM +0300, Leon Romanovsky wrote:
> From: yonatanc <yonatanc-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> 
> The RXE coupled with dummy device causes to the kernel panic attached
> below.  The panic happens when ib_register_device tries to set dma_mask
> by accessing a NULLed parent device.
> 
> The RXE does not actually use DMA, so we can set the dma_mask
> to architecture value.
> 
> [16240.199689] RIP: 0010:ib_register_device+0x468/0x5a0 [ib_core]
> [16240.205289] RSP: 0018:ffffc9000220fc10 EFLAGS: 00010246
> [16240.209909] RAX: 0000000000000024 RBX: ffff880220d1a2a8 RCX: 0000000000000000
> [16240.212244] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000009
> [16240.214385] RBP: ffffc9000220fcb0 R08: 0000000000000000 R09: 000000000000023f
> [16240.254465] R10: 0000000000000007 R11: 0000000000000000 R12: 0000000000000000
> [16240.259467] R13: 0000000000000000 R14: 0000000000000000 R15: ffff880220d1a2a8
> [16240.263314] FS:  00007fd8ecca0740(0000) GS:ffff8802364c0000(0000) knlGS:0000000000000000
> [16240.267292] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [16240.273503] CR2: 0000000000000218 CR3: 00000002253ba000 CR4: 00000000000006e0
> [16240.277066] Call Trace:
> [16240.281836]  ? __kmalloc+0x26f/0x280
> [16240.286596]  rxe_register_device+0x297/0x300 [rdma_rxe]
> [16240.291377]  rxe_add+0x535/0x5b0 [rdma_rxe]
> [16240.297586]  rxe_net_add+0x3e/0xc0 [rdma_rxe]
> [16240.302375]  rxe_param_set_add+0x65/0x144 [rdma_rxe]
> [16240.307769]  param_attr_store+0x68/0xd0
> [16240.311640]  module_attr_store+0x1d/0x30
> [16240.316421]  sysfs_kf_write+0x3a/0x50
> [16240.317802]  kernfs_fop_write+0xff/0x180
> [16240.322989]  __vfs_write+0x37/0x140
> [16240.328164]  ? handle_mm_fault+0xce/0x240
> [16240.333340]  vfs_write+0xb2/0x1b0
> [16240.335013]  SyS_write+0x55/0xc0
> [16240.340632]  entry_SYSCALL_64_fastpath+0x1a/0xa9
> 
> Fixes: 8700e3e7c485 ("Soft RoCE driver")
> Signed-off-by: Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> Reviewed-by: Moni Shoua <monis-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> Signed-off-by: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
> ---
>  drivers/infiniband/sw/rxe/rxe_verbs.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c
> index 83d709e74dfb..70fd060e30a7 100644
> --- a/drivers/infiniband/sw/rxe/rxe_verbs.c
> +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c
> @@ -1245,6 +1245,8 @@ int rxe_register_device(struct rxe_dev *rxe)
>  	addrconf_addr_eui48((unsigned char *)&dev->node_guid,
>  			    rxe->ndev->dev_addr);
>  	dev->dev.dma_ops = &dma_virt_ops;
> +	dma_coerce_mask_and_coherent(&dev->dev,
> +				     dma_get_required_mask(dev->dev.parent));
> 
>  	dev->uverbs_abi_ver = RXE_UVERBS_ABI_VERSION;
>  	dev->uverbs_cmd_mask = BIT_ULL(IB_USER_VERBS_CMD_GET_CONTEXT)
> --
> 2.13.1
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-next 2/2] IB/rxe: Set dma_mask and coherent_dma_mask

[Moni Shoua]

On Tue, Jun 27, 2017 at 1:04 PM, Yuval Shaia <yuval.shaia-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> wrote:
> Hi Yonatan and Leon,
> In one of my servers i got kernel oops also in ib_register_device when
> using dummy device (macvtap) with rxe so was blindly hoping this patch
> would solve it but is is not.
>
> Crash is in alloc_name somewhere in the "list_for_each_entry" loop, i think
> first line of it.
>
> Anyway, steps that i'm doing are:
> $ ip link add link eth0 name macvtap3 type macvtap mode bridge
> $ modprobe ib_core ib_umad rdma_ucm ib_uverbs rdma_rxe
> $ echo eth0 > /sys/module/rdma_rxe/parameters/add
> $ echo macvtap3 > /sys/module/rdma_rxe/parameters/add
> At this point the system crash.
>
> I'm using 4.12.0-rc6.
> This is 100% reproduced.
> Interesting thing is that i'm unable to reproduce it on my workstation.
>
thanks for the report Yuval
You said that it happens in only one of your servers but not in your
workstation, right?
Did you try to reproduce with the above scenario on different servers
with the same configuration?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-next 1/2] IB/rxe: Fix kernel panic from skb destructor

[Doug Ledford]

[-- Attachment #1.1: Type: text/plain, Size: 985 bytes --]

On 6/22/2017 10:09 AM, Leon Romanovsky wrote:
> From: Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> 
> In the time between rxe_send has finished and skb destructor
> called, the QP's ref count might be 0, leading to a possible
> QP destruction. This will lead to a kernel panic when the destructor
> dereferences the QP.
> 
> The operation of incrementing QP ref count at rxe_send and decrementing
> from skb destructor will prevent this crash.

This series has been applied to 4.13-rc, thanks.

FWIW Leon, if you have a series, even if it's a short 2 patch series, I
prefer a cover letter.  It doesn't need to be long, and can just be
something like "I have two bug fixes for rxe in this series", but I
dislike responding to a patch and saying I applied a series.

-- 
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
    GPG Key ID: B826A3330E572FDD
    Key fingerprint = AE6B 1BDA 122B 23B4 265B  1274 B826 A333 0E57 2FDD


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 884 bytes --]

Re: [PATCH rdma-next 1/2] IB/rxe: Fix kernel panic from skb destructor

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 1097 bytes --]

On Sat, Jul 22, 2017 at 01:14:26PM -0400, Doug Ledford wrote:
> On 6/22/2017 10:09 AM, Leon Romanovsky wrote:
> > From: Yonatan Cohen <yonatanc-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> >
> > In the time between rxe_send has finished and skb destructor
> > called, the QP's ref count might be 0, leading to a possible
> > QP destruction. This will lead to a kernel panic when the destructor
> > dereferences the QP.
> >
> > The operation of incrementing QP ref count at rxe_send and decrementing
> > from skb destructor will prevent this crash.
>
> This series has been applied to 4.13-rc, thanks.
>
> FWIW Leon, if you have a series, even if it's a short 2 patch series, I
> prefer a cover letter.  It doesn't need to be long, and can just be
> something like "I have two bug fixes for rxe in this series", but I
> dislike responding to a patch and saying I applied a series.

No problem, I'll do.

Thanks

>
> --
> Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>     GPG Key ID: B826A3330E572FDD
>     Key fingerprint = AE6B 1BDA 122B 23B4 265B  1274 B826 A333 0E57 2FDD
>




[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]