linux-rdma.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH rdma-next 0/3] CMA fixes
@ 2021-04-18 13:55 Leon Romanovsky
  2021-04-18 13:55 ` [PATCH rdma-next 1/3] RDMA/cma: Skip device which doesn't support CM Leon Romanovsky
                   ` (3 more replies)
  0 siblings, 4 replies; 14+ messages in thread
From: Leon Romanovsky @ 2021-04-18 13:55 UTC (permalink / raw)
  To: Doug Ledford, Jason Gunthorpe
  Cc: Leon Romanovsky, Krishna Kumar, linux-kernel, linux-rdma,
	Parav Pandit, Sean Hefty, Shay Drory

From: Leon Romanovsky <leonro@nvidia.com>

Another round of fixes to cma.c

Parav Pandit (1):
  RDMA/cma: Skip device which doesn't support CM

Shay Drory (2):
  RDMA/core: Fix check of device in rdma_listen()
  RDMA/core: Add CM to restrack after successful attachment to a device

 drivers/infiniband/core/cma.c | 28 +++++++++++++++++++++++++---
 1 file changed, 25 insertions(+), 3 deletions(-)

-- 
2.30.2


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH rdma-next 1/3] RDMA/cma: Skip device which doesn't support CM
  2021-04-18 13:55 [PATCH rdma-next 0/3] CMA fixes Leon Romanovsky
@ 2021-04-18 13:55 ` Leon Romanovsky
  2021-04-18 13:55 ` [PATCH rdma-next 2/3] RDMA/core: Fix check of device in rdma_listen() Leon Romanovsky
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 14+ messages in thread
From: Leon Romanovsky @ 2021-04-18 13:55 UTC (permalink / raw)
  To: Doug Ledford, Jason Gunthorpe; +Cc: Parav Pandit, linux-rdma

From: Parav Pandit <parav@nvidia.com>

A switchdev RDMA device do not support IB CM. When such device is added
to the RDMA CM's device list, when application invokes rdma_listen(),
cma attempts to listen to such device, however it has IB CM attribute
disabled
.
Due to this, rdma_listen() call fails to listen for other non
switchdev devices as well.

A below error message can be seen.

infiniband mlx5_0: RDMA CMA: cma_listen_on_dev, error -38

A failing call flow is below.

rdma_listen()
  cma_listen_on_all()
    cma_listen_on_dev()
      _cma_attach_to_dev()
      rdma_listen() <- fails on a specific switchdev device

Hence, when a IB device doesn't support IB CM or IW CM, avoid adding
such device to the cma list.

Signed-off-by: Parav Pandit <parav@nvidia.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
---
 drivers/infiniband/core/cma.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index bb7bd024f3bd..2dc302a83014 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -4900,6 +4900,17 @@ static void cma_process_remove(struct cma_device *cma_dev)
 	wait_for_completion(&cma_dev->comp);
 }
 
+static bool cma_supported(struct ib_device *device)
+{
+	u32 i;
+
+	rdma_for_each_port(device, i) {
+		if (rdma_cap_ib_cm(device, i) || rdma_cap_iw_cm(device, i))
+			return true;
+	}
+	return false;
+}
+
 static int cma_add_one(struct ib_device *device)
 {
 	struct rdma_id_private *to_destroy;
@@ -4909,6 +4920,9 @@ static int cma_add_one(struct ib_device *device)
 	int ret;
 	u32 i;
 
+	if (!cma_supported(device))
+		return -EOPNOTSUPP;
+
 	cma_dev = kmalloc(sizeof(*cma_dev), GFP_KERNEL);
 	if (!cma_dev)
 		return -ENOMEM;
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [PATCH rdma-next 2/3] RDMA/core: Fix check of device in rdma_listen()
  2021-04-18 13:55 [PATCH rdma-next 0/3] CMA fixes Leon Romanovsky
  2021-04-18 13:55 ` [PATCH rdma-next 1/3] RDMA/cma: Skip device which doesn't support CM Leon Romanovsky
@ 2021-04-18 13:55 ` Leon Romanovsky
  2021-04-22 11:28   ` Jason Gunthorpe
  2021-04-18 13:55 ` [PATCH rdma-next 3/3] RDMA/core: Add CM to restrack after successful attachment to a device Leon Romanovsky
  2021-04-21 23:59 ` [PATCH rdma-next 0/3] CMA fixes Jason Gunthorpe
  3 siblings, 1 reply; 14+ messages in thread
From: Leon Romanovsky @ 2021-04-18 13:55 UTC (permalink / raw)
  To: Doug Ledford, Jason Gunthorpe
  Cc: Shay Drory, Krishna Kumar, linux-rdma, Sean Hefty

From: Shay Drory <shayd@nvidia.com>

rdma_listen() checks if device already attached to rdma_id_priv,
based on the response the its decide to what to listen, however
this is different when the listeners are canceled.

This leads to a mismatch between rdma_listen() and cma_cancel_operation(),
and causes to bellow wild-memory-access. Fix it by aligning rdma_listen()
according to the cma_cancel_operation().

Found by syzcaller:
BUG: KASAN: wild-memory-access in __list_del include/linux/list.h:112 [inline]
BUG: KASAN: wild-memory-access in __list_del_entry include/linux/list.h:135 [inline]
BUG: KASAN: wild-memory-access in list_del include/linux/list.h:146 [inline]
BUG: KASAN: wild-memory-access in cma_cancel_listens drivers/infiniband/core/cma.c:1767 [inline]
BUG: KASAN: wild-memory-access in cma_cancel_operation drivers/infiniband/core/cma.c:1795 [inline]
BUG: KASAN: wild-memory-access in cma_cancel_operation+0x1f4/0x4b0 drivers/infiniband/core/cma.c:1783
Write of size 8 at addr dead000000000108 by task syz-executor716/334

CPU: 0 PID: 334 Comm: syz-executor716 Not tainted 5.11.0+ #271
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
Call Trace:
 __dump_stack lib/dump_stack.c:79 [inline]
 dump_stack+0xbe/0xf9 lib/dump_stack.c:120
 __kasan_report mm/kasan/report.c:400 [inline]
 kasan_report.cold+0x5f/0xd5 mm/kasan/report.c:413
 __list_del include/linux/list.h:112 [inline]
 __list_del_entry include/linux/list.h:135 [inline]
 list_del include/linux/list.h:146 [inline]
 cma_cancel_listens drivers/infiniband/core/cma.c:1767 [inline]
 cma_cancel_operation drivers/infiniband/core/cma.c:1795 [inline]
 cma_cancel_operation+0x1f4/0x4b0 drivers/infiniband/core/cma.c:1783
 _destroy_id+0x29/0x460 drivers/infiniband/core/cma.c:1862
 ucma_close_id+0x36/0x50 drivers/infiniband/core/ucma.c:185
 ucma_destroy_private_ctx+0x58d/0x5b0 drivers/infiniband/core/ucma.c:576
 ucma_close+0x91/0xd0 drivers/infiniband/core/ucma.c:1797
 __fput+0x169/0x540 fs/file_table.c:280
 task_work_run+0xb7/0x100 kernel/task_work.c:140
 exit_task_work include/linux/task_work.h:30 [inline]
 do_exit+0x7da/0x17f0 kernel/exit.c:825
 do_group_exit+0x9e/0x190 kernel/exit.c:922
 __do_sys_exit_group kernel/exit.c:933 [inline]
 __se_sys_exit_group kernel/exit.c:931 [inline]
 __x64_sys_exit_group+0x2d/0x30 kernel/exit.c:931
 do_syscall_64+0x2d/0x40 arch/x86/entry/common.c:46
 entry_SYSCALL_64_after_hwframe+0x44/0xa9
RIP: 0033:0x44a326
Code: Unable to access opcode bytes at RIP 0x44a2fc.
RSP: 002b:00007ffd14306748 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
RAX: ffffffffffffffda RBX: 00000000006c4490 RCX: 000000000044a326
RDX: 0000000000000001 RSI: 000000000000003c RDI: 0000000000000001
RBP: 0000000000000001 R08: 00000000000000e7 R09: ffffffffffffffc0
R10: bb1414ac000000c2 R11: 0000000000000246 R12: 00000000006c4490
R13: 0000000000000001 R14: 0000000000000000 R15: 0000000000000001
==================================================================

Fixes: 255d0c14b375 ("RDMA/cma: rdma_bind_addr() leaks a cma_dev reference count")
Signed-off-by: Shay Drory <shayd@nvidia.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
---
 drivers/infiniband/core/cma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 2dc302a83014..cc990adaf2b5 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -3768,7 +3768,7 @@ int rdma_listen(struct rdma_cm_id *id, int backlog)
 	}
 
 	id_priv->backlog = backlog;
-	if (id->device) {
+	if (id_priv->cma_dev) {
 		if (rdma_cap_ib_cm(id->device, 1)) {
 			ret = cma_ib_listen(id_priv);
 			if (ret)
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [PATCH rdma-next 3/3] RDMA/core: Add CM to restrack after successful attachment to a device
  2021-04-18 13:55 [PATCH rdma-next 0/3] CMA fixes Leon Romanovsky
  2021-04-18 13:55 ` [PATCH rdma-next 1/3] RDMA/cma: Skip device which doesn't support CM Leon Romanovsky
  2021-04-18 13:55 ` [PATCH rdma-next 2/3] RDMA/core: Fix check of device in rdma_listen() Leon Romanovsky
@ 2021-04-18 13:55 ` Leon Romanovsky
  2021-04-21 23:59 ` [PATCH rdma-next 0/3] CMA fixes Jason Gunthorpe
  3 siblings, 0 replies; 14+ messages in thread
From: Leon Romanovsky @ 2021-04-18 13:55 UTC (permalink / raw)
  To: Doug Ledford, Jason Gunthorpe; +Cc: Shay Drory, linux-rdma

From: Shay Drory <shayd@nvidia.com>

The device attach triggers addition of CM_ID to the restrack DB.
However, when error occurs, we releasing this device, but defer CM_ID
release. This causes to the situation where restrack sees CM_ID that
is not valid anymore.

As a solution, add the CM_ID to the resource tracking DB only after
the attachment is finished.

Found by syzcaller:
infiniband syz0: added syz_tun
rdma_rxe: ignoring netdev event = 10 for syz_tun
infiniband syz0: set down
infiniband syz0: ib_query_port failed (-19)
restrack: ------------[ cut here    ]------------
infiniband syz0: BUG: RESTRACK detected leak of resources
restrack: User CM_ID object allocated by syz-executor716 is not freed
restrack: ------------[ cut here    ]------------

Fixes: b09c4d701220 ("RDMA/restrack: Improve readability in task name management")
Signed-off-by: Shay Drory <shayd@nvidia.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
---
 drivers/infiniband/core/cma.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index cc990adaf2b5..1db2279cff18 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -454,7 +454,6 @@ static void _cma_attach_to_dev(struct rdma_id_private *id_priv,
 	id_priv->id.route.addr.dev_addr.transport =
 		rdma_node_get_transport(cma_dev->device->node_type);
 	list_add_tail(&id_priv->list, &cma_dev->id_list);
-	rdma_restrack_add(&id_priv->res);
 
 	trace_cm_id_attach(id_priv, cma_dev->device);
 }
@@ -691,6 +690,7 @@ static int cma_ib_acquire_dev(struct rdma_id_private *id_priv,
 	mutex_lock(&lock);
 	cma_attach_to_dev(id_priv, listen_id_priv->cma_dev);
 	mutex_unlock(&lock);
+	rdma_restrack_add(&id_priv->res);
 	return 0;
 }
 
@@ -745,8 +745,10 @@ static int cma_iw_acquire_dev(struct rdma_id_private *id_priv,
 	}
 
 out:
-	if (!ret)
+	if (!ret) {
 		cma_attach_to_dev(id_priv, cma_dev);
+		rdma_restrack_add(&id_priv->res);
+	}
 
 	mutex_unlock(&lock);
 	return ret;
@@ -807,6 +809,7 @@ static int cma_resolve_ib_dev(struct rdma_id_private *id_priv)
 
 found:
 	cma_attach_to_dev(id_priv, cma_dev);
+	rdma_restrack_add(&id_priv->res);
 	mutex_unlock(&lock);
 	addr = (struct sockaddr_ib *)cma_src_addr(id_priv);
 	memcpy(&addr->sib_addr, &sgid, sizeof(sgid));
@@ -2526,6 +2529,7 @@ static int cma_listen_on_dev(struct rdma_id_private *id_priv,
 	       rdma_addr_size(cma_src_addr(id_priv)));
 
 	_cma_attach_to_dev(dev_id_priv, cma_dev);
+	rdma_restrack_add(&dev_id_priv->res);
 	cma_id_get(id_priv);
 	dev_id_priv->internal_id = 1;
 	dev_id_priv->afonly = id_priv->afonly;
@@ -3203,6 +3207,7 @@ static int cma_bind_loopback(struct rdma_id_private *id_priv)
 	ib_addr_set_pkey(&id_priv->id.route.addr.dev_addr, pkey);
 	id_priv->id.port_num = p;
 	cma_attach_to_dev(id_priv, cma_dev);
+	rdma_restrack_add(&id_priv->res);
 	cma_set_loopback(cma_src_addr(id_priv));
 out:
 	mutex_unlock(&lock);
@@ -3235,6 +3240,7 @@ static void addr_handler(int status, struct sockaddr *src_addr,
 		if (status)
 			pr_debug_ratelimited("RDMA CM: ADDR_ERROR: failed to acquire device. status %d\n",
 					     status);
+		rdma_restrack_add(&id_priv->res);
 	} else if (status) {
 		pr_debug_ratelimited("RDMA CM: ADDR_ERROR: failed to resolve IP. status %d\n", status);
 	}
@@ -3846,6 +3852,8 @@ int rdma_bind_addr(struct rdma_cm_id *id, struct sockaddr *addr)
 	if (ret)
 		goto err2;
 
+	if (!cma_any_addr(addr))
+		rdma_restrack_add(&id_priv->res);
 	return 0;
 err2:
 	if (id_priv->cma_dev)
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* Re: [PATCH rdma-next 0/3] CMA fixes
  2021-04-18 13:55 [PATCH rdma-next 0/3] CMA fixes Leon Romanovsky
                   ` (2 preceding siblings ...)
  2021-04-18 13:55 ` [PATCH rdma-next 3/3] RDMA/core: Add CM to restrack after successful attachment to a device Leon Romanovsky
@ 2021-04-21 23:59 ` Jason Gunthorpe
  2021-04-22  6:46   ` Leon Romanovsky
  3 siblings, 1 reply; 14+ messages in thread
From: Jason Gunthorpe @ 2021-04-21 23:59 UTC (permalink / raw)
  To: Leon Romanovsky
  Cc: Doug Ledford, Leon Romanovsky, Krishna Kumar, linux-kernel,
	linux-rdma, Parav Pandit, Sean Hefty, Shay Drory

On Sun, Apr 18, 2021 at 04:55:51PM +0300, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro@nvidia.com>
> 
> Another round of fixes to cma.c
> 
> Parav Pandit (1):
>   RDMA/cma: Skip device which doesn't support CM
> 
> Shay Drory (2):
>   RDMA/core: Add CM to restrack after successful attachment to a device

These two applied to for-next

>   RDMA/core: Fix check of device in rdma_listen()

Lets think about this one some more

Thanks,
Jason

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH rdma-next 0/3] CMA fixes
  2021-04-21 23:59 ` [PATCH rdma-next 0/3] CMA fixes Jason Gunthorpe
@ 2021-04-22  6:46   ` Leon Romanovsky
  0 siblings, 0 replies; 14+ messages in thread
From: Leon Romanovsky @ 2021-04-22  6:46 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Doug Ledford, Krishna Kumar, linux-kernel, linux-rdma,
	Parav Pandit, Sean Hefty, Shay Drory

On Wed, Apr 21, 2021 at 08:59:09PM -0300, Jason Gunthorpe wrote:
> On Sun, Apr 18, 2021 at 04:55:51PM +0300, Leon Romanovsky wrote:
> > From: Leon Romanovsky <leonro@nvidia.com>
> > 
> > Another round of fixes to cma.c
> > 
> > Parav Pandit (1):
> >   RDMA/cma: Skip device which doesn't support CM
> > 
> > Shay Drory (2):
> >   RDMA/core: Add CM to restrack after successful attachment to a device
> 
> These two applied to for-next
> 
> >   RDMA/core: Fix check of device in rdma_listen()
> 
> Lets think about this one some more

Sure, thanks

> 
> Thanks,
> Jason

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH rdma-next 2/3] RDMA/core: Fix check of device in rdma_listen()
  2021-04-18 13:55 ` [PATCH rdma-next 2/3] RDMA/core: Fix check of device in rdma_listen() Leon Romanovsky
@ 2021-04-22 11:28   ` Jason Gunthorpe
  2021-04-22 12:44     ` Shay Drory
  0 siblings, 1 reply; 14+ messages in thread
From: Jason Gunthorpe @ 2021-04-22 11:28 UTC (permalink / raw)
  To: Leon Romanovsky
  Cc: Doug Ledford, Shay Drory, Krishna Kumar, linux-rdma, Sean Hefty

On Sun, Apr 18, 2021 at 04:55:53PM +0300, Leon Romanovsky wrote:
> From: Shay Drory <shayd@nvidia.com>
> 
> rdma_listen() checks if device already attached to rdma_id_priv,
> based on the response the its decide to what to listen, however
> this is different when the listeners are canceled.
> 
> This leads to a mismatch between rdma_listen() and cma_cancel_operation(),
> and causes to bellow wild-memory-access. Fix it by aligning rdma_listen()
> according to the cma_cancel_operation().

So this is happening because the error unwind in rdma_bind_addr() is
taking the exit path and calling cma_release_dev()?

This allows rdma_listen() to be called with a bogus device pointer
which precipitates this UAF during destroy.

However, I think rdma_bind_addr() should not allow the bogus device
pointer to leak out at all, since the ULP could see it. It really is
invalid to have it present no matter what.

This would make cma_release_dev() and _cma_attach_to_dev()
symmetrical - what do you think?

diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 2dc302a83014ae..91f6d968b46f65 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -474,6 +474,7 @@ static void cma_release_dev(struct rdma_id_private *id_priv)
 	list_del(&id_priv->list);
 	cma_dev_put(id_priv->cma_dev);
 	id_priv->cma_dev = NULL;
+	id_priv->id.device = NULL;
 	if (id_priv->id.route.addr.dev_addr.sgid_attr) {
 		rdma_put_gid_attr(id_priv->id.route.addr.dev_addr.sgid_attr);
 		id_priv->id.route.addr.dev_addr.sgid_attr = NULL;

^ permalink raw reply related	[flat|nested] 14+ messages in thread

* Re: [PATCH rdma-next 2/3] RDMA/core: Fix check of device in rdma_listen()
  2021-04-22 11:28   ` Jason Gunthorpe
@ 2021-04-22 12:44     ` Shay Drory
  2021-04-22 12:51       ` Jason Gunthorpe
  0 siblings, 1 reply; 14+ messages in thread
From: Shay Drory @ 2021-04-22 12:44 UTC (permalink / raw)
  To: Jason Gunthorpe, Leon Romanovsky
  Cc: Doug Ledford, Krishna Kumar, linux-rdma, Sean Hefty

On 4/22/2021 14:28, Jason Gunthorpe wrote:

> On Sun, Apr 18, 2021 at 04:55:53PM +0300, Leon Romanovsky wrote:
>> From: Shay Drory <shayd@nvidia.com>
>>
>> rdma_listen() checks if device already attached to rdma_id_priv,
>> based on the response the its decide to what to listen, however
>> this is different when the listeners are canceled.
>>
>> This leads to a mismatch between rdma_listen() and cma_cancel_operation(),
>> and causes to bellow wild-memory-access. Fix it by aligning rdma_listen()
>> according to the cma_cancel_operation().
> So this is happening because the error unwind in rdma_bind_addr() is
> taking the exit path and calling cma_release_dev()?
>
> This allows rdma_listen() to be called with a bogus device pointer
> which precipitates this UAF during destroy.
>
> However, I think rdma_bind_addr() should not allow the bogus device
> pointer to leak out at all, since the ULP could see it. It really is
> invalid to have it present no matter what.
>
> This would make cma_release_dev() and _cma_attach_to_dev()
> symmetrical - what do you think?
>
> diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
> index 2dc302a83014ae..91f6d968b46f65 100644
> --- a/drivers/infiniband/core/cma.c
> +++ b/drivers/infiniband/core/cma.c
> @@ -474,6 +474,7 @@ static void cma_release_dev(struct rdma_id_private *id_priv)
>   	list_del(&id_priv->list);
>   	cma_dev_put(id_priv->cma_dev);
>   	id_priv->cma_dev = NULL;
> +	id_priv->id.device = NULL;
>   	if (id_priv->id.route.addr.dev_addr.sgid_attr) {
>   		rdma_put_gid_attr(id_priv->id.route.addr.dev_addr.sgid_attr);
>   		id_priv->id.route.addr.dev_addr.sgid_attr = NULL;

I try that. this will break restrack_del() since restrack_del() is using id_priv->id.device
and is being called before restrack_del():
_destroy_id <https://elixir.bootlin.com/linux/v5.11/C/ident/_destroy_id>() {
if (id_priv 
<https://elixir.bootlin.com/linux/v5.11/C/ident/id_priv>->cma_dev 
<https://elixir.bootlin.com/linux/v5.11/C/ident/cma_dev>) 
cma_release_dev 
<https://elixir.bootlin.com/linux/v5.11/C/ident/cma_release_dev>(id_priv 
<https://elixir.bootlin.com/linux/v5.11/C/ident/id_priv>); ... 
rdma_restrack_del 
<https://elixir.bootlin.com/linux/v5.11/C/ident/rdma_restrack_del>(&id_priv 
<https://elixir.bootlin.com/linux/v5.11/C/ident/id_priv>->res); }  


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH rdma-next 2/3] RDMA/core: Fix check of device in rdma_listen()
  2021-04-22 12:44     ` Shay Drory
@ 2021-04-22 12:51       ` Jason Gunthorpe
  2021-04-22 13:01         ` Leon Romanovsky
  0 siblings, 1 reply; 14+ messages in thread
From: Jason Gunthorpe @ 2021-04-22 12:51 UTC (permalink / raw)
  To: Shay Drory
  Cc: Leon Romanovsky, Doug Ledford, Krishna Kumar, linux-rdma, Sean Hefty

On Thu, Apr 22, 2021 at 03:44:55PM +0300, Shay Drory wrote:
> On 4/22/2021 14:28, Jason Gunthorpe wrote:
> 
> > On Sun, Apr 18, 2021 at 04:55:53PM +0300, Leon Romanovsky wrote:
> > > From: Shay Drory <shayd@nvidia.com>
> > > 
> > > rdma_listen() checks if device already attached to rdma_id_priv,
> > > based on the response the its decide to what to listen, however
> > > this is different when the listeners are canceled.
> > > 
> > > This leads to a mismatch between rdma_listen() and cma_cancel_operation(),
> > > and causes to bellow wild-memory-access. Fix it by aligning rdma_listen()
> > > according to the cma_cancel_operation().
> > So this is happening because the error unwind in rdma_bind_addr() is
> > taking the exit path and calling cma_release_dev()?
> > 
> > This allows rdma_listen() to be called with a bogus device pointer
> > which precipitates this UAF during destroy.
> > 
> > However, I think rdma_bind_addr() should not allow the bogus device
> > pointer to leak out at all, since the ULP could see it. It really is
> > invalid to have it present no matter what.
> > 
> > This would make cma_release_dev() and _cma_attach_to_dev()
> > symmetrical - what do you think?
> > 
> > diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
> > index 2dc302a83014ae..91f6d968b46f65 100644
> > +++ b/drivers/infiniband/core/cma.c
> > @@ -474,6 +474,7 @@ static void cma_release_dev(struct rdma_id_private *id_priv)
> >   	list_del(&id_priv->list);
> >   	cma_dev_put(id_priv->cma_dev);
> >   	id_priv->cma_dev = NULL;
> > +	id_priv->id.device = NULL;
> >   	if (id_priv->id.route.addr.dev_addr.sgid_attr) {
> >   		rdma_put_gid_attr(id_priv->id.route.addr.dev_addr.sgid_attr);
> >   		id_priv->id.route.addr.dev_addr.sgid_attr = NULL;
> 
> I try that. this will break restrack_del() since restrack_del() is
> using id_priv->id.device and is being called before restrack_del():

Oh that is another bug, once cma_release_dev() is called there is no
refcount protecting the id.device and any access to it is invalid.

The order of rdma_restrack_del should be moved to be ahead of the
cma_release_dev, and we also can't have a restrack without a cma_dev
in the first place

Jason

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH rdma-next 2/3] RDMA/core: Fix check of device in rdma_listen()
  2021-04-22 12:51       ` Jason Gunthorpe
@ 2021-04-22 13:01         ` Leon Romanovsky
  2021-04-22 13:02           ` Jason Gunthorpe
  0 siblings, 1 reply; 14+ messages in thread
From: Leon Romanovsky @ 2021-04-22 13:01 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Shay Drory, Doug Ledford, Krishna Kumar, linux-rdma, Sean Hefty

On Thu, Apr 22, 2021 at 09:51:35AM -0300, Jason Gunthorpe wrote:
> On Thu, Apr 22, 2021 at 03:44:55PM +0300, Shay Drory wrote:
> > On 4/22/2021 14:28, Jason Gunthorpe wrote:
> > 
> > > On Sun, Apr 18, 2021 at 04:55:53PM +0300, Leon Romanovsky wrote:
> > > > From: Shay Drory <shayd@nvidia.com>
> > > > 
> > > > rdma_listen() checks if device already attached to rdma_id_priv,
> > > > based on the response the its decide to what to listen, however
> > > > this is different when the listeners are canceled.
> > > > 
> > > > This leads to a mismatch between rdma_listen() and cma_cancel_operation(),
> > > > and causes to bellow wild-memory-access. Fix it by aligning rdma_listen()
> > > > according to the cma_cancel_operation().
> > > So this is happening because the error unwind in rdma_bind_addr() is
> > > taking the exit path and calling cma_release_dev()?
> > > 
> > > This allows rdma_listen() to be called with a bogus device pointer
> > > which precipitates this UAF during destroy.
> > > 
> > > However, I think rdma_bind_addr() should not allow the bogus device
> > > pointer to leak out at all, since the ULP could see it. It really is
> > > invalid to have it present no matter what.
> > > 
> > > This would make cma_release_dev() and _cma_attach_to_dev()
> > > symmetrical - what do you think?
> > > 
> > > diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
> > > index 2dc302a83014ae..91f6d968b46f65 100644
> > > +++ b/drivers/infiniband/core/cma.c
> > > @@ -474,6 +474,7 @@ static void cma_release_dev(struct rdma_id_private *id_priv)
> > >   	list_del(&id_priv->list);
> > >   	cma_dev_put(id_priv->cma_dev);
> > >   	id_priv->cma_dev = NULL;
> > > +	id_priv->id.device = NULL;
> > >   	if (id_priv->id.route.addr.dev_addr.sgid_attr) {
> > >   		rdma_put_gid_attr(id_priv->id.route.addr.dev_addr.sgid_attr);
> > >   		id_priv->id.route.addr.dev_addr.sgid_attr = NULL;
> > 
> > I try that. this will break restrack_del() since restrack_del() is
> > using id_priv->id.device and is being called before restrack_del():
> 
> Oh that is another bug, once cma_release_dev() is called there is no
> refcount protecting the id.device and any access to it is invalid.
> 
> The order of rdma_restrack_del should be moved to be ahead of the
> cma_release_dev, and we also can't have a restrack without a cma_dev
> in the first place

We have restrack per-cmd_id and not per-cma_dev.

> 
> Jason

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH rdma-next 2/3] RDMA/core: Fix check of device in rdma_listen()
  2021-04-22 13:01         ` Leon Romanovsky
@ 2021-04-22 13:02           ` Jason Gunthorpe
  2021-04-22 14:58             ` Leon Romanovsky
  0 siblings, 1 reply; 14+ messages in thread
From: Jason Gunthorpe @ 2021-04-22 13:02 UTC (permalink / raw)
  To: Leon Romanovsky
  Cc: Shay Drory, Doug Ledford, Krishna Kumar, linux-rdma, Sean Hefty

On Thu, Apr 22, 2021 at 04:01:20PM +0300, Leon Romanovsky wrote:
> On Thu, Apr 22, 2021 at 09:51:35AM -0300, Jason Gunthorpe wrote:
> > On Thu, Apr 22, 2021 at 03:44:55PM +0300, Shay Drory wrote:
> > > On 4/22/2021 14:28, Jason Gunthorpe wrote:
> > > 
> > > > On Sun, Apr 18, 2021 at 04:55:53PM +0300, Leon Romanovsky wrote:
> > > > > From: Shay Drory <shayd@nvidia.com>
> > > > > 
> > > > > rdma_listen() checks if device already attached to rdma_id_priv,
> > > > > based on the response the its decide to what to listen, however
> > > > > this is different when the listeners are canceled.
> > > > > 
> > > > > This leads to a mismatch between rdma_listen() and cma_cancel_operation(),
> > > > > and causes to bellow wild-memory-access. Fix it by aligning rdma_listen()
> > > > > according to the cma_cancel_operation().
> > > > So this is happening because the error unwind in rdma_bind_addr() is
> > > > taking the exit path and calling cma_release_dev()?
> > > > 
> > > > This allows rdma_listen() to be called with a bogus device pointer
> > > > which precipitates this UAF during destroy.
> > > > 
> > > > However, I think rdma_bind_addr() should not allow the bogus device
> > > > pointer to leak out at all, since the ULP could see it. It really is
> > > > invalid to have it present no matter what.
> > > > 
> > > > This would make cma_release_dev() and _cma_attach_to_dev()
> > > > symmetrical - what do you think?
> > > > 
> > > > diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
> > > > index 2dc302a83014ae..91f6d968b46f65 100644
> > > > +++ b/drivers/infiniband/core/cma.c
> > > > @@ -474,6 +474,7 @@ static void cma_release_dev(struct rdma_id_private *id_priv)
> > > >   	list_del(&id_priv->list);
> > > >   	cma_dev_put(id_priv->cma_dev);
> > > >   	id_priv->cma_dev = NULL;
> > > > +	id_priv->id.device = NULL;
> > > >   	if (id_priv->id.route.addr.dev_addr.sgid_attr) {
> > > >   		rdma_put_gid_attr(id_priv->id.route.addr.dev_addr.sgid_attr);
> > > >   		id_priv->id.route.addr.dev_addr.sgid_attr = NULL;
> > > 
> > > I try that. this will break restrack_del() since restrack_del() is
> > > using id_priv->id.device and is being called before restrack_del():
> > 
> > Oh that is another bug, once cma_release_dev() is called there is no
> > refcount protecting the id.device and any access to it is invalid.
> > 
> > The order of rdma_restrack_del should be moved to be ahead of the
> > cma_release_dev, and we also can't have a restrack without a cma_dev
> > in the first place
> 
> We have restrack per-cmd_id and not per-cma_dev.

No, restrack has this:

	dev = res_to_dev(res);
	if (WARN_ON(!dev))

And here dev will be NULL if cma_dev isn't set

Jasno

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH rdma-next 2/3] RDMA/core: Fix check of device in rdma_listen()
  2021-04-22 13:02           ` Jason Gunthorpe
@ 2021-04-22 14:58             ` Leon Romanovsky
  2021-04-22 16:18               ` Jason Gunthorpe
  0 siblings, 1 reply; 14+ messages in thread
From: Leon Romanovsky @ 2021-04-22 14:58 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Shay Drory, Doug Ledford, Krishna Kumar, linux-rdma, Sean Hefty

On Thu, Apr 22, 2021 at 10:02:18AM -0300, Jason Gunthorpe wrote:
> On Thu, Apr 22, 2021 at 04:01:20PM +0300, Leon Romanovsky wrote:
> > On Thu, Apr 22, 2021 at 09:51:35AM -0300, Jason Gunthorpe wrote:
> > > On Thu, Apr 22, 2021 at 03:44:55PM +0300, Shay Drory wrote:
> > > > On 4/22/2021 14:28, Jason Gunthorpe wrote:
> > > > 
> > > > > On Sun, Apr 18, 2021 at 04:55:53PM +0300, Leon Romanovsky wrote:
> > > > > > From: Shay Drory <shayd@nvidia.com>
> > > > > > 
> > > > > > rdma_listen() checks if device already attached to rdma_id_priv,
> > > > > > based on the response the its decide to what to listen, however
> > > > > > this is different when the listeners are canceled.
> > > > > > 
> > > > > > This leads to a mismatch between rdma_listen() and cma_cancel_operation(),
> > > > > > and causes to bellow wild-memory-access. Fix it by aligning rdma_listen()
> > > > > > according to the cma_cancel_operation().
> > > > > So this is happening because the error unwind in rdma_bind_addr() is
> > > > > taking the exit path and calling cma_release_dev()?
> > > > > 
> > > > > This allows rdma_listen() to be called with a bogus device pointer
> > > > > which precipitates this UAF during destroy.
> > > > > 
> > > > > However, I think rdma_bind_addr() should not allow the bogus device
> > > > > pointer to leak out at all, since the ULP could see it. It really is
> > > > > invalid to have it present no matter what.
> > > > > 
> > > > > This would make cma_release_dev() and _cma_attach_to_dev()
> > > > > symmetrical - what do you think?
> > > > > 
> > > > > diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
> > > > > index 2dc302a83014ae..91f6d968b46f65 100644
> > > > > +++ b/drivers/infiniband/core/cma.c
> > > > > @@ -474,6 +474,7 @@ static void cma_release_dev(struct rdma_id_private *id_priv)
> > > > >   	list_del(&id_priv->list);
> > > > >   	cma_dev_put(id_priv->cma_dev);
> > > > >   	id_priv->cma_dev = NULL;
> > > > > +	id_priv->id.device = NULL;
> > > > >   	if (id_priv->id.route.addr.dev_addr.sgid_attr) {
> > > > >   		rdma_put_gid_attr(id_priv->id.route.addr.dev_addr.sgid_attr);
> > > > >   		id_priv->id.route.addr.dev_addr.sgid_attr = NULL;
> > > > 
> > > > I try that. this will break restrack_del() since restrack_del() is
> > > > using id_priv->id.device and is being called before restrack_del():
> > > 
> > > Oh that is another bug, once cma_release_dev() is called there is no
> > > refcount protecting the id.device and any access to it is invalid.
> > > 
> > > The order of rdma_restrack_del should be moved to be ahead of the
> > > cma_release_dev, and we also can't have a restrack without a cma_dev
> > > in the first place
> > 
> > We have restrack per-cmd_id and not per-cma_dev.
> 
> No, restrack has this:
> 
> 	dev = res_to_dev(res);
> 	if (WARN_ON(!dev))
> 
> And here dev will be NULL if cma_dev isn't set

  127 static struct ib_device *res_to_dev(struct rdma_restrack_entry *res)
  128 {

<...>

  136         case RDMA_RESTRACK_CM_ID:
  137                 return container_of(res, struct rdma_id_private,
  138                                     res)->id.device;
                                                ^^^^^ it is not cma_dev
Thanks

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH rdma-next 2/3] RDMA/core: Fix check of device in rdma_listen()
  2021-04-22 14:58             ` Leon Romanovsky
@ 2021-04-22 16:18               ` Jason Gunthorpe
  2021-04-25 13:17                 ` Leon Romanovsky
  0 siblings, 1 reply; 14+ messages in thread
From: Jason Gunthorpe @ 2021-04-22 16:18 UTC (permalink / raw)
  To: Leon Romanovsky
  Cc: Shay Drory, Doug Ledford, Krishna Kumar, linux-rdma, Sean Hefty

On Thu, Apr 22, 2021 at 05:58:18PM +0300, Leon Romanovsky wrote:
> On Thu, Apr 22, 2021 at 10:02:18AM -0300, Jason Gunthorpe wrote:
> > On Thu, Apr 22, 2021 at 04:01:20PM +0300, Leon Romanovsky wrote:
> > > On Thu, Apr 22, 2021 at 09:51:35AM -0300, Jason Gunthorpe wrote:
> > > > On Thu, Apr 22, 2021 at 03:44:55PM +0300, Shay Drory wrote:
> > > > > On 4/22/2021 14:28, Jason Gunthorpe wrote:
> > > > > 
> > > > > > On Sun, Apr 18, 2021 at 04:55:53PM +0300, Leon Romanovsky wrote:
> > > > > > > From: Shay Drory <shayd@nvidia.com>
> > > > > > > 
> > > > > > > rdma_listen() checks if device already attached to rdma_id_priv,
> > > > > > > based on the response the its decide to what to listen, however
> > > > > > > this is different when the listeners are canceled.
> > > > > > > 
> > > > > > > This leads to a mismatch between rdma_listen() and cma_cancel_operation(),
> > > > > > > and causes to bellow wild-memory-access. Fix it by aligning rdma_listen()
> > > > > > > according to the cma_cancel_operation().
> > > > > > So this is happening because the error unwind in rdma_bind_addr() is
> > > > > > taking the exit path and calling cma_release_dev()?
> > > > > > 
> > > > > > This allows rdma_listen() to be called with a bogus device pointer
> > > > > > which precipitates this UAF during destroy.
> > > > > > 
> > > > > > However, I think rdma_bind_addr() should not allow the bogus device
> > > > > > pointer to leak out at all, since the ULP could see it. It really is
> > > > > > invalid to have it present no matter what.
> > > > > > 
> > > > > > This would make cma_release_dev() and _cma_attach_to_dev()
> > > > > > symmetrical - what do you think?
> > > > > > 
> > > > > > diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
> > > > > > index 2dc302a83014ae..91f6d968b46f65 100644
> > > > > > +++ b/drivers/infiniband/core/cma.c
> > > > > > @@ -474,6 +474,7 @@ static void cma_release_dev(struct rdma_id_private *id_priv)
> > > > > >   	list_del(&id_priv->list);
> > > > > >   	cma_dev_put(id_priv->cma_dev);
> > > > > >   	id_priv->cma_dev = NULL;
> > > > > > +	id_priv->id.device = NULL;
> > > > > >   	if (id_priv->id.route.addr.dev_addr.sgid_attr) {
> > > > > >   		rdma_put_gid_attr(id_priv->id.route.addr.dev_addr.sgid_attr);
> > > > > >   		id_priv->id.route.addr.dev_addr.sgid_attr = NULL;
> > > > > 
> > > > > I try that. this will break restrack_del() since restrack_del() is
> > > > > using id_priv->id.device and is being called before restrack_del():
> > > > 
> > > > Oh that is another bug, once cma_release_dev() is called there is no
> > > > refcount protecting the id.device and any access to it is invalid.
> > > > 
> > > > The order of rdma_restrack_del should be moved to be ahead of the
> > > > cma_release_dev, and we also can't have a restrack without a cma_dev
> > > > in the first place
> > > 
> > > We have restrack per-cmd_id and not per-cma_dev.
> > 
> > No, restrack has this:
> > 
> > 	dev = res_to_dev(res);
> > 	if (WARN_ON(!dev))
> > 
> > And here dev will be NULL if cma_dev isn't set
> 
>   127 static struct ib_device *res_to_dev(struct rdma_restrack_entry *res)
>   128 {
> 
> <...>
> 
>   136         case RDMA_RESTRACK_CM_ID:
>   137                 return container_of(res, struct rdma_id_private,
>   138                                     res)->id.device;
>                                                 ^^^^^ it is not cma_dev

The invariant is that 

   priv.id.device == priv.cma_dev->device

(this de-normalization of data exists only to allow priv to be in a
private header)

If cma_dev == NULL then id.device == NULL as cma_Dev was the thing
preventing the pointer from being free'd.

Jason

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH rdma-next 2/3] RDMA/core: Fix check of device in rdma_listen()
  2021-04-22 16:18               ` Jason Gunthorpe
@ 2021-04-25 13:17                 ` Leon Romanovsky
  0 siblings, 0 replies; 14+ messages in thread
From: Leon Romanovsky @ 2021-04-25 13:17 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Shay Drory, Doug Ledford, Krishna Kumar, linux-rdma, Sean Hefty

On Thu, Apr 22, 2021 at 01:18:09PM -0300, Jason Gunthorpe wrote:
> On Thu, Apr 22, 2021 at 05:58:18PM +0300, Leon Romanovsky wrote:
> > On Thu, Apr 22, 2021 at 10:02:18AM -0300, Jason Gunthorpe wrote:
> > > On Thu, Apr 22, 2021 at 04:01:20PM +0300, Leon Romanovsky wrote:
> > > > On Thu, Apr 22, 2021 at 09:51:35AM -0300, Jason Gunthorpe wrote:
> > > > > On Thu, Apr 22, 2021 at 03:44:55PM +0300, Shay Drory wrote:
> > > > > > On 4/22/2021 14:28, Jason Gunthorpe wrote:
> > > > > > 
> > > > > > > On Sun, Apr 18, 2021 at 04:55:53PM +0300, Leon Romanovsky wrote:
> > > > > > > > From: Shay Drory <shayd@nvidia.com>
> > > > > > > > 
> > > > > > > > rdma_listen() checks if device already attached to rdma_id_priv,
> > > > > > > > based on the response the its decide to what to listen, however
> > > > > > > > this is different when the listeners are canceled.
> > > > > > > > 
> > > > > > > > This leads to a mismatch between rdma_listen() and cma_cancel_operation(),
> > > > > > > > and causes to bellow wild-memory-access. Fix it by aligning rdma_listen()
> > > > > > > > according to the cma_cancel_operation().
> > > > > > > So this is happening because the error unwind in rdma_bind_addr() is
> > > > > > > taking the exit path and calling cma_release_dev()?
> > > > > > > 
> > > > > > > This allows rdma_listen() to be called with a bogus device pointer
> > > > > > > which precipitates this UAF during destroy.
> > > > > > > 
> > > > > > > However, I think rdma_bind_addr() should not allow the bogus device
> > > > > > > pointer to leak out at all, since the ULP could see it. It really is
> > > > > > > invalid to have it present no matter what.
> > > > > > > 
> > > > > > > This would make cma_release_dev() and _cma_attach_to_dev()
> > > > > > > symmetrical - what do you think?
> > > > > > > 
> > > > > > > diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
> > > > > > > index 2dc302a83014ae..91f6d968b46f65 100644
> > > > > > > +++ b/drivers/infiniband/core/cma.c
> > > > > > > @@ -474,6 +474,7 @@ static void cma_release_dev(struct rdma_id_private *id_priv)
> > > > > > >   	list_del(&id_priv->list);
> > > > > > >   	cma_dev_put(id_priv->cma_dev);
> > > > > > >   	id_priv->cma_dev = NULL;
> > > > > > > +	id_priv->id.device = NULL;
> > > > > > >   	if (id_priv->id.route.addr.dev_addr.sgid_attr) {
> > > > > > >   		rdma_put_gid_attr(id_priv->id.route.addr.dev_addr.sgid_attr);
> > > > > > >   		id_priv->id.route.addr.dev_addr.sgid_attr = NULL;
> > > > > > 
> > > > > > I try that. this will break restrack_del() since restrack_del() is
> > > > > > using id_priv->id.device and is being called before restrack_del():
> > > > > 
> > > > > Oh that is another bug, once cma_release_dev() is called there is no
> > > > > refcount protecting the id.device and any access to it is invalid.
> > > > > 
> > > > > The order of rdma_restrack_del should be moved to be ahead of the
> > > > > cma_release_dev, and we also can't have a restrack without a cma_dev
> > > > > in the first place
> > > > 
> > > > We have restrack per-cmd_id and not per-cma_dev.
> > > 
> > > No, restrack has this:
> > > 
> > > 	dev = res_to_dev(res);
> > > 	if (WARN_ON(!dev))
> > > 
> > > And here dev will be NULL if cma_dev isn't set
> > 
> >   127 static struct ib_device *res_to_dev(struct rdma_restrack_entry *res)
> >   128 {
> > 
> > <...>
> > 
> >   136         case RDMA_RESTRACK_CM_ID:
> >   137                 return container_of(res, struct rdma_id_private,
> >   138                                     res)->id.device;
> >                                                 ^^^^^ it is not cma_dev
> 
> The invariant is that 
> 
>    priv.id.device == priv.cma_dev->device
> 
> (this de-normalization of data exists only to allow priv to be in a
> private header)
> 
> If cma_dev == NULL then id.device == NULL as cma_Dev was the thing
> preventing the pointer from being free'd.

Let's see what I can do here.

Thanks

> 
> Jason

^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2021-04-25 13:17 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-18 13:55 [PATCH rdma-next 0/3] CMA fixes Leon Romanovsky
2021-04-18 13:55 ` [PATCH rdma-next 1/3] RDMA/cma: Skip device which doesn't support CM Leon Romanovsky
2021-04-18 13:55 ` [PATCH rdma-next 2/3] RDMA/core: Fix check of device in rdma_listen() Leon Romanovsky
2021-04-22 11:28   ` Jason Gunthorpe
2021-04-22 12:44     ` Shay Drory
2021-04-22 12:51       ` Jason Gunthorpe
2021-04-22 13:01         ` Leon Romanovsky
2021-04-22 13:02           ` Jason Gunthorpe
2021-04-22 14:58             ` Leon Romanovsky
2021-04-22 16:18               ` Jason Gunthorpe
2021-04-25 13:17                 ` Leon Romanovsky
2021-04-18 13:55 ` [PATCH rdma-next 3/3] RDMA/core: Add CM to restrack after successful attachment to a device Leon Romanovsky
2021-04-21 23:59 ` [PATCH rdma-next 0/3] CMA fixes Jason Gunthorpe
2021-04-22  6:46   ` Leon Romanovsky

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).