* Re: + kmod-avoid-deadlock-by-recursive-kmod-call.patch added to -mm tree
@ 2012-01-26 17:56 Oleg Nesterov
2012-01-27 2:55 ` Rusty Russell
0 siblings, 1 reply; 14+ messages in thread
From: Oleg Nesterov @ 2012-01-26 17:56 UTC (permalink / raw)
To: Tetsuo Handa
Cc: Andrew Morton, Arjan van de Ven, Rusty Russell, Tejun Heo, linux-kernel
> @@ -449,6 +460,16 @@ int call_usermodehelper_exec(struct subp
> retval = -EBUSY;
> goto out;
> }
> + /*
> + * Worker thread must not wait for khelper thread at below
> + * wait_for_completion() if the thread was created with CLONE_VFORK
> + * flag, for khelper thread is already waiting for the thread at
> + * wait_for_completion() in do_fork().
> + */
> + if (wait != UMH_NO_WAIT && current == kmod_thread_locker) {
> + retval = -EBUSY;
> + goto out;
> + }
So, this is because khelper_wq's max_active == 1.
Can't we simply kill khelper_wq and use system_unbound_wq instead?
Oleg.
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: + kmod-avoid-deadlock-by-recursive-kmod-call.patch added to -mm tree 2012-01-26 17:56 + kmod-avoid-deadlock-by-recursive-kmod-call.patch added to -mm tree Oleg Nesterov @ 2012-01-27 2:55 ` Rusty Russell 2012-01-27 14:32 ` Oleg Nesterov 0 siblings, 1 reply; 14+ messages in thread From: Rusty Russell @ 2012-01-27 2:55 UTC (permalink / raw) To: Oleg Nesterov, Tetsuo Handa Cc: Andrew Morton, Arjan van de Ven, Tejun Heo, linux-kernel On Thu, 26 Jan 2012 18:56:12 +0100, Oleg Nesterov <oleg@redhat.com> wrote: > > @@ -449,6 +460,16 @@ int call_usermodehelper_exec(struct subp > > retval = -EBUSY; > > goto out; > > } > > + /* > > + * Worker thread must not wait for khelper thread at below > > + * wait_for_completion() if the thread was created with CLONE_VFORK > > + * flag, for khelper thread is already waiting for the thread at > > + * wait_for_completion() in do_fork(). > > + */ > > + if (wait != UMH_NO_WAIT && current == kmod_thread_locker) { > > + retval = -EBUSY; > > + goto out; > > + } > > So, this is because khelper_wq's max_active == 1. > > Can't we simply kill khelper_wq and use system_unbound_wq instead? I'd prefer that, because then we'd hit the existing "too many modprobes" check. Thanks, Rusty. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: + kmod-avoid-deadlock-by-recursive-kmod-call.patch added to -mm tree 2012-01-27 2:55 ` Rusty Russell @ 2012-01-27 14:32 ` Oleg Nesterov 2012-01-29 0:49 ` Rusty Russell 0 siblings, 1 reply; 14+ messages in thread From: Oleg Nesterov @ 2012-01-27 14:32 UTC (permalink / raw) To: Rusty Russell Cc: Tetsuo Handa, Andrew Morton, Arjan van de Ven, Tejun Heo, linux-kernel On 01/27, Rusty Russell wrote: > > On Thu, 26 Jan 2012 18:56:12 +0100, Oleg Nesterov <oleg@redhat.com> wrote: > > > @@ -449,6 +460,16 @@ int call_usermodehelper_exec(struct subp > > > retval = -EBUSY; > > > goto out; > > > } > > > + /* > > > + * Worker thread must not wait for khelper thread at below > > > + * wait_for_completion() if the thread was created with CLONE_VFORK > > > + * flag, for khelper thread is already waiting for the thread at > > > + * wait_for_completion() in do_fork(). > > > + */ > > > + if (wait != UMH_NO_WAIT && current == kmod_thread_locker) { > > > + retval = -EBUSY; > > > + goto out; > > > + } > > > > So, this is because khelper_wq's max_active == 1. > > > > Can't we simply kill khelper_wq and use system_unbound_wq instead? > > I'd prefer that, because then we'd hit the existing "too many modprobes" > check. Hmm. Why? I mean, why do you think that s/khelper_wq/system_unbound_wq/ leads to recursive __request_module's ? Note that that this patch (which adds kmod_thread_locker) can not limit the recursive modprobe loop. OK, yes, with system_unbound_wq we can hit this warning if we have max_modprobes UMH_WAIT_EXEC's resulting in __request_module at the same time, but probably this is good? I guess I missed something, could you explain? Just curious. Oleg. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: + kmod-avoid-deadlock-by-recursive-kmod-call.patch added to -mm tree 2012-01-27 14:32 ` Oleg Nesterov @ 2012-01-29 0:49 ` Rusty Russell 2012-01-29 16:31 ` Oleg Nesterov 0 siblings, 1 reply; 14+ messages in thread From: Rusty Russell @ 2012-01-29 0:49 UTC (permalink / raw) To: Oleg Nesterov Cc: Tetsuo Handa, Andrew Morton, Arjan van de Ven, Tejun Heo, linux-kernel On Fri, 27 Jan 2012 15:32:34 +0100, Oleg Nesterov <oleg@redhat.com> wrote: > On 01/27, Rusty Russell wrote: > > On Thu, 26 Jan 2012 18:56:12 +0100, Oleg Nesterov <oleg@redhat.com> wrote: > > > Can't we simply kill khelper_wq and use system_unbound_wq instead? > > > > I'd prefer that, because then we'd hit the existing "too many modprobes" > > check. > > Hmm. Why? I mean, why do you think that s/khelper_wq/system_unbound_wq/ > leads to recursive __request_module's ? > > Note that that this patch (which adds kmod_thread_locker) can not limit > the recursive modprobe loop. > > > OK, yes, with system_unbound_wq we can hit this warning if we have > max_modprobes UMH_WAIT_EXEC's resulting in __request_module at the > same time, but probably this is good? Yes, that's what I'm saying. We already have a check against too many modprobes, it might be best to use it. Cheers, Rusty. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: + kmod-avoid-deadlock-by-recursive-kmod-call.patch added to -mm tree 2012-01-29 0:49 ` Rusty Russell @ 2012-01-29 16:31 ` Oleg Nesterov 2012-01-29 23:26 ` Rusty Russell 0 siblings, 1 reply; 14+ messages in thread From: Oleg Nesterov @ 2012-01-29 16:31 UTC (permalink / raw) To: Rusty Russell Cc: Tetsuo Handa, Andrew Morton, Arjan van de Ven, Tejun Heo, linux-kernel On 01/29, Rusty Russell wrote: > > On Fri, 27 Jan 2012 15:32:34 +0100, Oleg Nesterov <oleg@redhat.com> wrote: > > On 01/27, Rusty Russell wrote: > > > On Thu, 26 Jan 2012 18:56:12 +0100, Oleg Nesterov <oleg@redhat.com> wrote: > > > > Can't we simply kill khelper_wq and use system_unbound_wq instead? > > > > > > I'd prefer that, because then we'd hit the existing "too many modprobes" > > > check. > > > > Hmm. Why? I mean, why do you think that s/khelper_wq/system_unbound_wq/ > > leads to recursive __request_module's ? > > > > Note that that this patch (which adds kmod_thread_locker) can not limit > > the recursive modprobe loop. > > > > > > OK, yes, with system_unbound_wq we can hit this warning if we have > > max_modprobes UMH_WAIT_EXEC's resulting in __request_module at the > > same time, but probably this is good? > > Yes, that's what I'm saying. > > We already have a check against too many modprobes, it might be best to > use it. Confused... in this case I do not understand why do you dislike the idea to kill khelper_wq. Help! Oleg. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: + kmod-avoid-deadlock-by-recursive-kmod-call.patch added to -mm tree 2012-01-29 16:31 ` Oleg Nesterov @ 2012-01-29 23:26 ` Rusty Russell 2012-01-30 0:25 ` Tejun Heo 2012-01-30 12:38 ` + kmod-avoid-deadlock-by-recursive-kmod-call.patch added to -mm tree Oleg Nesterov 0 siblings, 2 replies; 14+ messages in thread From: Rusty Russell @ 2012-01-29 23:26 UTC (permalink / raw) To: Oleg Nesterov Cc: Tetsuo Handa, Andrew Morton, Arjan van de Ven, Tejun Heo, linux-kernel On Sun, 29 Jan 2012 17:31:41 +0100, Oleg Nesterov <oleg@redhat.com> wrote: > Confused... in this case I do not understand why do you dislike the > idea to kill khelper_wq. Yes, you are confused. I was agreeing with you: On Thu, 26 Jan 2012 18:56:12 +0100, Oleg Nesterov <oleg@redhat.com> wrote: > Can't we simply kill khelper_wq and use system_unbound_wq instead? I'd prefer that, because then we'd hit the existing "too many modprobes" check. Thanks, Rusty. I thought "I'd prefer that" was pretty clear?? Thanks, Rusty. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: + kmod-avoid-deadlock-by-recursive-kmod-call.patch added to -mm tree 2012-01-29 23:26 ` Rusty Russell @ 2012-01-30 0:25 ` Tejun Heo 2012-01-30 13:03 ` Oleg Nesterov 2012-01-30 12:38 ` + kmod-avoid-deadlock-by-recursive-kmod-call.patch added to -mm tree Oleg Nesterov 1 sibling, 1 reply; 14+ messages in thread From: Tejun Heo @ 2012-01-30 0:25 UTC (permalink / raw) To: Rusty Russell Cc: Oleg Nesterov, Tetsuo Handa, Andrew Morton, Arjan van de Ven, linux-kernel On Mon, Jan 30, 2012 at 09:56:44AM +1030, Rusty Russell wrote: > On Sun, 29 Jan 2012 17:31:41 +0100, Oleg Nesterov <oleg@redhat.com> wrote: > > Confused... in this case I do not understand why do you dislike the > > idea to kill khelper_wq. > > Yes, you are confused. I was agreeing with you: > > On Thu, 26 Jan 2012 18:56:12 +0100, Oleg Nesterov <oleg@redhat.com> wrote: > > Can't we simply kill khelper_wq and use system_unbound_wq instead? BTW, why does it have to be unbound_wq? Is it expected consume large amount of CPU cycles? Thanks. -- tejun ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: + kmod-avoid-deadlock-by-recursive-kmod-call.patch added to -mm tree 2012-01-30 0:25 ` Tejun Heo @ 2012-01-30 13:03 ` Oleg Nesterov 2012-01-30 17:28 ` Tejun Heo 0 siblings, 1 reply; 14+ messages in thread From: Oleg Nesterov @ 2012-01-30 13:03 UTC (permalink / raw) To: Tejun Heo Cc: Rusty Russell, Tetsuo Handa, Andrew Morton, Arjan van de Ven, linux-kernel On 01/29, Tejun Heo wrote: > > BTW, why does it have to be unbound_wq? Perhaps we can use another system_wq, but afaics WQ_UNBOUND makes sense in this case. I mean, there is no reason to bind this work to any CPU. See also below. > Is it expected consume large > amount of CPU cycles? Currently __call_usermodehelper() does kernel_thread(), this is almost all. But it can block waiting for kernel_execve(). Not sure this really makes sense, but if we kill khelper_wq perhaps we can simplify this code a bit. We can change __call_usermodehelper() if (wait == UMH_WAIT_PROC) - pid = kernel_thread(wait_for_helper, sub_info, - CLONE_FS | CLONE_FILES | SIGCHLD); + wait_for_helper(...); else IOW, the worker thread itself can do the UMH_WAIT_PROC work. This makes this work really "long running", but then we can kill sub_info->complete and use flush_work(). Oleg. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: + kmod-avoid-deadlock-by-recursive-kmod-call.patch added to -mm tree 2012-01-30 13:03 ` Oleg Nesterov @ 2012-01-30 17:28 ` Tejun Heo 2012-02-03 18:00 ` Oleg Nesterov 0 siblings, 1 reply; 14+ messages in thread From: Tejun Heo @ 2012-01-30 17:28 UTC (permalink / raw) To: Oleg Nesterov Cc: Rusty Russell, Tetsuo Handa, Andrew Morton, Arjan van de Ven, linux-kernel Hello, On Mon, Jan 30, 2012 at 02:03:35PM +0100, Oleg Nesterov wrote: > Perhaps we can use another system_wq, but afaics WQ_UNBOUND makes sense > in this case. I mean, there is no reason to bind this work to any CPU. > See also below. I've been trying to nudge people away from using special wqs or flags unless really necessary. Other than non-reentrancy and strict ordering, all behaviors are mostly for optimization and using them incorrectly / spuriously usually doesn't cause any visible failure, making it very easy to get them wrong and if you have enough of wrong / unnecessary usages in tree, the whole thing gets really confusing and difficult to update in the future. > > Is it expected consume large > > amount of CPU cycles? > > Currently __call_usermodehelper() does kernel_thread(), this is almost > all. But it can block waiting for kernel_execve(). Blocking is completely fine on any workqueue. The only reason to require the use of unbound_wq is if work items would burn a lot of CPU cycles. In such cases, we want to let the scheduler have full jurisdiction instead of wq regulating concurrency. > Not sure this really makes sense, but if we kill khelper_wq perhaps we > can simplify this code a bit. We can change __call_usermodehelper() > > if (wait == UMH_WAIT_PROC) > - pid = kernel_thread(wait_for_helper, sub_info, > - CLONE_FS | CLONE_FILES | SIGCHLD); > + wait_for_helper(...); > else > > IOW, the worker thread itself can do the UMH_WAIT_PROC work. This makes > this work really "long running", but then we can kill sub_info->complete > and use flush_work(). Again, long-running in the sense that the work item spending a lot of time sleeping should be fine on system_wq or any other wq with default attributes. AFAICS, the things to consider here are... * If work items are expected to consume large amount of CPU cycles (as in crypto work items), consider using system_unbound_wq / WQ_UNBOUND. * If per-domain concurrency limit is necessary (ie. the number of concurrent work items doing this particular task should be limited rather than consuming global system_wq limit), a dedicated workqueue would be better. Thanks. -- tejun ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: + kmod-avoid-deadlock-by-recursive-kmod-call.patch added to -mm tree 2012-01-30 17:28 ` Tejun Heo @ 2012-02-03 18:00 ` Oleg Nesterov 2012-02-03 19:26 ` Tejun Heo 2012-02-04 12:56 ` + kmod-avoid-deadlock-by-recursive-kmod-call.patch added to-mm tree Tetsuo Handa 0 siblings, 2 replies; 14+ messages in thread From: Oleg Nesterov @ 2012-02-03 18:00 UTC (permalink / raw) To: Tejun Heo Cc: Rusty Russell, Tetsuo Handa, Andrew Morton, Arjan van de Ven, linux-kernel Hi Tejun, On 01/30, Tejun Heo wrote: > > On Mon, Jan 30, 2012 at 02:03:35PM +0100, Oleg Nesterov wrote: > > Perhaps we can use another system_wq, but afaics WQ_UNBOUND makes sense > > in this case. I mean, there is no reason to bind this work to any CPU. > > See also below. > > I've been trying to nudge people away from using special wqs or flags > unless really necessary. Other than non-reentrancy and strict > ordering, all behaviors are mostly for optimization and using them > incorrectly / spuriously usually doesn't cause any visible failure, > making it very easy to get them wrong and if you have enough of wrong > / unnecessary usages in tree, the whole thing gets really confusing > and difficult to update in the future. You know, I am a bit suprized. To me, it is the !WQ_UNBOUND case is "special". IOW, I think we need some reason to bind the work to the specific CPU. > > > Is it expected consume large > > > amount of CPU cycles? > > > > Currently __call_usermodehelper() does kernel_thread(), this is almost > > all. But it can block waiting for kernel_execve(). > > Blocking is completely fine on any workqueue. I understand. But, the blocked worker "consumes" nr_active/worker. > The only reason to > require the use of unbound_wq is if work items would burn a lot of CPU > cycles. In such cases, we want to let the scheduler have full > jurisdiction instead of wq regulating concurrency. I am starting to think I do not understand this code at all. OK, perhaps unbound_wq should be used for cpu-intensive works only. But why do you think that we should use a !WQ_UNBOUND workque instead of khelper_wq? And why "a lot of CPU" is the only reason for WQ_UNBOUND? > * If work items are expected to consume large amount of CPU cycles (as > in crypto work items), consider using system_unbound_wq / WQ_UNBOUND. > > * If per-domain concurrency limit is necessary (ie. the number of > concurrent work items doing this particular task should be limited > rather than consuming global system_wq limit), a dedicated workqueue > would be better. So I don't understand whether you like the idea to kill khelper_wq and use some system_ wq or not (and fix the bug). I do not really like the current patch. If nothing else, what if UMH_WAIT_EXEC request actually needs another UMH_WAIT_EXEC/PROC request to succeed? Tetsuo, we spent a lot of time discussing other problems. What do you think about s/khelper/system/ instead of this patch? Oleg. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: + kmod-avoid-deadlock-by-recursive-kmod-call.patch added to -mm tree 2012-02-03 18:00 ` Oleg Nesterov @ 2012-02-03 19:26 ` Tejun Heo 2012-02-04 12:56 ` + kmod-avoid-deadlock-by-recursive-kmod-call.patch added to-mm tree Tetsuo Handa 1 sibling, 0 replies; 14+ messages in thread From: Tejun Heo @ 2012-02-03 19:26 UTC (permalink / raw) To: Oleg Nesterov Cc: Rusty Russell, Tetsuo Handa, Andrew Morton, Arjan van de Ven, linux-kernel Hello, Oleg. On Fri, Feb 03, 2012 at 07:00:30PM +0100, Oleg Nesterov wrote: > > I've been trying to nudge people away from using special wqs or flags > > unless really necessary. Other than non-reentrancy and strict > > ordering, all behaviors are mostly for optimization and using them > > incorrectly / spuriously usually doesn't cause any visible failure, > > making it very easy to get them wrong and if you have enough of wrong > > / unnecessary usages in tree, the whole thing gets really confusing > > and difficult to update in the future. > > You know, I am a bit suprized. To me, it is the !WQ_UNBOUND case is > "special". IOW, I think we need some reason to bind the work to the > specific CPU. It's part historical due to the way workqueue was originally implemented and part because work items are expected to consume little CPU cycles and access data which the issuer accessed, but the question really isn't about whether unbound would be marginally better or not. We may change the default behavior in the future as implementation evolves or hardware environment changes and having more things which claim special treatment without clear reason makes things needlessly complicated, so the question to ask is whether the default behavior doesn't fit enough that the extra flag is really required. > > Blocking is completely fine on any workqueue. > > I understand. But, the blocked worker "consumes" nr_active/worker. If it's expected to consume high number of nr_active and needs limiting, creating a dedicated wq as an isolation domain would be a good idea. It's not like wqs are expensive or complex to use after all. > > The only reason to > > require the use of unbound_wq is if work items would burn a lot of CPU > > cycles. In such cases, we want to let the scheduler have full > > jurisdiction instead of wq regulating concurrency. > > I am starting to think I do not understand this code at all. OK, > perhaps unbound_wq should be used for cpu-intensive works only. > > But why do you think that we should use a !WQ_UNBOUND workque > instead of khelper_wq? And why "a lot of CPU" is the only reason > for WQ_UNBOUND? As I wrote, it's not about "should use bound", it's about having enough justification for deviating from the default. Thanks. -- tejun ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: + kmod-avoid-deadlock-by-recursive-kmod-call.patch added to-mm tree 2012-02-03 18:00 ` Oleg Nesterov 2012-02-03 19:26 ` Tejun Heo @ 2012-02-04 12:56 ` Tetsuo Handa 2012-02-06 17:19 ` Oleg Nesterov 1 sibling, 1 reply; 14+ messages in thread From: Tetsuo Handa @ 2012-02-04 12:56 UTC (permalink / raw) To: oleg, tj; +Cc: rusty, akpm, arjan, linux-kernel Oleg Nesterov wrote: > What do you think about s/khelper/system/ instead of this patch? I'm not catching up with this topic. But I'm fine with any solution provided that it can handle some kernel function calls request_module() request_module() triggers call_usermodehelper("/sbin/modprobe", UMH_WAIT_PROC) /sbin/modprobe loads a kernel module loading a kernel module triggers kobject_uevent_env() kobject_uevent_env() calls call_usermodehelper("/sbin/hotplug", UMH_WAIT_EXEC) do_execve("/sbin/hotplug") calls request_module("binfmt-0000") request_module("binfmt-0000") triggers call_usermodehelper("/sbin/modprobe", UMH_WAIT_PROC) /sbin/modprobe fails to load binfmt-0000 module call trace on an UP machine. kmod-avoid-deadlock-by-recursive-kmod-call.patch is for avoiding call_usermodehelper("/sbin/modprobe", UMH_WAIT_PROC) called from call_usermodehelper("/sbin/hotplug", UMH_WAIT_EXEC). ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: + kmod-avoid-deadlock-by-recursive-kmod-call.patch added to-mm tree 2012-02-04 12:56 ` + kmod-avoid-deadlock-by-recursive-kmod-call.patch added to-mm tree Tetsuo Handa @ 2012-02-06 17:19 ` Oleg Nesterov 0 siblings, 0 replies; 14+ messages in thread From: Oleg Nesterov @ 2012-02-06 17:19 UTC (permalink / raw) To: Tetsuo Handa; +Cc: tj, rusty, akpm, arjan, linux-kernel On 02/04, Tetsuo Handa wrote: > > Oleg Nesterov wrote: > > What do you think about s/khelper/system/ instead of this patch? > > I'm not catching up with this topic. But I'm fine with any solution provided > that it can handle I think it should... Tetsuo, all, sorry I am busy today. I'll try to return tomorrow to continue this (and off-list) discussion. Oleg. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: + kmod-avoid-deadlock-by-recursive-kmod-call.patch added to -mm tree 2012-01-29 23:26 ` Rusty Russell 2012-01-30 0:25 ` Tejun Heo @ 2012-01-30 12:38 ` Oleg Nesterov 1 sibling, 0 replies; 14+ messages in thread From: Oleg Nesterov @ 2012-01-30 12:38 UTC (permalink / raw) To: Rusty Russell Cc: Tetsuo Handa, Andrew Morton, Arjan van de Ven, Tejun Heo, linux-kernel On 01/30, Rusty Russell wrote: > > On Sun, 29 Jan 2012 17:31:41 +0100, Oleg Nesterov <oleg@redhat.com> wrote: > > Confused... in this case I do not understand why do you dislike the > > idea to kill khelper_wq. > > Yes, you are confused. I was agreeing with you: Ah... > I thought "I'd prefer that" was pretty clear?? It was! Looking back, I do not understand how did I manage to misread your email. Thanks Rusty, Oleg. ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2012-02-06 19:16 UTC | newest] Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2012-01-26 17:56 + kmod-avoid-deadlock-by-recursive-kmod-call.patch added to -mm tree Oleg Nesterov 2012-01-27 2:55 ` Rusty Russell 2012-01-27 14:32 ` Oleg Nesterov 2012-01-29 0:49 ` Rusty Russell 2012-01-29 16:31 ` Oleg Nesterov 2012-01-29 23:26 ` Rusty Russell 2012-01-30 0:25 ` Tejun Heo 2012-01-30 13:03 ` Oleg Nesterov 2012-01-30 17:28 ` Tejun Heo 2012-02-03 18:00 ` Oleg Nesterov 2012-02-03 19:26 ` Tejun Heo 2012-02-04 12:56 ` + kmod-avoid-deadlock-by-recursive-kmod-call.patch added to-mm tree Tetsuo Handa 2012-02-06 17:19 ` Oleg Nesterov 2012-01-30 12:38 ` + kmod-avoid-deadlock-by-recursive-kmod-call.patch added to -mm tree Oleg Nesterov
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).