* systemd and NFS "bg" mounts. @ 2017-05-26 2:46 NeilBrown 2017-05-29 13:38 ` [systemd-devel] " Lennart Poettering 0 siblings, 1 reply; 29+ messages in thread From: NeilBrown @ 2017-05-26 2:46 UTC (permalink / raw) To: systemd-devel, linux-nfs [-- Attachment #1: Type: text/plain, Size: 4491 bytes --] Hi all, it appears that systemd doesn't play well with NFS "bg" mounts. I can see a few options for how to address this and wonder if anyone has any opinions. "bg" mounts will try to mount the filesystem just like normal, but if the server cannot be contacted before a "major timeout" (4.5 minutes by default for TCP), mount.nfs will fork and continue in the background. Meanwhile the original mount process reports success (even though the filesystem wasn't mounted). This allows the boot process to continue and succeed. Currently if you specify the "bg" option in /etc/fstab and are using systemd, the "bg" has no useful effect. systemd imposes its own timeout of 90 seconds (which is less than 4.5 minutes). After 90 seconds, systemd will kill the mount process and decide that the mount failed. This will lead to remote-fs.target not being reached, and boot not completing. If you set TimeoutSec=0 (aka "infinity") for the mount unit, either by hacking fstab-generator or adding "x-systemd.mount-timeout=infinity" if you have systemd 233 or later, then systemd won't kill the mount process, and after 4.5 minutes it will exit. (to quote a comment in systemd/src/core/mount.c " /bin/mount lies to us and is broken" :-) This is better, but the background mount.nfs can persist for a long time. I don't think it persists forever, but at least half an hour I think. When the foo.mount unit is stopped, the mount.nfs process isn't killed. I don't think this is a major problem, but it is unfortunate and could be confusing. During testing I've had multiple mount.nfs background processes all attached to the one .mount unit. What should we do about bg NFS mounts with systemd? Some options: - declare "bg" to be not-supported. If you don't need the filesystem to be present for boot, then use x-systemd.automount, or some other automount scheme. If we did this, we probably need to make it very obvious that "bg" mounts aren't supported - maybe a log message that appears when you do "systemctl status ..." ?? - decide that "bg" is really just a lame attempt at automounting, and that now we have real automounting, "bg" can be translated to that. So systemd-fstab-generator would treat "bg" like "x-systemd.automount" and otherwise strip it from the list of options. - do our best to really support "bg". That means, at least, applying a much larger timeout to "bg" mounts, and preferably killing any background processes when a mount unit is stopped. Below is a little patch which does this last bit, but I'm not sure it is generally safe. A side question is: should this knowledge about NFS be encoded in systemd, or should nfs-utils add the necessary knowledge? i.e. we could add an nfs-fstab-generator to nfs-utils which creates drop-ins to modify the behaviour of the drop-ins provided by systemd-fstab-generator. Adding the TimeoutSec= would be easy. Stripping the "bg" would be possible. Changing the remote-fs.target.requires/foo.mount symlink to be remote-fs.target.requires/foo.automount would be problematic though. Could we teach systemd-fstab-generator to ignore $TYPE filesystems if TYPE-fstab-generator existed? Or should we just build all this filesystem-specific knowledge into systemd? Thanks for your thoughts, NeilBrown hackish patch to kill backgrounded mount.nfs processes: diff --git a/src/core/mount.c b/src/core/mount.c index ca0c4b0d5eed..91939b48d11a 100644 --- a/src/core/mount.c +++ b/src/core/mount.c @@ -883,6 +883,18 @@ static void mount_enter_unmounting(Mount *m) { MOUNT_UNMOUNTING_SIGKILL)) m->n_retry_umount = 0; + if (m->result == MOUNT_SUCCESS && + !m->from_proc_self_mountinfo) { + /* There is no mountpoint, but mount seemed to succeed. + * Could be a bg mount.nfs. + * In any case, kill any processes that might be hanging + * around, they cannot be doing anything useful. + */ + sd_bus_error error = SD_BUS_ERROR_NULL; + unit_kill_common(UNIT(m), KILL_ALL, SIGTERM, -1, -1, &error); + } + + m->control_command_id = MOUNT_EXEC_UNMOUNT; m->control_command = m->exec_command + MOUNT_EXEC_UNMOUNT; [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: [systemd-devel] systemd and NFS "bg" mounts. 2017-05-26 2:46 systemd and NFS "bg" mounts NeilBrown @ 2017-05-29 13:38 ` Lennart Poettering 2017-05-29 22:05 ` NeilBrown 2017-05-29 22:19 ` [PATCH] nfs.man: document incompatibility between "bg" option and systemd NeilBrown 0 siblings, 2 replies; 29+ messages in thread From: Lennart Poettering @ 2017-05-29 13:38 UTC (permalink / raw) To: NeilBrown; +Cc: systemd-devel, linux-nfs On Fri, 26.05.17 12:46, NeilBrown (neilb@suse.com) wrote: > > Hi all, > it appears that systemd doesn't play well with NFS "bg" mounts. > I can see a few options for how to address this and wonder if anyone > has any opinions. Yeah, this has come up before. Long story short: "bg" is simply not compatible with systemd. We assume that the /bin/mount's effect is visible in /proc/self/mountinfo, and everything else is considered a bug, i.e. /bin/mount lying to us. And I think that's a pretty rational assumption and requirement to make. I am not particularly convinced the "bg" usecase is really such a great idea, since it is necessarily racy: you never know whether mount is actually in effect or not, even though /bin/mount claims so. I am pretty sure other options (such as autofs mounts, which are dead-easy to use in system: just replace the "bg" mount option in fstab by "x-systemd.automount") are much better approaches to the problem at hand: they also make your local system less dependent on remote network access, but they do give proper guarantees about their runtime: when the autofs mount is established the path is available. Hence I am tempted to treat the issue as a documentation and warning issue: accept that "bg" is not supported, but document this better. In addition, we should probably log about "bg" being broken in the fstab-generator. I file a bug about that now: https://github.com/systemd/systemd/issues/6046 > This is better, but the background mount.nfs can persist for a long > time. I don't think it persists forever, but at least half an hour I > think. > > When the foo.mount unit is stopped, the mount.nfs process isn't > killed. Hmm, if you see this, then this would be a bug: mount units that are stopped should not leave processes around. > I don't think this is a major problem, but it is unfortunate and could > be confusing. During testing I've had multiple mount.nfs background > processes all attached to the one .mount unit. Humpf, could you file a bug? While I think the "bg" concept is broken, as discussed above, having FUSE mounts with processes in the background is actually supported, and we should clean them up properly when the mount unit is stopped. Hmm, maybe mount.nfs isn't properly killable? i.e. systemd tries to kill it, but it refuses to be killed? > What should we do about bg NFS mounts with systemd? > Some options: > - declare "bg" to be not-supported. If you don't need the filesystem > to be present for boot, then use x-systemd.automount, or some other > automount scheme. > If we did this, we probably need to make it very obvious that "bg" > mounts aren't supported - maybe a log message that appears when > you do "systemctl status ..." ?? I am all for this, as suggested above. I'd only log from fstab-generator though. (That said, if we want something stronger, we could also add the fact that we stripped "bg" from the mount optoins to the DEscription= of the generated mount unit.) > - decide that "bg" is really just a lame attempt at automounting, and > that now we have real automounting, "bg" can be translated to that. > So systemd-fstab-generator would treat "bg" like > "x-systemd.automount" and otherwise strip it from the list of > options. I am a bit afraid of this I must say. The semantics are different enough to likely cause more problems then we'd solve with this. Not supporting this at all sounds like the much better approach here: let's strip "bg" when specified. > - do our best to really support "bg". That means, at least, applying > a much larger timeout to "bg" mounts, and preferably killing any > background processes when a mount unit is stopped. Below is a > little patch which does this last bit, but I'm not sure it is generally > safe. As mentioned I think this would just trade one race for a couple of new ones, and that appears to be a bad idea to me. > A side question is: should this knowledge about NFS be encoded in > systemd, or should nfs-utils add the necessary knowledge? I am pretty sure we should keep special understanding of NFS at a minimum in PID 1, but I think we can be less strict in fstab-generator, as its primary job is compat with UNIX anyway. > > i.e. we could add an nfs-fstab-generator to nfs-utils which creates > drop-ins to modify the behaviour of the drop-ins provided by > systemd-fstab-generator. > Adding the TimeoutSec= would be easy. Stripping the "bg" would be > possible. > Changing the remote-fs.target.requires/foo.mount symlink to be > remote-fs.target.requires/foo.automount would be problematic > though. Well, I'd be fine with letting NFS do its own handling of the NFS /etc/fstab entries, but I think the special casing of "bg" is fine to simply add to the existing generator in systemd. > Could we teach systemd-fstab-generator to ignore $TYPE filesystems > if TYPE-fstab-generator existed? Well, generators can override each other in very limited ways (as there are three different output directories a gnerator can write to, which are inserted at different places in the unit file search path), we could build on that. That said, I think adding this to fstab-generator in systemd is OK. > Or should we just build all this filesystem-specific knowledge into > systemd? For now, I think adding this to systemd's fstab-generator would be fine. Hope this makes sense, Lennart -- Lennart Poettering, Red Hat ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [systemd-devel] systemd and NFS "bg" mounts. 2017-05-29 13:38 ` [systemd-devel] " Lennart Poettering @ 2017-05-29 22:05 ` NeilBrown 2017-05-29 22:19 ` [PATCH] nfs.man: document incompatibility between "bg" option and systemd NeilBrown 1 sibling, 0 replies; 29+ messages in thread From: NeilBrown @ 2017-05-29 22:05 UTC (permalink / raw) To: Lennart Poettering; +Cc: systemd-devel, linux-nfs [-- Attachment #1: Type: text/plain, Size: 7559 bytes --] On Mon, May 29 2017, Lennart Poettering wrote: > On Fri, 26.05.17 12:46, NeilBrown (neilb@suse.com) wrote: > >> >> Hi all, >> it appears that systemd doesn't play well with NFS "bg" mounts. >> I can see a few options for how to address this and wonder if anyone >> has any opinions. > > Yeah, this has come up before. Long story short: "bg" is simply not > compatible with systemd. We assume that the /bin/mount's effect is > visible in /proc/self/mountinfo, and everything else is considered a > bug, i.e. /bin/mount lying to us. And I think that's a pretty rational > assumption and requirement to make. > > I am not particularly convinced the "bg" usecase is really such a > great idea, since it is necessarily racy: you never know whether mount > is actually in effect or not, even though /bin/mount claims so. I am > pretty sure other options (such as autofs mounts, which are dead-easy > to use in system: just replace the "bg" mount option in fstab by > "x-systemd.automount") are much better approaches to the problem at > hand: they also make your local system less dependent on remote > network access, but they do give proper guarantees about their > runtime: when the autofs mount is established the path is available. > > Hence I am tempted to treat the issue as a documentation and warning > issue: accept that "bg" is not supported, but document this better. In > addition, we should probably log about "bg" being broken in the > fstab-generator. I file a bug about that now: > > https://github.com/systemd/systemd/issues/6046 There is a weird distorted sort of justice here. When NFS first appeared, it broke various long-standing Unix practices, such as O_EXCL|O_CREAT for lock files. Now systemd appears and breaks a long-standing NFS practice: bg mounts. I hoped we could find a way to make them work, but I won't cry over their demise. I much prefer automount .... I think all NFS mounts should be automounts. I see this is already documented in systemd.mount: The NFS mount option bg for NFS background mounts as documented in nfs(5) is not supported in /etc/fstab entries. I wonder how many people actually read that. We should probably add symmetric documentation to nfs(5) Both should give clear pointers to x-systemd.automount. > >> This is better, but the background mount.nfs can persist for a long >> time. I don't think it persists forever, but at least half an hour I >> think. >> >> When the foo.mount unit is stopped, the mount.nfs process isn't >> killed. > > Hmm, if you see this, then this would be a bug: mount units that are > stopped should not leave processes around. > >> I don't think this is a major problem, but it is unfortunate and could >> be confusing. During testing I've had multiple mount.nfs background >> processes all attached to the one .mount unit. > > Humpf, could you file a bug? https://github.com/systemd/systemd/issues/6048 > > While I think the "bg" concept is broken, as discussed above, having > FUSE mounts with processes in the background is actually supported, > and we should clean them up properly when the mount unit is stopped. > > Hmm, maybe mount.nfs isn't properly killable? i.e. systemd tries to > kill it, but it refuses to be killed? mount.nfs responds cleanly to SIGTERM. > >> What should we do about bg NFS mounts with systemd? >> Some options: >> - declare "bg" to be not-supported. If you don't need the filesystem >> to be present for boot, then use x-systemd.automount, or some other >> automount scheme. >> If we did this, we probably need to make it very obvious that "bg" >> mounts aren't supported - maybe a log message that appears when >> you do "systemctl status ..." ?? > > I am all for this, as suggested above. I'd only log from > fstab-generator though. (That said, if we want something stronger, we > could also add the fact that we stripped "bg" from the mount optoins > to the DEscription= of the generated mount unit.) That last bit sounds like a very good idea. Stripping "bg" could be seen as a "surprising" thing for fstab-generator to do. Making it as obvious as possible to the sys-admin would be a good thing (and would probably make support personnel happy too). > >> - decide that "bg" is really just a lame attempt at automounting, and >> that now we have real automounting, "bg" can be translated to that. >> So systemd-fstab-generator would treat "bg" like >> "x-systemd.automount" and otherwise strip it from the list of >> options. > > I am a bit afraid of this I must say. The semantics are different > enough to likely cause more problems then we'd solve with this. Not > supporting this at all sounds like the much better approach here: > let's strip "bg" when specified. > >> - do our best to really support "bg". That means, at least, applying >> a much larger timeout to "bg" mounts, and preferably killing any >> background processes when a mount unit is stopped. Below is a >> little patch which does this last bit, but I'm not sure it is generally >> safe. > > As mentioned I think this would just trade one race for a couple of > new ones, and that appears to be a bad idea to me. > >> A side question is: should this knowledge about NFS be encoded in >> systemd, or should nfs-utils add the necessary knowledge? > > I am pretty sure we should keep special understanding of NFS at a > minimum in PID 1, but I think we can be less strict in > fstab-generator, as its primary job is compat with UNIX anyway. I was thinking about which source package the knowledge would be in, and hence which set of maintainers would over-see it. I don't expect systemd maintainers to be fully in-touch with the details of NFS, but then NFS developers cannot be fully in-touch with how systemd works. Apart from some documentation changes, we probably don't need to put anything new in nfs-utils at the moment. > >> >> i.e. we could add an nfs-fstab-generator to nfs-utils which creates >> drop-ins to modify the behaviour of the drop-ins provided by >> systemd-fstab-generator. >> Adding the TimeoutSec= would be easy. Stripping the "bg" would be >> possible. >> Changing the remote-fs.target.requires/foo.mount symlink to be >> remote-fs.target.requires/foo.automount would be problematic >> though. > > Well, I'd be fine with letting NFS do its own handling of the NFS > /etc/fstab entries, but I think the special casing of "bg" is fine to > simply add to the existing generator in systemd. > >> Could we teach systemd-fstab-generator to ignore $TYPE filesystems >> if TYPE-fstab-generator existed? > > Well, generators can override each other in very limited ways (as > there are three different output directories a gnerator can write to, > which are inserted at different places in the unit file search path), > we could build on that. That said, I think adding this to > fstab-generator in systemd is OK. Ahh, of course. That is what normal-dir / early-dir / late-dir is for. I'll keep that in mind in case I do ever need it. > >> Or should we just build all this filesystem-specific knowledge into >> systemd? > > For now, I think adding this to systemd's fstab-generator would be fine. > > Hope this makes sense, Yes it does. It wasn't the outcome I was hoping for, but it is hard to argue against it. Thanks a lot, NeilBrown [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCH] nfs.man: document incompatibility between "bg" option and systemd. 2017-05-29 13:38 ` [systemd-devel] " Lennart Poettering 2017-05-29 22:05 ` NeilBrown @ 2017-05-29 22:19 ` NeilBrown 2017-05-30 4:47 ` Niels de Vos ` (4 more replies) 1 sibling, 5 replies; 29+ messages in thread From: NeilBrown @ 2017-05-29 22:19 UTC (permalink / raw) To: Steve Dickson; +Cc: systemd-devel, linux-nfs, Lennart Poettering [-- Attachment #1: Type: text/plain, Size: 1499 bytes --] Systemd does not, and will not, support "bg" correctly. It has other, better, ways to handle "background" mounting. Explain this. See also https://github.com/systemd/systemd/issues/6046 Signed-off-by: NeilBrown <neilb@suse.com> --- utils/mount/nfs.man | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/utils/mount/nfs.man b/utils/mount/nfs.man index cc6e992ed807..7e76492d454f 100644 --- a/utils/mount/nfs.man +++ b/utils/mount/nfs.man @@ -372,6 +372,21 @@ Alternatively these issues can be addressed using an automounter (refer to .BR automount (8) for details). +.IP +When +.B systemd +is used to mount the filesystems listed in +.IR /etc/fstab , +the +.B bg +option is not supported, and may be stripped from the option list. +Similar functionality can be achieved by providing the +.B x-system.automount +option. This will cause +.B systemd +to attempt to mount the filesystem when the mountpoint is first +accessed, rather than during system boot. The mount still happens in +the "background", though in a different way. .TP 1.5i .BR rdirplus " / " nordirplus Selects whether to use NFS v3 or v4 READDIRPLUS requests. @@ -1810,7 +1825,8 @@ such as security negotiation, server referrals, and named attributes. .BR rpc.idmapd (8), .BR rpc.gssd (8), .BR rpc.svcgssd (8), -.BR kerberos (1) +.BR kerberos (1), +.BR systemd.mount (5) . .sp RFC 768 for the UDP specification. .br -- 2.12.2 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: [PATCH] nfs.man: document incompatibility between "bg" option and systemd. 2017-05-29 22:19 ` [PATCH] nfs.man: document incompatibility between "bg" option and systemd NeilBrown @ 2017-05-30 4:47 ` Niels de Vos 2017-05-30 7:40 ` [systemd-devel] " Michael Biebl ` (3 subsequent siblings) 4 siblings, 0 replies; 29+ messages in thread From: Niels de Vos @ 2017-05-30 4:47 UTC (permalink / raw) To: NeilBrown; +Cc: Steve Dickson, systemd-devel, linux-nfs, Lennart Poettering On Tue, May 30, 2017 at 08:19:16AM +1000, NeilBrown wrote: > > Systemd does not, and will not, support "bg" correctly. > It has other, better, ways to handle "background" mounting. > > Explain this. > > See also https://github.com/systemd/systemd/issues/6046 > > Signed-off-by: NeilBrown <neilb@suse.com> > --- > utils/mount/nfs.man | 18 +++++++++++++++++- > 1 file changed, 17 insertions(+), 1 deletion(-) > > diff --git a/utils/mount/nfs.man b/utils/mount/nfs.man > index cc6e992ed807..7e76492d454f 100644 > --- a/utils/mount/nfs.man > +++ b/utils/mount/nfs.man > @@ -372,6 +372,21 @@ Alternatively these issues can be addressed > using an automounter (refer to > .BR automount (8) > for details). > +.IP > +When > +.B systemd > +is used to mount the filesystems listed in > +.IR /etc/fstab , > +the > +.B bg > +option is not supported, and may be stripped from the option list. > +Similar functionality can be achieved by providing the > +.B x-system.automount > +option. This will cause > +.B systemd > +to attempt to mount the filesystem when the mountpoint is first > +accessed, rather than during system boot. The mount still happens in > +the "background", though in a different way. > .TP 1.5i > .BR rdirplus " / " nordirplus > Selects whether to use NFS v3 or v4 READDIRPLUS requests. > @@ -1810,7 +1825,8 @@ such as security negotiation, server referrals, and named attributes. > .BR rpc.idmapd (8), > .BR rpc.gssd (8), > .BR rpc.svcgssd (8), > -.BR kerberos (1) > +.BR kerberos (1), > +.BR systemd.mount (5) . > .sp > RFC 768 for the UDP specification. > .br > -- > 2.12.2 > I like this, it makes it so much easier for users to find the better alternative. FWIW, Reviewed-by: Niels de Vos <ndevos@redha.com> ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [systemd-devel] [PATCH] nfs.man: document incompatibility between "bg" option and systemd. 2017-05-29 22:19 ` [PATCH] nfs.man: document incompatibility between "bg" option and systemd NeilBrown 2017-05-30 4:47 ` Niels de Vos @ 2017-05-30 7:40 ` Michael Biebl 2017-05-30 8:55 ` NeilBrown 2017-05-30 12:43 ` Lennart Poettering ` (2 subsequent siblings) 4 siblings, 1 reply; 29+ messages in thread From: Michael Biebl @ 2017-05-30 7:40 UTC (permalink / raw) To: NeilBrown; +Cc: Steve Dickson, linux-nfs, systemd-devel 2017-05-30 0:19 GMT+02:00 NeilBrown <neilb@suse.com>: > +.B bg > +option is not supported, and may be stripped from the option list. Either systemd is updated to actually strip the bg option or not. The documentation should reflect that. I don't think we should be vague about this, as it would only be confusing. Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [systemd-devel] [PATCH] nfs.man: document incompatibility between "bg" option and systemd. 2017-05-30 7:40 ` [systemd-devel] " Michael Biebl @ 2017-05-30 8:55 ` NeilBrown 2017-05-30 9:15 ` Michael Biebl 0 siblings, 1 reply; 29+ messages in thread From: NeilBrown @ 2017-05-30 8:55 UTC (permalink / raw) To: Michael Biebl; +Cc: Steve Dickson, linux-nfs, systemd-devel [-- Attachment #1: Type: text/plain, Size: 819 bytes --] On Tue, May 30 2017, Michael Biebl wrote: > 2017-05-30 0:19 GMT+02:00 NeilBrown <neilb@suse.com>: > >> +.B bg >> +option is not supported, and may be stripped from the option list. > > Either systemd is updated to actually strip the bg option or not. The > documentation should reflect that. > I don't think we should be vague about this, as it would only be confusing. It depends on which version of systemd is in use. Even if/when systemd is updated to strip the 'bg', it would not be correct to say "and will be stripped..." as that isn't true for all versions. We could spell it out "and may be stripped from the option list, depending on which version of systemd is installed", but I'm not sure that really helps... "... and some versions of systemd will strip 'bg' from the option list" ?? Thanks, NeilBrown [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [systemd-devel] [PATCH] nfs.man: document incompatibility between "bg" option and systemd. 2017-05-30 8:55 ` NeilBrown @ 2017-05-30 9:15 ` Michael Biebl 2017-05-30 12:45 ` Lennart Poettering 0 siblings, 1 reply; 29+ messages in thread From: Michael Biebl @ 2017-05-30 9:15 UTC (permalink / raw) To: NeilBrown; +Cc: Steve Dickson, linux-nfs, systemd-devel 2017-05-30 10:55 GMT+02:00 NeilBrown <neilb@suse.com>: > On Tue, May 30 2017, Michael Biebl wrote: > >> 2017-05-30 0:19 GMT+02:00 NeilBrown <neilb@suse.com>: >> >>> +.B bg >>> +option is not supported, and may be stripped from the option list. >> >> Either systemd is updated to actually strip the bg option or not. The >> documentation should reflect that. >> I don't think we should be vague about this, as it would only be confusing. > > It depends on which version of systemd is in use. > Even if/when systemd is updated to strip the 'bg', it would not > be correct to say "and will be stripped..." as that isn't true for all > versions. > We could spell it out "and may be stripped from the option list, > depending on which version of systemd is installed", but I'm not sure > that really helps... > "... and some versions of systemd will strip 'bg' from the option list" > ?? So far, no version of systemd strips bg from the option list. If in version X systemd actually starts stripping bg, we should explicitly mention that version in the man page imho. -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [systemd-devel] [PATCH] nfs.man: document incompatibility between "bg" option and systemd. 2017-05-30 9:15 ` Michael Biebl @ 2017-05-30 12:45 ` Lennart Poettering 0 siblings, 0 replies; 29+ messages in thread From: Lennart Poettering @ 2017-05-30 12:45 UTC (permalink / raw) To: Michael Biebl; +Cc: NeilBrown, linux-nfs, systemd-devel On Tue, 30.05.17 11:15, Michael Biebl (mbiebl@gmail.com) wrote: > 2017-05-30 10:55 GMT+02:00 NeilBrown <neilb@suse.com>: > > On Tue, May 30 2017, Michael Biebl wrote: > > > >> 2017-05-30 0:19 GMT+02:00 NeilBrown <neilb@suse.com>: > >> > >>> +.B bg > >>> +option is not supported, and may be stripped from the option list. > >> > >> Either systemd is updated to actually strip the bg option or not. The > >> documentation should reflect that. > >> I don't think we should be vague about this, as it would only be confusing. > > > > It depends on which version of systemd is in use. > > Even if/when systemd is updated to strip the 'bg', it would not > > be correct to say "and will be stripped..." as that isn't true for all > > versions. > > We could spell it out "and may be stripped from the option list, > > depending on which version of systemd is installed", but I'm not sure > > that really helps... > > "... and some versions of systemd will strip 'bg' from the option list" > > ?? > > So far, no version of systemd strips bg from the option list. > If in version X systemd actually starts stripping bg, we should > explicitly mention that version in the man page imho. I figure it's up to the NFS folks to decide on this, but if I were them I would not mention any version numbers. That just gets out of date. Instead, document the new behaviour and indicate in the README file that the documentation assumes systemd with a certain version is used, and then let the packers figure out the rest, i.e. if they want to place some kind of weak package version dependency or not. Lennart -- Lennart Poettering, Red Hat ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH] nfs.man: document incompatibility between "bg" option and systemd. 2017-05-29 22:19 ` [PATCH] nfs.man: document incompatibility between "bg" option and systemd NeilBrown 2017-05-30 4:47 ` Niels de Vos 2017-05-30 7:40 ` [systemd-devel] " Michael Biebl @ 2017-05-30 12:43 ` Lennart Poettering 2017-06-06 18:07 ` Steve Dickson 2017-07-04 22:20 ` NeilBrown 4 siblings, 0 replies; 29+ messages in thread From: Lennart Poettering @ 2017-05-30 12:43 UTC (permalink / raw) To: NeilBrown; +Cc: Steve Dickson, systemd-devel, linux-nfs On Tue, 30.05.17 08:19, NeilBrown (neilb@suse.com) wrote: > > Systemd does not, and will not, support "bg" correctly. > It has other, better, ways to handle "background" mounting. > > Explain this. > > See also https://github.com/systemd/systemd/issues/6046 > > Signed-off-by: NeilBrown <neilb@suse.com> > --- > utils/mount/nfs.man | 18 +++++++++++++++++- > 1 file changed, 17 insertions(+), 1 deletion(-) > > diff --git a/utils/mount/nfs.man b/utils/mount/nfs.man > index cc6e992ed807..7e76492d454f 100644 > --- a/utils/mount/nfs.man > +++ b/utils/mount/nfs.man > @@ -372,6 +372,21 @@ Alternatively these issues can be addressed > using an automounter (refer to > .BR automount (8) > for details). > +.IP > +When > +.B systemd > +is used to mount the filesystems listed in > +.IR /etc/fstab , > +the > +.B bg > +option is not supported, and may be stripped from the option list. > +Similar functionality can be achieved by providing the > +.B x-system.automount The option is called "x-systemd.automount", i.e. one more "d". > +option. This will cause > +.B systemd > +to attempt to mount the filesystem when the mountpoint is first > +accessed, rather than during system boot. The mount still happens in > +the "background", though in a different way. Looks good to me otherwise! Lennart -- Lennart Poettering, Red Hat ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH] nfs.man: document incompatibility between "bg" option and systemd. 2017-05-29 22:19 ` [PATCH] nfs.man: document incompatibility between "bg" option and systemd NeilBrown ` (2 preceding siblings ...) 2017-05-30 12:43 ` Lennart Poettering @ 2017-06-06 18:07 ` Steve Dickson 2017-06-06 19:57 ` [systemd-devel] " Michael Biebl ` (2 more replies) 2017-07-04 22:20 ` NeilBrown 4 siblings, 3 replies; 29+ messages in thread From: Steve Dickson @ 2017-06-06 18:07 UTC (permalink / raw) To: NeilBrown; +Cc: systemd-devel, linux-nfs, Lennart Poettering Hello, On 05/29/2017 06:19 PM, NeilBrown wrote: > > Systemd does not, and will not, support "bg" correctly. > It has other, better, ways to handle "background" mounting. The only problem with this is bg mounts still work at least up to 4.11 kernel... It appears there is a problem with a 4.12 kernel. The mount no longer errors out with ECONNREFUSED it just hangs in the kernel trying forever... It sounds like a bug to me but maybe that change was intentional.. Anna?? Trond??? So I'm a bit hesitant to commit this since not accurate, yet. Finally, the whole idea of systemd randomly/silently strip off mount options is crazy... IMHO... Just because a concept that has been around for years does not fix well in the systemd world it gets rip out??? IDK... but I think we can do better than that. Note, the 'bg' is used by clients that do want their booting to hang by servers that are down so if the option is rip out, boots will start hang. This will make it very difficult to debug since the bg will still exist in fstab. Again, the whole concept of systemd messing with mounts options is just not a good one... IMHO.. steved. > > Explain this. > > See also https://github.com/systemd/systemd/issues/6046 > > Signed-off-by: NeilBrown <neilb@suse.com> > --- > utils/mount/nfs.man | 18 +++++++++++++++++- > 1 file changed, 17 insertions(+), 1 deletion(-) > > diff --git a/utils/mount/nfs.man b/utils/mount/nfs.man > index cc6e992ed807..7e76492d454f 100644 > --- a/utils/mount/nfs.man > +++ b/utils/mount/nfs.man > @@ -372,6 +372,21 @@ Alternatively these issues can be addressed > using an automounter (refer to > .BR automount (8) > for details). > +.IP > +When > +.B systemd > +is used to mount the filesystems listed in > +.IR /etc/fstab , > +the > +.B bg > +option is not supported, and may be stripped from the option list. > +Similar functionality can be achieved by providing the > +.B x-system.automount > +option. This will cause > +.B systemd > +to attempt to mount the filesystem when the mountpoint is first > +accessed, rather than during system boot. The mount still happens in > +the "background", though in a different way. > .TP 1.5i > .BR rdirplus " / " nordirplus > Selects whether to use NFS v3 or v4 READDIRPLUS requests. > @@ -1810,7 +1825,8 @@ such as security negotiation, server referrals, and named attributes. > .BR rpc.idmapd (8), > .BR rpc.gssd (8), > .BR rpc.svcgssd (8), > -.BR kerberos (1) > +.BR kerberos (1), > +.BR systemd.mount (5) . > .sp > RFC 768 for the UDP specification. > .br > ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [systemd-devel] [PATCH] nfs.man: document incompatibility between "bg" option and systemd. 2017-06-06 18:07 ` Steve Dickson @ 2017-06-06 19:57 ` Michael Biebl 2017-06-07 8:13 ` Lennart Poettering 2017-06-06 21:49 ` NeilBrown 2017-06-07 8:12 ` Lennart Poettering 2 siblings, 1 reply; 29+ messages in thread From: Michael Biebl @ 2017-06-06 19:57 UTC (permalink / raw) To: Steve Dickson; +Cc: NeilBrown, linux-nfs, systemd-devel 2017-06-06 20:07 GMT+02:00 Steve Dickson <SteveD@redhat.com>: > Finally, the whole idea of systemd randomly/silently > strip off mount options is crazy... IMHO... Personally, I would prefer if systemd simply logged a warning/error message but would *not* strip the bg option. -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [systemd-devel] [PATCH] nfs.man: document incompatibility between "bg" option and systemd. 2017-06-06 19:57 ` [systemd-devel] " Michael Biebl @ 2017-06-07 8:13 ` Lennart Poettering 2017-06-07 9:42 ` Steve Dickson 0 siblings, 1 reply; 29+ messages in thread From: Lennart Poettering @ 2017-06-07 8:13 UTC (permalink / raw) To: Michael Biebl; +Cc: Steve Dickson, linux-nfs, systemd-devel, NeilBrown On Tue, 06.06.17 21:57, Michael Biebl (mbiebl@gmail.com) wrote: > 2017-06-06 20:07 GMT+02:00 Steve Dickson <SteveD@redhat.com>: > > Finally, the whole idea of systemd randomly/silently > > strip off mount options is crazy... IMHO... > > Personally, I would prefer if systemd simply logged a warning/error > message but would *not* strip the bg option. What good does that do if "bg" doesn't actually work properly? Lennart -- Lennart Poettering, Red Hat ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [systemd-devel] [PATCH] nfs.man: document incompatibility between "bg" option and systemd. 2017-06-07 8:13 ` Lennart Poettering @ 2017-06-07 9:42 ` Steve Dickson 0 siblings, 0 replies; 29+ messages in thread From: Steve Dickson @ 2017-06-07 9:42 UTC (permalink / raw) To: Lennart Poettering, Michael Biebl; +Cc: linux-nfs, systemd-devel, NeilBrown On 06/07/2017 04:13 AM, Lennart Poettering wrote: > On Tue, 06.06.17 21:57, Michael Biebl (mbiebl@gmail.com) wrote: > >> 2017-06-06 20:07 GMT+02:00 Steve Dickson <SteveD@redhat.com>: >>> Finally, the whole idea of systemd randomly/silently >>> strip off mount options is crazy... IMHO... >> >> Personally, I would prefer if systemd simply logged a warning/error >> message but would *not* strip the bg option. > > What good does that do if "bg" doesn't actually work properly? At the moment it works just fine... steved. > > Lennart > ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH] nfs.man: document incompatibility between "bg" option and systemd. 2017-06-06 18:07 ` Steve Dickson 2017-06-06 19:57 ` [systemd-devel] " Michael Biebl @ 2017-06-06 21:49 ` NeilBrown 2017-06-07 10:08 ` Steve Dickson 2017-06-07 8:12 ` Lennart Poettering 2 siblings, 1 reply; 29+ messages in thread From: NeilBrown @ 2017-06-06 21:49 UTC (permalink / raw) To: Steve Dickson; +Cc: systemd-devel, linux-nfs, Lennart Poettering [-- Attachment #1: Type: text/plain, Size: 5035 bytes --] On Tue, Jun 06 2017, Steve Dickson wrote: > Hello, > > On 05/29/2017 06:19 PM, NeilBrown wrote: >> >> Systemd does not, and will not, support "bg" correctly. >> It has other, better, ways to handle "background" mounting. > The only problem with this is bg mounts still work at least > up to 4.11 kernel... I don't think this is correct. In the default configuration, "mount -t nfs -o bg ...." runs for longer than 90 seconds, so systemd kill it. A working "bg" mount would successfully mount the filesystem is the server came back after 5 minutes. If you use current systemd in the default configuration, it won't. bg mounts do work sometimes, but I don't think they are reliable, and there seems to be no interest in changing this. Maybe the text should say "Systemd does not, and will not, reliably support "bg" mounts...". > > It appears there is a problem with a 4.12 kernel. The mount no > longer errors out with ECONNREFUSED it just hangs in the > kernel trying forever... It sounds like a bug to me but > maybe that change was intentional.. Anna?? Trond??? Might be related to my patch [PATCH] SUNRPC: ensure correct error is reported by xs_tcp_setup_socket() sent 25th May. > > So I'm a bit hesitant to commit this since not accurate, yet. > > Finally, the whole idea of systemd randomly/silently > strip off mount options is crazy... IMHO... It isn't exactly systemd, it is systemd-fstab-generator. The options lists in /etc/fstab are not all equal. Some are relevant to /bin/mount, some to mount.nfs, some to the kernel. I think /bin/mount processes the option lists before passing it to mount.nfs. There is no intrinsic reason that systemd-fstab-generator shouldn't do the same thing. > > Just because a concept that has been around for years > does not fix well in the systemd world it gets > rip out??? IDK... but I think we can do better than that. I could suggest that automount is uniformly better than bg. Give how long automount has been around, and how easy it is to use with systemd, it might be time to start encouraging people to stop using the inferior technology. I could say that, but I'm not 100% sure. The difference between automount and bg is that with bg it is easy to see if the mount has succeeded yet - just look for an empty directory. With automount, you'll typically get a delay at that point. We could possibly wind down that delay... > > Note, the 'bg' is used by clients that do want their > booting to hang by servers that are down so if the > option is rip out, boots will start hang. This > will make it very difficult to debug since the bg > will still exist in fstab. Not exactly. Current default behaviour is that systemd will wait 90 seconds, then kill the mount program and fail the boot. If we strip out "bg", the same thing will happen. I'm OK with the patch not being applied just yet. I think we need to resolve this issue, but it isn't 100% clear to me what the best resolution would be. So I'm happy to see a conversation happening and opinions being shared. I'd be particularly pleased if you could double check how "bg" is currently handled on some systemd-enabled system that you use. Does the mount program get killed like I see? Does boot succeed if there is a bg mount from an unresponsive server? Thanks, NeilBrown > > Again, the whole concept of systemd messing with mounts options > is just not a good one... IMHO.. > > steved. > >> >> Explain this. >> >> See also https://github.com/systemd/systemd/issues/6046 >> >> Signed-off-by: NeilBrown <neilb@suse.com> >> --- >> utils/mount/nfs.man | 18 +++++++++++++++++- >> 1 file changed, 17 insertions(+), 1 deletion(-) >> >> diff --git a/utils/mount/nfs.man b/utils/mount/nfs.man >> index cc6e992ed807..7e76492d454f 100644 >> --- a/utils/mount/nfs.man >> +++ b/utils/mount/nfs.man >> @@ -372,6 +372,21 @@ Alternatively these issues can be addressed >> using an automounter (refer to >> .BR automount (8) >> for details). >> +.IP >> +When >> +.B systemd >> +is used to mount the filesystems listed in >> +.IR /etc/fstab , >> +the >> +.B bg >> +option is not supported, and may be stripped from the option list. >> +Similar functionality can be achieved by providing the >> +.B x-system.automount >> +option. This will cause >> +.B systemd >> +to attempt to mount the filesystem when the mountpoint is first >> +accessed, rather than during system boot. The mount still happens in >> +the "background", though in a different way. >> .TP 1.5i >> .BR rdirplus " / " nordirplus >> Selects whether to use NFS v3 or v4 READDIRPLUS requests. >> @@ -1810,7 +1825,8 @@ such as security negotiation, server referrals, and named attributes. >> .BR rpc.idmapd (8), >> .BR rpc.gssd (8), >> .BR rpc.svcgssd (8), >> -.BR kerberos (1) >> +.BR kerberos (1), >> +.BR systemd.mount (5) . >> .sp >> RFC 768 for the UDP specification. >> .br >> [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH] nfs.man: document incompatibility between "bg" option and systemd. 2017-06-06 21:49 ` NeilBrown @ 2017-06-07 10:08 ` Steve Dickson 2017-06-07 12:02 ` Lennart Poettering 0 siblings, 1 reply; 29+ messages in thread From: Steve Dickson @ 2017-06-07 10:08 UTC (permalink / raw) To: NeilBrown; +Cc: systemd-devel, linux-nfs, Lennart Poettering On 06/06/2017 05:49 PM, NeilBrown wrote: > On Tue, Jun 06 2017, Steve Dickson wrote: > >> Hello, >> >> On 05/29/2017 06:19 PM, NeilBrown wrote: >>> >>> Systemd does not, and will not, support "bg" correctly. >>> It has other, better, ways to handle "background" mounting. >> The only problem with this is bg mounts still work at least >> up to 4.11 kernel... > > I don't think this is correct. > In the default configuration, "mount -t nfs -o bg ...." > runs for longer than 90 seconds, so systemd kill it. I must be missing something... it seems to be working for me # mount -vvv -o bg rhel7srv:/home/tmp /mnt/tmp mount.nfs: trying text-based options 'bg,vers=4.1,addr=172.31.1.60,clientaddr=172.31.1.58' mount.nfs: mount(2): Connection refused mount.nfs: trying text-based options 'bg,addr=172.31.1.60' mount.nfs: prog 100003, trying vers=3, prot=6 mount.nfs: trying 172.31.1.60 prog 100003 vers 3 prot TCP port 2049 mount.nfs: portmap query failed: RPC: Remote system error - Connection refused mount.nfs: backgrounding "rhel7srv:/home/tmp" mount.nfs: mount options: "rw,bg" # ps ax | grep mount.nfs 2270 ? Ss 0:00 /sbin/mount.nfs rhel7srv:/home/tmp /mnt/tmp -v -o rw,bg > > A working "bg" mount would successfully mount the filesystem is the > server came back after 5 minutes. If you use current systemd in the > default configuration, it won't. When I add a bg mount to fstab again... it works just fine. This is with the latest upstream nfs-utils. > > bg mounts do work sometimes, but I don't think they are reliable, and > there seems to be no interest in changing this. > Maybe the text should say "Systemd does not, and will not, reliably > support "bg" mounts...". not reliable maybe... I'm just doing very simple testing... > > >> >> It appears there is a problem with a 4.12 kernel. The mount no >> longer errors out with ECONNREFUSED it just hangs in the >> kernel trying forever... It sounds like a bug to me but >> maybe that change was intentional.. Anna?? Trond??? > > Might be related to my patch > [PATCH] SUNRPC: ensure correct error is reported by xs_tcp_setup_socket() > > sent 25th May. I'll take a look.. thanks! > >> >> So I'm a bit hesitant to commit this since not accurate, yet. >> >> Finally, the whole idea of systemd randomly/silently >> strip off mount options is crazy... IMHO... > > It isn't exactly systemd, it is systemd-fstab-generator. > The options lists in /etc/fstab are not all equal. Some > are relevant to /bin/mount, some to mount.nfs, some to the kernel. > I think /bin/mount processes the option lists before passing it > to mount.nfs. There is no intrinsic reason that systemd-fstab-generator > shouldn't do the same thing. OK. > >> >> Just because a concept that has been around for years >> does not fix well in the systemd world it gets >> rip out??? IDK... but I think we can do better than that. > > I could suggest that automount is uniformly better than bg. Give how > long automount has been around, and how easy it is to use with systemd, > it might be time to start encouraging people to stop using the inferior > technology. Yes... bg mounts are a poor man's automount... > > I could say that, but I'm not 100% sure. The difference between > automount and bg is that with bg it is easy to see if the mount has > succeeded yet - just look for an empty directory. With automount, > you'll typically get a delay at that point. We could possibly wind down > that delay... > >> >> Note, the 'bg' is used by clients that do want their >> booting to hang by servers that are down so if the >> option is rip out, boots will start hang. This >> will make it very difficult to debug since the bg >> will still exist in fstab. > > Not exactly. > Current default behaviour is that systemd will wait 90 seconds, then > kill the mount program and fail the boot. If we strip out "bg", the > same thing will happen. Again.. I'm not seeing this 90 sec delay when I add a bg mount to /etc/fstab. > > I'm OK with the patch not being applied just yet. I think we need to > resolve this issue, but it isn't 100% clear to me what the best > resolution would be. So I'm happy to see a conversation happening and > opinions being shared. > I'd be particularly pleased if you could double check how "bg" is > currently handled on some systemd-enabled system that you use. > Does the mount program get killed like I see? No. after adding a bg mount to fstab and rebooting (with the server down) I see the following mount in backgroup 1104 ? Ss 0:00 /sbin/mount.nfs nfssrv:/home/tmp /mnt/tmp -o rw,bg > Does boot succeed if there is a bg mount from an unresponsive server? Yes. Then when I bring up the server the mount succeeds steved. P.S. I'm taking some PTO today so I will not be back in the office until later today or tomorrow... steved. > > Thanks, > NeilBrown > > >> >> Again, the whole concept of systemd messing with mounts options >> is just not a good one... IMHO.. >> >> steved. >> >>> >>> Explain this. >>> >>> See also https://github.com/systemd/systemd/issues/6046 >>> >>> Signed-off-by: NeilBrown <neilb@suse.com> >>> --- >>> utils/mount/nfs.man | 18 +++++++++++++++++- >>> 1 file changed, 17 insertions(+), 1 deletion(-) >>> >>> diff --git a/utils/mount/nfs.man b/utils/mount/nfs.man >>> index cc6e992ed807..7e76492d454f 100644 >>> --- a/utils/mount/nfs.man >>> +++ b/utils/mount/nfs.man >>> @@ -372,6 +372,21 @@ Alternatively these issues can be addressed >>> using an automounter (refer to >>> .BR automount (8) >>> for details). >>> +.IP >>> +When >>> +.B systemd >>> +is used to mount the filesystems listed in >>> +.IR /etc/fstab , >>> +the >>> +.B bg >>> +option is not supported, and may be stripped from the option list. >>> +Similar functionality can be achieved by providing the >>> +.B x-system.automount >>> +option. This will cause >>> +.B systemd >>> +to attempt to mount the filesystem when the mountpoint is first >>> +accessed, rather than during system boot. The mount still happens in >>> +the "background", though in a different way. >>> .TP 1.5i >>> .BR rdirplus " / " nordirplus >>> Selects whether to use NFS v3 or v4 READDIRPLUS requests. >>> @@ -1810,7 +1825,8 @@ such as security negotiation, server referrals, and named attributes. >>> .BR rpc.idmapd (8), >>> .BR rpc.gssd (8), >>> .BR rpc.svcgssd (8), >>> -.BR kerberos (1) >>> +.BR kerberos (1), >>> +.BR systemd.mount (5) . >>> .sp >>> RFC 768 for the UDP specification. >>> .br >>> ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH] nfs.man: document incompatibility between "bg" option and systemd. 2017-06-07 10:08 ` Steve Dickson @ 2017-06-07 12:02 ` Lennart Poettering 2017-06-07 19:48 ` Steve Dickson 0 siblings, 1 reply; 29+ messages in thread From: Lennart Poettering @ 2017-06-07 12:02 UTC (permalink / raw) To: Steve Dickson; +Cc: NeilBrown, systemd-devel, linux-nfs On Wed, 07.06.17 06:08, Steve Dickson (SteveD@RedHat.com) wrote: > > > On 06/06/2017 05:49 PM, NeilBrown wrote: > > On Tue, Jun 06 2017, Steve Dickson wrote: > > > >> Hello, > >> > >> On 05/29/2017 06:19 PM, NeilBrown wrote: > >>> > >>> Systemd does not, and will not, support "bg" correctly. > >>> It has other, better, ways to handle "background" mounting. > >> The only problem with this is bg mounts still work at least > >> up to 4.11 kernel... > > > > I don't think this is correct. > > In the default configuration, "mount -t nfs -o bg ...." > > runs for longer than 90 seconds, so systemd kill it. > I must be missing something... it seems to be working for me > > # mount -vvv -o bg rhel7srv:/home/tmp /mnt/tmp > mount.nfs: trying text-based options 'bg,vers=4.1,addr=172.31.1.60,clientaddr=172.31.1.58' > mount.nfs: mount(2): Connection refused > mount.nfs: trying text-based options 'bg,addr=172.31.1.60' > mount.nfs: prog 100003, trying vers=3, prot=6 > mount.nfs: trying 172.31.1.60 prog 100003 vers 3 prot TCP port 2049 > mount.nfs: portmap query failed: RPC: Remote system error - Connection refused > mount.nfs: backgrounding "rhel7srv:/home/tmp" > mount.nfs: mount options: "rw,bg" We are talking about mounts done through /etc/fstab, i.e. the ones systemd actually manages. Lennart -- Lennart Poettering, Red Hat ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH] nfs.man: document incompatibility between "bg" option and systemd. 2017-06-07 12:02 ` Lennart Poettering @ 2017-06-07 19:48 ` Steve Dickson 2017-06-08 5:16 ` NeilBrown 0 siblings, 1 reply; 29+ messages in thread From: Steve Dickson @ 2017-06-07 19:48 UTC (permalink / raw) To: Lennart Poettering; +Cc: NeilBrown, systemd-devel, linux-nfs On 06/07/2017 08:02 AM, Lennart Poettering wrote: > On Wed, 07.06.17 06:08, Steve Dickson (SteveD@RedHat.com) wrote: > >> >> >> On 06/06/2017 05:49 PM, NeilBrown wrote: >>> On Tue, Jun 06 2017, Steve Dickson wrote: >>> >>>> Hello, >>>> >>>> On 05/29/2017 06:19 PM, NeilBrown wrote: >>>>> >>>>> Systemd does not, and will not, support "bg" correctly. >>>>> It has other, better, ways to handle "background" mounting. >>>> The only problem with this is bg mounts still work at least >>>> up to 4.11 kernel... >>> >>> I don't think this is correct. >>> In the default configuration, "mount -t nfs -o bg ...." >>> runs for longer than 90 seconds, so systemd kill it. >> I must be missing something... it seems to be working for me >> >> # mount -vvv -o bg rhel7srv:/home/tmp /mnt/tmp >> mount.nfs: trying text-based options 'bg,vers=4.1,addr=172.31.1.60,clientaddr=172.31.1.58' >> mount.nfs: mount(2): Connection refused >> mount.nfs: trying text-based options 'bg,addr=172.31.1.60' >> mount.nfs: prog 100003, trying vers=3, prot=6 >> mount.nfs: trying 172.31.1.60 prog 100003 vers 3 prot TCP port 2049 >> mount.nfs: portmap query failed: RPC: Remote system error - Connection refused >> mount.nfs: backgrounding "rhel7srv:/home/tmp" >> mount.nfs: mount options: "rw,bg" > > We are talking about mounts done through /etc/fstab, i.e. the ones > systemd actually manages. I guess I was not clear... After adding a bg mount to fstab and reboot, mounting a server that is not up, there is a mount in background that looks like # ps ax | grep mount 1104 ? Ss 0:00 /sbin/mount.nfs nfssrv:/home/tmp /mnt/tmp -o rw,bg Looking at the remote-fs.target status: # systemctl status remote-fs.target * remote-fs.target - Remote File Systems Loaded: loaded (/usr/lib/systemd/system/remote-fs.target; enabled; vendor preset: enabled) Active: active since Tue 2017-06-06 12:36:51 EDT; 12min ago Docs: man:systemd.special(7) Jun 06 12:36:51 f26.boston.devel.redhat.com systemd[1]: Reached target Remote File Systems. It appears to be successful steved. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH] nfs.man: document incompatibility between "bg" option and systemd. 2017-06-07 19:48 ` Steve Dickson @ 2017-06-08 5:16 ` NeilBrown 2017-06-08 15:36 ` J. Bruce Fields 2017-06-08 20:24 ` Steve Dickson 0 siblings, 2 replies; 29+ messages in thread From: NeilBrown @ 2017-06-08 5:16 UTC (permalink / raw) To: Steve Dickson, Lennart Poettering; +Cc: systemd-devel, linux-nfs [-- Attachment #1: Type: text/plain, Size: 6983 bytes --] On Wed, Jun 07 2017, Steve Dickson wrote: > On 06/07/2017 08:02 AM, Lennart Poettering wrote: >> On Wed, 07.06.17 06:08, Steve Dickson (SteveD@RedHat.com) wrote: >> >>> >>> >>> On 06/06/2017 05:49 PM, NeilBrown wrote: >>>> On Tue, Jun 06 2017, Steve Dickson wrote: >>>> >>>>> Hello, >>>>> >>>>> On 05/29/2017 06:19 PM, NeilBrown wrote: >>>>>> >>>>>> Systemd does not, and will not, support "bg" correctly. >>>>>> It has other, better, ways to handle "background" mounting. >>>>> The only problem with this is bg mounts still work at least >>>>> up to 4.11 kernel... >>>> >>>> I don't think this is correct. >>>> In the default configuration, "mount -t nfs -o bg ...." >>>> runs for longer than 90 seconds, so systemd kill it. >>> I must be missing something... it seems to be working for me >>> >>> # mount -vvv -o bg rhel7srv:/home/tmp /mnt/tmp >>> mount.nfs: trying text-based options 'bg,vers=4.1,addr=172.31.1.60,clientaddr=172.31.1.58' >>> mount.nfs: mount(2): Connection refused >>> mount.nfs: trying text-based options 'bg,addr=172.31.1.60' >>> mount.nfs: prog 100003, trying vers=3, prot=6 >>> mount.nfs: trying 172.31.1.60 prog 100003 vers 3 prot TCP port 2049 >>> mount.nfs: portmap query failed: RPC: Remote system error - Connection refused >>> mount.nfs: backgrounding "rhel7srv:/home/tmp" >>> mount.nfs: mount options: "rw,bg" >> >> We are talking about mounts done through /etc/fstab, i.e. the ones >> systemd actually manages. > I guess I was not clear... After adding a bg mount to fstab and > reboot, mounting a server that is not up, there is a mount in > background that looks like > > # ps ax | grep mount > 1104 ? Ss 0:00 /sbin/mount.nfs nfssrv:/home/tmp /mnt/tmp -o rw,bg > > Looking at the remote-fs.target status: > > # systemctl status remote-fs.target > * remote-fs.target - Remote File Systems > Loaded: loaded (/usr/lib/systemd/system/remote-fs.target; enabled; vendor preset: enabled) > Active: active since Tue 2017-06-06 12:36:51 EDT; 12min ago > Docs: man:systemd.special(7) > > Jun 06 12:36:51 f26.boston.devel.redhat.com systemd[1]: Reached target Remote File Systems. > > It appears to be successful Strange ... not for me. I have a recent compiled-from-source upstream systemd and a very recent upstream kernel. I add a line to /etc/fstab 192.168.1.111:/nowhere /mnt nfs bg 0 0 and reboot (192.168.1.111 is on a different subnet to the VM I am testing in, but no host responds to the address). There is a 1m30s wait while trying to mount /mnt, then boot completes (I was wrong when I said that it didn't). ● mnt.mount - /mnt Loaded: loaded (/etc/fstab; generated; vendor preset: enabled) Active: failed (Result: timeout) since Thu 2017-06-08 11:13:52 AEST; 1min 24s ago Where: /mnt What: 192.168.1.111:/nowhere Docs: man:fstab(5) man:systemd-fstab-generator(8) Process: 333 ExecMount=/bin/mount 192.168.1.111:/nowhere /mnt -t nfs -o bg (code=killed, signal=TERM) Jun 08 11:12:22 debian systemd[1]: Mounting /mnt... Jun 08 11:13:52 debian systemd[1]: mnt.mount: Mounting timed out. Stopping. Jun 08 11:13:52 debian systemd[1]: mnt.mount: Mount process exited, code=killed status=15 Jun 08 11:13:52 debian systemd[1]: Failed to mount /mnt. Jun 08 11:13:52 debian systemd[1]: mnt.mount: Unit entered failed state. The /bin/mount process has been killed (SIGTERM) after the 90 second timeout ● remote-fs.target - Remote File Systems Loaded: loaded (/lib/systemd/system/remote-fs.target; enabled; vendor preset: enabled) Drop-In: /run/systemd/generator/remote-fs.target.d └─50-insserv.conf.conf Active: inactive (dead) Docs: man:systemd.special(7) Jun 08 11:13:52 debian systemd[1]: Dependency failed for Remote File Systems. Jun 08 11:13:52 debian systemd[1]: remote-fs.target: Job remote-fs.target/start failed with result 'dependency'. remote-fs.target has not been reached. Because remote-fs.target is WantedBy multi-user.target, but need RequiredBy it, boot completes. But if anything did Require remote-fs.target, it would fail if "bg" mounts were not mounted within 90 seconds. Looking back over your log messages: >>> mount.nfs: portmap query failed: RPC: Remote system error - Connection refused >>> mount.nfs: backgrounding "rhel7srv:/home/tmp" it appears that the fg mount attempt failed quickly (ECONNREFUSED), so it background the process immediately. Systemd sees that as success (despite the fact that the filesystem isn't actually mounted) and doesn't clean up the background processes (which is arguably a bug). If you try to mount from a server which doesn't even return ECONNREFUSED or EHOSTUNREACH, such that the first mount attempt takes longer than 90 seconds, then you should get the same results as me. Your results go some way to explaining why Lennart hasn't received many complaints, but I'm convinced that the current situation is not ideal (because SUSE has received some complaints). I've been pondering the possibility of making "bg" work properly with systemd and I think I've found a promising approach. It involves having systemd take responsibility for the "run in the background" part. If we get systemd-fstab-generator to translate "bg" to "retry=10000", then "mount.nfs" will behave like the background version of "mount.nfs -o bg". i.e. it will retry for one month (nearly). If there is already a 'retry=' option, we just strip the "bg". For this to work, we would need to add TimeoutSec=infinity to the .mount unit file so that systemd doesn't kill the mount. We would also need to add (the effect of) "nofail", so that systemd doesn't wait for the mount to complete... Except that the current implementation of "nofail" is faulty. It removes the default "Before=remote-fs.mount", which has the unwanted consequence of unmounting the filesystem too early at shutdown. I have a solution for that too (which I'll submit a push request for shortly). My solution to "nofail" is to treat it much like "automount", but instead of using an automount unit to trigger the mount, we use a timer unit (with OnActiveSec=0). By triggering the mount unit with a timer instead of Wanting it directly, it gets run in a separate transaction. This means that the "Before=remote-fs.target" doesn't have the effect of delaying remote-fs.target. Before/After only order jobs within the one transaction. When it comes time to shutdown, remote-fs.target and the foo.mount will be in the same transaction, so the Before= will ensure foo.mount isn't unmounted until after remote-fs.target has been allows to finish, which is after any services that might be using the filesystem. So I think I've found a solution for systemd to handle "bg" nfs mounts correctly. I'll submit some pull requests for consideration. Thanks, NeilBrown [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH] nfs.man: document incompatibility between "bg" option and systemd. 2017-06-08 5:16 ` NeilBrown @ 2017-06-08 15:36 ` J. Bruce Fields 2017-06-08 21:54 ` NeilBrown 2017-06-08 20:24 ` Steve Dickson 1 sibling, 1 reply; 29+ messages in thread From: J. Bruce Fields @ 2017-06-08 15:36 UTC (permalink / raw) To: NeilBrown; +Cc: Steve Dickson, Lennart Poettering, systemd-devel, linux-nfs On Thu, Jun 08, 2017 at 03:16:52PM +1000, NeilBrown wrote: > So I think I've found a solution for systemd to handle "bg" nfs mounts > correctly. I'll submit some pull requests for consideration. Out of curiosity, after that change is there still any reason you'd recommend any new user actually use "bg" (as opposed to an automount)? I appreciate the effort to keep existing systems working, I'm just curious. --b. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH] nfs.man: document incompatibility between "bg" option and systemd. 2017-06-08 15:36 ` J. Bruce Fields @ 2017-06-08 21:54 ` NeilBrown 0 siblings, 0 replies; 29+ messages in thread From: NeilBrown @ 2017-06-08 21:54 UTC (permalink / raw) To: J. Bruce Fields Cc: Steve Dickson, Lennart Poettering, systemd-devel, linux-nfs [-- Attachment #1: Type: text/plain, Size: 2069 bytes --] On Thu, Jun 08 2017, J. Bruce Fields wrote: > On Thu, Jun 08, 2017 at 03:16:52PM +1000, NeilBrown wrote: >> So I think I've found a solution for systemd to handle "bg" nfs mounts >> correctly. I'll submit some pull requests for consideration. > > Out of curiosity, after that change is there still any reason you'd > recommend any new user actually use "bg" (as opposed to an automount)? Me? Recommend? Who would listen? Who would even hear? For the last few years I've been recommending that automount should be used for *all* NFS mounts at every opportunity. I think I've had two opportunities. But no, I would not recommend "bg". I would recommend automount and then when they reported problems, I would help fix them. I would be much happier recommending automount if it were easier. Setting up /etc/auto.direct with automountd is fairly easy, but you need to actually enable it but modifying auto.master or auto.master.d, which is slightly annoying. systemd does make it easier to do direct mounts, but it is ugly. You need to include "comment=systemd.automount" or "x-systemd.automount" in /etc/fstab instead of just "automount" or "ondemand". I understand exactly why they did that and I cannot fault the logic. But it still looks clumsy. With systemd, you cannot divorce the timeout that an application has to wait when accessing the mountpoint while the server is down, from the timeout imposed on the mount program. i.e., mount cannot keep trying in the background. - that could be useful if you want really-short timeouts... at least it seems to me that they should be separate. The timeout is configured differently if mounting from a device, or mounting from anything else such as NFS. The first uses x-systemd.device-timeout. The other needs x-systemd.mount-timeout. But I'm ranting... I should probably shut up and send patches. A generator for /etc/fstab.auto?? > > I appreciate the effort to keep existing systems working, I'm just > curious. Compatibility with existing practice is certainly the main driver. Thanks, NeilBrown [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH] nfs.man: document incompatibility between "bg" option and systemd. 2017-06-08 5:16 ` NeilBrown 2017-06-08 15:36 ` J. Bruce Fields @ 2017-06-08 20:24 ` Steve Dickson 1 sibling, 0 replies; 29+ messages in thread From: Steve Dickson @ 2017-06-08 20:24 UTC (permalink / raw) To: NeilBrown, Lennart Poettering; +Cc: systemd-devel, linux-nfs On 06/08/2017 01:16 AM, NeilBrown wrote: > On Wed, Jun 07 2017, Steve Dickson wrote: > >> On 06/07/2017 08:02 AM, Lennart Poettering wrote: >>> On Wed, 07.06.17 06:08, Steve Dickson (SteveD@RedHat.com) wrote: >>> >>>> >>>> >>>> On 06/06/2017 05:49 PM, NeilBrown wrote: >>>>> On Tue, Jun 06 2017, Steve Dickson wrote: >>>>> >>>>>> Hello, >>>>>> >>>>>> On 05/29/2017 06:19 PM, NeilBrown wrote: >>>>>>> >>>>>>> Systemd does not, and will not, support "bg" correctly. >>>>>>> It has other, better, ways to handle "background" mounting. >>>>>> The only problem with this is bg mounts still work at least >>>>>> up to 4.11 kernel... >>>>> >>>>> I don't think this is correct. >>>>> In the default configuration, "mount -t nfs -o bg ...." >>>>> runs for longer than 90 seconds, so systemd kill it. >>>> I must be missing something... it seems to be working for me >>>> >>>> # mount -vvv -o bg rhel7srv:/home/tmp /mnt/tmp >>>> mount.nfs: trying text-based options 'bg,vers=4.1,addr=172.31.1.60,clientaddr=172.31.1.58' >>>> mount.nfs: mount(2): Connection refused >>>> mount.nfs: trying text-based options 'bg,addr=172.31.1.60' >>>> mount.nfs: prog 100003, trying vers=3, prot=6 >>>> mount.nfs: trying 172.31.1.60 prog 100003 vers 3 prot TCP port 2049 >>>> mount.nfs: portmap query failed: RPC: Remote system error - Connection refused >>>> mount.nfs: backgrounding "rhel7srv:/home/tmp" >>>> mount.nfs: mount options: "rw,bg" >>> >>> We are talking about mounts done through /etc/fstab, i.e. the ones >>> systemd actually manages. >> I guess I was not clear... After adding a bg mount to fstab and >> reboot, mounting a server that is not up, there is a mount in >> background that looks like >> >> # ps ax | grep mount >> 1104 ? Ss 0:00 /sbin/mount.nfs nfssrv:/home/tmp /mnt/tmp -o rw,bg >> >> Looking at the remote-fs.target status: >> >> # systemctl status remote-fs.target >> * remote-fs.target - Remote File Systems >> Loaded: loaded (/usr/lib/systemd/system/remote-fs.target; enabled; vendor preset: enabled) >> Active: active since Tue 2017-06-06 12:36:51 EDT; 12min ago >> Docs: man:systemd.special(7) >> >> Jun 06 12:36:51 f26.boston.devel.redhat.com systemd[1]: Reached target Remote File Systems. >> >> It appears to be successful > > Strange ... not for me. > > I have a recent compiled-from-source upstream systemd and a very recent > upstream kernel. > > I add a line to /etc/fstab > > 192.168.1.111:/nowhere /mnt nfs bg 0 0 > > and reboot (192.168.1.111 is on a different subnet to the VM I am > testing in, but no host responds to the address). > > There is a 1m30s wait while trying to mount /mnt, then boot completes > (I was wrong when I said that it didn't). > > ● mnt.mount - /mnt > Loaded: loaded (/etc/fstab; generated; vendor preset: enabled) > Active: failed (Result: timeout) since Thu 2017-06-08 11:13:52 AEST; 1min 24s ago > Where: /mnt > What: 192.168.1.111:/nowhere > Docs: man:fstab(5) > man:systemd-fstab-generator(8) > Process: 333 ExecMount=/bin/mount 192.168.1.111:/nowhere /mnt -t nfs -o bg (code=killed, signal=TERM) > > Jun 08 11:12:22 debian systemd[1]: Mounting /mnt... > Jun 08 11:13:52 debian systemd[1]: mnt.mount: Mounting timed out. Stopping. > Jun 08 11:13:52 debian systemd[1]: mnt.mount: Mount process exited, code=killed status=15 > Jun 08 11:13:52 debian systemd[1]: Failed to mount /mnt. > Jun 08 11:13:52 debian systemd[1]: mnt.mount: Unit entered failed state. > > > The /bin/mount process has been killed (SIGTERM) after the 90 second > timeout > > ● remote-fs.target - Remote File Systems > Loaded: loaded (/lib/systemd/system/remote-fs.target; enabled; vendor preset: enabled) > Drop-In: /run/systemd/generator/remote-fs.target.d > └─50-insserv.conf.conf > Active: inactive (dead) > Docs: man:systemd.special(7) > > Jun 08 11:13:52 debian systemd[1]: Dependency failed for Remote File Systems. > Jun 08 11:13:52 debian systemd[1]: remote-fs.target: Job remote-fs.target/start failed with result 'dependency'. > > > remote-fs.target has not been reached. I'm seeing this now... the server has to be up to cause this hang. > > Because remote-fs.target is WantedBy multi-user.target, but need > RequiredBy it, boot completes. > But if anything did Require remote-fs.target, it would fail if "bg" > mounts were not mounted within 90 seconds. > > > Looking back over your log messages: > >>>> mount.nfs: portmap query failed: RPC: Remote system error - Connection refused >>>> mount.nfs: backgrounding "rhel7srv:/home/tmp" > > it appears that the fg mount attempt failed quickly (ECONNREFUSED), so it > background the process immediately. Systemd sees that as success > (despite the fact that the filesystem isn't actually mounted) and > doesn't clean up the background processes (which is arguably a bug). No... Systemd is doing the right thing in this case... Letting bg mounts work... > > If you try to mount from a server which doesn't even return ECONNREFUSED or > EHOSTUNREACH, such that the first mount attempt takes longer than 90 > seconds, then you should get the same results as me. This is clearly a bug in systemd! The 'bg' mounts work with the server down but hangs when the server is up. How can't this be a bug in systemd?? Why is systemd even looking at or interpreting the mount options??? This clear an overreach of systemd, IMHO. Then just to blindly rip them out... WOW! I'm only assuming... but it appears, do to this overreach, that systemd actually think it know how to mount a file system better than the actual mount command tailor for particular that file system and if it doesn't like an mount option it just going to rip it out! Again, I hope this is not the case because if its... that's just crazy!!! > > Your results go some way to explaining why Lennart hasn't received many > complaints, but I'm convinced that the current situation is not ideal > (because SUSE has received some complaints). It just proves bg mounts are being used. > > I've been pondering the possibility of making "bg" work properly with > systemd and I think I've found a promising approach. It involves having > systemd take responsibility for the "run in the background" part. > > If we get systemd-fstab-generator to translate "bg" to "retry=10000", > then "mount.nfs" will behave like the background version of > "mount.nfs -o bg". i.e. it will retry for one month (nearly). If there is > already a 'retry=' option, we just strip the "bg". > > For this to work, we would need to add > TimeoutSec=infinity > to the .mount unit file so that systemd doesn't kill the mount. > We would also need to add (the effect of) "nofail", so that systemd > doesn't wait for the mount to complete... > Except that the current implementation of "nofail" is faulty. > It removes the default "Before=remote-fs.mount", which has the unwanted > consequence of unmounting the filesystem too early at shutdown. > > I have a solution for that too (which I'll submit a push request for > shortly). > My solution to "nofail" is to treat it much like "automount", but > instead of using an automount unit to trigger the mount, we use > a timer unit (with OnActiveSec=0). > By triggering the mount unit with a timer instead of Wanting it > directly, it gets run in a separate transaction. This means that the > "Before=remote-fs.target" doesn't have the effect of delaying > remote-fs.target. Before/After only order jobs within the one > transaction. > When it comes time to shutdown, remote-fs.target and the foo.mount will > be in the same transaction, so the Before= will ensure foo.mount > isn't unmounted until after remote-fs.target has been allows to finish, > which is after any services that might be using the filesystem. > > So I think I've found a solution for systemd to handle "bg" nfs mounts > correctly. I'll submit some pull requests for consideration. Neil, your an excellent engineer so I'm sure you will craft something very cool... but I truly feel you are breaking one of the NFS communities long standing golden rule... We never add fix to client that fixes a bug in the server... Fix the server... That is the case here, IMHO... There is a bug in systemd that is not letting a mount command succeed... It's clearly their bug, let them fix it... Again, why systemd looking at mount options is just mind boggling... steved. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH] nfs.man: document incompatibility between "bg" option and systemd. 2017-06-06 18:07 ` Steve Dickson 2017-06-06 19:57 ` [systemd-devel] " Michael Biebl 2017-06-06 21:49 ` NeilBrown @ 2017-06-07 8:12 ` Lennart Poettering 2017-06-07 10:04 ` Steve Dickson 2 siblings, 1 reply; 29+ messages in thread From: Lennart Poettering @ 2017-06-07 8:12 UTC (permalink / raw) To: Steve Dickson; +Cc: NeilBrown, systemd-devel, linux-nfs On Tue, 06.06.17 14:07, Steve Dickson (SteveD@RedHat.com) wrote: > Hello, > > On 05/29/2017 06:19 PM, NeilBrown wrote: > > > > Systemd does not, and will not, support "bg" correctly. > > It has other, better, ways to handle "background" mounting. > The only problem with this is bg mounts still work at least > up to 4.11 kernel... > > It appears there is a problem with a 4.12 kernel. The mount no > longer errors out with ECONNREFUSED it just hangs in the > kernel trying forever... It sounds like a bug to me but > maybe that change was intentional.. Anna?? Trond??? > > So I'm a bit hesitant to commit this since not accurate, yet. > > Finally, the whole idea of systemd randomly/silently > strip off mount options is crazy... IMHO... > > Just because a concept that has been around for years > does not fix well in the systemd world it gets > rip out??? IDK... but I think we can do better than that. Well "bg" doesn't really work on systemd systems, and this was never different, hence I think it's only fair to document that it is incompatible with systemd. In addition, I have the suspicion it is not used very widely, since I never actually got complaints about it. Lennart -- Lennart Poettering, Red Hat ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH] nfs.man: document incompatibility between "bg" option and systemd. 2017-06-07 8:12 ` Lennart Poettering @ 2017-06-07 10:04 ` Steve Dickson 2017-06-07 16:08 ` J. Bruce Fields 0 siblings, 1 reply; 29+ messages in thread From: Steve Dickson @ 2017-06-07 10:04 UTC (permalink / raw) To: Lennart Poettering; +Cc: NeilBrown, systemd-devel, linux-nfs On 06/07/2017 04:12 AM, Lennart Poettering wrote: > On Tue, 06.06.17 14:07, Steve Dickson (SteveD@RedHat.com) wrote: > >> Hello, >> >> On 05/29/2017 06:19 PM, NeilBrown wrote: >>> >>> Systemd does not, and will not, support "bg" correctly. >>> It has other, better, ways to handle "background" mounting. >> The only problem with this is bg mounts still work at least >> up to 4.11 kernel... >> >> It appears there is a problem with a 4.12 kernel. The mount no >> longer errors out with ECONNREFUSED it just hangs in the >> kernel trying forever... It sounds like a bug to me but >> maybe that change was intentional.. Anna?? Trond??? >> >> So I'm a bit hesitant to commit this since not accurate, yet. >> >> Finally, the whole idea of systemd randomly/silently >> strip off mount options is crazy... IMHO... >> >> Just because a concept that has been around for years >> does not fix well in the systemd world it gets >> rip out??? IDK... but I think we can do better than that. > > Well "bg" doesn't really work on systemd systems, and this was never > different, hence I think it's only fair to document that it is > incompatible with systemd. I'm seeing it work just fine... /etc/fstab: nfssrv:/home/tmp /mnt/tmp nfs bg 0 0 nfssrv is down reboot client login to client # ps ax | grep mount 980 ? Ss 0:00 /sbin/mount.nfs nfssrv:/home/tmp /mnt/tmp -o rw,bg > In addition, I have the suspicion it is not > used very widely, since I never actually got complaints about it. Since it seems to be still working we probably would not hear any complaints about it... steved. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH] nfs.man: document incompatibility between "bg" option and systemd. 2017-06-07 10:04 ` Steve Dickson @ 2017-06-07 16:08 ` J. Bruce Fields 2017-06-08 20:34 ` Steve Dickson 0 siblings, 1 reply; 29+ messages in thread From: J. Bruce Fields @ 2017-06-07 16:08 UTC (permalink / raw) To: Steve Dickson; +Cc: Lennart Poettering, NeilBrown, systemd-devel, linux-nfs On Wed, Jun 07, 2017 at 06:04:12AM -0400, Steve Dickson wrote: > > > On 06/07/2017 04:12 AM, Lennart Poettering wrote: > > On Tue, 06.06.17 14:07, Steve Dickson (SteveD@RedHat.com) wrote: > > > >> Hello, > >> > >> On 05/29/2017 06:19 PM, NeilBrown wrote: > >>> > >>> Systemd does not, and will not, support "bg" correctly. > >>> It has other, better, ways to handle "background" mounting. > >> The only problem with this is bg mounts still work at least > >> up to 4.11 kernel... > >> > >> It appears there is a problem with a 4.12 kernel. The mount no > >> longer errors out with ECONNREFUSED it just hangs in the > >> kernel trying forever... It sounds like a bug to me but > >> maybe that change was intentional.. Anna?? Trond??? > >> > >> So I'm a bit hesitant to commit this since not accurate, yet. > >> > >> Finally, the whole idea of systemd randomly/silently > >> strip off mount options is crazy... IMHO... > >> > >> Just because a concept that has been around for years > >> does not fix well in the systemd world it gets > >> rip out??? IDK... but I think we can do better than that. > > > > Well "bg" doesn't really work on systemd systems, and this was never > > different, hence I think it's only fair to document that it is > > incompatible with systemd. > I'm seeing it work just fine... > > /etc/fstab: nfssrv:/home/tmp /mnt/tmp nfs bg 0 0 > nfssrv is down > reboot client > login to client > # ps ax | grep mount > 980 ? Ss 0:00 /sbin/mount.nfs nfssrv:/home/tmp /mnt/tmp -o rw,bg Right, but I think we also need to see a "systemctl status remote-fs.target", or something, to verify whether that's the forked background process or just the foreground process that's still hanging up some part of the boot process (even though it's gotten far enough along that you can log in--unless logins aren't permitted till remote fs's are mounted, I don't know.) --b. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH] nfs.man: document incompatibility between "bg" option and systemd. 2017-06-07 16:08 ` J. Bruce Fields @ 2017-06-08 20:34 ` Steve Dickson 0 siblings, 0 replies; 29+ messages in thread From: Steve Dickson @ 2017-06-08 20:34 UTC (permalink / raw) To: J. Bruce Fields; +Cc: Lennart Poettering, NeilBrown, systemd-devel, linux-nfs On 06/07/2017 12:08 PM, J. Bruce Fields wrote: > On Wed, Jun 07, 2017 at 06:04:12AM -0400, Steve Dickson wrote: >> # ps ax | grep mount >> 980 ? Ss 0:00 /sbin/mount.nfs nfssrv:/home/tmp /mnt/tmp -o rw,bg > > Right, but I think we also need to see a "systemctl status > remote-fs.target", or something, to verify whether that's the forked > background process or just the foreground process that's still hanging > up some part of the boot process (even though it's gotten far enough > along that you can log in--unless logins aren't permitted till remote > fs's are mounted, I don't know.) It succeeds... # systemctl status remote-fs.target * remote-fs.target - Remote File Systems Loaded: loaded (/usr/lib/systemd/system/remote-fs.target; enabled; vendor preset: enabled) Active: active since Tue 2017-06-06 12:36:51 EDT; 12min ago Docs: man:systemd.special(7) Jun 06 12:36:51 f26 systemd[1]: Reached target Remote File Systems. The reason being, as Neil pointed out, the mount.nfs gets the ECONNREFUSED right away because the server is down. So a child is quickly forked that continues to try the mount... Basically sneaking around systemd back... Which is hard to do... these day 8-) steved. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH] nfs.man: document incompatibility between "bg" option and systemd. 2017-05-29 22:19 ` [PATCH] nfs.man: document incompatibility between "bg" option and systemd NeilBrown ` (3 preceding siblings ...) 2017-06-06 18:07 ` Steve Dickson @ 2017-07-04 22:20 ` NeilBrown 2017-07-10 15:26 ` Steve Dickson 4 siblings, 1 reply; 29+ messages in thread From: NeilBrown @ 2017-07-04 22:20 UTC (permalink / raw) To: Steve Dickson; +Cc: systemd-devel, linux-nfs, Lennart Poettering [-- Attachment #1: Type: text/plain, Size: 456 bytes --] On Tue, May 30 2017, NeilBrown wrote: > Systemd does not, and will not, support "bg" correctly. > It has other, better, ways to handle "background" mounting. For those who aren't closely watching systemd development, a patch was recently accepted which causes systemd to work correctly with NFS bg mounts. So the above "and will not" was, happily, not correct. So this patch to the nfs documentation is no longer relevant. Thanks everyone, NeilBrown [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH] nfs.man: document incompatibility between "bg" option and systemd. 2017-07-04 22:20 ` NeilBrown @ 2017-07-10 15:26 ` Steve Dickson 2017-07-10 22:56 ` NeilBrown 0 siblings, 1 reply; 29+ messages in thread From: Steve Dickson @ 2017-07-10 15:26 UTC (permalink / raw) To: NeilBrown; +Cc: systemd-devel, linux-nfs, Lennart Poettering Hey Neil, On 07/04/2017 06:20 PM, NeilBrown wrote: > On Tue, May 30 2017, NeilBrown wrote: > >> Systemd does not, and will not, support "bg" correctly. >> It has other, better, ways to handle "background" mounting. > > For those who aren't closely watching systemd development, a > patch was recently accepted which causes systemd to work correctly with > NFS bg mounts. So the above "and will not" was, happily, not correct. Could you please post a pointer to the thread? tia, steved. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH] nfs.man: document incompatibility between "bg" option and systemd. 2017-07-10 15:26 ` Steve Dickson @ 2017-07-10 22:56 ` NeilBrown 0 siblings, 0 replies; 29+ messages in thread From: NeilBrown @ 2017-07-10 22:56 UTC (permalink / raw) To: Steve Dickson; +Cc: systemd-devel, linux-nfs, Lennart Poettering [-- Attachment #1: Type: text/plain, Size: 1578 bytes --] On Mon, Jul 10 2017, Steve Dickson wrote: > Hey Neil, > > On 07/04/2017 06:20 PM, NeilBrown wrote: >> On Tue, May 30 2017, NeilBrown wrote: >> >>> Systemd does not, and will not, support "bg" correctly. >>> It has other, better, ways to handle "background" mounting. >> >> For those who aren't closely watching systemd development, a >> patch was recently accepted which causes systemd to work correctly with >> NFS bg mounts. So the above "and will not" was, happily, not correct. > Could you please post a pointer to the thread? The main commmit is https://github.com/systemd/systemd/commit/65e1dee7dcf1668c25c32f0238c935708dbffbcf The link in the title leads to https://github.com/systemd/systemd/pull/6103 which is the discussion of the pull request. The "Fixes" link at the bottom leads to https://github.com/systemd/systemd/issues/6046 which is an 'issue' that started out as "we should warn if 'bg' is used" but became "here is a fix so bg works". I've been wondering if we should add text to nfs.5 suggesting that automount is often a good match for NFS, and can particularly be useful where "bg" is currently used. Not sure what to do about timeouts though. systemd currently waits for the mount command to fail, or kills it after 90 seconds. It doesn't return an error to accesses of the mount point (typically ENODEV) until mount exits. I think I would prefer a shorter timeout before the error, but a longer timeout before killing mount. But I'm not really sure. Does anyone have opinions? Thanks, NeilBrown [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 29+ messages in thread
end of thread, other threads:[~2017-07-10 22:56 UTC | newest] Thread overview: 29+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-05-26 2:46 systemd and NFS "bg" mounts NeilBrown 2017-05-29 13:38 ` [systemd-devel] " Lennart Poettering 2017-05-29 22:05 ` NeilBrown 2017-05-29 22:19 ` [PATCH] nfs.man: document incompatibility between "bg" option and systemd NeilBrown 2017-05-30 4:47 ` Niels de Vos 2017-05-30 7:40 ` [systemd-devel] " Michael Biebl 2017-05-30 8:55 ` NeilBrown 2017-05-30 9:15 ` Michael Biebl 2017-05-30 12:45 ` Lennart Poettering 2017-05-30 12:43 ` Lennart Poettering 2017-06-06 18:07 ` Steve Dickson 2017-06-06 19:57 ` [systemd-devel] " Michael Biebl 2017-06-07 8:13 ` Lennart Poettering 2017-06-07 9:42 ` Steve Dickson 2017-06-06 21:49 ` NeilBrown 2017-06-07 10:08 ` Steve Dickson 2017-06-07 12:02 ` Lennart Poettering 2017-06-07 19:48 ` Steve Dickson 2017-06-08 5:16 ` NeilBrown 2017-06-08 15:36 ` J. Bruce Fields 2017-06-08 21:54 ` NeilBrown 2017-06-08 20:24 ` Steve Dickson 2017-06-07 8:12 ` Lennart Poettering 2017-06-07 10:04 ` Steve Dickson 2017-06-07 16:08 ` J. Bruce Fields 2017-06-08 20:34 ` Steve Dickson 2017-07-04 22:20 ` NeilBrown 2017-07-10 15:26 ` Steve Dickson 2017-07-10 22:56 ` NeilBrown
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.