numeric UIDs

numeric UIDs

[Victor Mataré]

Hello,

I still hope I'm mistaken in assuming I have to go back to NFSv3 if I want to 
skip NFSv4 UID mapping altogether and just use the numeric UIDs the way 
they're stored on-disk. However if that's actually true, I'd like to try and 
make a case for implementing an option to turn off UID mapping completely (or 
at least for unknown UIDs). If this is already work in progress, just ignore 
this mail.

Thing is, the forced UID mapping seems to make tasks like backing up data a 
little inconvenient. You might want to preserve UIDs that are only known to 
the client.
But when you copy an entire root filesystem, it becomes outright destructive, 
because the rootfs will probably have several accounts that the server can't 
be expected know. Just imagine a server that's used for maintenance (like 
backing up and replacing hard drives) of random (foreign) systems. Idmapd will 
map all unknown UIDs to a single value and thereby destroy that information.

I think I read somewhere that the Sun people already have a way of handling 
this. Any chance Linux could do that, too?

Please excuse me if I'm barking up the wrong tree. If this has already been 
discussed, I'd appreciate a pointer.

Thanks,
Victor

Re: numeric UIDs

[Jim Rees]

Victor Mataré wrote:

  I'd like to try and 
  make a case for implementing an option to turn off UID mapping completely (or 
  at least for unknown UIDs).

I've always thought this would be useful too, just as an option for auth_sys
of course.  Some people think it's a security problem but since there is no
security in an auth_sys mount I don't see what the issue is.

Re: numeric UIDs

[J. Bruce Fields]

On Tue, Aug 03, 2010 at 12:43:18PM -0400, Jim Rees wrote:
> Victor Mataré wrote:
> 
>   I'd like to try and 
>   make a case for implementing an option to turn off UID mapping completely (or 
>   at least for unknown UIDs).
> 
> I've always thought this would be useful too, just as an option for auth_sys
> of course.  Some people think it's a security problem but since there is no
> security in an auth_sys mount I don't see what the issue is.

I'm not sure what the security flavor has to do with it.

I'll admit I like the idea of a v2/v3-compatibility mode that uses
(ascii-encoded) uid's only, though I think it's a technical violation of
the spec.

--b.

Re: numeric UIDs

[J. Bruce Fields]

On Tue, Aug 03, 2010 at 04:01:32AM +0200, Victor Mataré wrote:
> Hello,
> 
> I still hope I'm mistaken in assuming I have to go back to NFSv3 if I want to 
> skip NFSv4 UID mapping altogether and just use the numeric UIDs the way 
> they're stored on-disk. However if that's actually true, I'd like to try and 
> make a case for implementing an option to turn off UID mapping completely (or 
> at least for unknown UIDs). If this is already work in progress, just ignore 
> this mail.
> 
> Thing is, the forced UID mapping seems to make tasks like backing up data a 
> little inconvenient. You might want to preserve UIDs that are only known to 
> the client.
> But when you copy an entire root filesystem, it becomes outright destructive, 
> because the rootfs will probably have several accounts that the server can't 
> be expected know. Just imagine a server that's used for maintenance (like 
> backing up and replacing hard drives) of random (foreign) systems. Idmapd will 
> map all unknown UIDs to a single value and thereby destroy that information.
> 
> I think I read somewhere

Pointer?

--b.

> that the Sun people already have a way of handling 
> this. Any chance Linux could do that, too?
> 
> Please excuse me if I'm barking up the wrong tree. If this has already been 
> discussed, I'd appreciate a pointer.
> 
> Thanks,
> Victor
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: numeric UIDs

[Daniel.Muntz]

I'll fourth this motion.  The spec goes out of its way to declare this a violation.  IMHO, the NFSv4.[0-n] specs should adopt the convention that a uid string consisting of [0-9]+ be interpreted as the string representation of a numeric UID--just as valid as a "user@domain" string.  There are systems for which the name mapping is unnecessary, or even detrimental to implement.

  -Dan 

-----Original Message-----
From: linux-nfs-owner@vger.kernel.org [mailto:linux-nfs-owner@vger.kernel.org] On Behalf Of J. Bruce Fields
Sent: Tuesday, August 03, 2010 12:22 PM
To: Jim Rees
Cc: linux-nfs@vger.kernel.org
Subject: Re: numeric UIDs

On Tue, Aug 03, 2010 at 12:43:18PM -0400, Jim Rees wrote:
> Victor Mataré wrote:
> 
>   I'd like to try and 
>   make a case for implementing an option to turn off UID mapping completely (or 
>   at least for unknown UIDs).
> 
> I've always thought this would be useful too, just as an option for auth_sys
> of course.  Some people think it's a security problem but since there is no
> security in an auth_sys mount I don't see what the issue is.

I'm not sure what the security flavor has to do with it.

I'll admit I like the idea of a v2/v3-compatibility mode that uses
(ascii-encoded) uid's only, though I think it's a technical violation of
the spec.

--b.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: numeric UIDs

[Jim Rees]

Daniel.Muntz@emc.com wrote:

  I'll fourth this motion.  The spec goes out of its way to declare this a
  violation.  IMHO, the NFSv4.[0-n] specs should adopt the convention that a
  uid string consisting of [0-9]+ be interpreted as the string
  representation of a numeric UID--just as valid as a "user@domain" string.

I argued for this as an option in the early days but was shouted down.
Sorry I can't remember the details, it was many years ago.

Re: numeric UIDs

[Trond Myklebust]

On Tue, 2010-08-03 at 17:57 -0400, Jim Rees wrote:
> Daniel.Muntz@emc.com wrote:
> 
>   I'll fourth this motion.  The spec goes out of its way to declare this a
>   violation.  IMHO, the NFSv4.[0-n] specs should adopt the convention that a
>   uid string consisting of [0-9]+ be interpreted as the string
>   representation of a numeric UID--just as valid as a "user@domain" string.
> 
> I argued for this as an option in the early days but was shouted down.
> Sorry I can't remember the details, it was many years ago.

Why is nobody talking about fixing AUTH_SYS? The alternative to using
numeric uids/gids in NFS would be to use user@domain/group@domain in the
credential.

I believe that Nico had some proposals for RPCSEC_GSSv3 that addresses
this issue. If adopted, it would even be backwards compatible with
NFSv4.0.

Trond

Re: numeric UIDs

[J. Bruce Fields]

On Tue, Aug 03, 2010 at 06:15:19PM -0400, Trond Myklebust wrote:
> On Tue, 2010-08-03 at 17:57 -0400, Jim Rees wrote:
> > Daniel.Muntz@emc.com wrote:
> > 
> >   I'll fourth this motion.  The spec goes out of its way to declare this a
> >   violation.  IMHO, the NFSv4.[0-n] specs should adopt the convention that a
> >   uid string consisting of [0-9]+ be interpreted as the string
> >   representation of a numeric UID--just as valid as a "user@domain" string.
> > 
> > I argued for this as an option in the early days but was shouted down.
> > Sorry I can't remember the details, it was many years ago.
> 
> Why is nobody talking about fixing AUTH_SYS? The alternative to using
> numeric uids/gids in NFS would be to use user@domain/group@domain in the
> credential.

I'm not sure what that does to address complaints like original
poster's:

	http://marc.info/?l=linux-nfs&m=128080127215350&w=2

And I'd like it to be possible to make the NFSv3->NFSv4 upgrade as
transparent as possible.

--b.

> 
> I believe that Nico had some proposals for RPCSEC_GSSv3 that addresses
> this issue. If adopted, it would even be backwards compatible with
> NFSv4.0.
> 
> Trond
> 

Re: numeric UIDs

[Trond Myklebust]

On Tue, 2010-08-03 at 18:23 -0400, J. Bruce Fields wrote:
> On Tue, Aug 03, 2010 at 06:15:19PM -0400, Trond Myklebust wrote:
> > On Tue, 2010-08-03 at 17:57 -0400, Jim Rees wrote:
> > > Daniel.Muntz@emc.com wrote:
> > > 
> > >   I'll fourth this motion.  The spec goes out of its way to declare this a
> > >   violation.  IMHO, the NFSv4.[0-n] specs should adopt the convention that a
> > >   uid string consisting of [0-9]+ be interpreted as the string
> > >   representation of a numeric UID--just as valid as a "user@domain" string.
> > > 
> > > I argued for this as an option in the early days but was shouted down.
> > > Sorry I can't remember the details, it was many years ago.
> > 
> > Why is nobody talking about fixing AUTH_SYS? The alternative to using
> > numeric uids/gids in NFS would be to use user@domain/group@domain in the
> > credential.
> 
> I'm not sure what that does to address complaints like original
> poster's:
> 
> 	http://marc.info/?l=linux-nfs&m=128080127215350&w=2
> 
> And I'd like it to be possible to make the NFSv3->NFSv4 upgrade as
> transparent as possible.

1) RFC3530 does allow a workaround for cases where the _server_ doesn't
have a mapping from uid/gid -> name. We just haven't implemented it on
Linux servers (or clients).

2) Why is AUTH_SYS so sacrosanct? We know it has a bunch of problems,
not least the one that limits ngroups <= 16, and the fact that it relies
on uids (as opposed to login names) being the same on client and server
so why not try to fix those limitations?

3) SECINFO can advertise a new auth mechanism without any problems. If
done right, there would be no need to change default mount options on
the client.

Trond

Re: numeric UIDs

[J. Bruce Fields]

On Tue, Aug 03, 2010 at 06:31:15PM -0400, Trond Myklebust wrote:
> On Tue, 2010-08-03 at 18:23 -0400, J. Bruce Fields wrote:
> > On Tue, Aug 03, 2010 at 06:15:19PM -0400, Trond Myklebust wrote:
> > > On Tue, 2010-08-03 at 17:57 -0400, Jim Rees wrote:
> > > > Daniel.Muntz@emc.com wrote:
> > > > 
> > > >   I'll fourth this motion.  The spec goes out of its way to declare this a
> > > >   violation.  IMHO, the NFSv4.[0-n] specs should adopt the convention that a
> > > >   uid string consisting of [0-9]+ be interpreted as the string
> > > >   representation of a numeric UID--just as valid as a "user@domain" string.
> > > > 
> > > > I argued for this as an option in the early days but was shouted down.
> > > > Sorry I can't remember the details, it was many years ago.
> > > 
> > > Why is nobody talking about fixing AUTH_SYS? The alternative to using
> > > numeric uids/gids in NFS would be to use user@domain/group@domain in the
> > > credential.
> > 
> > I'm not sure what that does to address complaints like original
> > poster's:
> > 
> > 	http://marc.info/?l=linux-nfs&m=128080127215350&w=2
> > 
> > And I'd like it to be possible to make the NFSv3->NFSv4 upgrade as
> > transparent as possible.
> 
> 1) RFC3530 does allow a workaround for cases where the _server_ doesn't
> have a mapping from uid/gid -> name. We just haven't implemented it on
> Linux servers (or clients).

Yeah, somebody should.

> 2) Why is AUTH_SYS so sacrosanct?

Because it's what almost everyone uses.

> We know it has a bunch of problems,
> not least the one that limits ngroups <= 16, and the fact that it relies
> on uids (as opposed to login names) being the same on client and server
> so why not try to fix those limitations?

Sure, that would be great.

Again, that doesn't address the complaints above.

--b.

> 3) SECINFO can advertise a new auth mechanism without any problems. If
> done right, there would be no need to change default mount options on
> the client.

Re: numeric UIDs

[Trond Myklebust]

On Tue, 2010-08-03 at 18:42 -0400, J. Bruce Fields wrote:
> On Tue, Aug 03, 2010 at 06:31:15PM -0400, Trond Myklebust wrote:
> > On Tue, 2010-08-03 at 18:23 -0400, J. Bruce Fields wrote:
> > > On Tue, Aug 03, 2010 at 06:15:19PM -0400, Trond Myklebust wrote:
> > > > On Tue, 2010-08-03 at 17:57 -0400, Jim Rees wrote:
> > > > > Daniel.Muntz@emc.com wrote:
> > > > > 
> > > > >   I'll fourth this motion.  The spec goes out of its way to declare this a
> > > > >   violation.  IMHO, the NFSv4.[0-n] specs should adopt the convention that a
> > > > >   uid string consisting of [0-9]+ be interpreted as the string
> > > > >   representation of a numeric UID--just as valid as a "user@domain" string.
> > > > > 
> > > > > I argued for this as an option in the early days but was shouted down.
> > > > > Sorry I can't remember the details, it was many years ago.
> > > > 
> > > > Why is nobody talking about fixing AUTH_SYS? The alternative to using
> > > > numeric uids/gids in NFS would be to use user@domain/group@domain in the
> > > > credential.
> > > 
> > > I'm not sure what that does to address complaints like original
> > > poster's:
> > > 
> > > 	http://marc.info/?l=linux-nfs&m=128080127215350&w=2
> > > 
> > > And I'd like it to be possible to make the NFSv3->NFSv4 upgrade as
> > > transparent as possible.
> > 
> > 1) RFC3530 does allow a workaround for cases where the _server_ doesn't
> > have a mapping from uid/gid -> name. We just haven't implemented it on
> > Linux servers (or clients).
> 
> Yeah, somebody should.
> 
> > 2) Why is AUTH_SYS so sacrosanct?
> 
> Because it's what almost everyone uses.

No. It's the _default_. ...and a really really bad default.

The sane thing to do is to change that default instead of trying to add
in yet another ptolemaic epicycle to "fix" just one of the many
problems...
For NFSv4, an auth system that is based on user/groupnames and domains
is a sane fix. It can be made to work with legacy NFSv2/v3 uids and gids
by setting up /etc/passwords, /etc/groups and/or ldap appropriately (as
should already be the case if you are using NFSv2/v3).

> > We know it has a bunch of problems,
> > not least the one that limits ngroups <= 16, and the fact that it relies
> > on uids (as opposed to login names) being the same on client and server
> > so why not try to fix those limitations?
> 
> Sure, that would be great.
> 
> Again, that doesn't address the complaints above.

Yes it does.

It is far better to return an error when the NFS server doesn't
recognize the user and group name rather than to try to use a uid and
gid that is only valid on the local client. Better still, the server
could allocate a new uid or gid (just like Samba does) when it
encounters an unknown user or group name, instead of returning the
error.

The server can still default to the workaround of using the ascii
uid/gid when it is trying to export a file for which it has no
user/group.

It boils down to the fact that we all log in to our clients with
_usernames_, not uids. Trying to make the uid the fundamental object is
the mistake that got us here in the first place.

Trond

Re: numeric UIDs

[David Brodbeck]

On Aug 3, 2010, at 7:02 PM, Trond Myklebust wrote:

> On Tue, 2010-08-03 at 18:42 -0400, J. Bruce Fields wrote:
>> On Tue, Aug 03, 2010 at 06:31:15PM -0400, Trond Myklebust wrote:
>>> On Tue, 2010-08-03 at 18:23 -0400, J. Bruce Fields wrote:
>>>> On Tue, Aug 03, 2010 at 06:15:19PM -0400, Trond Myklebust wrote:
>> 
>>> 2) Why is AUTH_SYS so sacrosanct?
>> 
>> Because it's what almost everyone uses.
> 
> No. It's the _default_. ...and a really really bad default.

The problem is the only supported alternative is to set up Kerberos.  This is a lot of work, especially for established sites where it essentially requires every user to change their password during the migration.  It also creates problems with ticket expiration if you have daemons or batch jobs that need continuous access to NFS filesystems.

I've been looking at it for a while, because the 16-group limit is a problem for us, but it's a huge ball of wax.  I understand the security benefits, but the sheer complexity of setting it up and then coming up with workarounds for ticket expiration has me a bit cowed.

-- 

David Brodbeck
System Administrator, Linguistics
University of Washington

Re: numeric UIDs

[Andy Adamson]

On Aug 4, 2010, at 1:06 PM, David Brodbeck wrote:

> 
> On Aug 3, 2010, at 7:02 PM, Trond Myklebust wrote:
> 
>> On Tue, 2010-08-03 at 18:42 -0400, J. Bruce Fields wrote:
>>> On Tue, Aug 03, 2010 at 06:31:15PM -0400, Trond Myklebust wrote:
>>>> On Tue, 2010-08-03 at 18:23 -0400, J. Bruce Fields wrote:
>>>>> On Tue, Aug 03, 2010 at 06:15:19PM -0400, Trond Myklebust wrote:
>>> 
>>>> 2) Why is AUTH_SYS so sacrosanct?
>>> 
>>> Because it's what almost everyone uses.
>> 
>> No. It's the _default_. ...and a really really bad default.
> 
> The problem is the only supported alternative is to set up Kerberos.  This is a lot of work, especially for established sites where it essentially requires every user to change their password during the migration.  It also creates problems with ticket expiration if you have daemons or batch jobs that need continuous access to NFS filesystems.

Changing passwords is a good thing - should be done on a regular basis anyway. Ticket expiration is handled by using a keytab and a cron job to refresh the keytab.  That said, Kerberos does need to be easier (automatic) to set up than it is.

-->Andy

> 
> I've been looking at it for a while, because the 16-group limit is a problem for us, but it's a huge ball of wax.  I understand the security benefits, but the sheer complexity of setting it up and then coming up with workarounds for ticket expiration has me a bit cowed.
> 
> -- 
> 
> David Brodbeck
> System Administrator, Linguistics
> University of Washington
> 
> 
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: numeric UIDs

[David Brodbeck]

On Aug 4, 2010, at 11:30 AM, Andy Adamson wrote:

> 
> On Aug 4, 2010, at 1:06 PM, David Brodbeck wrote:
> 
>> 
>> On Aug 3, 2010, at 7:02 PM, Trond Myklebust wrote:
>> 
>>> On Tue, 2010-08-03 at 18:42 -0400, J. Bruce Fields wrote:
>>>> On Tue, Aug 03, 2010 at 06:31:15PM -0400, Trond Myklebust wrote:
>>>>> On Tue, 2010-08-03 at 18:23 -0400, J. Bruce Fields wrote:
>>>>>> On Tue, Aug 03, 2010 at 06:15:19PM -0400, Trond Myklebust wrote:
>>>> 
>>>>> 2) Why is AUTH_SYS so sacrosanct?
>>>> 
>>>> Because it's what almost everyone uses.
>>> 
>>> No. It's the _default_. ...and a really really bad default.
>> 
>> The problem is the only supported alternative is to set up Kerberos.  This is a lot of work, especially for established sites where it essentially requires every user to change their password during the migration.  It also creates problems with ticket expiration if you have daemons or batch jobs that need continuous access to NFS filesystems.
> 
> Changing passwords is a good thing - should be done on a regular basis anyway.

True, but getting all the users of an academic system -- some of whom may not be active in any given term -- to change their passwords by a particular flag day is not an easy task.  Doable, sure, but painful.

> Ticket expiration is handled by using a keytab and a cron job to refresh the keytab.

That feels like a hack, but it ought to be workable for daemons like apache.  A bigger problem is Condor batch jobs, which are user-submitted and may run for days or even weeks, potentially after sitting in a queue for several hours.  It's not clear to me how to make this work reliably in a Kerberos environment.  The Kerberos security model seems to implicitly assume that any file access will happen within a short period of time after an interactive login, and that isn't always the case for our site.

There also appear to be outstanding issues where one user's expired Kerberos ticket can cause NFS reads by other users to hang; e.g.:
https://bugzilla.redhat.com/show_bug.cgi?id=537193

-- 

David Brodbeck
System Administrator, Linguistics
University of Washington

Re: numeric UIDs

[J. Bruce Fields]

On Tue, Aug 03, 2010 at 10:02:16PM -0400, Trond Myklebust wrote:
> On Tue, 2010-08-03 at 18:42 -0400, J. Bruce Fields wrote:
> > On Tue, Aug 03, 2010 at 06:31:15PM -0400, Trond Myklebust wrote:
> > > We know it has a bunch of problems,
> > > not least the one that limits ngroups <= 16, and the fact that it relies
> > > on uids (as opposed to login names) being the same on client and server
> > > so why not try to fix those limitations?
> > 
> > Sure, that would be great.
> > 
> > Again, that doesn't address the complaints above.
> 
> Yes it does.

See the stated scenario:

	http://marc.info/?l=linux-nfs&m=128080127215350&w=2

It's a dumb client making a copy of a filesystem over NFS for backup.

It's not true that this case could be dealt with by an auth_sys
replacement that uses names instead of id's.

(You could argue that it's a hypothetical case, crazy, not important, or
whatever--just not that it has much to do with the authentication
flavor.

Personally I think it *is* of at least some importance, since anyone
depending on that sort of behavior will see their systems stop working
if they switch from v2/v3 to v4.  The v2/v3 install base being massive
compared to v4's, the success of v4+ depends in part on reducing the
chances of that kind of thing happening.)

--b.

Re: numeric UIDs

[Neil Brown]

On Wed, 4 Aug 2010 14:32:06 -0700
David Brodbeck <brodbd@u.washington.edu> wrote:

> 
> On Aug 4, 2010, at 11:30 AM, Andy Adamson wrote:
> 
> > 
> > On Aug 4, 2010, at 1:06 PM, David Brodbeck wrote:
> > 
> >> 
> >> On Aug 3, 2010, at 7:02 PM, Trond Myklebust wrote:
> >> 
> >>> On Tue, 2010-08-03 at 18:42 -0400, J. Bruce Fields wrote:
> >>>> On Tue, Aug 03, 2010 at 06:31:15PM -0400, Trond Myklebust wrote:
> >>>>> On Tue, 2010-08-03 at 18:23 -0400, J. Bruce Fields wrote:
> >>>>>> On Tue, Aug 03, 2010 at 06:15:19PM -0400, Trond Myklebust wrote:
> >>>> 
> >>>>> 2) Why is AUTH_SYS so sacrosanct?
> >>>> 
> >>>> Because it's what almost everyone uses.
> >>> 
> >>> No. It's the _default_. ...and a really really bad default.
> >> 
> >> The problem is the only supported alternative is to set up Kerberos.  This is a lot of work, especially for established sites where it essentially requires every user to change their password during the migration.  It also creates problems with ticket expiration if you have daemons or batch jobs that need continuous access to NFS filesystems.
> > 
> > Changing passwords is a good thing - should be done on a regular basis anyway.
> 
> True, 

Not true.  Forced password changing encourages poor choice of passwords and
other poor practices.
Much better to choose a really good password and only change it when you have
reason to believe that it has been compromised, or when you get bored of the
old one.

(better still is two-factor authentication of course).

NeilBrown

Re: numeric UIDs

[Neil Brown]

On Wed, 4 Aug 2010 10:06:05 -0700
David Brodbeck <brodbd@u.washington.edu> wrote:

> I've been looking at it for a while, because the 16-group limit is a problem for us, but it's a huge ball of wax.  I understand the security benefits, but the sheer complexity of setting it up and then coming up with workarounds for ticket expiration has me a bit cowed.
> 

The 16-group limit is easily avoidable if you use Linux as your NFS server
and a given uid maps to the same gids on both client and server.
Just run mounted with "--manage-gids".
Then the gid list on an incoming request will be ignored, and replace with a
gid list obtained by doing a password/group lookup on the server.

NeilBrown

Re: numeric UIDs

[Neil Brown]

On Thu, 5 Aug 2010 11:34:21 -0400
"J. Bruce Fields" <bfields@fieldses.org> wrote:

> On Tue, Aug 03, 2010 at 10:02:16PM -0400, Trond Myklebust wrote:
> > On Tue, 2010-08-03 at 18:42 -0400, J. Bruce Fields wrote:
> > > On Tue, Aug 03, 2010 at 06:31:15PM -0400, Trond Myklebust wrote:
> > > > We know it has a bunch of problems,
> > > > not least the one that limits ngroups <= 16, and the fact that it relies
> > > > on uids (as opposed to login names) being the same on client and server
> > > > so why not try to fix those limitations?
> > > 
> > > Sure, that would be great.
> > > 
> > > Again, that doesn't address the complaints above.
> > 
> > Yes it does.
> 
> See the stated scenario:
> 
> 	http://marc.info/?l=linux-nfs&m=128080127215350&w=2
> 
> It's a dumb client making a copy of a filesystem over NFS for backup.
> 
> It's not true that this case could be dealt with by an auth_sys
> replacement that uses names instead of id's.
> 
> (You could argue that it's a hypothetical case, crazy, not important, or
> whatever--just not that it has much to do with the authentication
> flavor.
> 
> Personally I think it *is* of at least some importance, since anyone
> depending on that sort of behavior will see their systems stop working
> if they switch from v2/v3 to v4.  The v2/v3 install base being massive
> compared to v4's, the success of v4+ depends in part on reducing the
> chances of that kind of thing happening.)

I agree.  And surely it can all be solved in idmapd.

On the server, tell idmapd to map all users to "NUMERIC_USER:%d" and all
groups to "NUMERIC_GROUP:%d" (or whatever) for some given clients (i.e. stop
ignoring the 'authentication name'.  And of course map those names back to
numbers.

I don't know if the client can easily differentiate based on which server it
is talking to, but there is probably less need there (and maybe it can
anyway).

It shouldn't take more that half an hour to hack something into
idmapd.c:nfsdcb() for the server side and nfscb for the client side - or
for a quicker hack, just go directly to imconv and ignore the client name on
the server.  (all this in nfs-utils of course).

NeilBrown

Re: numeric UIDs

[Andy Adamson]

On Aug 11, 2010, at 7:06 PM, Neil Brown wrote:

> On Wed, 4 Aug 2010 14:32:06 -0700
> David Brodbeck <brodbd@u.washington.edu> wrote:
> 
>> 
>> On Aug 4, 2010, at 11:30 AM, Andy Adamson wrote:
>> 
>>> 
>>> On Aug 4, 2010, at 1:06 PM, David Brodbeck wrote:
>>> 
>>>> 
>>>> On Aug 3, 2010, at 7:02 PM, Trond Myklebust wrote:
>>>> 
>>>>> On Tue, 2010-08-03 at 18:42 -0400, J. Bruce Fields wrote:
>>>>>> On Tue, Aug 03, 2010 at 06:31:15PM -0400, Trond Myklebust wrote:
>>>>>>> On Tue, 2010-08-03 at 18:23 -0400, J. Bruce Fields wrote:
>>>>>>>> On Tue, Aug 03, 2010 at 06:15:19PM -0400, Trond Myklebust wrote:
>>>>>> 
>>>>>>> 2) Why is AUTH_SYS so sacrosanct?
>>>>>> 
>>>>>> Because it's what almost everyone uses.
>>>>> 
>>>>> No. It's the _default_. ...and a really really bad default.
>>>> 
>>>> The problem is the only supported alternative is to set up Kerberos.  This is a lot of work, especially for established sites where it essentially requires every user to change their password during the migration.  It also creates problems with ticket expiration if you have daemons or batch jobs that need continuous access to NFS filesystems.
>>> 
>>> Changing passwords is a good thing - should be done on a regular basis anyway.
>> 
>> True, 
> 
> Not true.  Forced password changing encourages poor choice of passwords and
> other poor practices.

This is not rocket science. Picking a good password is easy, once you know how to do it - after all, it's not like there's a shortage of good passwords!

> Much better to choose a really good password and only change it when you have
> reason to believe that it has been compromised,

well, then of course it's too late  :)

> or when you get bored of the
> old one.
> 
> (better still is two-factor authentication of course).
> 
> NeilBrown
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: numeric UIDs

[Steve Dickson]

On 08/03/2010 06:42 PM, J. Bruce Fields wrote:
> On Tue, Aug 03, 2010 at 06:31:15PM -0400, Trond Myklebust wrote:
>> On Tue, 2010-08-03 at 18:23 -0400, J. Bruce Fields wrote:
>>> On Tue, Aug 03, 2010 at 06:15:19PM -0400, Trond Myklebust wrote:
>>>> On Tue, 2010-08-03 at 17:57 -0400, Jim Rees wrote:
>>>>> Daniel.Muntz@emc.com wrote:
>>>>>
>>>>>   I'll fourth this motion.  The spec goes out of its way to declare this a
>>>>>   violation.  IMHO, the NFSv4.[0-n] specs should adopt the convention that a
>>>>>   uid string consisting of [0-9]+ be interpreted as the string
>>>>>   representation of a numeric UID--just as valid as a "user@domain" string.
>>>>>
>>>>> I argued for this as an option in the early days but was shouted down.
>>>>> Sorry I can't remember the details, it was many years ago.
>>>>
>>>> Why is nobody talking about fixing AUTH_SYS? The alternative to using
>>>> numeric uids/gids in NFS would be to use user@domain/group@domain in the
>>>> credential.
>>>
>>> I'm not sure what that does to address complaints like original
>>> poster's:
>>>
>>> 	http://marc.info/?l=linux-nfs&m=128080127215350&w=2
>>>
>>> And I'd like it to be possible to make the NFSv3->NFSv4 upgrade as
>>> transparent as possible.
>>
>> 1) RFC3530 does allow a workaround for cases where the _server_ doesn't
>> have a mapping from uid/gid -> name. We just haven't implemented it on
>> Linux servers (or clients).
> 
> Yeah, somebody should.

I'm assuming you are talking about in the about page 47, third
paragraph that starts with "To provide a greater degree of compatibility..."
or are you referring to a different part of the RFC?


steved.

Re: numeric UIDs

[Steve Dickson]

On 08/11/2010 07:22 PM, Neil Brown wrote:
> 
> I agree.  And surely it can all be solved in idmapd.
> 
> On the server, tell idmapd to map all users to "NUMERIC_USER:%d" and all
> groups to "NUMERIC_GROUP:%d" (or whatever) for some given clients (i.e. stop
> ignoring the 'authentication name'.  And of course map those names back to
> numbers.
> 
> I don't know if the client can easily differentiate based on which server it
> is talking to, but there is probably less need there (and maybe it can
> anyway).
> 
> It shouldn't take more that half an hour to hack something into
> idmapd.c:nfsdcb() for the server side and nfscb for the client side - or
> for a quicker hack, just go directly to imconv and ignore the client name on
> the server.  (all this in nfs-utils of course).
I took a look... and you are right it would not be that difficult to
hack something up... but would this only be a Linux to Linux thing? 
Or am I missing something?

steved.

Re: numeric UIDs

[J. Bruce Fields]

On Fri, Aug 13, 2010 at 10:43:06AM -0400, Steve Dickson wrote:
> 
> 
> On 08/11/2010 07:22 PM, Neil Brown wrote:
> > 
> > I agree.  And surely it can all be solved in idmapd.
> > 
> > On the server, tell idmapd to map all users to "NUMERIC_USER:%d" and all
> > groups to "NUMERIC_GROUP:%d" (or whatever) for some given clients (i.e. stop
> > ignoring the 'authentication name'.  And of course map those names back to
> > numbers.
> > 
> > I don't know if the client can easily differentiate based on which server it
> > is talking to, but there is probably less need there (and maybe it can
> > anyway).
> > 
> > It shouldn't take more that half an hour to hack something into
> > idmapd.c:nfsdcb() for the server side and nfscb for the client side - or
> > for a quicker hack, just go directly to imconv and ignore the client name on
> > the server.  (all this in nfs-utils of course).
> I took a look... and you are right it would not be that difficult to
> hack something up... but would this only be a Linux to Linux thing? 
> Or am I missing something?

There are four cases where translation can be done:

	Sending id from server to client (ls, stat, getacl):
		1. server uid -> string
		2. string -> client uid
	Sending id from client to server (chown, setacl):
		3. client uid -> string
		4. string -> client uid

Cases 1 and 2 are uncontroversial.  Definitely map ascii-fied integers
in both of those cases.

Case 4 violates the SHOULD on page 47.  Which would make case 3 useless
if all servers respect that SHOULD.  I think we should ignore the SHOULD
and implement 3 and 4 too, but Trond may not agree.

I suppose we could make this all configurable, and then argue about what
the defaults should be.  If we implement all this in idmapd then that's
easy.

I don't know what other clients and servers do.  Probably 1 and 2 at
least, but maybe it's something to check at the next bakeathon.

Do we actually use an @-less "nobody" as suggested in the last
paragraph?  If not that might be something else to fix.

--b.

Re: numeric UIDs

[Steve Dickson]

On 08/13/2010 12:31 PM, J. Bruce Fields wrote:
> On Fri, Aug 13, 2010 at 10:43:06AM -0400, Steve Dickson wrote:
>>
>>
>> On 08/11/2010 07:22 PM, Neil Brown wrote:
>>>
>>> I agree.  And surely it can all be solved in idmapd.
>>>
>>> On the server, tell idmapd to map all users to "NUMERIC_USER:%d" and all
>>> groups to "NUMERIC_GROUP:%d" (or whatever) for some given clients (i.e. stop
>>> ignoring the 'authentication name'.  And of course map those names back to
>>> numbers.
>>>
>>> I don't know if the client can easily differentiate based on which server it
>>> is talking to, but there is probably less need there (and maybe it can
>>> anyway).
>>>
>>> It shouldn't take more that half an hour to hack something into
>>> idmapd.c:nfsdcb() for the server side and nfscb for the client side - or
>>> for a quicker hack, just go directly to imconv and ignore the client name on
>>> the server.  (all this in nfs-utils of course).
>> I took a look... and you are right it would not be that difficult to
>> hack something up... but would this only be a Linux to Linux thing? 
>> Or am I missing something?
> 
> There are four cases where translation can be done:
> 
> 	Sending id from server to client (ls, stat, getacl):
> 		1. server uid -> string
> 		2. string -> client uid
> 	Sending id from client to server (chown, setacl):
> 		3. client uid -> string
> 		4. string -> client uid
> 
> Cases 1 and 2 are uncontroversial.  Definitely map ascii-fied integers
> in both of those cases.
Does "ascii-fied integers" mean "3606" (a mapping without the @domain part)?

> 
> Case 4 violates the SHOULD on page 47.  Which would make case 3 useless
> if all servers respect that SHOULD.  I think we should ignore the SHOULD
> and implement 3 and 4 too, but Trond may not agree.
> 
> I suppose we could make this all configurable, and then argue about what
> the defaults should be.  If we implement all this in idmapd then that's
> easy.
I guess... I would think whatever make the v2/v3 to v4 transition
seamless would be the best default... 
 
> 
> I don't know what other clients and servers do.  Probably 1 and 2 at
> least, but maybe it's something to check at the next bakeathon.
> 
> Do we actually use an @-less "nobody" as suggested in the last
> paragraph?  If not that might be something else to fix.
It appears we do... see idtonameres()....

steved.

Re: numeric UIDs

[J. Bruce Fields]

On Fri, Aug 13, 2010 at 01:30:46PM -0400, Steve Dickson wrote:
> 
> 
> On 08/13/2010 12:31 PM, J. Bruce Fields wrote:
> > There are four cases where translation can be done:
> > 
> > 	Sending id from server to client (ls, stat, getacl):
> > 		1. server uid -> string
> > 		2. string -> client uid
> > 	Sending id from client to server (chown, setacl):
> > 		3. client uid -> string
> > 		4. string -> client uid
> > 
> > Cases 1 and 2 are uncontroversial.  Definitely map ascii-fied integers
> > in both of those cases.
> Does "ascii-fied integers" mean "3606" (a mapping without the @domain part)?

That's what I meant, yes.

> 
> > 
> > Case 4 violates the SHOULD on page 47.  Which would make case 3 useless
> > if all servers respect that SHOULD.  I think we should ignore the SHOULD
> > and implement 3 and 4 too, but Trond may not agree.
> > 
> > I suppose we could make this all configurable, and then argue about what
> > the defaults should be.  If we implement all this in idmapd then that's
> > easy.
> I guess... I would think whatever make the v2/v3 to v4 transition
> seamless would be the best default... 
>  
> > 
> > I don't know what other clients and servers do.  Probably 1 and 2 at
> > least, but maybe it's something to check at the next bakeathon.
> > 
> > Do we actually use an @-less "nobody" as suggested in the last
> > paragraph?  If not that might be something else to fix.
> It appears we do... see idtonameres()....

OK, good.

--b.

Re: numeric UIDs

[Chuck Lever]

On Aug 13, 2010, at 12:31 PM, J. Bruce Fields wrote:

> On Fri, Aug 13, 2010 at 10:43:06AM -0400, Steve Dickson wrote:
>> 
>> 
>> On 08/11/2010 07:22 PM, Neil Brown wrote:
>>> 
>>> I agree.  And surely it can all be solved in idmapd.
>>> 
>>> On the server, tell idmapd to map all users to "NUMERIC_USER:%d" and all
>>> groups to "NUMERIC_GROUP:%d" (or whatever) for some given clients (i.e. stop
>>> ignoring the 'authentication name'.  And of course map those names back to
>>> numbers.
>>> 
>>> I don't know if the client can easily differentiate based on which server it
>>> is talking to, but there is probably less need there (and maybe it can
>>> anyway).
>>> 
>>> It shouldn't take more that half an hour to hack something into
>>> idmapd.c:nfsdcb() for the server side and nfscb for the client side - or
>>> for a quicker hack, just go directly to imconv and ignore the client name on
>>> the server.  (all this in nfs-utils of course).
>> I took a look... and you are right it would not be that difficult to
>> hack something up... but would this only be a Linux to Linux thing? 
>> Or am I missing something?
> 
> There are four cases where translation can be done:
> 
> 	Sending id from server to client (ls, stat, getacl):
> 		1. server uid -> string
> 		2. string -> client uid
> 	Sending id from client to server (chown, setacl):
> 		3. client uid -> string
> 		4. string -> client uid
> 
> Cases 1 and 2 are uncontroversial.  Definitely map ascii-fied integers
> in both of those cases.
> 
> Case 4 violates the SHOULD on page 47.  Which would make case 3 useless
> if all servers respect that SHOULD.  I think we should ignore the SHOULD
> and implement 3 and 4 too, but Trond may not agree.

Is this something that should be taken up in 3530bis?

> I suppose we could make this all configurable, and then argue about what
> the defaults should be.  If we implement all this in idmapd then that's
> easy.
> 
> I don't know what other clients and servers do.  Probably 1 and 2 at
> least, but maybe it's something to check at the next bakeathon.
> 
> Do we actually use an @-less "nobody" as suggested in the last
> paragraph?  If not that might be something else to fix.
> 
> --b.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

Re: numeric UIDs

[Neil Brown]

On Fri, 13 Aug 2010 10:43:06 -0400
Steve Dickson <SteveD@redhat.com> wrote:

> 
> 
> On 08/11/2010 07:22 PM, Neil Brown wrote:
> > 
> > I agree.  And surely it can all be solved in idmapd.
> > 
> > On the server, tell idmapd to map all users to "NUMERIC_USER:%d" and all
> > groups to "NUMERIC_GROUP:%d" (or whatever) for some given clients (i.e. stop
> > ignoring the 'authentication name'.  And of course map those names back to
> > numbers.
> > 
> > I don't know if the client can easily differentiate based on which server it
> > is talking to, but there is probably less need there (and maybe it can
> > anyway).
> > 
> > It shouldn't take more that half an hour to hack something into
> > idmapd.c:nfsdcb() for the server side and nfscb for the client side - or
> > for a quicker hack, just go directly to imconv and ignore the client name on
> > the server.  (all this in nfs-utils of course).
> I took a look... and you are right it would not be that difficult to
> hack something up... but would this only be a Linux to Linux thing? 
> Or am I missing something?
> 
> steved.

Yes, I was thinking only Linux to Linux.

But if it works, is well designed, and if there is a customer demand, then one
can expect it to spread (which I think is a much better way of creating
standards than the IETF process..)

NeilBrown

Re: numeric UIDs

[Tom Haynes]

Chuck Lever wrote:
> On Aug 13, 2010, at 12:31 PM, J. Bruce Fields wrote:
>
>   
>> On Fri, Aug 13, 2010 at 10:43:06AM -0400, Steve Dickson wrote:
>>     
>>> On 08/11/2010 07:22 PM, Neil Brown wrote:
>>>       
>>>> I agree.  And surely it can all be solved in idmapd.
>>>>
>>>> On the server, tell idmapd to map all users to "NUMERIC_USER:%d" and all
>>>> groups to "NUMERIC_GROUP:%d" (or whatever) for some given clients (i.e. stop
>>>> ignoring the 'authentication name'.  And of course map those names back to
>>>> numbers.
>>>>
>>>> I don't know if the client can easily differentiate based on which server it
>>>> is talking to, but there is probably less need there (and maybe it can
>>>> anyway).
>>>>
>>>> It shouldn't take more that half an hour to hack something into
>>>> idmapd.c:nfsdcb() for the server side and nfscb for the client side - or
>>>> for a quicker hack, just go directly to imconv and ignore the client name on
>>>> the server.  (all this in nfs-utils of course).
>>>>         
>>> I took a look... and you are right it would not be that difficult to
>>> hack something up... but would this only be a Linux to Linux thing? 
>>> Or am I missing something?
>>>       
>> There are four cases where translation can be done:
>>
>> 	Sending id from server to client (ls, stat, getacl):
>> 		1. server uid -> string
>> 		2. string -> client uid
>> 	Sending id from client to server (chown, setacl):
>> 		3. client uid -> string
>> 		4. string -> client uid
>>
>> Cases 1 and 2 are uncontroversial.  Definitely map ascii-fied integers
>> in both of those cases.
>>
>> Case 4 violates the SHOULD on page 47.  Which would make case 3 useless
>> if all servers respect that SHOULD.  I think we should ignore the SHOULD
>> and implement 3 and 4 too, but Trond may not agree.
>>     
>
>   

So how would that happen?

If we send "2525" and we can locally map uid 2525 to 'bfields", does 
that mean the client is
attempting to subvert the normal process?

Or do we have to send uid 2525 to our id mapper to see if a reverse 
mapping applies?

What if there exists a thud@remote with that uid, but the mapping was 
really bfields@crimson?
We could claim it was a violation, which is why we would want the client 
to provide the
string and not the uid.


> Is this something that should be taken up in 3530bis?
>
>   

The issue should at least be presented on the working group alias.


>> I suppose we could make this all configurable, and then argue about what
>> the defaults should be.  If we implement all this in idmapd then that's
>> easy.
>>
>> I don't know what other clients and servers do.  Probably 1 and 2 at
>> least, but maybe it's something to check at the next bakeathon.
>>
>> Do we actually use an @-less "nobody" as suggested in the last
>> paragraph?  If not that might be something else to fix.
>>
>> --b.
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>     
>
> --
> Chuck Lever
> chuck[dot]lever[at]oracle[dot]com
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>   

Re: numeric UIDs

[Tom Haynes]

J. Bruce Fields wrote:
> On Tue, Aug 03, 2010 at 04:01:32AM +0200, Victor Mataré wrote:
>   
>> Hello,
>>
>> I still hope I'm mistaken in assuming I have to go back to NFSv3 if I want to 
>> skip NFSv4 UID mapping altogether and just use the numeric UIDs the way 
>> they're stored on-disk. However if that's actually true, I'd like to try and 
>> make a case for implementing an option to turn off UID mapping completely (or 
>> at least for unknown UIDs). If this is already work in progress, just ignore 
>> this mail.
>>
>> Thing is, the forced UID mapping seems to make tasks like backing up data a 
>> little inconvenient. You might want to preserve UIDs that are only known to 
>> the client.
>> But when you copy an entire root filesystem, it becomes outright destructive, 
>> because the rootfs will probably have several accounts that the server can't 
>> be expected know. Just imagine a server that's used for maintenance (like 
>> backing up and replacing hard drives) of random (foreign) systems. Idmapd will 
>> map all unknown UIDs to a single value and thereby destroy that information.
>>
>> I think I read somewhere
>>     
>
> Pointer?
>
>   

http://www.unix.com/man-page/OpenSolaris/4/nfs/

       If  a  domainname  is still not obtained following all of the preceding
       steps, nfsmapid will have no domain configured.	This  results  in  the
       following behavior:

	   o	  Outbound  "owner"  and  "owner_group"  attribute strings are
		  encoded as literal id's.  For  example,  the	UID  12345  is
		  encoded as 12345.

	   o	  nfsmapid   ignores  the  "domain"  portion  of  the  inbound
		  attribute string and performs name service lookups only  for
		  the  user  or  group.  If the user/group exists in the local
		  system name service databases, then the proper uid/gid  will
		  be mapped even when no domain has been configured.

		  This	 behavior   implies   that   the  same	administrative
		  user/group domain exists between  NFSv4  client  and	server
		  (that is, the same uid/gid's for users/groups on both client
		  and server). In the  case  of  overlapping  id  spaces,  the
		  inbound  attribute string could potentially be mapped to the
		  wrong id. However, this is not functionally  different  from
		  mapping  the	inbound string to nobody, yet provides greater
		  flexibility.





> --b.
>
>   
>> that the Sun people already have a way of handling 
>> this. Any chance Linux could do that, too?
>>
>> Please excuse me if I'm barking up the wrong tree. If this has already been 
>> discussed, I'd appreciate a pointer.
>>
>> Thanks,
>> Victor
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>     
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>   

Re: numeric UIDs

[J. Bruce Fields]

On Tue, Aug 17, 2010 at 12:46:09PM -0500, Tom Haynes wrote:
> Chuck Lever wrote:
> >On Aug 13, 2010, at 12:31 PM, J. Bruce Fields wrote:
> >>There are four cases where translation can be done:
> >>
> >>	Sending id from server to client (ls, stat, getacl):
> >>		1. server uid -> string
> >>		2. string -> client uid
> >>	Sending id from client to server (chown, setacl):
> >>		3. client uid -> string
> >>		4. string -> client uid
> >>
> >>Cases 1 and 2 are uncontroversial.  Definitely map ascii-fied integers
> >>in both of those cases.
> >>
> >>Case 4 violates the SHOULD on page 47.  Which would make case 3 useless
> >>if all servers respect that SHOULD.  I think we should ignore the SHOULD
> >>and implement 3 and 4 too, but Trond may not agree.
> >
> 
> So how would that happen?

What's the antecedent to "that"?

> If we send "2525" and we can locally map uid 2525 to 'bfields", does
> that mean the client is
> attempting to subvert the normal process?

I don't understand what you mean by "subvert the normal process", nor
what you see as the threat here.

> Or do we have to send uid 2525 to our id mapper to see if a reverse
> mapping applies?

Checking for a reverse mapping doesn't sound like a good idea to me.

> What if there exists a thud@remote with that uid, but the mapping
> was really bfields@crimson?

In the case of a user upgrading from NFSv3 to NFSv4, that's the behavior
they've always had, so presumably they can live with it.  I'd prefer to
avoid situations where something that previously worked over v3 fails
when you upgrade the protocol version.

I assume that most users arrive at NFSv4 by an upgrade from a previous
version of NFS.

So my priorities would be 1) to ensure the NFSv3->NFSv4 upgrade goes
smoothly, then 2) to make it easy for users to switch from ids to
strings, rather than forcing both at once.

--b.

Re: numeric UIDs

[J. Bruce Fields]

On Tue, Aug 17, 2010 at 12:48:32PM -0500, Tom Haynes wrote:
> J. Bruce Fields wrote:
> >On Tue, Aug 03, 2010 at 04:01:32AM +0200, Victor Mataré wrote:
> >>Hello,
> >>
> >>I still hope I'm mistaken in assuming I have to go back to NFSv3
> >>if I want to skip NFSv4 UID mapping altogether and just use the
> >>numeric UIDs the way they're stored on-disk. However if that's
> >>actually true, I'd like to try and make a case for implementing
> >>an option to turn off UID mapping completely (or at least for
> >>unknown UIDs). If this is already work in progress, just ignore
> >>this mail.
> >>
> >>Thing is, the forced UID mapping seems to make tasks like
> >>backing up data a little inconvenient. You might want to
> >>preserve UIDs that are only known to the client.
> >>But when you copy an entire root filesystem, it becomes outright
> >>destructive, because the rootfs will probably have several
> >>accounts that the server can't be expected know. Just imagine a
> >>server that's used for maintenance (like backing up and
> >>replacing hard drives) of random (foreign) systems. Idmapd will
> >>map all unknown UIDs to a single value and thereby destroy that
> >>information.
> >>
> >>I think I read somewhere
> >
> >Pointer?
> >
> 
> http://www.unix.com/man-page/OpenSolaris/4/nfs/
> 
>       If  a  domainname  is still not obtained following all of the preceding
>       steps, nfsmapid will have no domain configured.	This  results  in  the
>       following behavior:
> 
> 	   o	  Outbound  "owner"  and  "owner_group"  attribute strings are
> 		  encoded as literal id's.  For  example,  the	UID  12345  is
> 		  encoded as 12345.
> 
> 	   o	  nfsmapid   ignores  the  "domain"  portion  of  the  inbound
> 		  attribute string and performs name service lookups only  for
> 		  the  user  or  group.  If the user/group exists in the local
> 		  system name service databases, then the proper uid/gid  will
> 		  be mapped even when no domain has been configured.
> 
> 		  This	 behavior   implies   that   the  same	administrative
> 		  user/group domain exists between  NFSv4  client  and	server
> 		  (that is, the same uid/gid's for users/groups on both client
> 		  and server). In the  case  of  overlapping  id  spaces,  the
> 		  inbound  attribute string could potentially be mapped to the
> 		  wrong id. However, this is not functionally  different  from
> 		  mapping  the	inbound string to nobody, yet provides greater
> 		  flexibility.

Thanks!

So the server rejects any attempts to set a numeric uid?

That means the sort of dumb-backup scenario required above would, as in
Linux, succeed over v3, but fail over v4?

And have there been reports of users hitting that issue, or does it
remain a completely hypothetical problem?

--b.

Re: numeric UIDs

[Tom Haynes]

J. Bruce Fields wrote:
> On Tue, Aug 17, 2010 at 12:46:09PM -0500, Tom Haynes wrote:
>   
>> Chuck Lever wrote:
>>     
>>> On Aug 13, 2010, at 12:31 PM, J. Bruce Fields wrote:
>>>       
>>>> There are four cases where translation can be done:
>>>>
>>>> 	Sending id from server to client (ls, stat, getacl):
>>>> 		1. server uid -> string
>>>> 		2. string -> client uid
>>>> 	Sending id from client to server (chown, setacl):
>>>> 		3. client uid -> string
>>>> 		4. string -> client uid
>>>>
>>>> Cases 1 and 2 are uncontroversial.  Definitely map ascii-fied integers
>>>> in both of those cases.
>>>>
>>>> Case 4 violates the SHOULD on page 47.  Which would make case 3 useless
>>>> if all servers respect that SHOULD.  I think we should ignore the SHOULD
>>>> and implement 3 and 4 too, but Trond may not agree.
>>>>         
>> So how would that happen?
>>     
>
> What's the antecedent to "that"?
>
>   

The SHOULD you quote:

   To avoid this mechanism being used to subvert user and
   group translation, so that a client might pass all of the owners and
   groups in numeric form, a server SHOULD return an NFS4ERR_BADOWNER
   error when there is a valid translation for the user or owner
   designated in this way.  In that case, the client must use the
   appropriate name@domain string and not the special form for
   compatibility.



>> If we send "2525" and we can locally map uid 2525 to 'bfields", does
>> that mean the client is
>> attempting to subvert the normal process?
>>     
>
> I don't understand what you mean by "subvert the normal process", nor
> what you see as the threat here.
>
>   

How do you detect the SHOULD?


>> Or do we have to send uid 2525 to our id mapper to see if a reverse
>> mapping applies?
>>     
>
> Checking for a reverse mapping doesn't sound like a good idea to me.
>
>   

No, it doesn't. My point is that the SHOULD is hard to enforce. How does 
the server
determine that there is a "valid translation"?


>> What if there exists a thud@remote with that uid, but the mapping
>> was really bfields@crimson?
>>     
>
> In the case of a user upgrading from NFSv3 to NFSv4, that's the behavior
> they've always had, so presumably they can live with it.  I'd prefer to
> avoid situations where something that previously worked over v3 fails
> when you upgrade the protocol version.
>
> I assume that most users arrive at NFSv4 by an upgrade from a previous
> version of NFS.
>
> So my priorities would be 1) to ensure the NFSv3->NFSv4 upgrade goes
> smoothly, then 2) to make it easy for users to switch from ids to
> strings, rather than forcing both at once.
>
>   

I'd say that problem breaks down to being able to correctly configure 
your id domain or to
allowing a server which is not connected to NIS/LDAP to be able to 
accept random users.
With NFSv3, a server will happily serve up data in these situations.

While various ways pop to mind to hack this up, why not bring it up to 
the working group
and get consensus? Perhaps whomever struck down Jim Rees will recall why 
it is such
a bad idea and convince us all to stay away from it. Or perhaps after 
years of enduring
customers who can't do the above, they will capitulate.

Re: numeric UIDs

[J. Bruce Fields]

On Tue, Aug 17, 2010 at 01:43:45PM -0500, Tom Haynes wrote:
> J. Bruce Fields wrote:
> >On Tue, Aug 17, 2010 at 12:46:09PM -0500, Tom Haynes wrote:
> >>Chuck Lever wrote:
> >>>On Aug 13, 2010, at 12:31 PM, J. Bruce Fields wrote:
> >>>>There are four cases where translation can be done:
> >>>>
> >>>>	Sending id from server to client (ls, stat, getacl):
> >>>>		1. server uid -> string
> >>>>		2. string -> client uid
> >>>>	Sending id from client to server (chown, setacl):
> >>>>		3. client uid -> string
> >>>>		4. string -> client uid
> >>>>
> >>>>Cases 1 and 2 are uncontroversial.  Definitely map ascii-fied integers
> >>>>in both of those cases.
> >>>>
> >>>>Case 4 violates the SHOULD on page 47.  Which would make case 3 useless
> >>>>if all servers respect that SHOULD.  I think we should ignore the SHOULD
> >>>>and implement 3 and 4 too, but Trond may not agree.
> >>So how would that happen?
> >
> >What's the antecedent to "that"?
> >
> 
> The SHOULD you quote:
> 
>   To avoid this mechanism being used to subvert user and
>   group translation, so that a client might pass all of the owners and
>   groups in numeric form, a server SHOULD return an NFS4ERR_BADOWNER
>   error when there is a valid translation for the user or owner
>   designated in this way.  In that case, the client must use the
>   appropriate name@domain string and not the special form for
>   compatibility.
> 
> 
> 
> >>If we send "2525" and we can locally map uid 2525 to 'bfields", does
> >>that mean the client is
> >>attempting to subvert the normal process?
> >
> >I don't understand what you mean by "subvert the normal process", nor
> >what you see as the threat here.
> 
> How do you detect the SHOULD?

I'm suggesting not trying to detect it.

I'm not convinced the SHOULD is a good idea, and I think it would be
acceptable, for backwards-compaibility reasons, for a client to "pass
all owners and groups in numeric form".

> >>Or do we have to send uid 2525 to our id mapper to see if a reverse
> >>mapping applies?
> >
> >Checking for a reverse mapping doesn't sound like a good idea to me.
> >
> 
> No, it doesn't. My point is that the SHOULD is hard to enforce. How
> does the server
> determine that there is a "valid translation"?

Agreed.

> >>What if there exists a thud@remote with that uid, but the mapping
> >>was really bfields@crimson?
> >
> >In the case of a user upgrading from NFSv3 to NFSv4, that's the behavior
> >they've always had, so presumably they can live with it.  I'd prefer to
> >avoid situations where something that previously worked over v3 fails
> >when you upgrade the protocol version.
> >
> >I assume that most users arrive at NFSv4 by an upgrade from a previous
> >version of NFS.
> >
> >So my priorities would be 1) to ensure the NFSv3->NFSv4 upgrade goes
> >smoothly, then 2) to make it easy for users to switch from ids to
> >strings, rather than forcing both at once.
> >
> 
> I'd say that problem breaks down to being able to correctly
> configure your id domain or to
> allowing a server which is not connected to NIS/LDAP to be able to
> accept random users.
> With NFSv3, a server will happily serve up data in these situations.
> 
> While various ways pop to mind to hack this up, why not bring it up
> to the working group
> and get consensus? Perhaps whomever struck down Jim Rees will recall
> why it is such
> a bad idea and convince us all to stay away from it. Or perhaps
> after years of enduring
> customers who can't do the above, they will capitulate.

OK.

--b.

Re: numeric UIDs

[Tom Haynes]

J. Bruce Fields wrote:
> On Tue, Aug 17, 2010 at 12:48:32PM -0500, Tom Haynes wrote:
>   
>> J. Bruce Fields wrote:
>>     
>>> On Tue, Aug 03, 2010 at 04:01:32AM +0200, Victor Mataré wrote:
>>>       
>>>> Hello,
>>>>
>>>> I still hope I'm mistaken in assuming I have to go back to NFSv3
>>>> if I want to skip NFSv4 UID mapping altogether and just use the
>>>> numeric UIDs the way they're stored on-disk. However if that's
>>>> actually true, I'd like to try and make a case for implementing
>>>> an option to turn off UID mapping completely (or at least for
>>>> unknown UIDs). If this is already work in progress, just ignore
>>>> this mail.
>>>>
>>>> Thing is, the forced UID mapping seems to make tasks like
>>>> backing up data a little inconvenient. You might want to
>>>> preserve UIDs that are only known to the client.
>>>> But when you copy an entire root filesystem, it becomes outright
>>>> destructive, because the rootfs will probably have several
>>>> accounts that the server can't be expected know. Just imagine a
>>>> server that's used for maintenance (like backing up and
>>>> replacing hard drives) of random (foreign) systems. Idmapd will
>>>> map all unknown UIDs to a single value and thereby destroy that
>>>> information.
>>>>
>>>> I think I read somewhere
>>>>         
>>> Pointer?
>>>
>>>       
>> http://www.unix.com/man-page/OpenSolaris/4/nfs/
>>
>>       If  a  domainname  is still not obtained following all of the preceding
>>       steps, nfsmapid will have no domain configured.	This  results  in  the
>>       following behavior:
>>
>> 	   o	  Outbound  "owner"  and  "owner_group"  attribute strings are
>> 		  encoded as literal id's.  For  example,  the	UID  12345  is
>> 		  encoded as 12345.
>>
>> 	   o	  nfsmapid   ignores  the  "domain"  portion  of  the  inbound
>> 		  attribute string and performs name service lookups only  for
>> 		  the  user  or  group.  If the user/group exists in the local
>> 		  system name service databases, then the proper uid/gid  will
>> 		  be mapped even when no domain has been configured.
>>
>> 		  This	 behavior   implies   that   the  same	administrative
>> 		  user/group domain exists between  NFSv4  client  and	server
>> 		  (that is, the same uid/gid's for users/groups on both client
>> 		  and server). In the  case  of  overlapping  id  spaces,  the
>> 		  inbound  attribute string could potentially be mapped to the
>> 		  wrong id. However, this is not functionally  different  from
>> 		  mapping  the	inbound string to nobody, yet provides greater
>> 		  flexibility.
>>     
>
> Thanks!
>
> So the server rejects any attempts to set a numeric uid?
>   

No, only ones which do not have a valid mapping in /etc/passwd.

> That means the sort of dumb-backup scenario required above would, as in
> Linux, succeed over v3, but fail over v4?
>   

Yes, it appears so.

> And have there been reports of users hitting that issue, or does it
> remain a completely hypothetical problem?
>
> --b.
>   

It is all hypothetical. I think you would have to work pretty hard to 
get a system in such a state.

And the key would be whether or not the admin populates the local 
/etc/passwd.

Re: numeric UIDs

[J. Bruce Fields]

Tom Haynes said:
> > While various ways pop to mind to hack this up, why not bring it up
> > to the working group and get consensus? Perhaps whomever struck down
> > Jim Rees will recall why it is such a bad idea and convince us all
> > to stay away from it. Or perhaps after years of enduring customers
> > who can't do the above, they will capitulate.

Adding nfsv4 list to the cc.

So, to summarize: every now and then somebody asks why the Linux NFSv4
implementation doesn't allow (ascii-encoded) numerical uid's and gid's;
most recently:

> On Tue, Aug 03, 2010 at 04:01:32AM +0200, Victor Mataré wrote:
>> I still hope I'm mistaken in assuming I have to go back to NFSv3 if I
>> want to skip NFSv4 UID mapping altogether and just use the numeric
>> UIDs the way they're stored on-disk....

They went on to describe a situation (apparently hypothetical) where NFS
is used to perform backups of a filesystem when the server storing the
backup (and maybe even the client performing the backup) lack mappings
referred to by the filesystem.

This is a use case that would work over v3 but likely not v4, as v4
implementors are advised against allowing this on page 47 of rfc 3530:

> >   To avoid this mechanism being used to subvert user and
> >   group translation, so that a client might pass all of the owners and
> >   groups in numeric form, a server SHOULD return an NFS4ERR_BADOWNER
> >   error when there is a valid translation for the user or owner
> >   designated in this way.  In that case, the client must use the
> >   appropriate name@domain string and not the special form for
> >   compatibility.

String uid's have advantages, but I've been arguing that:

> > >In the case of a user upgrading from NFSv3 to NFSv4, that's the behavior
> > >they've always had, so presumably they can live with it.  I'd prefer to
> > >avoid situations where something that previously worked over v3 fails
> > >when you upgrade the protocol version.
> > >
> > >I assume that most users arrive at NFSv4 by an upgrade from a previous
> > >version of NFS.
> > >
> > >So my priorities would be 1) to ensure the NFSv3->NFSv4 upgrade goes
> > >smoothly, then 2) to make it easy for users to switch from ids to
> > >strings, rather than forcing both at once.

So the question is whether there's still consensus for that SHOULD.

As Tom says:
> > I'd say that problem breaks down to being able to correctly
> > configure your id domain or to allowing a server which is not
> > connected to NIS/LDAP to be able to accept random users.  With
> > NFSv3, a server will happily serve up data in these situations.

So I suppose it comes down in part to a question of whether users of the
various implementations have actually seen these problems in practice
and, if so, whether diagnosing the problem, and setting up the required
mapping, has been an obstacle to NFSv4 adoption.

--b.

Re: numeric UIDs

[David Brodbeck]

On Aug 17, 2010, at 12:00 PM, Tom Haynes wrote:

> J. Bruce Fields wrote:
>> 
>> So the server rejects any attempts to set a numeric uid?
>>  
> 
> No, only ones which do not have a valid mapping in /etc/passwd.
> 
>> That means the sort of dumb-backup scenario required above would, as in
>> Linux, succeed over v3, but fail over v4?
>>  
> 
> Yes, it appears so.
> 
>> And have there been reports of users hitting that issue, or does it
>> remain a completely hypothetical problem?
>> 
>> --b.
>>  
> 
> It is all hypothetical. I think you would have to work pretty hard to get a system in such a state.

It comes up pretty regularly when extracting "foreign" tar files on an NFSv4 mount, ones that contain numeric UIDs that don't correspond to a valid user on your system.  You get one error message printed for each file as tar tries to set the numeric UID.  Of course, in that case you can get around it by specifying --no-same-owner, and in most applications it doesn't matter.  It's noticeably different behavior than NFSv3, though.

-- 

David Brodbeck
System Administrator, Linguistics
University of Washington