* Suggested new user link command @ 2018-05-01 8:03 Tony Wallace 2018-05-01 9:03 ` Bernd Petrovitsch 0 siblings, 1 reply; 8+ messages in thread From: Tony Wallace @ 2018-05-01 8:03 UTC (permalink / raw) To: linux-kernel I am suggesting a new command for linking files. Currently there is no easy way to link a file with a known inode number to its correct position in the directory tree. int ilink(const int inode, const char *newpath) The current alternative to this command is to find the file path associated with an inode using the find command and then once found using a standard link command. Obviously this is very inefficient. I am certainly willing to give a patch a go, but I would like approval in principle first. Tony Wallace ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Suggested new user link command 2018-05-01 8:03 Suggested new user link command Tony Wallace @ 2018-05-01 9:03 ` Bernd Petrovitsch 2018-05-01 10:58 ` Tony Wallace 0 siblings, 1 reply; 8+ messages in thread From: Bernd Petrovitsch @ 2018-05-01 9:03 UTC (permalink / raw) To: Tony Wallace, linux-kernel Hi On Tue, 2018-05-01 at 20:03 +1200, Tony Wallace wrote: > I am suggesting a new command for linking files. Currently there is > no > easy way to link a file with a known inode number to its correct > position in the directory tree. > > > int ilink(const int inode, const char *newpath) ... avoiding any permission checks on all paths to the existing names. > The current alternative to this command is to find the file path > associated with an inode using the find command and then once found > using a standard link command. Obviously this is very inefficient. That's the price for security as it requires proper permissions. Or is this a root-only syscall? MfG, Bernd -- Bernd Petrovitsch Email : bernd@petrovitsch.priv.at LUGA : http://www.luga.at ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Suggested new user link command 2018-05-01 9:03 ` Bernd Petrovitsch @ 2018-05-01 10:58 ` Tony Wallace 2018-05-01 11:04 ` Richard Weinberger 0 siblings, 1 reply; 8+ messages in thread From: Tony Wallace @ 2018-05-01 10:58 UTC (permalink / raw) To: Bernd Petrovitsch, linux-kernel Good point. But there are gid and uid fields in inode disc record. https://ext4.wiki.kernel.org/index.php/Ext4_Disk_Layout#Inode_Table I assume these can be use to ensure that the directory in which it is to be placed has permissions to accept the inode. If that is not the case then it would have to be a root only syscall. Tony On 01/05/18 21:03, Bernd Petrovitsch wrote: > Hi > > On Tue, 2018-05-01 at 20:03 +1200, Tony Wallace wrote: >> I am suggesting a new command for linking files. Currently there is >> no >> easy way to link a file with a known inode number to its correct >> position in the directory tree. >> >> >> int ilink(const int inode, const char *newpath) > ... avoiding any permission checks on all paths to the existing names. > >> The current alternative to this command is to find the file path >> associated with an inode using the find command and then once found >> using a standard link command. Obviously this is very inefficient. > That's the price for security as it requires proper permissions. > Or is this a root-only syscall? > > MfG, > Bernd ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Suggested new user link command 2018-05-01 10:58 ` Tony Wallace @ 2018-05-01 11:04 ` Richard Weinberger 2018-05-01 12:17 ` Tony Wallace 0 siblings, 1 reply; 8+ messages in thread From: Richard Weinberger @ 2018-05-01 11:04 UTC (permalink / raw) To: Tony Wallace; +Cc: Bernd Petrovitsch, LKML On Tue, May 1, 2018 at 12:58 PM, Tony Wallace <tony@tony.gen.nz> wrote: > Good point. But there are gid and uid fields in inode disc record. > > https://ext4.wiki.kernel.org/index.php/Ext4_Disk_Layout#Inode_Table > > I assume these can be use to ensure that the directory in which it is to > be placed has permissions to accept the inode. If that is not the case > then it would have to be a root only syscall. Before we dig into details, please come up with a very good use case. :-) But I'm with Bernd, I don't think it is worth the permissions hassle. -- Thanks, //richard ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Suggested new user link command 2018-05-01 11:04 ` Richard Weinberger @ 2018-05-01 12:17 ` Tony Wallace 2018-05-01 13:35 ` Bernd Petrovitsch 0 siblings, 1 reply; 8+ messages in thread From: Tony Wallace @ 2018-05-01 12:17 UTC (permalink / raw) To: Richard Weinberger; +Cc: Bernd Petrovitsch, LKML Two issues here: 1) Use case (which I have) 2) Permissions 1) Use case I am trying to build a backup system. To avoid duplication of files over multiple backups I take an Md5 check sum of file contents. Files with the same sum are hardlinked together. Files are linked in to a standard directory structure a new link for each backup that the file is part of. When all backups pointing to a file are deleted the reference count drops to zero and the file is deleted. We can keep a database of checksums and there related inode numbers for linking purposes. So why not have some reference copy to link against it would take no extra space. Well it doesn't, but it keeps at least one copy of the file on disk forever and the reference count never drops to zero. Using one of the backup copies to link to (as stored as the reference copy in the database) will not work as it could be deleted at any time. I have seen on stack overflow others wanting to do this also. 2) Permissions To maintain security there are two requirements: 2.1) The effective user must have rights to the inode, that is they must either own it or be root 2.2) The effective user must have rights file creation rights to the directory where it is being linked If you say no, that is fine, but I do think this idea has merit and can be done without compromising the system. I am off to bed, it is the middle of the night here in New Zealand. Tony On 01/05/18 23:04, Richard Weinberger wrote: > On Tue, May 1, 2018 at 12:58 PM, Tony Wallace <tony@tony.gen.nz> wrote: >> Good point. But there are gid and uid fields in inode disc record. >> >> https://ext4.wiki.kernel.org/index.php/Ext4_Disk_Layout#Inode_Table >> >> I assume these can be use to ensure that the directory in which it is to >> be placed has permissions to accept the inode. If that is not the case >> then it would have to be a root only syscall. > Before we dig into details, please come up with a very good use case. :-) > But I'm with Bernd, I don't think it is worth the permissions hassle. > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Suggested new user link command 2018-05-01 12:17 ` Tony Wallace @ 2018-05-01 13:35 ` Bernd Petrovitsch 2018-05-01 18:41 ` Tony Wallace 0 siblings, 1 reply; 8+ messages in thread From: Bernd Petrovitsch @ 2018-05-01 13:35 UTC (permalink / raw) To: Tony Wallace, Richard Weinberger; +Cc: LKML Hi all! Top-quoting is evil BTW. On Wed, 2018-05-02 at 00:17 +1200, Tony Wallace wrote: > Two issues here: > 1) Use case (which I have) > 2) Permissions > > 1) Use case > > I am trying to build a backup system. To avoid duplication of files > over multiple backups I take an Md5 check sum of file contents. Files > with the same sum are hardlinked together. Files are linked in to a > standard directory structure a new link for each backup that the file is > part of. When all backups pointing to a file are deleted the reference > count drops to zero and the file is deleted. We can keep a database of > checksums and there related inode numbers for linking purposes. So why a) You can store one of the filenames instead of the inode number. b) You can keep an extra directory with a hardlink named as the inode number (and delete the entries there if the link count drops to 1). > not have some reference copy to link against it would take no extra > space. Well it doesn't, but it keeps at least one copy of the file on You have a (disk) space problems on an backup system? I don't think so, Tim;-) > disk forever and the reference count never drops to zero. Using one of > the backup copies to link to (as stored as the reference copy in the > database) will not work as it could be deleted at any time. > > I have seen on stack overflow others wanting to do this also. "Do. Or do not. There is no try." - Yoda SCNR ..... > 2) Permissions > > To maintain security there are two requirements: > 2.1) The effective user must have rights to the inode, that is they must > either own it or be root > 2.2) The effective user must have rights file creation rights to the > directory where it is being linked Obviously (und useful). And on a backup system, there is no problem about that (because the backup software probably runs as root anyways because otherwise 2.1) below will limit the deduplication severely). But for a (to be mainlined/accepted) new syscall, one should think about all situations/use cases and not just one. Additionally to the 2 items above, one needs also x-permissions on *all* directories from / to one existing hardlink in the traditional case and such a syscall bypasses that. Think about it: Everyone can write a progrm to try link all inodes from 0 to ~0 to a directory entry and gets all files with restrictions 2.1) and 2.2) from below. ATM it is enough to `chmod o= ~` to keep all others from all files in my $HOME. Afterwards it's no longer that easy. > If you say no, that is fine, but I do think this idea has merit and can > be done without compromising the system. I'm no one to say no (or yes;-) here to anything;-) I'm just thinking about the implications. And you can always implement a patch and if it's ignored/not accepted, you can use it locally anyways - no one can stop that:-) One more - more constructive - thing: Perhaps it is more acceptable/useful if there is a mount option which must be activated on the backup filesystems and that is not activated anywhere else. MfG, Bernd -- Bernd Petrovitsch Email : bernd@petrovitsch.priv.at LUGA : http://www.luga.at ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Suggested new user link command 2018-05-01 13:35 ` Bernd Petrovitsch @ 2018-05-01 18:41 ` Tony Wallace 2018-05-05 5:10 ` Eric W. Biederman 0 siblings, 1 reply; 8+ messages in thread From: Tony Wallace @ 2018-05-01 18:41 UTC (permalink / raw) To: Bernd Petrovitsch, Richard Weinberger; +Cc: LKML On 02/05/18 01:35, Bernd Petrovitsch wrote: > Hi all! > > Top-quoting is evil BTW. > > On Wed, 2018-05-02 at 00:17 +1200, Tony Wallace wrote: >> Two issues here: >> 1) Use case (which I have) >> 2) Permissions >> >> 1) Use case >> >> I am trying to build a backup system. To avoid duplication of files >> over multiple backups I take an Md5 check sum of file contents. Files >> with the same sum are hardlinked together. Files are linked in to a >> standard directory structure a new link for each backup that the file is >> part of. When all backups pointing to a file are deleted the reference >> count drops to zero and the file is deleted. We can keep a database of >> checksums and there related inode numbers for linking purposes. So why > a) You can store one of the filenames instead of the inode number. > b) You can keep an extra directory with a hardlink named as the inode > number (and delete the entries there if the link count drops to 1). > >> not have some reference copy to link against it would take no extra >> space. Well it doesn't, but it keeps at least one copy of the file on > You have a (disk) space problems on an backup system? > I don't think so, Tim;-) > >> disk forever and the reference count never drops to zero. Using one of >> the backup copies to link to (as stored as the reference copy in the >> database) will not work as it could be deleted at any time. >> >> I have seen on stack overflow others wanting to do this also. > "Do. Or do not. There is no try." - Yoda > SCNR ..... > >> 2) Permissions >> >> To maintain security there are two requirements: >> 2.1) The effective user must have rights to the inode, that is they must >> either own it or be root >> 2.2) The effective user must have rights file creation rights to the >> directory where it is being linked > Obviously (und useful). And on a backup system, there is no problem > about that (because the backup software probably runs as root anyways > because otherwise 2.1) below will limit the deduplication severely). > > But for a (to be mainlined/accepted) new syscall, one should think > about all situations/use cases and not just one. > > Additionally to the 2 items above, one needs also x-permissions on > *all* directories from / to one existing hardlink in the traditional > case and such a syscall bypasses that. > Think about it: Everyone can write a progrm to try link all inodes from > 0 to ~0 to a directory entry and gets all files with restrictions 2.1) > and 2.2) from below. > ATM it is enough to `chmod o= ~` to keep all others from all files in > my $HOME. Afterwards it's no longer that easy. > >> If you say no, that is fine, but I do think this idea has merit and can >> be done without compromising the system. > I'm no one to say no (or yes;-) here to anything;-) I'm just thinking > about the implications. > > And you can always implement a patch and if it's ignored/not accepted, > you can use it locally anyways - no one can stop that:-) > > One more - more constructive - thing: Perhaps it is more > acceptable/useful if there is a mount option which must be activated on > the backup filesystems and that is not activated anywhere else. > > MfG, > Bernd I want to thank everyone for their time. I have taken note of your comments. I believe that there is the need for a companion command istat that obtains the stat data from an inode. Istat may be useful in constructing ilink. For my proposed use case complexity is minimised, and effectiveness is maximised by making both istat and ilink root only system calls, and then doing my backup as root. I do not know how a mount option would work, and for my own use it is again probably unnecessary complexity, but accept it may be necessary if released more generally. I will be dropping the matter now, at least until I have some code to show, but if anyone has any more thoughts feel free to drop me an email. MfG Tony ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Suggested new user link command 2018-05-01 18:41 ` Tony Wallace @ 2018-05-05 5:10 ` Eric W. Biederman 0 siblings, 0 replies; 8+ messages in thread From: Eric W. Biederman @ 2018-05-05 5:10 UTC (permalink / raw) To: Tony Wallace; +Cc: Bernd Petrovitsch, Richard Weinberger, LKML Tony Wallace <tony@tony.gen.nz> writes: > On 02/05/18 01:35, Bernd Petrovitsch wrote: >> Hi all! >> >> Top-quoting is evil BTW. >> >> On Wed, 2018-05-02 at 00:17 +1200, Tony Wallace wrote: >>> Two issues here: >>> 1) Use case (which I have) >>> 2) Permissions >>> >>> 1) Use case >>> >>> I am trying to build a backup system. To avoid duplication of files >>> over multiple backups I take an Md5 check sum of file contents. Files >>> with the same sum are hardlinked together. Files are linked in to a >>> standard directory structure a new link for each backup that the file is >>> part of. When all backups pointing to a file are deleted the reference >>> count drops to zero and the file is deleted. We can keep a database of >>> checksums and there related inode numbers for linking purposes. So why >> a) You can store one of the filenames instead of the inode number. >> b) You can keep an extra directory with a hardlink named as the inode >> number (and delete the entries there if the link count drops to 1). >> >>> not have some reference copy to link against it would take no extra >>> space. Well it doesn't, but it keeps at least one copy of the file on >> You have a (disk) space problems on an backup system? >> I don't think so, Tim;-) >> >>> disk forever and the reference count never drops to zero. Using one of >>> the backup copies to link to (as stored as the reference copy in the >>> database) will not work as it could be deleted at any time. >>> >>> I have seen on stack overflow others wanting to do this also. >> "Do. Or do not. There is no try." - Yoda >> SCNR ..... >> >>> 2) Permissions >>> >>> To maintain security there are two requirements: >>> 2.1) The effective user must have rights to the inode, that is they must >>> either own it or be root >>> 2.2) The effective user must have rights file creation rights to the >>> directory where it is being linked >> Obviously (und useful). And on a backup system, there is no problem >> about that (because the backup software probably runs as root anyways >> because otherwise 2.1) below will limit the deduplication severely). >> >> But for a (to be mainlined/accepted) new syscall, one should think >> about all situations/use cases and not just one. >> >> Additionally to the 2 items above, one needs also x-permissions on >> *all* directories from / to one existing hardlink in the traditional >> case and such a syscall bypasses that. >> Think about it: Everyone can write a progrm to try link all inodes from >> 0 to ~0 to a directory entry and gets all files with restrictions 2.1) >> and 2.2) from below. >> ATM it is enough to `chmod o= ~` to keep all others from all files in >> my $HOME. Afterwards it's no longer that easy. >> >>> If you say no, that is fine, but I do think this idea has merit and can >>> be done without compromising the system. >> I'm no one to say no (or yes;-) here to anything;-) I'm just thinking >> about the implications. >> >> And you can always implement a patch and if it's ignored/not accepted, >> you can use it locally anyways - no one can stop that:-) >> >> One more - more constructive - thing: Perhaps it is more >> acceptable/useful if there is a mount option which must be activated on >> the backup filesystems and that is not activated anywhere else. >> >> MfG, >> Bernd > > I want to thank everyone for their time. I have taken note of your > comments. I believe that there is the need for a companion command > istat that obtains the stat data from an inode. Istat may be useful in > constructing ilink. For my proposed use case complexity is minimised, > and effectiveness is maximised by making both istat and ilink root only > system calls, and then doing my backup as root. I do not know how a > mount option would work, and for my own use it is again probably > unnecessary complexity, but accept it may be necessary if released more > generally. > > I will be dropping the matter now, at least until I have some code to > show, but if anyone has any more thoughts feel free to drop me an > email. Actually the functionality you are looking for has in some sense already been implemented, and in a way that does not assume a strictly posix filesystem. The system calls are: name_to_handle_at open_by_handle_at Good luck, Eric ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2018-05-05 5:10 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2018-05-01 8:03 Suggested new user link command Tony Wallace 2018-05-01 9:03 ` Bernd Petrovitsch 2018-05-01 10:58 ` Tony Wallace 2018-05-01 11:04 ` Richard Weinberger 2018-05-01 12:17 ` Tony Wallace 2018-05-01 13:35 ` Bernd Petrovitsch 2018-05-01 18:41 ` Tony Wallace 2018-05-05 5:10 ` Eric W. Biederman
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.